[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 13:04:07.700690 14748 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.103.62:38771
I20260709 13:04:07.702307 14748 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 13:04:07.703394 14748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:07.720849 14757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:07.720841 14755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:07.720845 14754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:07.722178 14748 server_base.cc:1061] running on GCE node
I20260709 13:04:08.916750 14748 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:08.916978 14748 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:08.917260 14748 hybrid_clock.cc:648] HybridClock initialized: now 1783602248917227 us; error 0 us; skew 500 ppm
I20260709 13:04:08.918107 14748 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:08.924569 14748 webserver.cc:533] Webserver started at http://127.14.103.62:36959/ using document root <none> and password file <none>
I20260709 13:04:08.925441 14748 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:08.925673 14748 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:08.926139 14748 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:08.932004 14748 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602247531521-14748-0/minicluster-data/master-0-root/instance:
uuid: "d9948df4059e4dc0b63aebc1c6baa8e0"
format_stamp: "Formatted at 2026-07-09 13:04:08 on dist-test-slave-fpp7"
I20260709 13:04:08.939971 14748 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.003s	sys 0.004s
I20260709 13:04:08.945129 14763 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:08.946302 14748 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:04:08.946710 14748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602247531521-14748-0/minicluster-data/master-0-root
uuid: "d9948df4059e4dc0b63aebc1c6baa8e0"
format_stamp: "Formatted at 2026-07-09 13:04:08 on dist-test-slave-fpp7"
I20260709 13:04:08.947064 14748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602247531521-14748-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602247531521-14748-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602247531521-14748-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:09.000100 14748 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:09.001627 14748 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 13:04:09.002115 14748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:09.074479 14748 rpc_server.cc:307] RPC server started. Bound to: 127.14.103.62:38771
I20260709 13:04:09.074563 14814 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.103.62:38771 every 8 connection(s)
I20260709 13:04:09.082729 14815 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:09.104975 14815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9948df4059e4dc0b63aebc1c6baa8e0: Bootstrap starting.
I20260709 13:04:09.111021 14815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9948df4059e4dc0b63aebc1c6baa8e0: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:09.113009 14815 log.cc:826] T 00000000000000000000000000000000 P d9948df4059e4dc0b63aebc1c6baa8e0: Log is configured to *not* fsync() on all Append() calls
I20260709 13:04:09.117686 14815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9948df4059e4dc0b63aebc1c6baa8e0: No bootstrap required, opened a new log
I20260709 13:04:09.138213 14815 raft_consensus.cc:359] T 00000000000000000000000000000000 P d9948df4059e4dc0b63aebc1c6baa8e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9948df4059e4dc0b63aebc1c6baa8e0" member_type: VOTER }
I20260709 13:04:09.138760 14815 raft_consensus.cc:385] T 00000000000000000000000000000000 P d9948df4059e4dc0b63aebc1c6baa8e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:09.139063 14815 raft_consensus.cc:740] T 00000000000000000000000000000000 P d9948df4059e4dc0b63aebc1c6baa8e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9948df4059e4dc0b63aebc1c6baa8e0, State: Initialized, Role: FOLLOWER
I20260709 13:04:09.140092 14815 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9948df4059e4dc0b63aebc1c6baa8e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9948df4059e4dc0b63aebc1c6baa8e0" member_type: VOTER }
I20260709 13:04:09.140568 14815 raft_consensus.cc:399] T 00000000000000000000000000000000 P d9948df4059e4dc0b63aebc1c6baa8e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:04:09.140929 14815 raft_consensus.cc:493] T 00000000000000000000000000000000 P d9948df4059e4dc0b63aebc1c6baa8e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:04:09.141283 14815 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d9948df4059e4dc0b63aebc1c6baa8e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:09.145519 14815 raft_consensus.cc:515] T 00000000000000000000000000000000 P d9948df4059e4dc0b63aebc1c6baa8e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9948df4059e4dc0b63aebc1c6baa8e0" member_type: VOTER }
I20260709 13:04:09.146306 14815 leader_election.cc:304] T 00000000000000000000000000000000 P d9948df4059e4dc0b63aebc1c6baa8e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9948df4059e4dc0b63aebc1c6baa8e0; no voters: 
I20260709 13:04:09.148061 14815 leader_election.cc:290] T 00000000000000000000000000000000 P d9948df4059e4dc0b63aebc1c6baa8e0 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:04:09.148412 14818 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d9948df4059e4dc0b63aebc1c6baa8e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:09.151103 14818 raft_consensus.cc:697] T 00000000000000000000000000000000 P d9948df4059e4dc0b63aebc1c6baa8e0 [term 1 LEADER]: Becoming Leader. State: Replica: d9948df4059e4dc0b63aebc1c6baa8e0, State: Running, Role: LEADER
I20260709 13:04:09.152056 14818 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9948df4059e4dc0b63aebc1c6baa8e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9948df4059e4dc0b63aebc1c6baa8e0" member_type: VOTER }
I20260709 13:04:09.153062 14815 sys_catalog.cc:565] T 00000000000000000000000000000000 P d9948df4059e4dc0b63aebc1c6baa8e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:04:09.164054 14820 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9948df4059e4dc0b63aebc1c6baa8e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9948df4059e4dc0b63aebc1c6baa8e0. Latest consensus state: current_term: 1 leader_uuid: "d9948df4059e4dc0b63aebc1c6baa8e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9948df4059e4dc0b63aebc1c6baa8e0" member_type: VOTER } }
I20260709 13:04:09.164819 14820 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9948df4059e4dc0b63aebc1c6baa8e0 [sys.catalog]: This master's current role is: LEADER
I20260709 13:04:09.166954 14819 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9948df4059e4dc0b63aebc1c6baa8e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9948df4059e4dc0b63aebc1c6baa8e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9948df4059e4dc0b63aebc1c6baa8e0" member_type: VOTER } }
I20260709 13:04:09.167548 14819 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9948df4059e4dc0b63aebc1c6baa8e0 [sys.catalog]: This master's current role is: LEADER
I20260709 13:04:09.168922 14828 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:04:09.180368 14828 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:04:09.183000 14748 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:04:09.198011 14828 catalog_manager.cc:1383] Generated new cluster ID: ace75a3960dd44e9a3e069f699c210f8
I20260709 13:04:09.198305 14828 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:04:09.237730 14828 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:04:09.239434 14828 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:04:09.271873 14828 catalog_manager.cc:6070] T 00000000000000000000000000000000 P d9948df4059e4dc0b63aebc1c6baa8e0: Generated new TSK 0
I20260709 13:04:09.272846 14828 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:04:09.314344 14748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:09.324223 14837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:09.325449 14838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:09.328914 14840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:09.328944 14748 server_base.cc:1061] running on GCE node
I20260709 13:04:09.330080 14748 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:09.330360 14748 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:09.330566 14748 hybrid_clock.cc:648] HybridClock initialized: now 1783602249330548 us; error 0 us; skew 500 ppm
I20260709 13:04:09.331355 14748 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:09.335526 14748 webserver.cc:533] Webserver started at http://127.14.103.1:35393/ using document root <none> and password file <none>
I20260709 13:04:09.336256 14748 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:09.336495 14748 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:09.336833 14748 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:09.338330 14748 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602247531521-14748-0/minicluster-data/ts-0-root/instance:
uuid: "39f9e4e2209748a69d253ea46414e575"
format_stamp: "Formatted at 2026-07-09 13:04:09 on dist-test-slave-fpp7"
I20260709 13:04:09.345304 14748 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.005s	sys 0.004s
I20260709 13:04:09.350451 14845 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:09.351312 14748 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.002s
I20260709 13:04:09.351656 14748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602247531521-14748-0/minicluster-data/ts-0-root
uuid: "39f9e4e2209748a69d253ea46414e575"
format_stamp: "Formatted at 2026-07-09 13:04:09 on dist-test-slave-fpp7"
I20260709 13:04:09.352067 14748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602247531521-14748-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602247531521-14748-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602247531521-14748-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:09.377590 14748 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:09.379348 14748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:09.382153 14748 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:09.387542 14748 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:09.387897 14748 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:04:09.388196 14748 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:09.388422 14748 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:09.479657 14748 rpc_server.cc:307] RPC server started. Bound to: 127.14.103.1:33371
I20260709 13:04:09.479810 14907 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.103.1:33371 every 8 connection(s)
I20260709 13:04:09.490518 14748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:09.500488 14912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:09.502491 14913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:09.512619 14915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:09.512850 14748 server_base.cc:1061] running on GCE node
I20260709 13:04:09.514001 14748 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:09.514278 14748 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:09.514505 14748 hybrid_clock.cc:648] HybridClock initialized: now 1783602249514485 us; error 0 us; skew 500 ppm
I20260709 13:04:09.515079 14748 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:09.517011 14908 heartbeater.cc:344] Connected to a master server at 127.14.103.62:38771
I20260709 13:04:09.517468 14908 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:09.518575 14908 heartbeater.cc:507] Master 127.14.103.62:38771 requested a full tablet report, sending...
I20260709 13:04:09.519619 14748 webserver.cc:533] Webserver started at http://127.14.103.2:42031/ using document root <none> and password file <none>
I20260709 13:04:09.520334 14748 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:09.520581 14748 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:09.520918 14748 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:09.521904 14780 ts_manager.cc:194] Registered new tserver with Master: 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371)
I20260709 13:04:09.522380 14748 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602247531521-14748-0/minicluster-data/ts-1-root/instance:
uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
format_stamp: "Formatted at 2026-07-09 13:04:09 on dist-test-slave-fpp7"
I20260709 13:04:09.524205 14780 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48296
I20260709 13:04:09.529148 14748 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.005s	sys 0.004s
I20260709 13:04:09.534036 14920 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:09.535352 14748 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.000s
I20260709 13:04:09.535720 14748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602247531521-14748-0/minicluster-data/ts-1-root
uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
format_stamp: "Formatted at 2026-07-09 13:04:09 on dist-test-slave-fpp7"
I20260709 13:04:09.536073 14748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602247531521-14748-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602247531521-14748-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602247531521-14748-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:09.572260 14748 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:09.573549 14748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:09.575650 14748 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:09.578542 14748 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:09.578792 14748 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:09.579047 14748 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:09.579221 14748 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:09.679960 14748 rpc_server.cc:307] RPC server started. Bound to: 127.14.103.2:33423
I20260709 13:04:09.680086 14982 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.103.2:33423 every 8 connection(s)
I20260709 13:04:09.688020 14748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:09.697321 14986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:09.701792 14983 heartbeater.cc:344] Connected to a master server at 127.14.103.62:38771
I20260709 13:04:09.702231 14983 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:09.703186 14983 heartbeater.cc:507] Master 127.14.103.62:38771 requested a full tablet report, sending...
I20260709 13:04:09.705555 14780 ts_manager.cc:194] Registered new tserver with Master: 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2:33423)
W20260709 13:04:09.706633 14989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:09.707226 14748 server_base.cc:1061] running on GCE node
I20260709 13:04:09.707455 14780 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48312
W20260709 13:04:09.707235 14987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:09.709003 14748 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:09.709276 14748 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:09.709486 14748 hybrid_clock.cc:648] HybridClock initialized: now 1783602249709466 us; error 0 us; skew 500 ppm
I20260709 13:04:09.710207 14748 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:09.713191 14748 webserver.cc:533] Webserver started at http://127.14.103.3:38427/ using document root <none> and password file <none>
I20260709 13:04:09.713682 14748 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:09.713905 14748 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:09.714177 14748 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:09.715236 14748 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602247531521-14748-0/minicluster-data/ts-2-root/instance:
uuid: "435d239f6f2441ac9d0e835010bfd2b5"
format_stamp: "Formatted at 2026-07-09 13:04:09 on dist-test-slave-fpp7"
I20260709 13:04:09.720062 14748 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 13:04:09.724244 14994 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:09.725106 14748 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.002s
I20260709 13:04:09.725437 14748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602247531521-14748-0/minicluster-data/ts-2-root
uuid: "435d239f6f2441ac9d0e835010bfd2b5"
format_stamp: "Formatted at 2026-07-09 13:04:09 on dist-test-slave-fpp7"
I20260709 13:04:09.725781 14748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602247531521-14748-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602247531521-14748-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602247531521-14748-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:09.750234 14748 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:09.751886 14748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:09.754539 14748 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:09.758245 14748 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:09.758507 14748 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:09.758792 14748 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:09.758988 14748 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:09.843580 14748 rpc_server.cc:307] RPC server started. Bound to: 127.14.103.3:45113
I20260709 13:04:09.844112 15056 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.103.3:45113 every 8 connection(s)
I20260709 13:04:09.861481 15057 heartbeater.cc:344] Connected to a master server at 127.14.103.62:38771
I20260709 13:04:09.861922 15057 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:09.862898 15057 heartbeater.cc:507] Master 127.14.103.62:38771 requested a full tablet report, sending...
I20260709 13:04:09.865219 14780 ts_manager.cc:194] Registered new tserver with Master: 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3:45113)
I20260709 13:04:09.865478 14748 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016466455s
I20260709 13:04:09.866955 14780 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48318
I20260709 13:04:09.914146 14780 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48324:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:04:09.918089 14780 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:04:10.012326 15021 tablet_service.cc:1511] Processing CreateTablet for tablet f54152e1e2cf41a3970778369876d656 (DEFAULT_TABLE table=test-workload [id=d01f9ed16e5e46889b0d9b1073103b8f]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:04:10.013253 15019 tablet_service.cc:1511] Processing CreateTablet for tablet 32bc5a8acc774a1d9c7e99691a591567 (DEFAULT_TABLE table=test-workload [id=d01f9ed16e5e46889b0d9b1073103b8f]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:04:10.014179 15021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f54152e1e2cf41a3970778369876d656. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:10.016134 15018 tablet_service.cc:1511] Processing CreateTablet for tablet bb4dacc583b6453cb577a9e46c45c3af (DEFAULT_TABLE table=test-workload [id=d01f9ed16e5e46889b0d9b1073103b8f]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:04:10.017580 15018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb4dacc583b6453cb577a9e46c45c3af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:10.020311 15019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32bc5a8acc774a1d9c7e99691a591567. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:10.019933 15022 tablet_service.cc:1511] Processing CreateTablet for tablet 810f60f24e33413985aa63f396a2b55a (DEFAULT_TABLE table=test-workload [id=d01f9ed16e5e46889b0d9b1073103b8f]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:04:10.019971 15020 tablet_service.cc:1511] Processing CreateTablet for tablet c7c38457995f4e988d3ccf6bd94e6c37 (DEFAULT_TABLE table=test-workload [id=d01f9ed16e5e46889b0d9b1073103b8f]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:04:10.023633 15020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7c38457995f4e988d3ccf6bd94e6c37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:10.025336 15022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 810f60f24e33413985aa63f396a2b55a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:10.043146 15017 tablet_service.cc:1511] Processing CreateTablet for tablet 14bf28d1d32a48e7973484fb93521945 (DEFAULT_TABLE table=test-workload [id=d01f9ed16e5e46889b0d9b1073103b8f]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:04:10.045279 15017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14bf28d1d32a48e7973484fb93521945. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:10.048733 14947 tablet_service.cc:1511] Processing CreateTablet for tablet f54152e1e2cf41a3970778369876d656 (DEFAULT_TABLE table=test-workload [id=d01f9ed16e5e46889b0d9b1073103b8f]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:04:10.050843 14947 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f54152e1e2cf41a3970778369876d656. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:10.052058 14948 tablet_service.cc:1511] Processing CreateTablet for tablet 810f60f24e33413985aa63f396a2b55a (DEFAULT_TABLE table=test-workload [id=d01f9ed16e5e46889b0d9b1073103b8f]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:04:10.054008 14948 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 810f60f24e33413985aa63f396a2b55a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:10.085510 14946 tablet_service.cc:1511] Processing CreateTablet for tablet c7c38457995f4e988d3ccf6bd94e6c37 (DEFAULT_TABLE table=test-workload [id=d01f9ed16e5e46889b0d9b1073103b8f]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:04:10.085860 14945 tablet_service.cc:1511] Processing CreateTablet for tablet 32bc5a8acc774a1d9c7e99691a591567 (DEFAULT_TABLE table=test-workload [id=d01f9ed16e5e46889b0d9b1073103b8f]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:04:10.087045 14946 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7c38457995f4e988d3ccf6bd94e6c37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:10.087868 14945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32bc5a8acc774a1d9c7e99691a591567. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:10.090399 15016 tablet_service.cc:1511] Processing CreateTablet for tablet be17f0536ffd4425a0db52b530c5f3ef (DEFAULT_TABLE table=test-workload [id=d01f9ed16e5e46889b0d9b1073103b8f]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:04:10.091917 15016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be17f0536ffd4425a0db52b530c5f3ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:10.094977 15015 tablet_service.cc:1511] Processing CreateTablet for tablet 7481805a7cfe4c2c8da2b33cf58c7ea5 (DEFAULT_TABLE table=test-workload [id=d01f9ed16e5e46889b0d9b1073103b8f]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:04:10.096485 15015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7481805a7cfe4c2c8da2b33cf58c7ea5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:10.098106 14942 tablet_service.cc:1511] Processing CreateTablet for tablet be17f0536ffd4425a0db52b530c5f3ef (DEFAULT_TABLE table=test-workload [id=d01f9ed16e5e46889b0d9b1073103b8f]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:04:10.099517 14942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be17f0536ffd4425a0db52b530c5f3ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:10.099779 14940 tablet_service.cc:1511] Processing CreateTablet for tablet 90b851c6655d47c09fca18bbfc1a4d95 (DEFAULT_TABLE table=test-workload [id=d01f9ed16e5e46889b0d9b1073103b8f]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:04:10.101099 14940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90b851c6655d47c09fca18bbfc1a4d95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:10.105130 14941 tablet_service.cc:1511] Processing CreateTablet for tablet 7481805a7cfe4c2c8da2b33cf58c7ea5 (DEFAULT_TABLE table=test-workload [id=d01f9ed16e5e46889b0d9b1073103b8f]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:04:10.105592 15014 tablet_service.cc:1511] Processing CreateTablet for tablet 90b851c6655d47c09fca18bbfc1a4d95 (DEFAULT_TABLE table=test-workload [id=d01f9ed16e5e46889b0d9b1073103b8f]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:04:10.106581 14941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7481805a7cfe4c2c8da2b33cf58c7ea5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:10.111585 15014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90b851c6655d47c09fca18bbfc1a4d95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:10.115957 14944 tablet_service.cc:1511] Processing CreateTablet for tablet bb4dacc583b6453cb577a9e46c45c3af (DEFAULT_TABLE table=test-workload [id=d01f9ed16e5e46889b0d9b1073103b8f]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:04:10.116166 14943 tablet_service.cc:1511] Processing CreateTablet for tablet 14bf28d1d32a48e7973484fb93521945 (DEFAULT_TABLE table=test-workload [id=d01f9ed16e5e46889b0d9b1073103b8f]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:04:10.120623 14943 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14bf28d1d32a48e7973484fb93521945. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:10.129674 14944 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb4dacc583b6453cb577a9e46c45c3af. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:04:10.173938 14827 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:10.179322 14872 tablet_service.cc:1511] Processing CreateTablet for tablet f54152e1e2cf41a3970778369876d656 (DEFAULT_TABLE table=test-workload [id=d01f9ed16e5e46889b0d9b1073103b8f]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:04:10.186715 14872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f54152e1e2cf41a3970778369876d656. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:10.184082 14873 tablet_service.cc:1511] Processing CreateTablet for tablet 810f60f24e33413985aa63f396a2b55a (DEFAULT_TABLE table=test-workload [id=d01f9ed16e5e46889b0d9b1073103b8f]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:04:10.181260 14871 tablet_service.cc:1511] Processing CreateTablet for tablet c7c38457995f4e988d3ccf6bd94e6c37 (DEFAULT_TABLE table=test-workload [id=d01f9ed16e5e46889b0d9b1073103b8f]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:04:10.202931 14871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7c38457995f4e988d3ccf6bd94e6c37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:10.206646 14870 tablet_service.cc:1511] Processing CreateTablet for tablet 32bc5a8acc774a1d9c7e99691a591567 (DEFAULT_TABLE table=test-workload [id=d01f9ed16e5e46889b0d9b1073103b8f]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:04:10.221366 14873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 810f60f24e33413985aa63f396a2b55a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:10.224854 15075 tablet_bootstrap.cc:492] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5: Bootstrap starting.
I20260709 13:04:10.227465 14869 tablet_service.cc:1511] Processing CreateTablet for tablet bb4dacc583b6453cb577a9e46c45c3af (DEFAULT_TABLE table=test-workload [id=d01f9ed16e5e46889b0d9b1073103b8f]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:04:10.209415 14868 tablet_service.cc:1511] Processing CreateTablet for tablet 14bf28d1d32a48e7973484fb93521945 (DEFAULT_TABLE table=test-workload [id=d01f9ed16e5e46889b0d9b1073103b8f]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:04:10.230824 14865 tablet_service.cc:1511] Processing CreateTablet for tablet 90b851c6655d47c09fca18bbfc1a4d95 (DEFAULT_TABLE table=test-workload [id=d01f9ed16e5e46889b0d9b1073103b8f]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:04:10.231281 14866 tablet_service.cc:1511] Processing CreateTablet for tablet 7481805a7cfe4c2c8da2b33cf58c7ea5 (DEFAULT_TABLE table=test-workload [id=d01f9ed16e5e46889b0d9b1073103b8f]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:04:10.231419 14867 tablet_service.cc:1511] Processing CreateTablet for tablet be17f0536ffd4425a0db52b530c5f3ef (DEFAULT_TABLE table=test-workload [id=d01f9ed16e5e46889b0d9b1073103b8f]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:04:10.242285 14870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32bc5a8acc774a1d9c7e99691a591567. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:10.244990 14869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb4dacc583b6453cb577a9e46c45c3af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:10.250140 14865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90b851c6655d47c09fca18bbfc1a4d95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:10.253566 14867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be17f0536ffd4425a0db52b530c5f3ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:10.254812 14868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14bf28d1d32a48e7973484fb93521945. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:10.263386 14866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7481805a7cfe4c2c8da2b33cf58c7ea5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:10.264919 15076 tablet_bootstrap.cc:492] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575: Bootstrap starting.
I20260709 13:04:10.288237 15077 tablet_bootstrap.cc:492] T c7c38457995f4e988d3ccf6bd94e6c37 P 09d3fa2b7ef942ae858e2c5727d99ce5: Bootstrap starting.
I20260709 13:04:10.343040 15077 tablet_bootstrap.cc:654] T c7c38457995f4e988d3ccf6bd94e6c37 P 09d3fa2b7ef942ae858e2c5727d99ce5: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:10.350302 15076 tablet_bootstrap.cc:654] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:10.350960 15075 tablet_bootstrap.cc:654] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:10.366444 15077 tablet_bootstrap.cc:492] T c7c38457995f4e988d3ccf6bd94e6c37 P 09d3fa2b7ef942ae858e2c5727d99ce5: No bootstrap required, opened a new log
I20260709 13:04:10.366925 15077 ts_tablet_manager.cc:1403] T c7c38457995f4e988d3ccf6bd94e6c37 P 09d3fa2b7ef942ae858e2c5727d99ce5: Time spent bootstrapping tablet: real 0.081s	user 0.018s	sys 0.007s
I20260709 13:04:10.369745 15077 raft_consensus.cc:359] T c7c38457995f4e988d3ccf6bd94e6c37 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.370440 15077 raft_consensus.cc:385] T c7c38457995f4e988d3ccf6bd94e6c37 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:10.370609 15076 tablet_bootstrap.cc:492] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575: No bootstrap required, opened a new log
I20260709 13:04:10.370798 15077 raft_consensus.cc:740] T c7c38457995f4e988d3ccf6bd94e6c37 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d3fa2b7ef942ae858e2c5727d99ce5, State: Initialized, Role: FOLLOWER
I20260709 13:04:10.371114 15076 ts_tablet_manager.cc:1403] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575: Time spent bootstrapping tablet: real 0.120s	user 0.012s	sys 0.006s
I20260709 13:04:10.371558 15077 consensus_queue.cc:260] T c7c38457995f4e988d3ccf6bd94e6c37 P 09d3fa2b7ef942ae858e2c5727d99ce5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.374029 15075 tablet_bootstrap.cc:492] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5: No bootstrap required, opened a new log
I20260709 13:04:10.374169 14983 heartbeater.cc:499] Master 127.14.103.62:38771 was elected leader, sending a full tablet report...
I20260709 13:04:10.374478 15075 ts_tablet_manager.cc:1403] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5: Time spent bootstrapping tablet: real 0.160s	user 0.004s	sys 0.015s
I20260709 13:04:10.376473 15077 ts_tablet_manager.cc:1434] T c7c38457995f4e988d3ccf6bd94e6c37 P 09d3fa2b7ef942ae858e2c5727d99ce5: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:04:10.377590 15077 tablet_bootstrap.cc:492] T 32bc5a8acc774a1d9c7e99691a591567 P 09d3fa2b7ef942ae858e2c5727d99ce5: Bootstrap starting.
I20260709 13:04:10.377421 15075 raft_consensus.cc:359] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } }
I20260709 13:04:10.381517 15075 raft_consensus.cc:385] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:10.381829 15075 raft_consensus.cc:740] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 435d239f6f2441ac9d0e835010bfd2b5, State: Initialized, Role: FOLLOWER
I20260709 13:04:10.382527 15075 consensus_queue.cc:260] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } }
I20260709 13:04:10.385360 15077 tablet_bootstrap.cc:654] T 32bc5a8acc774a1d9c7e99691a591567 P 09d3fa2b7ef942ae858e2c5727d99ce5: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:10.388335 15076 raft_consensus.cc:359] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.389334 15076 raft_consensus.cc:385] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:10.389709 15076 raft_consensus.cc:740] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39f9e4e2209748a69d253ea46414e575, State: Initialized, Role: FOLLOWER
I20260709 13:04:10.391022 15057 heartbeater.cc:499] Master 127.14.103.62:38771 was elected leader, sending a full tablet report...
I20260709 13:04:10.392066 15075 ts_tablet_manager.cc:1434] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5: Time spent starting tablet: real 0.017s	user 0.001s	sys 0.014s
I20260709 13:04:10.393085 15075 tablet_bootstrap.cc:492] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5: Bootstrap starting.
I20260709 13:04:10.390628 15076 consensus_queue.cc:260] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.397851 15077 tablet_bootstrap.cc:492] T 32bc5a8acc774a1d9c7e99691a591567 P 09d3fa2b7ef942ae858e2c5727d99ce5: No bootstrap required, opened a new log
I20260709 13:04:10.398569 15077 ts_tablet_manager.cc:1403] T 32bc5a8acc774a1d9c7e99691a591567 P 09d3fa2b7ef942ae858e2c5727d99ce5: Time spent bootstrapping tablet: real 0.021s	user 0.019s	sys 0.000s
I20260709 13:04:10.402372 15077 raft_consensus.cc:359] T 32bc5a8acc774a1d9c7e99691a591567 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.403522 15077 raft_consensus.cc:385] T 32bc5a8acc774a1d9c7e99691a591567 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:10.403914 15077 raft_consensus.cc:740] T 32bc5a8acc774a1d9c7e99691a591567 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d3fa2b7ef942ae858e2c5727d99ce5, State: Initialized, Role: FOLLOWER
I20260709 13:04:10.404590 14908 heartbeater.cc:499] Master 127.14.103.62:38771 was elected leader, sending a full tablet report...
I20260709 13:04:10.405716 15076 ts_tablet_manager.cc:1434] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.006s
I20260709 13:04:10.406603 15076 tablet_bootstrap.cc:492] T 90b851c6655d47c09fca18bbfc1a4d95 P 39f9e4e2209748a69d253ea46414e575: Bootstrap starting.
I20260709 13:04:10.408877 15077 consensus_queue.cc:260] T 32bc5a8acc774a1d9c7e99691a591567 P 09d3fa2b7ef942ae858e2c5727d99ce5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.411103 15077 ts_tablet_manager.cc:1434] T 32bc5a8acc774a1d9c7e99691a591567 P 09d3fa2b7ef942ae858e2c5727d99ce5: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:04:10.412117 15077 tablet_bootstrap.cc:492] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5: Bootstrap starting.
I20260709 13:04:10.415239 15075 tablet_bootstrap.cc:654] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:10.418799 15077 tablet_bootstrap.cc:654] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:10.425280 15075 tablet_bootstrap.cc:492] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5: No bootstrap required, opened a new log
I20260709 13:04:10.425793 15075 ts_tablet_manager.cc:1403] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5: Time spent bootstrapping tablet: real 0.033s	user 0.010s	sys 0.003s
I20260709 13:04:10.428709 15075 raft_consensus.cc:359] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.429193 15077 tablet_bootstrap.cc:492] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5: No bootstrap required, opened a new log
I20260709 13:04:10.429453 15075 raft_consensus.cc:385] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:10.429829 15075 raft_consensus.cc:740] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 435d239f6f2441ac9d0e835010bfd2b5, State: Initialized, Role: FOLLOWER
I20260709 13:04:10.430013 15077 ts_tablet_manager.cc:1403] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5: Time spent bootstrapping tablet: real 0.018s	user 0.009s	sys 0.006s
I20260709 13:04:10.430609 15075 consensus_queue.cc:260] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.432993 15075 ts_tablet_manager.cc:1434] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 13:04:10.433665 15077 raft_consensus.cc:359] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.434453 15077 raft_consensus.cc:385] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:10.434821 15077 raft_consensus.cc:740] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d3fa2b7ef942ae858e2c5727d99ce5, State: Initialized, Role: FOLLOWER
I20260709 13:04:10.433960 15075 tablet_bootstrap.cc:492] T c7c38457995f4e988d3ccf6bd94e6c37 P 435d239f6f2441ac9d0e835010bfd2b5: Bootstrap starting.
W20260709 13:04:10.436718 14984 tablet.cc:2406] T 32bc5a8acc774a1d9c7e99691a591567 P 09d3fa2b7ef942ae858e2c5727d99ce5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:04:10.435906 15077 consensus_queue.cc:260] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.437318 15076 tablet_bootstrap.cc:654] T 90b851c6655d47c09fca18bbfc1a4d95 P 39f9e4e2209748a69d253ea46414e575: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:10.444912 15077 ts_tablet_manager.cc:1434] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.001s
I20260709 13:04:10.445147 15075 tablet_bootstrap.cc:654] T c7c38457995f4e988d3ccf6bd94e6c37 P 435d239f6f2441ac9d0e835010bfd2b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:10.446067 15077 tablet_bootstrap.cc:492] T be17f0536ffd4425a0db52b530c5f3ef P 09d3fa2b7ef942ae858e2c5727d99ce5: Bootstrap starting.
I20260709 13:04:10.452766 15077 tablet_bootstrap.cc:654] T be17f0536ffd4425a0db52b530c5f3ef P 09d3fa2b7ef942ae858e2c5727d99ce5: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:10.466497 15075 tablet_bootstrap.cc:492] T c7c38457995f4e988d3ccf6bd94e6c37 P 435d239f6f2441ac9d0e835010bfd2b5: No bootstrap required, opened a new log
I20260709 13:04:10.467424 15075 ts_tablet_manager.cc:1403] T c7c38457995f4e988d3ccf6bd94e6c37 P 435d239f6f2441ac9d0e835010bfd2b5: Time spent bootstrapping tablet: real 0.034s	user 0.020s	sys 0.006s
I20260709 13:04:10.470252 15075 raft_consensus.cc:359] T c7c38457995f4e988d3ccf6bd94e6c37 P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.470983 15075 raft_consensus.cc:385] T c7c38457995f4e988d3ccf6bd94e6c37 P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:10.471280 15075 raft_consensus.cc:740] T c7c38457995f4e988d3ccf6bd94e6c37 P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 435d239f6f2441ac9d0e835010bfd2b5, State: Initialized, Role: FOLLOWER
I20260709 13:04:10.472019 15075 consensus_queue.cc:260] T c7c38457995f4e988d3ccf6bd94e6c37 P 435d239f6f2441ac9d0e835010bfd2b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.474488 15075 ts_tablet_manager.cc:1434] T c7c38457995f4e988d3ccf6bd94e6c37 P 435d239f6f2441ac9d0e835010bfd2b5: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:04:10.475448 15075 tablet_bootstrap.cc:492] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5: Bootstrap starting.
I20260709 13:04:10.476452 15076 tablet_bootstrap.cc:492] T 90b851c6655d47c09fca18bbfc1a4d95 P 39f9e4e2209748a69d253ea46414e575: No bootstrap required, opened a new log
I20260709 13:04:10.476969 15076 ts_tablet_manager.cc:1403] T 90b851c6655d47c09fca18bbfc1a4d95 P 39f9e4e2209748a69d253ea46414e575: Time spent bootstrapping tablet: real 0.071s	user 0.014s	sys 0.009s
I20260709 13:04:10.479941 15076 raft_consensus.cc:359] T 90b851c6655d47c09fca18bbfc1a4d95 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.481940 15075 tablet_bootstrap.cc:654] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:10.482095 15077 tablet_bootstrap.cc:492] T be17f0536ffd4425a0db52b530c5f3ef P 09d3fa2b7ef942ae858e2c5727d99ce5: No bootstrap required, opened a new log
I20260709 13:04:10.482784 15077 ts_tablet_manager.cc:1403] T be17f0536ffd4425a0db52b530c5f3ef P 09d3fa2b7ef942ae858e2c5727d99ce5: Time spent bootstrapping tablet: real 0.037s	user 0.013s	sys 0.005s
I20260709 13:04:10.486028 15077 raft_consensus.cc:359] T be17f0536ffd4425a0db52b530c5f3ef P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.486670 15077 raft_consensus.cc:385] T be17f0536ffd4425a0db52b530c5f3ef P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:10.486945 15077 raft_consensus.cc:740] T be17f0536ffd4425a0db52b530c5f3ef P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d3fa2b7ef942ae858e2c5727d99ce5, State: Initialized, Role: FOLLOWER
I20260709 13:04:10.483049 15076 raft_consensus.cc:385] T 90b851c6655d47c09fca18bbfc1a4d95 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:10.487903 15076 raft_consensus.cc:740] T 90b851c6655d47c09fca18bbfc1a4d95 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39f9e4e2209748a69d253ea46414e575, State: Initialized, Role: FOLLOWER
I20260709 13:04:10.487579 15077 consensus_queue.cc:260] T be17f0536ffd4425a0db52b530c5f3ef P 09d3fa2b7ef942ae858e2c5727d99ce5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.490093 15077 ts_tablet_manager.cc:1434] T be17f0536ffd4425a0db52b530c5f3ef P 09d3fa2b7ef942ae858e2c5727d99ce5: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.004s
I20260709 13:04:10.491029 15077 tablet_bootstrap.cc:492] T f54152e1e2cf41a3970778369876d656 P 09d3fa2b7ef942ae858e2c5727d99ce5: Bootstrap starting.
I20260709 13:04:10.497589 15077 tablet_bootstrap.cc:654] T f54152e1e2cf41a3970778369876d656 P 09d3fa2b7ef942ae858e2c5727d99ce5: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:10.498179 15075 tablet_bootstrap.cc:492] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5: No bootstrap required, opened a new log
I20260709 13:04:10.498638 15075 ts_tablet_manager.cc:1403] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.011s
I20260709 13:04:10.501566 15075 raft_consensus.cc:359] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.502277 15075 raft_consensus.cc:385] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:10.502557 15075 raft_consensus.cc:740] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 435d239f6f2441ac9d0e835010bfd2b5, State: Initialized, Role: FOLLOWER
I20260709 13:04:10.503232 15075 consensus_queue.cc:260] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.505656 15075 ts_tablet_manager.cc:1434] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:04:10.506583 15075 tablet_bootstrap.cc:492] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5: Bootstrap starting.
I20260709 13:04:10.503505 15076 consensus_queue.cc:260] T 90b851c6655d47c09fca18bbfc1a4d95 P 39f9e4e2209748a69d253ea46414e575 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.513145 15075 tablet_bootstrap.cc:654] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:10.514767 15077 tablet_bootstrap.cc:492] T f54152e1e2cf41a3970778369876d656 P 09d3fa2b7ef942ae858e2c5727d99ce5: No bootstrap required, opened a new log
I20260709 13:04:10.515288 15077 ts_tablet_manager.cc:1403] T f54152e1e2cf41a3970778369876d656 P 09d3fa2b7ef942ae858e2c5727d99ce5: Time spent bootstrapping tablet: real 0.024s	user 0.015s	sys 0.008s
I20260709 13:04:10.518311 15077 raft_consensus.cc:359] T f54152e1e2cf41a3970778369876d656 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.519073 15077 raft_consensus.cc:385] T f54152e1e2cf41a3970778369876d656 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:10.519449 15077 raft_consensus.cc:740] T f54152e1e2cf41a3970778369876d656 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d3fa2b7ef942ae858e2c5727d99ce5, State: Initialized, Role: FOLLOWER
I20260709 13:04:10.519614 15076 ts_tablet_manager.cc:1434] T 90b851c6655d47c09fca18bbfc1a4d95 P 39f9e4e2209748a69d253ea46414e575: Time spent starting tablet: real 0.042s	user 0.006s	sys 0.000s
I20260709 13:04:10.520644 15076 tablet_bootstrap.cc:492] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575: Bootstrap starting.
I20260709 13:04:10.520309 15077 consensus_queue.cc:260] T f54152e1e2cf41a3970778369876d656 P 09d3fa2b7ef942ae858e2c5727d99ce5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.522728 15077 ts_tablet_manager.cc:1434] T f54152e1e2cf41a3970778369876d656 P 09d3fa2b7ef942ae858e2c5727d99ce5: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:04:10.523718 15077 tablet_bootstrap.cc:492] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5: Bootstrap starting.
I20260709 13:04:10.528201 15075 tablet_bootstrap.cc:492] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5: No bootstrap required, opened a new log
I20260709 13:04:10.529192 15075 ts_tablet_manager.cc:1403] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.012s
I20260709 13:04:10.531566 15077 tablet_bootstrap.cc:654] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:10.532256 15075 raft_consensus.cc:359] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.533015 15075 raft_consensus.cc:385] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:10.533334 15075 raft_consensus.cc:740] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 435d239f6f2441ac9d0e835010bfd2b5, State: Initialized, Role: FOLLOWER
I20260709 13:04:10.534092 15075 consensus_queue.cc:260] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.536532 15075 ts_tablet_manager.cc:1434] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:04:10.537437 15075 tablet_bootstrap.cc:492] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5: Bootstrap starting.
I20260709 13:04:10.543298 15077 tablet_bootstrap.cc:492] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5: No bootstrap required, opened a new log
I20260709 13:04:10.544085 15077 ts_tablet_manager.cc:1403] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5: Time spent bootstrapping tablet: real 0.021s	user 0.007s	sys 0.011s
I20260709 13:04:10.544230 15082 raft_consensus.cc:493] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.544703 15082 raft_consensus.cc:515] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } }
I20260709 13:04:10.547612 15082 leader_election.cc:290] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2:33423), 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371)
I20260709 13:04:10.543831 15075 tablet_bootstrap.cc:654] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:10.547065 15077 raft_consensus.cc:359] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.556092 15077 raft_consensus.cc:385] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:10.557663 15077 raft_consensus.cc:740] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d3fa2b7ef942ae858e2c5727d99ce5, State: Initialized, Role: FOLLOWER
I20260709 13:04:10.559262 15082 raft_consensus.cc:493] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.559733 15082 raft_consensus.cc:515] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.561015 15077 consensus_queue.cc:260] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.562146 15082 leader_election.cc:290] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371), 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2:33423)
I20260709 13:04:10.569223 15075 tablet_bootstrap.cc:492] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5: No bootstrap required, opened a new log
I20260709 13:04:10.569664 15075 ts_tablet_manager.cc:1403] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5: Time spent bootstrapping tablet: real 0.032s	user 0.005s	sys 0.012s
I20260709 13:04:10.569818 15077 ts_tablet_manager.cc:1434] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.001s
I20260709 13:04:10.571795 15077 tablet_bootstrap.cc:492] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5: Bootstrap starting.
I20260709 13:04:10.573957 15075 raft_consensus.cc:359] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.574642 15075 raft_consensus.cc:385] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:10.574924 15075 raft_consensus.cc:740] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 435d239f6f2441ac9d0e835010bfd2b5, State: Initialized, Role: FOLLOWER
I20260709 13:04:10.575656 15075 consensus_queue.cc:260] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.578045 15075 ts_tablet_manager.cc:1434] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5: Time spent starting tablet: real 0.008s	user 0.001s	sys 0.005s
I20260709 13:04:10.578930 15075 tablet_bootstrap.cc:492] T 90b851c6655d47c09fca18bbfc1a4d95 P 435d239f6f2441ac9d0e835010bfd2b5: Bootstrap starting.
I20260709 13:04:10.579487 15077 tablet_bootstrap.cc:654] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:10.581722 15076 tablet_bootstrap.cc:654] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:10.584894 15075 tablet_bootstrap.cc:654] T 90b851c6655d47c09fca18bbfc1a4d95 P 435d239f6f2441ac9d0e835010bfd2b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:10.598675 15076 tablet_bootstrap.cc:492] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575: No bootstrap required, opened a new log
I20260709 13:04:10.599269 15076 ts_tablet_manager.cc:1403] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575: Time spent bootstrapping tablet: real 0.079s	user 0.015s	sys 0.000s
I20260709 13:04:10.602169 14958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "810f60f24e33413985aa63f396a2b55a" candidate_uuid: "435d239f6f2441ac9d0e835010bfd2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" is_pre_election: true
W20260709 13:04:10.604146 14996 leader_election.cc:343] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2:33423): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:10.604138 14957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32bc5a8acc774a1d9c7e99691a591567" candidate_uuid: "435d239f6f2441ac9d0e835010bfd2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" is_pre_election: true
I20260709 13:04:10.604966 14957 raft_consensus.cc:2468] T 32bc5a8acc774a1d9c7e99691a591567 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 435d239f6f2441ac9d0e835010bfd2b5 in term 0.
I20260709 13:04:10.606143 14996 leader_election.cc:304] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d3fa2b7ef942ae858e2c5727d99ce5, 435d239f6f2441ac9d0e835010bfd2b5; no voters: 
I20260709 13:04:10.606596 14883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "810f60f24e33413985aa63f396a2b55a" candidate_uuid: "435d239f6f2441ac9d0e835010bfd2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f9e4e2209748a69d253ea46414e575" is_pre_election: true
I20260709 13:04:10.606973 15082 raft_consensus.cc:2804] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:10.607349 15082 raft_consensus.cc:493] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.607656 15082 raft_consensus.cc:3060] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:10.608186 14882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32bc5a8acc774a1d9c7e99691a591567" candidate_uuid: "435d239f6f2441ac9d0e835010bfd2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f9e4e2209748a69d253ea46414e575" is_pre_election: true
I20260709 13:04:10.607957 15076 raft_consensus.cc:359] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.608824 15076 raft_consensus.cc:385] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:10.609131 15076 raft_consensus.cc:740] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39f9e4e2209748a69d253ea46414e575, State: Initialized, Role: FOLLOWER
I20260709 13:04:10.609833 15076 consensus_queue.cc:260] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.612344 15076 ts_tablet_manager.cc:1434] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:04:10.613359 15076 tablet_bootstrap.cc:492] T bb4dacc583b6453cb577a9e46c45c3af P 39f9e4e2209748a69d253ea46414e575: Bootstrap starting.
I20260709 13:04:10.614619 15082 raft_consensus.cc:515] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
W20260709 13:04:10.616412 14996 leader_election.cc:343] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:10.625193 15084 raft_consensus.cc:493] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.626614 15084 raft_consensus.cc:515] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.628017 15082 leader_election.cc:290] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5 [CANDIDATE]: Term 1 election: Requested vote from peers 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371), 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2:33423)
I20260709 13:04:10.629390 14996 leader_election.cc:304] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 435d239f6f2441ac9d0e835010bfd2b5; no voters: 09d3fa2b7ef942ae858e2c5727d99ce5, 39f9e4e2209748a69d253ea46414e575
W20260709 13:04:10.630904 14996 leader_election.cc:343] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:10.633181 15077 tablet_bootstrap.cc:492] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5: No bootstrap required, opened a new log
I20260709 13:04:10.634505 15084 leader_election.cc:290] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2:33423), 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3:45113)
I20260709 13:04:10.632709 15082 raft_consensus.cc:2749] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:10.636627 15077 ts_tablet_manager.cc:1403] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5: Time spent bootstrapping tablet: real 0.065s	user 0.001s	sys 0.012s
I20260709 13:04:10.638345 15075 tablet_bootstrap.cc:492] T 90b851c6655d47c09fca18bbfc1a4d95 P 435d239f6f2441ac9d0e835010bfd2b5: No bootstrap required, opened a new log
I20260709 13:04:10.639251 14883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32bc5a8acc774a1d9c7e99691a591567" candidate_uuid: "435d239f6f2441ac9d0e835010bfd2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f9e4e2209748a69d253ea46414e575"
I20260709 13:04:10.640925 14957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32bc5a8acc774a1d9c7e99691a591567" candidate_uuid: "435d239f6f2441ac9d0e835010bfd2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
I20260709 13:04:10.641537 14957 raft_consensus.cc:3060] T 32bc5a8acc774a1d9c7e99691a591567 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:10.643265 15082 raft_consensus.cc:493] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.642413 15075 ts_tablet_manager.cc:1403] T 90b851c6655d47c09fca18bbfc1a4d95 P 435d239f6f2441ac9d0e835010bfd2b5: Time spent bootstrapping tablet: real 0.064s	user 0.008s	sys 0.010s
I20260709 13:04:10.650064 14957 raft_consensus.cc:2468] T 32bc5a8acc774a1d9c7e99691a591567 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 435d239f6f2441ac9d0e835010bfd2b5 in term 1.
I20260709 13:04:10.649760 15077 raft_consensus.cc:359] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } }
I20260709 13:04:10.650767 15077 raft_consensus.cc:385] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:10.651089 15077 raft_consensus.cc:740] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d3fa2b7ef942ae858e2c5727d99ce5, State: Initialized, Role: FOLLOWER
I20260709 13:04:10.643901 15082 raft_consensus.cc:515] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.661296 15081 raft_consensus.cc:493] T c7c38457995f4e988d3ccf6bd94e6c37 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.661806 15081 raft_consensus.cc:515] T c7c38457995f4e988d3ccf6bd94e6c37 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.674861 15081 leader_election.cc:290] T c7c38457995f4e988d3ccf6bd94e6c37 P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371), 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3:45113)
I20260709 13:04:10.673509 15082 leader_election.cc:290] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371), 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2:33423)
I20260709 13:04:10.663887 15077 consensus_queue.cc:260] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } }
I20260709 13:04:10.677026 15076 tablet_bootstrap.cc:654] T bb4dacc583b6453cb577a9e46c45c3af P 39f9e4e2209748a69d253ea46414e575: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:10.681656 15075 raft_consensus.cc:359] T 90b851c6655d47c09fca18bbfc1a4d95 P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.684010 15075 raft_consensus.cc:385] T 90b851c6655d47c09fca18bbfc1a4d95 P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:10.686018 15075 raft_consensus.cc:740] T 90b851c6655d47c09fca18bbfc1a4d95 P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 435d239f6f2441ac9d0e835010bfd2b5, State: Initialized, Role: FOLLOWER
I20260709 13:04:10.685490 15081 raft_consensus.cc:493] T be17f0536ffd4425a0db52b530c5f3ef P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.689357 15077 ts_tablet_manager.cc:1434] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5: Time spent starting tablet: real 0.050s	user 0.004s	sys 0.004s
I20260709 13:04:10.690587 15077 tablet_bootstrap.cc:492] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5: Bootstrap starting.
I20260709 13:04:10.687249 15075 consensus_queue.cc:260] T 90b851c6655d47c09fca18bbfc1a4d95 P 435d239f6f2441ac9d0e835010bfd2b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.688053 15081 raft_consensus.cc:515] T be17f0536ffd4425a0db52b530c5f3ef P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.709529 15075 ts_tablet_manager.cc:1434] T 90b851c6655d47c09fca18bbfc1a4d95 P 435d239f6f2441ac9d0e835010bfd2b5: Time spent starting tablet: real 0.056s	user 0.007s	sys 0.002s
I20260709 13:04:10.711752 15081 leader_election.cc:290] T be17f0536ffd4425a0db52b530c5f3ef P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371), 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3:45113)
I20260709 13:04:10.712093 15075 tablet_bootstrap.cc:492] T 14bf28d1d32a48e7973484fb93521945 P 435d239f6f2441ac9d0e835010bfd2b5: Bootstrap starting.
I20260709 13:04:10.722242 14883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17f0536ffd4425a0db52b530c5f3ef" candidate_uuid: "435d239f6f2441ac9d0e835010bfd2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f9e4e2209748a69d253ea46414e575" is_pre_election: true
I20260709 13:04:10.727779 14957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17f0536ffd4425a0db52b530c5f3ef" candidate_uuid: "435d239f6f2441ac9d0e835010bfd2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" is_pre_election: true
I20260709 13:04:10.728582 14958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7c38457995f4e988d3ccf6bd94e6c37" candidate_uuid: "39f9e4e2209748a69d253ea46414e575" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" is_pre_election: true
I20260709 13:04:10.730520 14957 raft_consensus.cc:2468] T be17f0536ffd4425a0db52b530c5f3ef P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 435d239f6f2441ac9d0e835010bfd2b5 in term 0.
I20260709 13:04:10.730633 14958 raft_consensus.cc:2468] T c7c38457995f4e988d3ccf6bd94e6c37 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39f9e4e2209748a69d253ea46414e575 in term 0.
I20260709 13:04:10.737659 14847 leader_election.cc:304] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d3fa2b7ef942ae858e2c5727d99ce5, 39f9e4e2209748a69d253ea46414e575; no voters: 
I20260709 13:04:10.734516 14996 leader_election.cc:304] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d3fa2b7ef942ae858e2c5727d99ce5, 435d239f6f2441ac9d0e835010bfd2b5; no voters: 
I20260709 13:04:10.739279 15084 raft_consensus.cc:2804] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:10.739944 15084 raft_consensus.cc:493] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.740361 15084 raft_consensus.cc:3060] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:10.740551 15104 raft_consensus.cc:2804] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:10.738235 15077 tablet_bootstrap.cc:654] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:10.741586 14996 leader_election.cc:343] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:10.745621 15104 raft_consensus.cc:697] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 LEADER]: Becoming Leader. State: Replica: 435d239f6f2441ac9d0e835010bfd2b5, State: Running, Role: LEADER
W20260709 13:04:10.748086 14996 leader_election.cc:343] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:10.750574 14996 leader_election.cc:304] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 09d3fa2b7ef942ae858e2c5727d99ce5, 435d239f6f2441ac9d0e835010bfd2b5; no voters: 39f9e4e2209748a69d253ea46414e575
I20260709 13:04:10.752425 15082 raft_consensus.cc:2804] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:10.753057 15082 raft_consensus.cc:493] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.753855 15082 raft_consensus.cc:3060] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:10.756083 15084 raft_consensus.cc:515] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.758656 15084 leader_election.cc:290] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575 [CANDIDATE]: Term 1 election: Requested vote from peers 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2:33423), 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3:45113)
I20260709 13:04:10.751716 15104 consensus_queue.cc:237] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.761914 14958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7c38457995f4e988d3ccf6bd94e6c37" candidate_uuid: "39f9e4e2209748a69d253ea46414e575" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
I20260709 13:04:10.762923 14958 raft_consensus.cc:3060] T c7c38457995f4e988d3ccf6bd94e6c37 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:10.767392 15082 raft_consensus.cc:515] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.770102 15075 tablet_bootstrap.cc:654] T 14bf28d1d32a48e7973484fb93521945 P 435d239f6f2441ac9d0e835010bfd2b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:10.770219 15082 leader_election.cc:290] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5 [CANDIDATE]: Term 1 election: Requested vote from peers 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371), 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2:33423)
I20260709 13:04:10.780599 14883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17f0536ffd4425a0db52b530c5f3ef" candidate_uuid: "435d239f6f2441ac9d0e835010bfd2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f9e4e2209748a69d253ea46414e575"
I20260709 13:04:10.783998 15107 raft_consensus.cc:493] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.784883 14957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17f0536ffd4425a0db52b530c5f3ef" candidate_uuid: "435d239f6f2441ac9d0e835010bfd2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
I20260709 13:04:10.784562 15107 raft_consensus.cc:515] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.787599 15076 tablet_bootstrap.cc:492] T bb4dacc583b6453cb577a9e46c45c3af P 39f9e4e2209748a69d253ea46414e575: No bootstrap required, opened a new log
I20260709 13:04:10.788657 15084 raft_consensus.cc:493] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.789713 14883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7c38457995f4e988d3ccf6bd94e6c37" candidate_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f9e4e2209748a69d253ea46414e575" is_pre_election: true
I20260709 13:04:10.789680 15084 raft_consensus.cc:515] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.789980 15107 leader_election.cc:290] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371), 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3:45113)
I20260709 13:04:10.790678 14883 raft_consensus.cc:2393] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 09d3fa2b7ef942ae858e2c5727d99ce5 in current term 1: Already voted for candidate 39f9e4e2209748a69d253ea46414e575 in this term.
I20260709 13:04:10.786837 14957 raft_consensus.cc:3060] T be17f0536ffd4425a0db52b530c5f3ef P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:10.791988 15084 leader_election.cc:290] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2:33423), 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3:45113)
I20260709 13:04:10.787999 15082 raft_consensus.cc:493] T c7c38457995f4e988d3ccf6bd94e6c37 P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.796243 14882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7481805a7cfe4c2c8da2b33cf58c7ea5" candidate_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f9e4e2209748a69d253ea46414e575" is_pre_election: true
I20260709 13:04:10.796135 14956 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54152e1e2cf41a3970778369876d656" candidate_uuid: "39f9e4e2209748a69d253ea46414e575" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" is_pre_election: true
I20260709 13:04:10.797902 14956 raft_consensus.cc:2468] T f54152e1e2cf41a3970778369876d656 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39f9e4e2209748a69d253ea46414e575 in term 0.
I20260709 13:04:10.794754 14883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17f0536ffd4425a0db52b530c5f3ef" candidate_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f9e4e2209748a69d253ea46414e575" is_pre_election: true
I20260709 13:04:10.794201 15076 ts_tablet_manager.cc:1403] T bb4dacc583b6453cb577a9e46c45c3af P 39f9e4e2209748a69d253ea46414e575: Time spent bootstrapping tablet: real 0.181s	user 0.021s	sys 0.016s
W20260709 13:04:10.794032 14996 leader_election.cc:343] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:10.800832 14922 leader_election.cc:343] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:10.800899 15032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7c38457995f4e988d3ccf6bd94e6c37" candidate_uuid: "39f9e4e2209748a69d253ea46414e575" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5" is_pre_election: true
I20260709 13:04:10.802451 14847 leader_election.cc:304] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d3fa2b7ef942ae858e2c5727d99ce5, 39f9e4e2209748a69d253ea46414e575; no voters: 
I20260709 13:04:10.799290 15082 raft_consensus.cc:515] T c7c38457995f4e988d3ccf6bd94e6c37 P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.803206 15084 raft_consensus.cc:2804] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:10.803565 15084 raft_consensus.cc:493] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.803750 15032 raft_consensus.cc:2468] T c7c38457995f4e988d3ccf6bd94e6c37 P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39f9e4e2209748a69d253ea46414e575 in term 0.
I20260709 13:04:10.803965 15084 raft_consensus.cc:3060] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:10.805131 14922 leader_election.cc:343] T be17f0536ffd4425a0db52b530c5f3ef P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:10.810166 15031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7c38457995f4e988d3ccf6bd94e6c37" candidate_uuid: "39f9e4e2209748a69d253ea46414e575" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5"
I20260709 13:04:10.811949 15031 raft_consensus.cc:3060] T c7c38457995f4e988d3ccf6bd94e6c37 P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:10.815095 14883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7c38457995f4e988d3ccf6bd94e6c37" candidate_uuid: "435d239f6f2441ac9d0e835010bfd2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f9e4e2209748a69d253ea46414e575" is_pre_election: true
I20260709 13:04:10.816162 14883 raft_consensus.cc:2393] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 435d239f6f2441ac9d0e835010bfd2b5 in current term 1: Already voted for candidate 39f9e4e2209748a69d253ea46414e575 in this term.
I20260709 13:04:10.816535 15032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54152e1e2cf41a3970778369876d656" candidate_uuid: "39f9e4e2209748a69d253ea46414e575" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5" is_pre_election: true
I20260709 13:04:10.825495 14958 raft_consensus.cc:2468] T c7c38457995f4e988d3ccf6bd94e6c37 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39f9e4e2209748a69d253ea46414e575 in term 1.
I20260709 13:04:10.827322 15077 tablet_bootstrap.cc:492] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5: No bootstrap required, opened a new log
I20260709 13:04:10.827816 15077 ts_tablet_manager.cc:1403] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5: Time spent bootstrapping tablet: real 0.137s	user 0.018s	sys 0.003s
I20260709 13:04:10.827987 14847 leader_election.cc:304] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d3fa2b7ef942ae858e2c5727d99ce5, 39f9e4e2209748a69d253ea46414e575; no voters: 
I20260709 13:04:10.829516 15082 leader_election.cc:290] T c7c38457995f4e988d3ccf6bd94e6c37 P 435d239f6f2441ac9d0e835010bfd2b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371), 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2:33423)
I20260709 13:04:10.830806 14957 raft_consensus.cc:2468] T be17f0536ffd4425a0db52b530c5f3ef P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 435d239f6f2441ac9d0e835010bfd2b5 in term 1.
W20260709 13:04:10.832885 14846 leader_election.cc:343] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3:45113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:10.834103 15032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7c38457995f4e988d3ccf6bd94e6c37" candidate_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5" is_pre_election: true
I20260709 13:04:10.836135 15084 raft_consensus.cc:515] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.840325 15032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17f0536ffd4425a0db52b530c5f3ef" candidate_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5" is_pre_election: true
I20260709 13:04:10.841079 15032 raft_consensus.cc:2393] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 09d3fa2b7ef942ae858e2c5727d99ce5 in current term 1: Already voted for candidate 435d239f6f2441ac9d0e835010bfd2b5 in this term.
I20260709 13:04:10.841641 15084 leader_election.cc:290] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575 [CANDIDATE]: Term 1 election: Requested vote from peers 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2:33423), 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3:45113)
I20260709 13:04:10.842279 15084 raft_consensus.cc:2804] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:10.842621 15030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7481805a7cfe4c2c8da2b33cf58c7ea5" candidate_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5" is_pre_election: true
I20260709 13:04:10.843327 15030 raft_consensus.cc:2468] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09d3fa2b7ef942ae858e2c5727d99ce5 in term 0.
I20260709 13:04:10.836741 14957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7c38457995f4e988d3ccf6bd94e6c37" candidate_uuid: "435d239f6f2441ac9d0e835010bfd2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" is_pre_election: true
I20260709 13:04:10.837353 15076 raft_consensus.cc:359] T bb4dacc583b6453cb577a9e46c45c3af P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.844159 15076 raft_consensus.cc:385] T bb4dacc583b6453cb577a9e46c45c3af P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:10.844378 14957 raft_consensus.cc:2393] T c7c38457995f4e988d3ccf6bd94e6c37 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 435d239f6f2441ac9d0e835010bfd2b5 in current term 1: Already voted for candidate 39f9e4e2209748a69d253ea46414e575 in this term.
I20260709 13:04:10.844527 15076 raft_consensus.cc:740] T bb4dacc583b6453cb577a9e46c45c3af P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39f9e4e2209748a69d253ea46414e575, State: Initialized, Role: FOLLOWER
I20260709 13:04:10.845325 15076 consensus_queue.cc:260] T bb4dacc583b6453cb577a9e46c45c3af P 39f9e4e2209748a69d253ea46414e575 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.839699 15031 raft_consensus.cc:2468] T c7c38457995f4e988d3ccf6bd94e6c37 P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39f9e4e2209748a69d253ea46414e575 in term 1.
I20260709 13:04:10.842792 15084 raft_consensus.cc:697] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575 [term 1 LEADER]: Becoming Leader. State: Replica: 39f9e4e2209748a69d253ea46414e575, State: Running, Role: LEADER
I20260709 13:04:10.845659 14921 leader_election.cc:304] T c7c38457995f4e988d3ccf6bd94e6c37 P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 09d3fa2b7ef942ae858e2c5727d99ce5; no voters: 39f9e4e2209748a69d253ea46414e575, 435d239f6f2441ac9d0e835010bfd2b5
I20260709 13:04:10.848614 15084 consensus_queue.cc:237] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.849501 15107 raft_consensus.cc:2749] T c7c38457995f4e988d3ccf6bd94e6c37 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:10.836381 14996 leader_election.cc:304] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 09d3fa2b7ef942ae858e2c5727d99ce5, 435d239f6f2441ac9d0e835010bfd2b5; no voters: 39f9e4e2209748a69d253ea46414e575
I20260709 13:04:10.848971 14921 leader_election.cc:304] T be17f0536ffd4425a0db52b530c5f3ef P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 09d3fa2b7ef942ae858e2c5727d99ce5; no voters: 39f9e4e2209748a69d253ea46414e575, 435d239f6f2441ac9d0e835010bfd2b5
I20260709 13:04:10.839371 15077 raft_consensus.cc:359] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.850266 15077 raft_consensus.cc:385] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:10.850589 15077 raft_consensus.cc:740] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d3fa2b7ef942ae858e2c5727d99ce5, State: Initialized, Role: FOLLOWER
I20260709 13:04:10.850811 14921 leader_election.cc:304] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 09d3fa2b7ef942ae858e2c5727d99ce5, 435d239f6f2441ac9d0e835010bfd2b5; no voters: 39f9e4e2209748a69d253ea46414e575
I20260709 13:04:10.851523 15081 raft_consensus.cc:2749] T be17f0536ffd4425a0db52b530c5f3ef P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:10.852070 15107 raft_consensus.cc:2804] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:10.852509 15107 raft_consensus.cc:493] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.851346 15077 consensus_queue.cc:260] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.852845 15107 raft_consensus.cc:3060] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:10.854699 15077 ts_tablet_manager.cc:1434] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.000s
I20260709 13:04:10.855727 15077 tablet_bootstrap.cc:492] T bb4dacc583b6453cb577a9e46c45c3af P 09d3fa2b7ef942ae858e2c5727d99ce5: Bootstrap starting.
I20260709 13:04:10.861666 15081 raft_consensus.cc:493] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.862134 15081 raft_consensus.cc:515] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } }
I20260709 13:04:10.864161 15081 leader_election.cc:290] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3:45113), 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371)
I20260709 13:04:10.865250 14883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "810f60f24e33413985aa63f396a2b55a" candidate_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f9e4e2209748a69d253ea46414e575" is_pre_election: true
W20260709 13:04:10.866583 14922 leader_election.cc:343] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:10.870903 15107 raft_consensus.cc:515] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.874831 15077 tablet_bootstrap.cc:654] T bb4dacc583b6453cb577a9e46c45c3af P 09d3fa2b7ef942ae858e2c5727d99ce5: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:10.875022 15082 raft_consensus.cc:2804] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:10.877197 15030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54152e1e2cf41a3970778369876d656" candidate_uuid: "39f9e4e2209748a69d253ea46414e575" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5"
W20260709 13:04:10.879568 14846 leader_election.cc:343] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3:45113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:10.876171 14996 leader_election.cc:304] T c7c38457995f4e988d3ccf6bd94e6c37 P 435d239f6f2441ac9d0e835010bfd2b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 435d239f6f2441ac9d0e835010bfd2b5; no voters: 09d3fa2b7ef942ae858e2c5727d99ce5, 39f9e4e2209748a69d253ea46414e575
I20260709 13:04:10.882866 15030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "810f60f24e33413985aa63f396a2b55a" candidate_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5" is_pre_election: true
I20260709 13:04:10.883639 15030 raft_consensus.cc:2468] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09d3fa2b7ef942ae858e2c5727d99ce5 in term 0.
I20260709 13:04:10.884970 14921 leader_election.cc:304] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 09d3fa2b7ef942ae858e2c5727d99ce5, 435d239f6f2441ac9d0e835010bfd2b5; no voters: 39f9e4e2209748a69d253ea46414e575
I20260709 13:04:10.886178 15081 raft_consensus.cc:2804] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:10.886564 15081 raft_consensus.cc:493] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.886883 15081 raft_consensus.cc:3060] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:10.888917 15116 raft_consensus.cc:2749] T c7c38457995f4e988d3ccf6bd94e6c37 P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:04:10.894574 14847 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:33423, user_credentials={real_user=slave}} blocked reactor thread for 63989.4us
I20260709 13:04:10.896808 15082 raft_consensus.cc:697] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 LEADER]: Becoming Leader. State: Replica: 435d239f6f2441ac9d0e835010bfd2b5, State: Running, Role: LEADER
I20260709 13:04:10.897198 14957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54152e1e2cf41a3970778369876d656" candidate_uuid: "39f9e4e2209748a69d253ea46414e575" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
I20260709 13:04:10.897907 14957 raft_consensus.cc:3060] T f54152e1e2cf41a3970778369876d656 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:10.898082 15082 consensus_queue.cc:237] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.898942 15081 raft_consensus.cc:515] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } }
I20260709 13:04:10.904686 15081 leader_election.cc:290] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 1 election: Requested vote from peers 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3:45113), 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371)
I20260709 13:04:10.908705 14883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "810f60f24e33413985aa63f396a2b55a" candidate_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f9e4e2209748a69d253ea46414e575"
I20260709 13:04:10.908983 15030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "810f60f24e33413985aa63f396a2b55a" candidate_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5"
I20260709 13:04:10.909498 15030 raft_consensus.cc:3060] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:10.911775 14922 leader_election.cc:343] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:10.920508 14957 raft_consensus.cc:2468] T f54152e1e2cf41a3970778369876d656 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39f9e4e2209748a69d253ea46414e575 in term 1.
I20260709 13:04:10.921030 15112 raft_consensus.cc:493] T 90b851c6655d47c09fca18bbfc1a4d95 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.922277 15081 raft_consensus.cc:493] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.923039 15081 raft_consensus.cc:515] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.925606 14883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7481805a7cfe4c2c8da2b33cf58c7ea5" candidate_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f9e4e2209748a69d253ea46414e575"
I20260709 13:04:10.926257 15081 leader_election.cc:290] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371), 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3:45113)
I20260709 13:04:10.922325 15112 raft_consensus.cc:515] T 90b851c6655d47c09fca18bbfc1a4d95 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.932812 14847 leader_election.cc:304] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 09d3fa2b7ef942ae858e2c5727d99ce5, 39f9e4e2209748a69d253ea46414e575; no voters: 435d239f6f2441ac9d0e835010bfd2b5
I20260709 13:04:10.935348 15031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90b851c6655d47c09fca18bbfc1a4d95" candidate_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5" is_pre_election: true
W20260709 13:04:10.937258 14996 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:33423, user_credentials={real_user=slave}} blocked reactor thread for 61199.8us
I20260709 13:04:10.937470 15031 raft_consensus.cc:2468] T 90b851c6655d47c09fca18bbfc1a4d95 P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09d3fa2b7ef942ae858e2c5727d99ce5 in term 0.
I20260709 13:04:10.932183 14780 catalog_manager.cc:5697] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575 reported cstate change: term changed from 0 to 1, leader changed from <none> to 39f9e4e2209748a69d253ea46414e575 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "39f9e4e2209748a69d253ea46414e575" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:10.938076 14883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90b851c6655d47c09fca18bbfc1a4d95" candidate_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f9e4e2209748a69d253ea46414e575" is_pre_election: true
I20260709 13:04:10.939657 15075 tablet_bootstrap.cc:492] T 14bf28d1d32a48e7973484fb93521945 P 435d239f6f2441ac9d0e835010bfd2b5: No bootstrap required, opened a new log
I20260709 13:04:10.940172 15075 ts_tablet_manager.cc:1403] T 14bf28d1d32a48e7973484fb93521945 P 435d239f6f2441ac9d0e835010bfd2b5: Time spent bootstrapping tablet: real 0.229s	user 0.009s	sys 0.015s
I20260709 13:04:10.940706 14921 leader_election.cc:304] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d3fa2b7ef942ae858e2c5727d99ce5, 435d239f6f2441ac9d0e835010bfd2b5; no voters: 
I20260709 13:04:10.942071 15081 raft_consensus.cc:2804] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:10.945019 15081 raft_consensus.cc:493] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.945947 15081 raft_consensus.cc:3060] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:10.942204 15084 raft_consensus.cc:2804] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:10.944113 14883 raft_consensus.cc:2468] T 90b851c6655d47c09fca18bbfc1a4d95 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09d3fa2b7ef942ae858e2c5727d99ce5 in term 0.
I20260709 13:04:10.950493 15031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7481805a7cfe4c2c8da2b33cf58c7ea5" candidate_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5"
I20260709 13:04:10.951056 15031 raft_consensus.cc:3060] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:10.945540 14922 leader_election.cc:343] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:10.944331 15107 leader_election.cc:290] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 1 election: Requested vote from peers 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371), 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3:45113)
I20260709 13:04:10.958806 15031 raft_consensus.cc:2468] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d3fa2b7ef942ae858e2c5727d99ce5 in term 1.
I20260709 13:04:10.946103 14778 catalog_manager.cc:5697] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3). New cstate: current_term: 1 leader_uuid: "435d239f6f2441ac9d0e835010bfd2b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:10.960294 15075 raft_consensus.cc:359] T 14bf28d1d32a48e7973484fb93521945 P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.961329 15075 raft_consensus.cc:385] T 14bf28d1d32a48e7973484fb93521945 P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:10.961753 15075 raft_consensus.cc:740] T 14bf28d1d32a48e7973484fb93521945 P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 435d239f6f2441ac9d0e835010bfd2b5, State: Initialized, Role: FOLLOWER
I20260709 13:04:10.959537 15084 raft_consensus.cc:697] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575 [term 1 LEADER]: Becoming Leader. State: Replica: 39f9e4e2209748a69d253ea46414e575, State: Running, Role: LEADER
I20260709 13:04:10.963191 15081 raft_consensus.cc:515] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.962881 15075 consensus_queue.cc:260] T 14bf28d1d32a48e7973484fb93521945 P 435d239f6f2441ac9d0e835010bfd2b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.961640 15076 ts_tablet_manager.cc:1434] T bb4dacc583b6453cb577a9e46c45c3af P 39f9e4e2209748a69d253ea46414e575: Time spent starting tablet: real 0.162s	user 0.021s	sys 0.033s
I20260709 13:04:10.967943 14921 leader_election.cc:304] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 09d3fa2b7ef942ae858e2c5727d99ce5, 435d239f6f2441ac9d0e835010bfd2b5; no voters: 39f9e4e2209748a69d253ea46414e575
I20260709 13:04:10.969854 15107 raft_consensus.cc:2804] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:10.964370 15030 raft_consensus.cc:2468] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d3fa2b7ef942ae858e2c5727d99ce5 in term 1.
I20260709 13:04:10.977049 15084 consensus_queue.cc:237] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:10.984261 15076 tablet_bootstrap.cc:492] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 39f9e4e2209748a69d253ea46414e575: Bootstrap starting.
I20260709 13:04:11.002274 15082 raft_consensus.cc:493] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.011987 15112 leader_election.cc:290] T 90b851c6655d47c09fca18bbfc1a4d95 P 39f9e4e2209748a69d253ea46414e575 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2:33423), 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3:45113)
I20260709 13:04:11.011907 14921 leader_election.cc:304] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 09d3fa2b7ef942ae858e2c5727d99ce5, 435d239f6f2441ac9d0e835010bfd2b5; no voters: 39f9e4e2209748a69d253ea46414e575
I20260709 13:04:11.013094 15075 ts_tablet_manager.cc:1434] T 14bf28d1d32a48e7973484fb93521945 P 435d239f6f2441ac9d0e835010bfd2b5: Time spent starting tablet: real 0.072s	user 0.008s	sys 0.000s
I20260709 13:04:11.015118 15077 tablet_bootstrap.cc:492] T bb4dacc583b6453cb577a9e46c45c3af P 09d3fa2b7ef942ae858e2c5727d99ce5: No bootstrap required, opened a new log
I20260709 13:04:11.015025 15030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90b851c6655d47c09fca18bbfc1a4d95" candidate_uuid: "39f9e4e2209748a69d253ea46414e575" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5" is_pre_election: true
I20260709 13:04:11.015578 15077 ts_tablet_manager.cc:1403] T bb4dacc583b6453cb577a9e46c45c3af P 09d3fa2b7ef942ae858e2c5727d99ce5: Time spent bootstrapping tablet: real 0.160s	user 0.007s	sys 0.015s
I20260709 13:04:11.016230 15075 tablet_bootstrap.cc:492] T f54152e1e2cf41a3970778369876d656 P 435d239f6f2441ac9d0e835010bfd2b5: Bootstrap starting.
I20260709 13:04:11.016454 15030 raft_consensus.cc:2468] T 90b851c6655d47c09fca18bbfc1a4d95 P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39f9e4e2209748a69d253ea46414e575 in term 0.
I20260709 13:04:11.017616 15121 raft_consensus.cc:493] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.017803 14957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90b851c6655d47c09fca18bbfc1a4d95" candidate_uuid: "39f9e4e2209748a69d253ea46414e575" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" is_pre_election: true
I20260709 13:04:11.018582 14957 raft_consensus.cc:2393] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 39f9e4e2209748a69d253ea46414e575 in current term 1: Already voted for candidate 09d3fa2b7ef942ae858e2c5727d99ce5 in this term.
I20260709 13:04:11.018127 15121 raft_consensus.cc:515] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:11.016070 15107 raft_consensus.cc:697] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 LEADER]: Becoming Leader. State: Replica: 09d3fa2b7ef942ae858e2c5727d99ce5, State: Running, Role: LEADER
I20260709 13:04:11.018754 14846 leader_election.cc:304] T 90b851c6655d47c09fca18bbfc1a4d95 P 39f9e4e2209748a69d253ea46414e575 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39f9e4e2209748a69d253ea46414e575, 435d239f6f2441ac9d0e835010bfd2b5; no voters: 
I20260709 13:04:11.010917 15082 raft_consensus.cc:515] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:11.022250 15112 raft_consensus.cc:2804] T 90b851c6655d47c09fca18bbfc1a4d95 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:11.023377 15112 raft_consensus.cc:493] T 90b851c6655d47c09fca18bbfc1a4d95 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:11.022140 15107 consensus_queue.cc:237] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:11.026681 14883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90b851c6655d47c09fca18bbfc1a4d95" candidate_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f9e4e2209748a69d253ea46414e575"
I20260709 13:04:11.025581 15112 raft_consensus.cc:3060] T 90b851c6655d47c09fca18bbfc1a4d95 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:11.031046 15081 leader_election.cc:290] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 1 election: Requested vote from peers 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371), 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3:45113)
I20260709 13:04:11.032335 14882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bf28d1d32a48e7973484fb93521945" candidate_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f9e4e2209748a69d253ea46414e575" is_pre_election: true
I20260709 13:04:11.033471 15082 leader_election.cc:290] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371), 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2:33423)
I20260709 13:04:11.030671 15077 raft_consensus.cc:359] T bb4dacc583b6453cb577a9e46c45c3af P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:11.034768 15077 raft_consensus.cc:385] T bb4dacc583b6453cb577a9e46c45c3af P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:11.035153 15077 raft_consensus.cc:740] T bb4dacc583b6453cb577a9e46c45c3af P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d3fa2b7ef942ae858e2c5727d99ce5, State: Initialized, Role: FOLLOWER
W20260709 13:04:11.038848 14922 leader_election.cc:343] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:11.038532 15077 consensus_queue.cc:260] T bb4dacc583b6453cb577a9e46c45c3af P 09d3fa2b7ef942ae858e2c5727d99ce5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:11.040086 14957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb4dacc583b6453cb577a9e46c45c3af" candidate_uuid: "435d239f6f2441ac9d0e835010bfd2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" is_pre_election: true
I20260709 13:04:11.042047 15030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90b851c6655d47c09fca18bbfc1a4d95" candidate_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5"
I20260709 13:04:11.042234 14957 raft_consensus.cc:2468] T bb4dacc583b6453cb577a9e46c45c3af P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 435d239f6f2441ac9d0e835010bfd2b5 in term 0.
I20260709 13:04:11.042578 15030 raft_consensus.cc:3060] T 90b851c6655d47c09fca18bbfc1a4d95 P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:11.043326 14996 leader_election.cc:304] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d3fa2b7ef942ae858e2c5727d99ce5, 435d239f6f2441ac9d0e835010bfd2b5; no voters: 
I20260709 13:04:11.043483 14882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb4dacc583b6453cb577a9e46c45c3af" candidate_uuid: "435d239f6f2441ac9d0e835010bfd2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f9e4e2209748a69d253ea46414e575" is_pre_election: true
I20260709 13:04:11.044107 14882 raft_consensus.cc:2468] T bb4dacc583b6453cb577a9e46c45c3af P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 435d239f6f2441ac9d0e835010bfd2b5 in term 0.
I20260709 13:04:11.044246 15082 raft_consensus.cc:2804] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:11.044588 15112 raft_consensus.cc:515] T 90b851c6655d47c09fca18bbfc1a4d95 P 39f9e4e2209748a69d253ea46414e575 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:11.044771 15082 raft_consensus.cc:493] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:11.045689 14883 raft_consensus.cc:2393] T 90b851c6655d47c09fca18bbfc1a4d95 P 39f9e4e2209748a69d253ea46414e575 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 09d3fa2b7ef942ae858e2c5727d99ce5 in current term 1: Already voted for candidate 39f9e4e2209748a69d253ea46414e575 in this term.
I20260709 13:04:11.045826 15082 raft_consensus.cc:3060] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:11.047291 15112 leader_election.cc:290] T 90b851c6655d47c09fca18bbfc1a4d95 P 39f9e4e2209748a69d253ea46414e575 [CANDIDATE]: Term 1 election: Requested vote from peers 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2:33423), 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3:45113)
I20260709 13:04:11.050043 15077 ts_tablet_manager.cc:1434] T bb4dacc583b6453cb577a9e46c45c3af P 09d3fa2b7ef942ae858e2c5727d99ce5: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.000s
I20260709 13:04:11.050196 14957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90b851c6655d47c09fca18bbfc1a4d95" candidate_uuid: "39f9e4e2209748a69d253ea46414e575" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
I20260709 13:04:11.051040 14957 raft_consensus.cc:2393] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 39f9e4e2209748a69d253ea46414e575 in current term 1: Already voted for candidate 09d3fa2b7ef942ae858e2c5727d99ce5 in this term.
I20260709 13:04:11.053449 15075 tablet_bootstrap.cc:654] T f54152e1e2cf41a3970778369876d656 P 435d239f6f2441ac9d0e835010bfd2b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:11.058960 15032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bf28d1d32a48e7973484fb93521945" candidate_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5" is_pre_election: true
I20260709 13:04:11.057654 15122 raft_consensus.cc:2804] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:11.058996 15031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90b851c6655d47c09fca18bbfc1a4d95" candidate_uuid: "39f9e4e2209748a69d253ea46414e575" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5"
I20260709 13:04:11.060905 15032 raft_consensus.cc:2468] T 14bf28d1d32a48e7973484fb93521945 P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09d3fa2b7ef942ae858e2c5727d99ce5 in term 0.
I20260709 13:04:11.063925 15122 raft_consensus.cc:697] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 LEADER]: Becoming Leader. State: Replica: 09d3fa2b7ef942ae858e2c5727d99ce5, State: Running, Role: LEADER
I20260709 13:04:11.064579 15030 raft_consensus.cc:2468] T 90b851c6655d47c09fca18bbfc1a4d95 P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d3fa2b7ef942ae858e2c5727d99ce5 in term 1.
I20260709 13:04:11.064652 14846 leader_election.cc:304] T 90b851c6655d47c09fca18bbfc1a4d95 P 39f9e4e2209748a69d253ea46414e575 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39f9e4e2209748a69d253ea46414e575; no voters: 09d3fa2b7ef942ae858e2c5727d99ce5, 435d239f6f2441ac9d0e835010bfd2b5
I20260709 13:04:11.065956 15084 raft_consensus.cc:2749] T 90b851c6655d47c09fca18bbfc1a4d95 P 39f9e4e2209748a69d253ea46414e575 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:04:11.066776 14921 leader_election.cc:304] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 09d3fa2b7ef942ae858e2c5727d99ce5, 435d239f6f2441ac9d0e835010bfd2b5; no voters: 39f9e4e2209748a69d253ea46414e575
I20260709 13:04:11.071421 15107 raft_consensus.cc:2804] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:11.065308 15122 consensus_queue.cc:237] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } }
I20260709 13:04:11.072886 15107 raft_consensus.cc:493] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:11.072364 14921 leader_election.cc:304] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d3fa2b7ef942ae858e2c5727d99ce5, 435d239f6f2441ac9d0e835010bfd2b5; no voters: 
I20260709 13:04:11.076959 15107 raft_consensus.cc:3060] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:11.078334 15081 raft_consensus.cc:2804] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:11.079272 15081 raft_consensus.cc:697] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 LEADER]: Becoming Leader. State: Replica: 09d3fa2b7ef942ae858e2c5727d99ce5, State: Running, Role: LEADER
I20260709 13:04:11.080082 15081 consensus_queue.cc:237] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:11.081748 15082 raft_consensus.cc:515] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:11.083840 15107 raft_consensus.cc:515] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:11.086400 15107 leader_election.cc:290] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 1 election: Requested vote from peers 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371), 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3:45113)
I20260709 13:04:11.087430 15030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bf28d1d32a48e7973484fb93521945" candidate_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5"
I20260709 13:04:11.087567 14882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bf28d1d32a48e7973484fb93521945" candidate_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f9e4e2209748a69d253ea46414e575"
I20260709 13:04:11.088020 15030 raft_consensus.cc:3060] T 14bf28d1d32a48e7973484fb93521945 P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:11.090121 14922 leader_election.cc:343] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:11.093335 15075 tablet_bootstrap.cc:492] T f54152e1e2cf41a3970778369876d656 P 435d239f6f2441ac9d0e835010bfd2b5: No bootstrap required, opened a new log
I20260709 13:04:11.093708 15075 ts_tablet_manager.cc:1403] T f54152e1e2cf41a3970778369876d656 P 435d239f6f2441ac9d0e835010bfd2b5: Time spent bootstrapping tablet: real 0.080s	user 0.011s	sys 0.003s
I20260709 13:04:11.094199 14882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb4dacc583b6453cb577a9e46c45c3af" candidate_uuid: "435d239f6f2441ac9d0e835010bfd2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f9e4e2209748a69d253ea46414e575"
I20260709 13:04:11.095073 14882 raft_consensus.cc:3060] T bb4dacc583b6453cb577a9e46c45c3af P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:11.097908 15075 raft_consensus.cc:359] T f54152e1e2cf41a3970778369876d656 P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:11.098673 15075 raft_consensus.cc:385] T f54152e1e2cf41a3970778369876d656 P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:11.098757 15030 raft_consensus.cc:2468] T 14bf28d1d32a48e7973484fb93521945 P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d3fa2b7ef942ae858e2c5727d99ce5 in term 1.
I20260709 13:04:11.099100 15075 raft_consensus.cc:740] T f54152e1e2cf41a3970778369876d656 P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 435d239f6f2441ac9d0e835010bfd2b5, State: Initialized, Role: FOLLOWER
I20260709 13:04:11.100274 15075 consensus_queue.cc:260] T f54152e1e2cf41a3970778369876d656 P 435d239f6f2441ac9d0e835010bfd2b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:11.106698 15082 leader_election.cc:290] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5 [CANDIDATE]: Term 1 election: Requested vote from peers 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371), 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2:33423)
I20260709 13:04:11.110191 14957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb4dacc583b6453cb577a9e46c45c3af" candidate_uuid: "435d239f6f2441ac9d0e835010bfd2b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
I20260709 13:04:11.110757 14957 raft_consensus.cc:3060] T bb4dacc583b6453cb577a9e46c45c3af P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:11.111073 15076 tablet_bootstrap.cc:654] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 39f9e4e2209748a69d253ea46414e575: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:11.111127 15075 ts_tablet_manager.cc:1434] T f54152e1e2cf41a3970778369876d656 P 435d239f6f2441ac9d0e835010bfd2b5: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.001s
I20260709 13:04:11.109104 14921 leader_election.cc:304] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 09d3fa2b7ef942ae858e2c5727d99ce5, 435d239f6f2441ac9d0e835010bfd2b5; no voters: 39f9e4e2209748a69d253ea46414e575
I20260709 13:04:11.115612 14882 raft_consensus.cc:2468] T bb4dacc583b6453cb577a9e46c45c3af P 39f9e4e2209748a69d253ea46414e575 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 435d239f6f2441ac9d0e835010bfd2b5 in term 1.
I20260709 13:04:11.116614 14996 leader_election.cc:304] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39f9e4e2209748a69d253ea46414e575, 435d239f6f2441ac9d0e835010bfd2b5; no voters: 
I20260709 13:04:11.121189 15117 raft_consensus.cc:2804] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:11.121701 14957 raft_consensus.cc:2468] T bb4dacc583b6453cb577a9e46c45c3af P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 435d239f6f2441ac9d0e835010bfd2b5 in term 1.
I20260709 13:04:11.122841 15117 raft_consensus.cc:697] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 LEADER]: Becoming Leader. State: Replica: 435d239f6f2441ac9d0e835010bfd2b5, State: Running, Role: LEADER
I20260709 13:04:11.124595 15122 raft_consensus.cc:2804] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:11.126034 15122 raft_consensus.cc:697] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 LEADER]: Becoming Leader. State: Replica: 09d3fa2b7ef942ae858e2c5727d99ce5, State: Running, Role: LEADER
I20260709 13:04:11.125823 15117 consensus_queue.cc:237] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:11.127028 15122 consensus_queue.cc:237] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:11.132057 15121 leader_election.cc:290] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371), 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3:45113)
I20260709 13:04:11.141776 14779 catalog_manager.cc:5697] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2). New cstate: current_term: 1 leader_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:11.146714 15076 tablet_bootstrap.cc:492] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 39f9e4e2209748a69d253ea46414e575: No bootstrap required, opened a new log
I20260709 13:04:11.147169 15076 ts_tablet_manager.cc:1403] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 39f9e4e2209748a69d253ea46414e575: Time spent bootstrapping tablet: real 0.170s	user 0.012s	sys 0.014s
I20260709 13:04:11.150520 15076 raft_consensus.cc:359] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:11.151623 15076 raft_consensus.cc:385] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:11.152132 15076 raft_consensus.cc:740] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39f9e4e2209748a69d253ea46414e575, State: Initialized, Role: FOLLOWER
I20260709 13:04:11.152951 15076 consensus_queue.cc:260] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 39f9e4e2209748a69d253ea46414e575 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:11.155438 15076 ts_tablet_manager.cc:1434] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 39f9e4e2209748a69d253ea46414e575: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 13:04:11.156421 15076 tablet_bootstrap.cc:492] T 32bc5a8acc774a1d9c7e99691a591567 P 39f9e4e2209748a69d253ea46414e575: Bootstrap starting.
I20260709 13:04:11.166577 15076 tablet_bootstrap.cc:654] T 32bc5a8acc774a1d9c7e99691a591567 P 39f9e4e2209748a69d253ea46414e575: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:11.176599 14827 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:11.179728 15076 tablet_bootstrap.cc:492] T 32bc5a8acc774a1d9c7e99691a591567 P 39f9e4e2209748a69d253ea46414e575: No bootstrap required, opened a new log
I20260709 13:04:11.180217 15076 ts_tablet_manager.cc:1403] T 32bc5a8acc774a1d9c7e99691a591567 P 39f9e4e2209748a69d253ea46414e575: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.004s
I20260709 13:04:11.176048 14780 catalog_manager.cc:5697] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575 reported cstate change: term changed from 0 to 1, leader changed from <none> to 39f9e4e2209748a69d253ea46414e575 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "39f9e4e2209748a69d253ea46414e575" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:11.183161 15076 raft_consensus.cc:359] T 32bc5a8acc774a1d9c7e99691a591567 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:11.184037 15076 raft_consensus.cc:385] T 32bc5a8acc774a1d9c7e99691a591567 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:11.184494 15076 raft_consensus.cc:740] T 32bc5a8acc774a1d9c7e99691a591567 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39f9e4e2209748a69d253ea46414e575, State: Initialized, Role: FOLLOWER
I20260709 13:04:11.185222 15076 consensus_queue.cc:260] T 32bc5a8acc774a1d9c7e99691a591567 P 39f9e4e2209748a69d253ea46414e575 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:11.187817 15076 ts_tablet_manager.cc:1434] T 32bc5a8acc774a1d9c7e99691a591567 P 39f9e4e2209748a69d253ea46414e575: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:04:11.188802 15076 tablet_bootstrap.cc:492] T 14bf28d1d32a48e7973484fb93521945 P 39f9e4e2209748a69d253ea46414e575: Bootstrap starting.
I20260709 13:04:11.193275 14779 catalog_manager.cc:5697] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3). New cstate: current_term: 1 leader_uuid: "435d239f6f2441ac9d0e835010bfd2b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:11.195575 14779 catalog_manager.cc:5697] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3). New cstate: current_term: 1 leader_uuid: "435d239f6f2441ac9d0e835010bfd2b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:11.197139 15076 tablet_bootstrap.cc:654] T 14bf28d1d32a48e7973484fb93521945 P 39f9e4e2209748a69d253ea46414e575: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:11.203253 15076 tablet_bootstrap.cc:492] T 14bf28d1d32a48e7973484fb93521945 P 39f9e4e2209748a69d253ea46414e575: No bootstrap required, opened a new log
I20260709 13:04:11.203750 15076 ts_tablet_manager.cc:1403] T 14bf28d1d32a48e7973484fb93521945 P 39f9e4e2209748a69d253ea46414e575: Time spent bootstrapping tablet: real 0.015s	user 0.010s	sys 0.003s
I20260709 13:04:11.206434 15076 raft_consensus.cc:359] T 14bf28d1d32a48e7973484fb93521945 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:11.207132 15076 raft_consensus.cc:385] T 14bf28d1d32a48e7973484fb93521945 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:11.209059 15076 raft_consensus.cc:740] T 14bf28d1d32a48e7973484fb93521945 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39f9e4e2209748a69d253ea46414e575, State: Initialized, Role: FOLLOWER
I20260709 13:04:11.209801 15076 consensus_queue.cc:260] T 14bf28d1d32a48e7973484fb93521945 P 39f9e4e2209748a69d253ea46414e575 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:11.212307 15076 ts_tablet_manager.cc:1434] T 14bf28d1d32a48e7973484fb93521945 P 39f9e4e2209748a69d253ea46414e575: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:04:11.213243 15076 tablet_bootstrap.cc:492] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575: Bootstrap starting.
I20260709 13:04:11.213212 14776 catalog_manager.cc:5697] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2). New cstate: current_term: 1 leader_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:11.217300 14776 catalog_manager.cc:5697] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2). New cstate: current_term: 1 leader_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:11.220082 15076 tablet_bootstrap.cc:654] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:11.219251 14776 catalog_manager.cc:5697] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2). New cstate: current_term: 1 leader_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:11.231310 15076 tablet_bootstrap.cc:492] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575: No bootstrap required, opened a new log
I20260709 13:04:11.231796 15076 ts_tablet_manager.cc:1403] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575: Time spent bootstrapping tablet: real 0.019s	user 0.015s	sys 0.000s
I20260709 13:04:11.234390 15076 raft_consensus.cc:359] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } }
I20260709 13:04:11.235041 15076 raft_consensus.cc:385] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:11.235316 15076 raft_consensus.cc:740] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39f9e4e2209748a69d253ea46414e575, State: Initialized, Role: FOLLOWER
I20260709 13:04:11.235980 15076 consensus_queue.cc:260] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } }
I20260709 13:04:11.237879 15076 ts_tablet_manager.cc:1434] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:04:11.238910 15076 tablet_bootstrap.cc:492] T be17f0536ffd4425a0db52b530c5f3ef P 39f9e4e2209748a69d253ea46414e575: Bootstrap starting.
I20260709 13:04:11.244511 15076 tablet_bootstrap.cc:654] T be17f0536ffd4425a0db52b530c5f3ef P 39f9e4e2209748a69d253ea46414e575: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:11.244897 15084 raft_consensus.cc:493] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.245450 15084 raft_consensus.cc:515] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } }
I20260709 13:04:11.247859 15084 leader_election.cc:290] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3:45113), 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2:33423)
I20260709 13:04:11.248588 15030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "810f60f24e33413985aa63f396a2b55a" candidate_uuid: "39f9e4e2209748a69d253ea46414e575" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5" is_pre_election: true
I20260709 13:04:11.249372 15030 raft_consensus.cc:2393] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 39f9e4e2209748a69d253ea46414e575 in current term 1: Already voted for candidate 09d3fa2b7ef942ae858e2c5727d99ce5 in this term.
I20260709 13:04:11.249541 15076 tablet_bootstrap.cc:492] T be17f0536ffd4425a0db52b530c5f3ef P 39f9e4e2209748a69d253ea46414e575: No bootstrap required, opened a new log
I20260709 13:04:11.249451 14957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "810f60f24e33413985aa63f396a2b55a" candidate_uuid: "39f9e4e2209748a69d253ea46414e575" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" is_pre_election: true
I20260709 13:04:11.250669 15076 ts_tablet_manager.cc:1403] T be17f0536ffd4425a0db52b530c5f3ef P 39f9e4e2209748a69d253ea46414e575: Time spent bootstrapping tablet: real 0.012s	user 0.010s	sys 0.000s
I20260709 13:04:11.251350 14847 leader_election.cc:304] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39f9e4e2209748a69d253ea46414e575; no voters: 09d3fa2b7ef942ae858e2c5727d99ce5, 435d239f6f2441ac9d0e835010bfd2b5
I20260709 13:04:11.252751 15084 raft_consensus.cc:3060] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:11.253199 15076 raft_consensus.cc:359] T be17f0536ffd4425a0db52b530c5f3ef P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:11.254348 15076 raft_consensus.cc:385] T be17f0536ffd4425a0db52b530c5f3ef P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:11.254643 15076 raft_consensus.cc:740] T be17f0536ffd4425a0db52b530c5f3ef P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39f9e4e2209748a69d253ea46414e575, State: Initialized, Role: FOLLOWER
I20260709 13:04:11.256049 15076 consensus_queue.cc:260] T be17f0536ffd4425a0db52b530c5f3ef P 39f9e4e2209748a69d253ea46414e575 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:11.258000 15076 ts_tablet_manager.cc:1434] T be17f0536ffd4425a0db52b530c5f3ef P 39f9e4e2209748a69d253ea46414e575: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:04:11.262212 15084 raft_consensus.cc:2749] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:11.276659 15084 raft_consensus.cc:493] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.277105 15084 raft_consensus.cc:515] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:11.279485 14957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7481805a7cfe4c2c8da2b33cf58c7ea5" candidate_uuid: "39f9e4e2209748a69d253ea46414e575" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" is_pre_election: true
I20260709 13:04:11.279754 15030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7481805a7cfe4c2c8da2b33cf58c7ea5" candidate_uuid: "39f9e4e2209748a69d253ea46414e575" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5" is_pre_election: true
I20260709 13:04:11.279840 15084 leader_election.cc:290] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 39f9e4e2209748a69d253ea46414e575 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2:33423), 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3:45113)
I20260709 13:04:11.280509 15030 raft_consensus.cc:2393] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 39f9e4e2209748a69d253ea46414e575 in current term 1: Already voted for candidate 09d3fa2b7ef942ae858e2c5727d99ce5 in this term.
I20260709 13:04:11.281427 14846 leader_election.cc:304] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 39f9e4e2209748a69d253ea46414e575 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39f9e4e2209748a69d253ea46414e575; no voters: 09d3fa2b7ef942ae858e2c5727d99ce5, 435d239f6f2441ac9d0e835010bfd2b5
I20260709 13:04:11.282073 15084 raft_consensus.cc:3060] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:11.289179 15084 raft_consensus.cc:2749] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 39f9e4e2209748a69d253ea46414e575 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:11.312325 15117 consensus_queue.cc:1048] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:11.325932 14883 raft_consensus.cc:3060] T be17f0536ffd4425a0db52b530c5f3ef P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:11.334149 15104 consensus_queue.cc:1048] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:11.336541 15104 consensus_queue.cc:1048] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:11.351625 14883 raft_consensus.cc:3060] T 32bc5a8acc774a1d9c7e99691a591567 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:11.363205 15104 consensus_queue.cc:1048] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:11.401829 15084 consensus_queue.cc:1048] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575 [LEADER]: Connected to new peer: Peer: permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:11.415520 15084 consensus_queue.cc:1048] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:11.415849 15114 consensus_queue.cc:1048] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:11.435953 15030 raft_consensus.cc:3060] T f54152e1e2cf41a3970778369876d656 P 435d239f6f2441ac9d0e835010bfd2b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:11.444247 15084 consensus_queue.cc:1048] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575 [LEADER]: Connected to new peer: Peer: permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:11.459306 15122 consensus_queue.cc:1048] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:11.473593 15122 consensus_queue.cc:1048] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:11.498202 15122 consensus_queue.cc:1048] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:11.525105 15121 consensus_queue.cc:1048] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:11.527699 14958 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "810f60f24e33413985aa63f396a2b55a"
dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
mode: GRACEFUL
new_leader_uuid: "39f9e4e2209748a69d253ea46414e575"
 from {username='slave'} at 127.0.0.1:44824
I20260709 13:04:11.528422 14958 raft_consensus.cc:606] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 LEADER]: Received request to transfer leadership to TS 39f9e4e2209748a69d253ea46414e575
I20260709 13:04:11.550393 15030 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "32bc5a8acc774a1d9c7e99691a591567"
dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5"
mode: GRACEFUL
new_leader_uuid: "39f9e4e2209748a69d253ea46414e575"
 from {username='slave'} at 127.0.0.1:58678
I20260709 13:04:11.551014 15030 raft_consensus.cc:606] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 LEADER]: Received request to transfer leadership to TS 39f9e4e2209748a69d253ea46414e575
I20260709 13:04:11.554132 15121 consensus_queue.cc:1048] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:11.556481 15032 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bb4dacc583b6453cb577a9e46c45c3af"
dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5"
mode: GRACEFUL
new_leader_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
 from {username='slave'} at 127.0.0.1:58678
I20260709 13:04:11.556981 15032 raft_consensus.cc:606] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 LEADER]: Received request to transfer leadership to TS 09d3fa2b7ef942ae858e2c5727d99ce5
I20260709 13:04:11.563603 15121 consensus_queue.cc:1048] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:11.566742 15032 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "be17f0536ffd4425a0db52b530c5f3ef"
dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5"
mode: GRACEFUL
new_leader_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
 from {username='slave'} at 127.0.0.1:58678
I20260709 13:04:11.567286 15032 raft_consensus.cc:606] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 LEADER]: Received request to transfer leadership to TS 09d3fa2b7ef942ae858e2c5727d99ce5
I20260709 13:04:11.581348 14958 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "90b851c6655d47c09fca18bbfc1a4d95"
dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
mode: GRACEFUL
new_leader_uuid: "435d239f6f2441ac9d0e835010bfd2b5"
 from {username='slave'} at 127.0.0.1:44824
I20260709 13:04:11.581948 14958 raft_consensus.cc:606] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 LEADER]: Received request to transfer leadership to TS 435d239f6f2441ac9d0e835010bfd2b5
I20260709 13:04:11.659301 15084 raft_consensus.cc:493] T 14bf28d1d32a48e7973484fb93521945 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.659826 15084 raft_consensus.cc:515] T 14bf28d1d32a48e7973484fb93521945 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:11.661563 15084 leader_election.cc:290] T 14bf28d1d32a48e7973484fb93521945 P 39f9e4e2209748a69d253ea46414e575 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2:33423), 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3:45113)
I20260709 13:04:11.662709 15030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bf28d1d32a48e7973484fb93521945" candidate_uuid: "39f9e4e2209748a69d253ea46414e575" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5" is_pre_election: true
I20260709 13:04:11.662709 14958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bf28d1d32a48e7973484fb93521945" candidate_uuid: "39f9e4e2209748a69d253ea46414e575" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" is_pre_election: true
I20260709 13:04:11.663647 15030 raft_consensus.cc:2393] T 14bf28d1d32a48e7973484fb93521945 P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 39f9e4e2209748a69d253ea46414e575 in current term 1: Already voted for candidate 09d3fa2b7ef942ae858e2c5727d99ce5 in this term.
I20260709 13:04:11.664649 14846 leader_election.cc:304] T 14bf28d1d32a48e7973484fb93521945 P 39f9e4e2209748a69d253ea46414e575 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39f9e4e2209748a69d253ea46414e575; no voters: 09d3fa2b7ef942ae858e2c5727d99ce5, 435d239f6f2441ac9d0e835010bfd2b5
I20260709 13:04:11.665381 15084 raft_consensus.cc:3060] T 14bf28d1d32a48e7973484fb93521945 P 39f9e4e2209748a69d253ea46414e575 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:11.670012 15084 raft_consensus.cc:2749] T 14bf28d1d32a48e7973484fb93521945 P 39f9e4e2209748a69d253ea46414e575 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:11.684929 15121 consensus_queue.cc:1048] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:11.690110 15104 consensus_queue.cc:1048] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:11.699569 15121 consensus_queue.cc:1048] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:11.705363 15104 raft_consensus.cc:993] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5: : Instructing follower 09d3fa2b7ef942ae858e2c5727d99ce5 to start an election
I20260709 13:04:11.706846 15117 raft_consensus.cc:1081] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 LEADER]: Signalling peer 09d3fa2b7ef942ae858e2c5727d99ce5 to start an election
I20260709 13:04:11.708823 14957 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bb4dacc583b6453cb577a9e46c45c3af"
dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
 from {username='slave'} at 127.0.0.1:44802
I20260709 13:04:11.709427 14957 raft_consensus.cc:493] T bb4dacc583b6453cb577a9e46c45c3af P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:11.709805 14957 raft_consensus.cc:3060] T bb4dacc583b6453cb577a9e46c45c3af P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:11.715041 15117 consensus_queue.cc:1048] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:11.722213 14957 raft_consensus.cc:515] T bb4dacc583b6453cb577a9e46c45c3af P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:11.723479 14958 raft_consensus.cc:1240] T bb4dacc583b6453cb577a9e46c45c3af P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 FOLLOWER]: Rejecting Update request from peer 435d239f6f2441ac9d0e835010bfd2b5 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:11.726668 14883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb4dacc583b6453cb577a9e46c45c3af" candidate_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "39f9e4e2209748a69d253ea46414e575"
I20260709 13:04:11.728174 15117 consensus_queue.cc:1059] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:11.729609 15082 raft_consensus.cc:3055] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:11.730089 15082 raft_consensus.cc:740] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 435d239f6f2441ac9d0e835010bfd2b5, State: Running, Role: LEADER
I20260709 13:04:11.731264 15030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb4dacc583b6453cb577a9e46c45c3af" candidate_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5"
I20260709 13:04:11.732146 14957 leader_election.cc:290] T bb4dacc583b6453cb577a9e46c45c3af P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 2 election: Requested vote from peers 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371), 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3:45113)
I20260709 13:04:11.731204 15082 consensus_queue.cc:260] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:11.733000 15082 raft_consensus.cc:3060] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:11.744042 15030 raft_consensus.cc:2468] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d3fa2b7ef942ae858e2c5727d99ce5 in term 2.
I20260709 13:04:11.745474 14921 leader_election.cc:304] T bb4dacc583b6453cb577a9e46c45c3af P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 09d3fa2b7ef942ae858e2c5727d99ce5, 435d239f6f2441ac9d0e835010bfd2b5; no voters: 39f9e4e2209748a69d253ea46414e575
I20260709 13:04:11.746155 15121 raft_consensus.cc:2804] T bb4dacc583b6453cb577a9e46c45c3af P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:11.746620 15121 raft_consensus.cc:697] T bb4dacc583b6453cb577a9e46c45c3af P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 LEADER]: Becoming Leader. State: Replica: 09d3fa2b7ef942ae858e2c5727d99ce5, State: Running, Role: LEADER
I20260709 13:04:11.747381 15121 consensus_queue.cc:237] T bb4dacc583b6453cb577a9e46c45c3af P 09d3fa2b7ef942ae858e2c5727d99ce5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:11.754787 14776 catalog_manager.cc:5697] T bb4dacc583b6453cb577a9e46c45c3af P 09d3fa2b7ef942ae858e2c5727d99ce5 reported cstate change: term changed from 1 to 2, leader changed from 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3) to 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2). New cstate: current_term: 2 leader_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:11.837713 15117 raft_consensus.cc:993] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5: : Instructing follower 09d3fa2b7ef942ae858e2c5727d99ce5 to start an election
I20260709 13:04:11.838265 15082 raft_consensus.cc:1081] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 LEADER]: Signalling peer 09d3fa2b7ef942ae858e2c5727d99ce5 to start an election
I20260709 13:04:11.839777 14958 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "be17f0536ffd4425a0db52b530c5f3ef"
dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
 from {username='slave'} at 127.0.0.1:44802
I20260709 13:04:11.840250 14958 raft_consensus.cc:493] T be17f0536ffd4425a0db52b530c5f3ef P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:11.840553 14958 raft_consensus.cc:3060] T be17f0536ffd4425a0db52b530c5f3ef P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:11.844774 14958 raft_consensus.cc:515] T be17f0536ffd4425a0db52b530c5f3ef P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:11.846323 14958 leader_election.cc:290] T be17f0536ffd4425a0db52b530c5f3ef P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 2 election: Requested vote from peers 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371), 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3:45113)
I20260709 13:04:11.847211 14882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17f0536ffd4425a0db52b530c5f3ef" candidate_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "39f9e4e2209748a69d253ea46414e575"
I20260709 13:04:11.847391 15030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be17f0536ffd4425a0db52b530c5f3ef" candidate_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5"
I20260709 13:04:11.847832 14882 raft_consensus.cc:3060] T be17f0536ffd4425a0db52b530c5f3ef P 39f9e4e2209748a69d253ea46414e575 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:11.847991 15030 raft_consensus.cc:3055] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:11.848397 15030 raft_consensus.cc:740] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 435d239f6f2441ac9d0e835010bfd2b5, State: Running, Role: LEADER
I20260709 13:04:11.849112 15030 consensus_queue.cc:260] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:11.850124 15030 raft_consensus.cc:3060] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:11.854197 14882 raft_consensus.cc:2468] T be17f0536ffd4425a0db52b530c5f3ef P 39f9e4e2209748a69d253ea46414e575 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d3fa2b7ef942ae858e2c5727d99ce5 in term 2.
I20260709 13:04:11.854231 15030 raft_consensus.cc:2468] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d3fa2b7ef942ae858e2c5727d99ce5 in term 2.
I20260709 13:04:11.855422 14922 leader_election.cc:304] T be17f0536ffd4425a0db52b530c5f3ef P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d3fa2b7ef942ae858e2c5727d99ce5, 39f9e4e2209748a69d253ea46414e575; no voters: 
I20260709 13:04:11.856252 15121 raft_consensus.cc:2804] T be17f0536ffd4425a0db52b530c5f3ef P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:11.856742 15121 raft_consensus.cc:697] T be17f0536ffd4425a0db52b530c5f3ef P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 LEADER]: Becoming Leader. State: Replica: 09d3fa2b7ef942ae858e2c5727d99ce5, State: Running, Role: LEADER
I20260709 13:04:11.857596 15121 consensus_queue.cc:237] T be17f0536ffd4425a0db52b530c5f3ef P 09d3fa2b7ef942ae858e2c5727d99ce5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:11.864717 14778 catalog_manager.cc:5697] T be17f0536ffd4425a0db52b530c5f3ef P 09d3fa2b7ef942ae858e2c5727d99ce5 reported cstate change: term changed from 1 to 2, leader changed from 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3) to 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2). New cstate: current_term: 2 leader_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:11.868664 15117 raft_consensus.cc:993] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5: : Instructing follower 39f9e4e2209748a69d253ea46414e575 to start an election
I20260709 13:04:11.869094 15117 raft_consensus.cc:1081] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 LEADER]: Signalling peer 39f9e4e2209748a69d253ea46414e575 to start an election
I20260709 13:04:11.871912 14882 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "32bc5a8acc774a1d9c7e99691a591567"
dest_uuid: "39f9e4e2209748a69d253ea46414e575"
 from {username='slave'} at 127.0.0.1:60516
I20260709 13:04:11.872864 14882 raft_consensus.cc:493] T 32bc5a8acc774a1d9c7e99691a591567 P 39f9e4e2209748a69d253ea46414e575 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:11.873278 14882 raft_consensus.cc:3060] T 32bc5a8acc774a1d9c7e99691a591567 P 39f9e4e2209748a69d253ea46414e575 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:11.877872 14882 raft_consensus.cc:515] T 32bc5a8acc774a1d9c7e99691a591567 P 39f9e4e2209748a69d253ea46414e575 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:11.879417 14882 leader_election.cc:290] T 32bc5a8acc774a1d9c7e99691a591567 P 39f9e4e2209748a69d253ea46414e575 [CANDIDATE]: Term 2 election: Requested vote from peers 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2:33423), 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3:45113)
I20260709 13:04:11.880264 14958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32bc5a8acc774a1d9c7e99691a591567" candidate_uuid: "39f9e4e2209748a69d253ea46414e575" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
I20260709 13:04:11.880483 15030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32bc5a8acc774a1d9c7e99691a591567" candidate_uuid: "39f9e4e2209748a69d253ea46414e575" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5"
I20260709 13:04:11.880846 14958 raft_consensus.cc:3060] T 32bc5a8acc774a1d9c7e99691a591567 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:11.881050 15030 raft_consensus.cc:3055] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:11.881423 15030 raft_consensus.cc:740] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 435d239f6f2441ac9d0e835010bfd2b5, State: Running, Role: LEADER
I20260709 13:04:11.882050 15030 consensus_queue.cc:260] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:11.882822 15030 raft_consensus.cc:3060] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:11.887306 15030 raft_consensus.cc:2468] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39f9e4e2209748a69d253ea46414e575 in term 2.
I20260709 13:04:11.887874 14958 raft_consensus.cc:2468] T 32bc5a8acc774a1d9c7e99691a591567 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39f9e4e2209748a69d253ea46414e575 in term 2.
I20260709 13:04:11.888309 14846 leader_election.cc:304] T 32bc5a8acc774a1d9c7e99691a591567 P 39f9e4e2209748a69d253ea46414e575 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39f9e4e2209748a69d253ea46414e575, 435d239f6f2441ac9d0e835010bfd2b5; no voters: 
I20260709 13:04:11.889135 15084 raft_consensus.cc:2804] T 32bc5a8acc774a1d9c7e99691a591567 P 39f9e4e2209748a69d253ea46414e575 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:11.889544 15084 raft_consensus.cc:697] T 32bc5a8acc774a1d9c7e99691a591567 P 39f9e4e2209748a69d253ea46414e575 [term 2 LEADER]: Becoming Leader. State: Replica: 39f9e4e2209748a69d253ea46414e575, State: Running, Role: LEADER
I20260709 13:04:11.890229 15084 consensus_queue.cc:237] T 32bc5a8acc774a1d9c7e99691a591567 P 39f9e4e2209748a69d253ea46414e575 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:11.896247 14778 catalog_manager.cc:5697] T 32bc5a8acc774a1d9c7e99691a591567 P 39f9e4e2209748a69d253ea46414e575 reported cstate change: term changed from 1 to 2, leader changed from 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3) to 39f9e4e2209748a69d253ea46414e575 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "39f9e4e2209748a69d253ea46414e575" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:12.012545 15122 raft_consensus.cc:993] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5: : Instructing follower 435d239f6f2441ac9d0e835010bfd2b5 to start an election
I20260709 13:04:12.013021 15121 raft_consensus.cc:1081] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 LEADER]: Signalling peer 435d239f6f2441ac9d0e835010bfd2b5 to start an election
I20260709 13:04:12.014540 15030 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "90b851c6655d47c09fca18bbfc1a4d95"
dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5"
 from {username='slave'} at 127.0.0.1:58670
I20260709 13:04:12.015115 15030 raft_consensus.cc:493] T 90b851c6655d47c09fca18bbfc1a4d95 P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:12.015491 15030 raft_consensus.cc:3060] T 90b851c6655d47c09fca18bbfc1a4d95 P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:12.022218 15030 raft_consensus.cc:515] T 90b851c6655d47c09fca18bbfc1a4d95 P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:12.024230 15030 leader_election.cc:290] T 90b851c6655d47c09fca18bbfc1a4d95 P 435d239f6f2441ac9d0e835010bfd2b5 [CANDIDATE]: Term 2 election: Requested vote from peers 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371), 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2:33423)
I20260709 13:04:12.025066 14882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90b851c6655d47c09fca18bbfc1a4d95" candidate_uuid: "435d239f6f2441ac9d0e835010bfd2b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "39f9e4e2209748a69d253ea46414e575"
I20260709 13:04:12.025722 14882 raft_consensus.cc:3060] T 90b851c6655d47c09fca18bbfc1a4d95 P 39f9e4e2209748a69d253ea46414e575 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:12.025650 14958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90b851c6655d47c09fca18bbfc1a4d95" candidate_uuid: "435d239f6f2441ac9d0e835010bfd2b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
I20260709 13:04:12.026341 14958 raft_consensus.cc:3055] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:12.026654 14958 raft_consensus.cc:740] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 09d3fa2b7ef942ae858e2c5727d99ce5, State: Running, Role: LEADER
I20260709 13:04:12.027467 14958 consensus_queue.cc:260] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:12.028805 14958 raft_consensus.cc:3060] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:12.033476 14882 raft_consensus.cc:2468] T 90b851c6655d47c09fca18bbfc1a4d95 P 39f9e4e2209748a69d253ea46414e575 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 435d239f6f2441ac9d0e835010bfd2b5 in term 2.
I20260709 13:04:12.033294 14958 raft_consensus.cc:2468] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 435d239f6f2441ac9d0e835010bfd2b5 in term 2.
I20260709 13:04:12.034536 14996 leader_election.cc:304] T 90b851c6655d47c09fca18bbfc1a4d95 P 435d239f6f2441ac9d0e835010bfd2b5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39f9e4e2209748a69d253ea46414e575, 435d239f6f2441ac9d0e835010bfd2b5; no voters: 
I20260709 13:04:12.035341 15082 raft_consensus.cc:2804] T 90b851c6655d47c09fca18bbfc1a4d95 P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:12.035856 15082 raft_consensus.cc:697] T 90b851c6655d47c09fca18bbfc1a4d95 P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 LEADER]: Becoming Leader. State: Replica: 435d239f6f2441ac9d0e835010bfd2b5, State: Running, Role: LEADER
I20260709 13:04:12.036974 15122 raft_consensus.cc:993] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5: : Instructing follower 39f9e4e2209748a69d253ea46414e575 to start an election
I20260709 13:04:12.036674 15082 consensus_queue.cc:237] T 90b851c6655d47c09fca18bbfc1a4d95 P 435d239f6f2441ac9d0e835010bfd2b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:12.037423 15121 raft_consensus.cc:1081] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 LEADER]: Signalling peer 39f9e4e2209748a69d253ea46414e575 to start an election
I20260709 13:04:12.039100 14883 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "810f60f24e33413985aa63f396a2b55a"
dest_uuid: "39f9e4e2209748a69d253ea46414e575"
 from {username='slave'} at 127.0.0.1:60528
I20260709 13:04:12.039623 14883 raft_consensus.cc:493] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:12.039997 14883 raft_consensus.cc:3060] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:12.044566 14778 catalog_manager.cc:5697] T 90b851c6655d47c09fca18bbfc1a4d95 P 435d239f6f2441ac9d0e835010bfd2b5 reported cstate change: term changed from 1 to 2, leader changed from 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2) to 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3). New cstate: current_term: 2 leader_uuid: "435d239f6f2441ac9d0e835010bfd2b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:12.045262 14883 raft_consensus.cc:515] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } }
I20260709 13:04:12.047996 14883 leader_election.cc:290] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [CANDIDATE]: Term 2 election: Requested vote from peers 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3:45113), 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2:33423)
I20260709 13:04:12.048244 14958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "810f60f24e33413985aa63f396a2b55a" candidate_uuid: "39f9e4e2209748a69d253ea46414e575" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
I20260709 13:04:12.048868 14958 raft_consensus.cc:3055] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:12.049233 14958 raft_consensus.cc:740] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 09d3fa2b7ef942ae858e2c5727d99ce5, State: Running, Role: LEADER
I20260709 13:04:12.050489 15030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "810f60f24e33413985aa63f396a2b55a" candidate_uuid: "39f9e4e2209748a69d253ea46414e575" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5"
I20260709 13:04:12.051061 15030 raft_consensus.cc:3060] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:12.050798 14958 consensus_queue.cc:260] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } }
I20260709 13:04:12.052018 14958 raft_consensus.cc:3060] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:12.057951 15030 raft_consensus.cc:2468] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39f9e4e2209748a69d253ea46414e575 in term 2.
I20260709 13:04:12.058974 14846 leader_election.cc:304] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39f9e4e2209748a69d253ea46414e575, 435d239f6f2441ac9d0e835010bfd2b5; no voters: 
I20260709 13:04:12.058988 14958 raft_consensus.cc:2468] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39f9e4e2209748a69d253ea46414e575 in term 2.
I20260709 13:04:12.059764 15084 raft_consensus.cc:2804] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:12.060264 15084 raft_consensus.cc:697] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [term 2 LEADER]: Becoming Leader. State: Replica: 39f9e4e2209748a69d253ea46414e575, State: Running, Role: LEADER
I20260709 13:04:12.061043 15084 consensus_queue.cc:237] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } }
I20260709 13:04:12.067191 14778 catalog_manager.cc:5697] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 reported cstate change: term changed from 1 to 2, leader changed from 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2) to 39f9e4e2209748a69d253ea46414e575 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "39f9e4e2209748a69d253ea46414e575" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:12.202147 14883 raft_consensus.cc:3060] T bb4dacc583b6453cb577a9e46c45c3af P 39f9e4e2209748a69d253ea46414e575 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:12.211203 14883 raft_consensus.cc:1275] T bb4dacc583b6453cb577a9e46c45c3af P 39f9e4e2209748a69d253ea46414e575 [term 2 FOLLOWER]: Refusing update from remote peer 09d3fa2b7ef942ae858e2c5727d99ce5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:12.212960 15122 consensus_queue.cc:1048] T bb4dacc583b6453cb577a9e46c45c3af P 09d3fa2b7ef942ae858e2c5727d99ce5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:12.223248 15030 raft_consensus.cc:1275] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 FOLLOWER]: Refusing update from remote peer 09d3fa2b7ef942ae858e2c5727d99ce5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:12.226042 15122 consensus_queue.cc:1048] T bb4dacc583b6453cb577a9e46c45c3af P 09d3fa2b7ef942ae858e2c5727d99ce5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:12.273744 15030 raft_consensus.cc:1275] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 FOLLOWER]: Refusing update from remote peer 39f9e4e2209748a69d253ea46414e575: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:12.274930 15084 consensus_queue.cc:1048] T 32bc5a8acc774a1d9c7e99691a591567 P 39f9e4e2209748a69d253ea46414e575 [LEADER]: Connected to new peer: Peer: permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:12.282853 14958 raft_consensus.cc:1275] T 32bc5a8acc774a1d9c7e99691a591567 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 FOLLOWER]: Refusing update from remote peer 39f9e4e2209748a69d253ea46414e575: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:12.284392 15112 consensus_queue.cc:1048] T 32bc5a8acc774a1d9c7e99691a591567 P 39f9e4e2209748a69d253ea46414e575 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:12.316622 15030 raft_consensus.cc:1275] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 FOLLOWER]: Refusing update from remote peer 09d3fa2b7ef942ae858e2c5727d99ce5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:12.317946 15169 consensus_queue.cc:1048] T be17f0536ffd4425a0db52b530c5f3ef P 09d3fa2b7ef942ae858e2c5727d99ce5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:12.325186 14883 raft_consensus.cc:1275] T be17f0536ffd4425a0db52b530c5f3ef P 39f9e4e2209748a69d253ea46414e575 [term 2 FOLLOWER]: Refusing update from remote peer 09d3fa2b7ef942ae858e2c5727d99ce5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:12.326539 15123 consensus_queue.cc:1048] T be17f0536ffd4425a0db52b530c5f3ef P 09d3fa2b7ef942ae858e2c5727d99ce5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:12.428260 14958 raft_consensus.cc:1275] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 FOLLOWER]: Refusing update from remote peer 435d239f6f2441ac9d0e835010bfd2b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:12.429587 15082 consensus_queue.cc:1048] T 90b851c6655d47c09fca18bbfc1a4d95 P 435d239f6f2441ac9d0e835010bfd2b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:12.436832 14883 raft_consensus.cc:1275] T 90b851c6655d47c09fca18bbfc1a4d95 P 39f9e4e2209748a69d253ea46414e575 [term 2 FOLLOWER]: Refusing update from remote peer 435d239f6f2441ac9d0e835010bfd2b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:12.438545 15170 consensus_queue.cc:1048] T 90b851c6655d47c09fca18bbfc1a4d95 P 435d239f6f2441ac9d0e835010bfd2b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:12.509951 15030 raft_consensus.cc:1275] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 FOLLOWER]: Refusing update from remote peer 39f9e4e2209748a69d253ea46414e575: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:12.511096 15112 consensus_queue.cc:1048] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [LEADER]: Connected to new peer: Peer: permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:12.517829 14958 raft_consensus.cc:1275] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 FOLLOWER]: Refusing update from remote peer 39f9e4e2209748a69d253ea46414e575: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:12.520649 15114 consensus_queue.cc:1048] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:14.609994 14748 auto_leader_rebalancer-test.cc:316] The leader distribution is 
I20260709 13:04:14.610951 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:14.632159 14958 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7481805a7cfe4c2c8da2b33cf58c7ea5"
dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
mode: GRACEFUL
new_leader_uuid: "435d239f6f2441ac9d0e835010bfd2b5"
 from {username='slave'} at 127.0.0.1:44830
I20260709 13:04:14.632781 14958 raft_consensus.cc:606] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 LEADER]: Received request to transfer leadership to TS 435d239f6f2441ac9d0e835010bfd2b5
I20260709 13:04:14.643301 14883 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "810f60f24e33413985aa63f396a2b55a"
dest_uuid: "39f9e4e2209748a69d253ea46414e575"
mode: GRACEFUL
new_leader_uuid: "435d239f6f2441ac9d0e835010bfd2b5"
 from {username='slave'} at 127.0.0.1:60532
I20260709 13:04:14.643855 14883 raft_consensus.cc:606] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [term 2 LEADER]: Received request to transfer leadership to TS 435d239f6f2441ac9d0e835010bfd2b5
I20260709 13:04:14.644872 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:04:14.645346 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:14.869953 15174 raft_consensus.cc:993] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5: : Instructing follower 435d239f6f2441ac9d0e835010bfd2b5 to start an election
I20260709 13:04:14.870460 15169 raft_consensus.cc:1081] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 LEADER]: Signalling peer 435d239f6f2441ac9d0e835010bfd2b5 to start an election
I20260709 13:04:14.871981 15032 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7481805a7cfe4c2c8da2b33cf58c7ea5"
dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5"
 from {username='slave'} at 127.0.0.1:58670
I20260709 13:04:14.872452 15032 raft_consensus.cc:493] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:14.872706 15032 raft_consensus.cc:3060] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:14.876938 15032 raft_consensus.cc:515] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:14.878397 15032 leader_election.cc:290] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5 [CANDIDATE]: Term 2 election: Requested vote from peers 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371), 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2:33423)
I20260709 13:04:14.879227 14883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7481805a7cfe4c2c8da2b33cf58c7ea5" candidate_uuid: "435d239f6f2441ac9d0e835010bfd2b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "39f9e4e2209748a69d253ea46414e575"
I20260709 13:04:14.879493 14958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7481805a7cfe4c2c8da2b33cf58c7ea5" candidate_uuid: "435d239f6f2441ac9d0e835010bfd2b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
I20260709 13:04:14.879654 14883 raft_consensus.cc:3060] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 39f9e4e2209748a69d253ea46414e575 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:14.880079 14958 raft_consensus.cc:3055] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:14.880431 14958 raft_consensus.cc:740] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 09d3fa2b7ef942ae858e2c5727d99ce5, State: Running, Role: LEADER
I20260709 13:04:14.881052 14958 consensus_queue.cc:260] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:14.881928 14958 raft_consensus.cc:3060] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:14.884229 14883 raft_consensus.cc:2468] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 39f9e4e2209748a69d253ea46414e575 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 435d239f6f2441ac9d0e835010bfd2b5 in term 2.
I20260709 13:04:14.885192 14996 leader_election.cc:304] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39f9e4e2209748a69d253ea46414e575, 435d239f6f2441ac9d0e835010bfd2b5; no voters: 
I20260709 13:04:14.885812 15173 raft_consensus.cc:2804] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:14.886731 14958 raft_consensus.cc:2468] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 435d239f6f2441ac9d0e835010bfd2b5 in term 2.
I20260709 13:04:14.886831 15173 raft_consensus.cc:697] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 LEADER]: Becoming Leader. State: Replica: 435d239f6f2441ac9d0e835010bfd2b5, State: Running, Role: LEADER
I20260709 13:04:14.888666 15173 consensus_queue.cc:237] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:14.897485 14776 catalog_manager.cc:5697] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5 reported cstate change: term changed from 1 to 2, leader changed from 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2) to 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3). New cstate: current_term: 2 leader_uuid: "435d239f6f2441ac9d0e835010bfd2b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:15.193405 15084 raft_consensus.cc:993] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575: : Instructing follower 435d239f6f2441ac9d0e835010bfd2b5 to start an election
I20260709 13:04:15.193823 15184 raft_consensus.cc:1081] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [term 2 LEADER]: Signalling peer 435d239f6f2441ac9d0e835010bfd2b5 to start an election
I20260709 13:04:15.195150 15032 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "810f60f24e33413985aa63f396a2b55a"
dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5"
 from {username='slave'} at 127.0.0.1:58658
I20260709 13:04:15.195652 15032 raft_consensus.cc:493] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:15.195992 15032 raft_consensus.cc:3060] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:15.200376 15032 raft_consensus.cc:515] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } }
I20260709 13:04:15.202227 15032 leader_election.cc:290] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [CANDIDATE]: Term 3 election: Requested vote from peers 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2:33423), 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371)
I20260709 13:04:15.202986 14957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "810f60f24e33413985aa63f396a2b55a" candidate_uuid: "435d239f6f2441ac9d0e835010bfd2b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
I20260709 13:04:15.203233 14883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "810f60f24e33413985aa63f396a2b55a" candidate_uuid: "435d239f6f2441ac9d0e835010bfd2b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "39f9e4e2209748a69d253ea46414e575"
I20260709 13:04:15.203739 14883 raft_consensus.cc:3055] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:15.204063 14883 raft_consensus.cc:740] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 39f9e4e2209748a69d253ea46414e575, State: Running, Role: LEADER
I20260709 13:04:15.204643 14957 raft_consensus.cc:3060] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:15.204974 14883 consensus_queue.cc:260] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } }
I20260709 13:04:15.206189 14883 raft_consensus.cc:3060] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:15.211356 14957 raft_consensus.cc:2468] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 435d239f6f2441ac9d0e835010bfd2b5 in term 3.
I20260709 13:04:15.212476 14996 leader_election.cc:304] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d3fa2b7ef942ae858e2c5727d99ce5, 435d239f6f2441ac9d0e835010bfd2b5; no voters: 
I20260709 13:04:15.212536 14883 raft_consensus.cc:2468] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 435d239f6f2441ac9d0e835010bfd2b5 in term 3.
I20260709 13:04:15.213217 15173 raft_consensus.cc:2804] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:15.213721 15173 raft_consensus.cc:697] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [term 3 LEADER]: Becoming Leader. State: Replica: 435d239f6f2441ac9d0e835010bfd2b5, State: Running, Role: LEADER
I20260709 13:04:15.214491 15173 consensus_queue.cc:237] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } }
I20260709 13:04:15.221374 14776 catalog_manager.cc:5697] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 reported cstate change: term changed from 2 to 3, leader changed from 39f9e4e2209748a69d253ea46414e575 (127.14.103.1) to 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3). New cstate: current_term: 3 leader_uuid: "435d239f6f2441ac9d0e835010bfd2b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:15.355846 14883 raft_consensus.cc:1275] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 39f9e4e2209748a69d253ea46414e575 [term 2 FOLLOWER]: Refusing update from remote peer 435d239f6f2441ac9d0e835010bfd2b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:15.357013 15173 consensus_queue.cc:1048] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:15.370188 14957 raft_consensus.cc:1275] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 FOLLOWER]: Refusing update from remote peer 435d239f6f2441ac9d0e835010bfd2b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:15.371749 15178 consensus_queue.cc:1048] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:15.646229 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:15.647120 14957 raft_consensus.cc:1275] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 3 FOLLOWER]: Refusing update from remote peer 435d239f6f2441ac9d0e835010bfd2b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:15.648453 15189 consensus_queue.cc:1048] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:15.658418 14882 raft_consensus.cc:1275] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [term 3 FOLLOWER]: Refusing update from remote peer 435d239f6f2441ac9d0e835010bfd2b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:15.659232 15032 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "810f60f24e33413985aa63f396a2b55a"
dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5"
mode: GRACEFUL
new_leader_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
 from {username='slave'} at 127.0.0.1:58678
I20260709 13:04:15.659878 15032 raft_consensus.cc:606] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [term 3 LEADER]: Received request to transfer leadership to TS 09d3fa2b7ef942ae858e2c5727d99ce5
I20260709 13:04:15.660039 15173 consensus_queue.cc:1048] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:15.662267 15189 raft_consensus.cc:993] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5: : Instructing follower 09d3fa2b7ef942ae858e2c5727d99ce5 to start an election
I20260709 13:04:15.662674 15189 raft_consensus.cc:1081] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [term 3 LEADER]: Signalling peer 09d3fa2b7ef942ae858e2c5727d99ce5 to start an election
I20260709 13:04:15.666518 14957 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "810f60f24e33413985aa63f396a2b55a"
dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
 from {username='slave'} at 127.0.0.1:44802
I20260709 13:04:15.667196 14957 raft_consensus.cc:493] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:15.671288 14957 raft_consensus.cc:3060] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:15.679039 14882 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f54152e1e2cf41a3970778369876d656"
dest_uuid: "39f9e4e2209748a69d253ea46414e575"
mode: GRACEFUL
new_leader_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
 from {username='slave'} at 127.0.0.1:60536
I20260709 13:04:15.679700 14882 raft_consensus.cc:606] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575 [term 1 LEADER]: Received request to transfer leadership to TS 09d3fa2b7ef942ae858e2c5727d99ce5
I20260709 13:04:15.679847 14957 raft_consensus.cc:515] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } }
I20260709 13:04:15.681887 14957 leader_election.cc:290] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 4 election: Requested vote from peers 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3:45113), 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371)
I20260709 13:04:15.682477 15032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "810f60f24e33413985aa63f396a2b55a" candidate_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5"
I20260709 13:04:15.683059 15032 raft_consensus.cc:3055] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:15.683372 15032 raft_consensus.cc:740] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 435d239f6f2441ac9d0e835010bfd2b5, State: Running, Role: LEADER
I20260709 13:04:15.683562 14882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "810f60f24e33413985aa63f396a2b55a" candidate_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "39f9e4e2209748a69d253ea46414e575"
I20260709 13:04:15.684120 14882 raft_consensus.cc:3060] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:15.684011 15032 consensus_queue.cc:260] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } }
I20260709 13:04:15.685032 15032 raft_consensus.cc:3060] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:15.689118 14882 raft_consensus.cc:2468] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d3fa2b7ef942ae858e2c5727d99ce5 in term 4.
I20260709 13:04:15.689769 15032 raft_consensus.cc:2468] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d3fa2b7ef942ae858e2c5727d99ce5 in term 4.
I20260709 13:04:15.685495 14883 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c7c38457995f4e988d3ccf6bd94e6c37"
dest_uuid: "39f9e4e2209748a69d253ea46414e575"
mode: GRACEFUL
new_leader_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
 from {username='slave'} at 127.0.0.1:60536
I20260709 13:04:15.690466 14883 raft_consensus.cc:606] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575 [term 1 LEADER]: Received request to transfer leadership to TS 09d3fa2b7ef942ae858e2c5727d99ce5
I20260709 13:04:15.690860 14922 leader_election.cc:304] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d3fa2b7ef942ae858e2c5727d99ce5, 39f9e4e2209748a69d253ea46414e575; no voters: 
I20260709 13:04:15.691886 15169 raft_consensus.cc:2804] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:15.693172 15169 raft_consensus.cc:697] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 4 LEADER]: Becoming Leader. State: Replica: 09d3fa2b7ef942ae858e2c5727d99ce5, State: Running, Role: LEADER
I20260709 13:04:15.693944 15169 consensus_queue.cc:237] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } }
I20260709 13:04:15.696167 14882 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "32bc5a8acc774a1d9c7e99691a591567"
dest_uuid: "39f9e4e2209748a69d253ea46414e575"
mode: GRACEFUL
new_leader_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
 from {username='slave'} at 127.0.0.1:60536
I20260709 13:04:15.696656 14882 raft_consensus.cc:606] T 32bc5a8acc774a1d9c7e99691a591567 P 39f9e4e2209748a69d253ea46414e575 [term 2 LEADER]: Received request to transfer leadership to TS 09d3fa2b7ef942ae858e2c5727d99ce5
I20260709 13:04:15.700615 14778 catalog_manager.cc:5697] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 reported cstate change: term changed from 3 to 4, leader changed from 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3) to 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2). New cstate: current_term: 4 leader_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:15.710151 15032 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7481805a7cfe4c2c8da2b33cf58c7ea5"
dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5"
mode: GRACEFUL
new_leader_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
 from {username='slave'} at 127.0.0.1:58678
I20260709 13:04:15.710588 15032 raft_consensus.cc:606] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 LEADER]: Received request to transfer leadership to TS 09d3fa2b7ef942ae858e2c5727d99ce5
I20260709 13:04:15.775233 15184 raft_consensus.cc:993] T 32bc5a8acc774a1d9c7e99691a591567 P 39f9e4e2209748a69d253ea46414e575: : Instructing follower 09d3fa2b7ef942ae858e2c5727d99ce5 to start an election
I20260709 13:04:15.775655 15184 raft_consensus.cc:1081] T 32bc5a8acc774a1d9c7e99691a591567 P 39f9e4e2209748a69d253ea46414e575 [term 2 LEADER]: Signalling peer 09d3fa2b7ef942ae858e2c5727d99ce5 to start an election
I20260709 13:04:15.777071 14957 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "32bc5a8acc774a1d9c7e99691a591567"
dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
 from {username='slave'} at 127.0.0.1:44818
I20260709 13:04:15.777545 14957 raft_consensus.cc:493] T 32bc5a8acc774a1d9c7e99691a591567 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:15.777853 14957 raft_consensus.cc:3060] T 32bc5a8acc774a1d9c7e99691a591567 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:15.782212 15178 raft_consensus.cc:993] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5: : Instructing follower 09d3fa2b7ef942ae858e2c5727d99ce5 to start an election
I20260709 13:04:15.782670 15173 raft_consensus.cc:1081] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 LEADER]: Signalling peer 09d3fa2b7ef942ae858e2c5727d99ce5 to start an election
I20260709 13:04:15.784060 14958 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7481805a7cfe4c2c8da2b33cf58c7ea5"
dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
 from {username='slave'} at 127.0.0.1:44802
I20260709 13:04:15.784590 14958 raft_consensus.cc:493] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:15.784356 14957 raft_consensus.cc:515] T 32bc5a8acc774a1d9c7e99691a591567 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:15.784963 14958 raft_consensus.cc:3060] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:15.786403 14957 leader_election.cc:290] T 32bc5a8acc774a1d9c7e99691a591567 P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 3 election: Requested vote from peers 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371), 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3:45113)
I20260709 13:04:15.787097 14883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32bc5a8acc774a1d9c7e99691a591567" candidate_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "39f9e4e2209748a69d253ea46414e575"
I20260709 13:04:15.787443 15032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32bc5a8acc774a1d9c7e99691a591567" candidate_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5"
I20260709 13:04:15.787657 14883 raft_consensus.cc:3055] T 32bc5a8acc774a1d9c7e99691a591567 P 39f9e4e2209748a69d253ea46414e575 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:15.787997 15032 raft_consensus.cc:3060] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:15.788034 14883 raft_consensus.cc:740] T 32bc5a8acc774a1d9c7e99691a591567 P 39f9e4e2209748a69d253ea46414e575 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 39f9e4e2209748a69d253ea46414e575, State: Running, Role: LEADER
I20260709 13:04:15.788944 14883 consensus_queue.cc:260] T 32bc5a8acc774a1d9c7e99691a591567 P 39f9e4e2209748a69d253ea46414e575 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:15.790127 14883 raft_consensus.cc:3060] T 32bc5a8acc774a1d9c7e99691a591567 P 39f9e4e2209748a69d253ea46414e575 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:15.791431 14958 raft_consensus.cc:515] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:15.794036 14881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7481805a7cfe4c2c8da2b33cf58c7ea5" candidate_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "39f9e4e2209748a69d253ea46414e575"
I20260709 13:04:15.794310 14958 leader_election.cc:290] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 3 election: Requested vote from peers 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371), 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3:45113)
I20260709 13:04:15.795279 15030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7481805a7cfe4c2c8da2b33cf58c7ea5" candidate_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5"
I20260709 13:04:15.795068 15032 raft_consensus.cc:2468] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d3fa2b7ef942ae858e2c5727d99ce5 in term 3.
I20260709 13:04:15.795814 15030 raft_consensus.cc:3055] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:15.796087 15030 raft_consensus.cc:740] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 435d239f6f2441ac9d0e835010bfd2b5, State: Running, Role: LEADER
I20260709 13:04:15.796396 14921 leader_election.cc:304] T 32bc5a8acc774a1d9c7e99691a591567 P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d3fa2b7ef942ae858e2c5727d99ce5, 435d239f6f2441ac9d0e835010bfd2b5; no voters: 
I20260709 13:04:15.796686 15030 consensus_queue.cc:260] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:15.797076 15169 raft_consensus.cc:2804] T 32bc5a8acc774a1d9c7e99691a591567 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:15.797518 15030 raft_consensus.cc:3060] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:15.797573 15169 raft_consensus.cc:697] T 32bc5a8acc774a1d9c7e99691a591567 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 3 LEADER]: Becoming Leader. State: Replica: 09d3fa2b7ef942ae858e2c5727d99ce5, State: Running, Role: LEADER
I20260709 13:04:15.798496 15169 consensus_queue.cc:237] T 32bc5a8acc774a1d9c7e99691a591567 P 09d3fa2b7ef942ae858e2c5727d99ce5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:15.799024 14883 raft_consensus.cc:2468] T 32bc5a8acc774a1d9c7e99691a591567 P 39f9e4e2209748a69d253ea46414e575 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d3fa2b7ef942ae858e2c5727d99ce5 in term 3.
I20260709 13:04:15.802418 15030 raft_consensus.cc:2468] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d3fa2b7ef942ae858e2c5727d99ce5 in term 3.
I20260709 13:04:15.803448 14921 leader_election.cc:304] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 09d3fa2b7ef942ae858e2c5727d99ce5, 435d239f6f2441ac9d0e835010bfd2b5; no voters: 39f9e4e2209748a69d253ea46414e575
I20260709 13:04:15.804239 15169 raft_consensus.cc:2804] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:15.804677 15169 raft_consensus.cc:697] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 3 LEADER]: Becoming Leader. State: Replica: 09d3fa2b7ef942ae858e2c5727d99ce5, State: Running, Role: LEADER
I20260709 13:04:15.805344 15169 consensus_queue.cc:237] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:15.805720 14779 catalog_manager.cc:5697] T 32bc5a8acc774a1d9c7e99691a591567 P 09d3fa2b7ef942ae858e2c5727d99ce5 reported cstate change: term changed from 2 to 3, leader changed from 39f9e4e2209748a69d253ea46414e575 (127.14.103.1) to 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2). New cstate: current_term: 3 leader_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:15.816570 14779 catalog_manager.cc:5697] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 reported cstate change: term changed from 2 to 3, leader changed from 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3) to 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2). New cstate: current_term: 3 leader_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:15.830551 15197 raft_consensus.cc:993] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575: : Instructing follower 09d3fa2b7ef942ae858e2c5727d99ce5 to start an election
I20260709 13:04:15.830968 15197 raft_consensus.cc:1081] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575 [term 1 LEADER]: Signalling peer 09d3fa2b7ef942ae858e2c5727d99ce5 to start an election
I20260709 13:04:15.832362 14958 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f54152e1e2cf41a3970778369876d656"
dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
 from {username='slave'} at 127.0.0.1:44818
I20260709 13:04:15.832809 14958 raft_consensus.cc:493] T f54152e1e2cf41a3970778369876d656 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:15.833088 14958 raft_consensus.cc:3060] T f54152e1e2cf41a3970778369876d656 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:15.837312 14958 raft_consensus.cc:515] T f54152e1e2cf41a3970778369876d656 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:15.839174 14958 leader_election.cc:290] T f54152e1e2cf41a3970778369876d656 P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 2 election: Requested vote from peers 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371), 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3:45113)
I20260709 13:04:15.839833 14883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54152e1e2cf41a3970778369876d656" candidate_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "39f9e4e2209748a69d253ea46414e575"
I20260709 13:04:15.840415 14883 raft_consensus.cc:3055] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:15.840539 15032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54152e1e2cf41a3970778369876d656" candidate_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5"
I20260709 13:04:15.840828 14883 raft_consensus.cc:740] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 39f9e4e2209748a69d253ea46414e575, State: Running, Role: LEADER
I20260709 13:04:15.841159 15032 raft_consensus.cc:3060] T f54152e1e2cf41a3970778369876d656 P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:15.841630 14883 consensus_queue.cc:260] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:15.842419 14883 raft_consensus.cc:3060] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:15.845667 15032 raft_consensus.cc:2468] T f54152e1e2cf41a3970778369876d656 P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d3fa2b7ef942ae858e2c5727d99ce5 in term 2.
I20260709 13:04:15.846148 14883 raft_consensus.cc:2468] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d3fa2b7ef942ae858e2c5727d99ce5 in term 2.
I20260709 13:04:15.846617 14921 leader_election.cc:304] T f54152e1e2cf41a3970778369876d656 P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d3fa2b7ef942ae858e2c5727d99ce5, 435d239f6f2441ac9d0e835010bfd2b5; no voters: 
I20260709 13:04:15.847404 15169 raft_consensus.cc:2804] T f54152e1e2cf41a3970778369876d656 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:15.847864 15169 raft_consensus.cc:697] T f54152e1e2cf41a3970778369876d656 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 LEADER]: Becoming Leader. State: Replica: 09d3fa2b7ef942ae858e2c5727d99ce5, State: Running, Role: LEADER
I20260709 13:04:15.848522 15169 consensus_queue.cc:237] T f54152e1e2cf41a3970778369876d656 P 09d3fa2b7ef942ae858e2c5727d99ce5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:15.854470 14779 catalog_manager.cc:5697] T f54152e1e2cf41a3970778369876d656 P 09d3fa2b7ef942ae858e2c5727d99ce5 reported cstate change: term changed from 1 to 2, leader changed from 39f9e4e2209748a69d253ea46414e575 (127.14.103.1) to 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2). New cstate: current_term: 2 leader_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:15.987810 15184 raft_consensus.cc:993] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575: : Instructing follower 09d3fa2b7ef942ae858e2c5727d99ce5 to start an election
I20260709 13:04:15.988237 15197 raft_consensus.cc:1081] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575 [term 1 LEADER]: Signalling peer 09d3fa2b7ef942ae858e2c5727d99ce5 to start an election
I20260709 13:04:15.989434 14958 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c7c38457995f4e988d3ccf6bd94e6c37"
dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
 from {username='slave'} at 127.0.0.1:44818
I20260709 13:04:15.989898 14958 raft_consensus.cc:493] T c7c38457995f4e988d3ccf6bd94e6c37 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:15.990207 14958 raft_consensus.cc:3060] T c7c38457995f4e988d3ccf6bd94e6c37 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:15.994417 14958 raft_consensus.cc:515] T c7c38457995f4e988d3ccf6bd94e6c37 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:15.995788 14958 leader_election.cc:290] T c7c38457995f4e988d3ccf6bd94e6c37 P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 2 election: Requested vote from peers 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371), 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3:45113)
I20260709 13:04:15.996713 14883 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7c38457995f4e988d3ccf6bd94e6c37" candidate_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "39f9e4e2209748a69d253ea46414e575"
I20260709 13:04:15.996807 15032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7c38457995f4e988d3ccf6bd94e6c37" candidate_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5"
I20260709 13:04:15.997159 14883 raft_consensus.cc:3055] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:15.997347 15032 raft_consensus.cc:3060] T c7c38457995f4e988d3ccf6bd94e6c37 P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:15.997478 14883 raft_consensus.cc:740] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 39f9e4e2209748a69d253ea46414e575, State: Running, Role: LEADER
I20260709 13:04:15.998186 14883 consensus_queue.cc:260] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:15.999285 14883 raft_consensus.cc:3060] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:16.002143 15032 raft_consensus.cc:2468] T c7c38457995f4e988d3ccf6bd94e6c37 P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d3fa2b7ef942ae858e2c5727d99ce5 in term 2.
I20260709 13:04:16.003067 14921 leader_election.cc:304] T c7c38457995f4e988d3ccf6bd94e6c37 P 09d3fa2b7ef942ae858e2c5727d99ce5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d3fa2b7ef942ae858e2c5727d99ce5, 435d239f6f2441ac9d0e835010bfd2b5; no voters: 
I20260709 13:04:16.003762 15169 raft_consensus.cc:2804] T c7c38457995f4e988d3ccf6bd94e6c37 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:16.003856 14883 raft_consensus.cc:2468] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d3fa2b7ef942ae858e2c5727d99ce5 in term 2.
I20260709 13:04:16.004182 15169 raft_consensus.cc:697] T c7c38457995f4e988d3ccf6bd94e6c37 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 LEADER]: Becoming Leader. State: Replica: 09d3fa2b7ef942ae858e2c5727d99ce5, State: Running, Role: LEADER
I20260709 13:04:16.004964 15169 consensus_queue.cc:237] T c7c38457995f4e988d3ccf6bd94e6c37 P 09d3fa2b7ef942ae858e2c5727d99ce5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:16.011159 14779 catalog_manager.cc:5697] T c7c38457995f4e988d3ccf6bd94e6c37 P 09d3fa2b7ef942ae858e2c5727d99ce5 reported cstate change: term changed from 1 to 2, leader changed from 39f9e4e2209748a69d253ea46414e575 (127.14.103.1) to 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2). New cstate: current_term: 2 leader_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:16.209151 15032 raft_consensus.cc:1275] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [term 4 FOLLOWER]: Refusing update from remote peer 09d3fa2b7ef942ae858e2c5727d99ce5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:16.210649 15169 consensus_queue.cc:1048] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:16.218982 14883 raft_consensus.cc:1275] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [term 4 FOLLOWER]: Refusing update from remote peer 09d3fa2b7ef942ae858e2c5727d99ce5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:16.220747 15169 consensus_queue.cc:1048] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:16.255951 15032 raft_consensus.cc:1275] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5 [term 3 FOLLOWER]: Refusing update from remote peer 09d3fa2b7ef942ae858e2c5727d99ce5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:16.257006 15169 consensus_queue.cc:1048] T 32bc5a8acc774a1d9c7e99691a591567 P 09d3fa2b7ef942ae858e2c5727d99ce5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:16.266194 14882 raft_consensus.cc:1275] T 32bc5a8acc774a1d9c7e99691a591567 P 39f9e4e2209748a69d253ea46414e575 [term 3 FOLLOWER]: Refusing update from remote peer 09d3fa2b7ef942ae858e2c5727d99ce5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:16.268100 15206 consensus_queue.cc:1048] T 32bc5a8acc774a1d9c7e99691a591567 P 09d3fa2b7ef942ae858e2c5727d99ce5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:16.279486 14882 raft_consensus.cc:1275] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575 [term 2 FOLLOWER]: Refusing update from remote peer 09d3fa2b7ef942ae858e2c5727d99ce5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:16.280747 15206 consensus_queue.cc:1048] T f54152e1e2cf41a3970778369876d656 P 09d3fa2b7ef942ae858e2c5727d99ce5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:16.289062 15032 raft_consensus.cc:1275] T f54152e1e2cf41a3970778369876d656 P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 FOLLOWER]: Refusing update from remote peer 09d3fa2b7ef942ae858e2c5727d99ce5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:16.291018 15169 consensus_queue.cc:1048] T f54152e1e2cf41a3970778369876d656 P 09d3fa2b7ef942ae858e2c5727d99ce5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:16.339957 15032 raft_consensus.cc:1275] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5 [term 3 FOLLOWER]: Refusing update from remote peer 09d3fa2b7ef942ae858e2c5727d99ce5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:16.341073 15169 consensus_queue.cc:1048] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:16.348168 14882 raft_consensus.cc:3060] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 39f9e4e2209748a69d253ea46414e575 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:16.354516 14882 raft_consensus.cc:1275] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 39f9e4e2209748a69d253ea46414e575 [term 3 FOLLOWER]: Refusing update from remote peer 09d3fa2b7ef942ae858e2c5727d99ce5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:16.355713 15202 consensus_queue.cc:1048] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:16.400203 14882 raft_consensus.cc:1275] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575 [term 2 FOLLOWER]: Refusing update from remote peer 09d3fa2b7ef942ae858e2c5727d99ce5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:16.401456 15202 consensus_queue.cc:1048] T c7c38457995f4e988d3ccf6bd94e6c37 P 09d3fa2b7ef942ae858e2c5727d99ce5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:16.409653 15032 raft_consensus.cc:1275] T c7c38457995f4e988d3ccf6bd94e6c37 P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 FOLLOWER]: Refusing update from remote peer 09d3fa2b7ef942ae858e2c5727d99ce5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:16.410943 15202 consensus_queue.cc:1048] T c7c38457995f4e988d3ccf6bd94e6c37 P 09d3fa2b7ef942ae858e2c5727d99ce5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:04:18.520393 14979 debug-util.cc:398] Leaking SignalData structure 0x7b080012b060 after lost signal to thread 14751
W20260709 13:04:18.521421 14979 debug-util.cc:398] Leaking SignalData structure 0x7b080012b740 after lost signal to thread 14814
W20260709 13:04:18.522341 14979 debug-util.cc:398] Leaking SignalData structure 0x7b0800041c40 after lost signal to thread 14907
W20260709 13:04:18.523152 14979 debug-util.cc:398] Leaking SignalData structure 0x7b080018a620 after lost signal to thread 14982
W20260709 13:04:18.524019 14979 debug-util.cc:398] Leaking SignalData structure 0x7b0800192540 after lost signal to thread 15056
I20260709 13:04:19.203328 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:19.214615 14958 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "14bf28d1d32a48e7973484fb93521945"
dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
mode: GRACEFUL
new_leader_uuid: "435d239f6f2441ac9d0e835010bfd2b5"
 from {username='slave'} at 127.0.0.1:44830
I20260709 13:04:19.215389 14958 raft_consensus.cc:606] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 LEADER]: Received request to transfer leadership to TS 435d239f6f2441ac9d0e835010bfd2b5
I20260709 13:04:19.218321 14958 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7481805a7cfe4c2c8da2b33cf58c7ea5"
dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
mode: GRACEFUL
new_leader_uuid: "39f9e4e2209748a69d253ea46414e575"
 from {username='slave'} at 127.0.0.1:44830
I20260709 13:04:19.218773 14958 raft_consensus.cc:606] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 3 LEADER]: Received request to transfer leadership to TS 39f9e4e2209748a69d253ea46414e575
I20260709 13:04:19.221396 14958 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "810f60f24e33413985aa63f396a2b55a"
dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
mode: GRACEFUL
new_leader_uuid: "39f9e4e2209748a69d253ea46414e575"
 from {username='slave'} at 127.0.0.1:44830
I20260709 13:04:19.221931 14958 raft_consensus.cc:606] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 4 LEADER]: Received request to transfer leadership to TS 39f9e4e2209748a69d253ea46414e575
I20260709 13:04:19.224361 14958 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bb4dacc583b6453cb577a9e46c45c3af"
dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
mode: GRACEFUL
new_leader_uuid: "39f9e4e2209748a69d253ea46414e575"
 from {username='slave'} at 127.0.0.1:44830
I20260709 13:04:19.224803 14958 raft_consensus.cc:606] T bb4dacc583b6453cb577a9e46c45c3af P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 LEADER]: Received request to transfer leadership to TS 39f9e4e2209748a69d253ea46414e575
I20260709 13:04:19.226989 14958 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f54152e1e2cf41a3970778369876d656"
dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
mode: GRACEFUL
new_leader_uuid: "435d239f6f2441ac9d0e835010bfd2b5"
 from {username='slave'} at 127.0.0.1:44830
I20260709 13:04:19.227387 14958 raft_consensus.cc:606] T f54152e1e2cf41a3970778369876d656 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 LEADER]: Received request to transfer leadership to TS 435d239f6f2441ac9d0e835010bfd2b5
I20260709 13:04:19.228468 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:04:19.228909 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:19.261529 15221 raft_consensus.cc:993] T f54152e1e2cf41a3970778369876d656 P 09d3fa2b7ef942ae858e2c5727d99ce5: : Instructing follower 435d239f6f2441ac9d0e835010bfd2b5 to start an election
I20260709 13:04:19.262143 15220 raft_consensus.cc:1081] T f54152e1e2cf41a3970778369876d656 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 LEADER]: Signalling peer 435d239f6f2441ac9d0e835010bfd2b5 to start an election
I20260709 13:04:19.263648 15027 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f54152e1e2cf41a3970778369876d656"
dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5"
 from {username='slave'} at 127.0.0.1:58670
I20260709 13:04:19.264166 15027 raft_consensus.cc:493] T f54152e1e2cf41a3970778369876d656 P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:19.264571 15027 raft_consensus.cc:3060] T f54152e1e2cf41a3970778369876d656 P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:19.270418 15027 raft_consensus.cc:515] T f54152e1e2cf41a3970778369876d656 P 435d239f6f2441ac9d0e835010bfd2b5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:19.272903 15027 leader_election.cc:290] T f54152e1e2cf41a3970778369876d656 P 435d239f6f2441ac9d0e835010bfd2b5 [CANDIDATE]: Term 3 election: Requested vote from peers 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371), 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2:33423)
I20260709 13:04:19.273991 14958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54152e1e2cf41a3970778369876d656" candidate_uuid: "435d239f6f2441ac9d0e835010bfd2b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
I20260709 13:04:19.274295 14882 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54152e1e2cf41a3970778369876d656" candidate_uuid: "435d239f6f2441ac9d0e835010bfd2b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "39f9e4e2209748a69d253ea46414e575"
I20260709 13:04:19.274744 14958 raft_consensus.cc:3055] T f54152e1e2cf41a3970778369876d656 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:19.274914 14882 raft_consensus.cc:3060] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:19.275094 14958 raft_consensus.cc:740] T f54152e1e2cf41a3970778369876d656 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 09d3fa2b7ef942ae858e2c5727d99ce5, State: Running, Role: LEADER
I20260709 13:04:19.275970 14958 consensus_queue.cc:260] T f54152e1e2cf41a3970778369876d656 P 09d3fa2b7ef942ae858e2c5727d99ce5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:19.277279 14958 raft_consensus.cc:3060] T f54152e1e2cf41a3970778369876d656 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:19.281747 14958 raft_consensus.cc:2468] T f54152e1e2cf41a3970778369876d656 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 435d239f6f2441ac9d0e835010bfd2b5 in term 3.
I20260709 13:04:19.282857 14996 leader_election.cc:304] T f54152e1e2cf41a3970778369876d656 P 435d239f6f2441ac9d0e835010bfd2b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d3fa2b7ef942ae858e2c5727d99ce5, 435d239f6f2441ac9d0e835010bfd2b5; no voters: 
I20260709 13:04:19.282961 14882 raft_consensus.cc:2468] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 435d239f6f2441ac9d0e835010bfd2b5 in term 3.
I20260709 13:04:19.283799 15173 raft_consensus.cc:2804] T f54152e1e2cf41a3970778369876d656 P 435d239f6f2441ac9d0e835010bfd2b5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:19.284884 15173 raft_consensus.cc:697] T f54152e1e2cf41a3970778369876d656 P 435d239f6f2441ac9d0e835010bfd2b5 [term 3 LEADER]: Becoming Leader. State: Replica: 435d239f6f2441ac9d0e835010bfd2b5, State: Running, Role: LEADER
I20260709 13:04:19.285933 15173 consensus_queue.cc:237] T f54152e1e2cf41a3970778369876d656 P 435d239f6f2441ac9d0e835010bfd2b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:19.295620 14779 catalog_manager.cc:5697] T f54152e1e2cf41a3970778369876d656 P 435d239f6f2441ac9d0e835010bfd2b5 reported cstate change: term changed from 2 to 3, leader changed from 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2) to 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3). New cstate: current_term: 3 leader_uuid: "435d239f6f2441ac9d0e835010bfd2b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:19.482931 15221 raft_consensus.cc:993] T bb4dacc583b6453cb577a9e46c45c3af P 09d3fa2b7ef942ae858e2c5727d99ce5: : Instructing follower 39f9e4e2209748a69d253ea46414e575 to start an election
I20260709 13:04:19.483429 15220 raft_consensus.cc:1081] T bb4dacc583b6453cb577a9e46c45c3af P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 LEADER]: Signalling peer 39f9e4e2209748a69d253ea46414e575 to start an election
I20260709 13:04:19.485097 14882 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bb4dacc583b6453cb577a9e46c45c3af"
dest_uuid: "39f9e4e2209748a69d253ea46414e575"
 from {username='slave'} at 127.0.0.1:60528
I20260709 13:04:19.485685 14882 raft_consensus.cc:493] T bb4dacc583b6453cb577a9e46c45c3af P 39f9e4e2209748a69d253ea46414e575 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:19.486063 14882 raft_consensus.cc:3060] T bb4dacc583b6453cb577a9e46c45c3af P 39f9e4e2209748a69d253ea46414e575 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:19.487759 15221 raft_consensus.cc:993] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5: : Instructing follower 39f9e4e2209748a69d253ea46414e575 to start an election
I20260709 13:04:19.488209 15220 raft_consensus.cc:1081] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 3 LEADER]: Signalling peer 39f9e4e2209748a69d253ea46414e575 to start an election
I20260709 13:04:19.489609 14881 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7481805a7cfe4c2c8da2b33cf58c7ea5"
dest_uuid: "39f9e4e2209748a69d253ea46414e575"
 from {username='slave'} at 127.0.0.1:60528
I20260709 13:04:19.490103 14881 raft_consensus.cc:493] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 39f9e4e2209748a69d253ea46414e575 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:19.490444 14881 raft_consensus.cc:3060] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 39f9e4e2209748a69d253ea46414e575 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:19.493997 14882 raft_consensus.cc:515] T bb4dacc583b6453cb577a9e46c45c3af P 39f9e4e2209748a69d253ea46414e575 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:19.496644 14881 raft_consensus.cc:515] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 39f9e4e2209748a69d253ea46414e575 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:19.496874 14882 leader_election.cc:290] T bb4dacc583b6453cb577a9e46c45c3af P 39f9e4e2209748a69d253ea46414e575 [CANDIDATE]: Term 3 election: Requested vote from peers 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2:33423), 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3:45113)
I20260709 13:04:19.497830 15031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb4dacc583b6453cb577a9e46c45c3af" candidate_uuid: "39f9e4e2209748a69d253ea46414e575" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5"
I20260709 13:04:19.498446 15031 raft_consensus.cc:3060] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:19.499286 14881 leader_election.cc:290] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 39f9e4e2209748a69d253ea46414e575 [CANDIDATE]: Term 4 election: Requested vote from peers 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2:33423), 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3:45113)
I20260709 13:04:19.500106 15030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7481805a7cfe4c2c8da2b33cf58c7ea5" candidate_uuid: "39f9e4e2209748a69d253ea46414e575" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5"
I20260709 13:04:19.500661 15030 raft_consensus.cc:3060] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:19.501942 14958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb4dacc583b6453cb577a9e46c45c3af" candidate_uuid: "39f9e4e2209748a69d253ea46414e575" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
I20260709 13:04:19.502471 14958 raft_consensus.cc:3055] T bb4dacc583b6453cb577a9e46c45c3af P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:19.502769 14958 raft_consensus.cc:740] T bb4dacc583b6453cb577a9e46c45c3af P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 09d3fa2b7ef942ae858e2c5727d99ce5, State: Running, Role: LEADER
I20260709 13:04:19.503487 14958 consensus_queue.cc:260] T bb4dacc583b6453cb577a9e46c45c3af P 09d3fa2b7ef942ae858e2c5727d99ce5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:19.504608 14958 raft_consensus.cc:3060] T bb4dacc583b6453cb577a9e46c45c3af P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:19.505357 15031 raft_consensus.cc:2468] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39f9e4e2209748a69d253ea46414e575 in term 3.
I20260709 13:04:19.506999 14846 leader_election.cc:304] T bb4dacc583b6453cb577a9e46c45c3af P 39f9e4e2209748a69d253ea46414e575 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39f9e4e2209748a69d253ea46414e575, 435d239f6f2441ac9d0e835010bfd2b5; no voters: 
I20260709 13:04:19.508544 15030 raft_consensus.cc:2468] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39f9e4e2209748a69d253ea46414e575 in term 4.
I20260709 13:04:19.512673 14846 leader_election.cc:304] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 39f9e4e2209748a69d253ea46414e575 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39f9e4e2209748a69d253ea46414e575, 435d239f6f2441ac9d0e835010bfd2b5; no voters: 
I20260709 13:04:19.512771 15228 raft_consensus.cc:2804] T bb4dacc583b6453cb577a9e46c45c3af P 39f9e4e2209748a69d253ea46414e575 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:19.514274 14957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7481805a7cfe4c2c8da2b33cf58c7ea5" candidate_uuid: "39f9e4e2209748a69d253ea46414e575" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
I20260709 13:04:19.514935 14957 raft_consensus.cc:3055] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:19.515237 14957 raft_consensus.cc:740] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 09d3fa2b7ef942ae858e2c5727d99ce5, State: Running, Role: LEADER
I20260709 13:04:19.516139 14957 consensus_queue.cc:260] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:19.520795 14957 raft_consensus.cc:3060] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:19.524657 15229 raft_consensus.cc:2804] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 39f9e4e2209748a69d253ea46414e575 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:19.525202 15229 raft_consensus.cc:697] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 39f9e4e2209748a69d253ea46414e575 [term 4 LEADER]: Becoming Leader. State: Replica: 39f9e4e2209748a69d253ea46414e575, State: Running, Role: LEADER
I20260709 13:04:19.527405 15229 consensus_queue.cc:237] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 39f9e4e2209748a69d253ea46414e575 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:19.529894 14958 raft_consensus.cc:2468] T bb4dacc583b6453cb577a9e46c45c3af P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39f9e4e2209748a69d253ea46414e575 in term 3.
I20260709 13:04:19.525202 15228 raft_consensus.cc:697] T bb4dacc583b6453cb577a9e46c45c3af P 39f9e4e2209748a69d253ea46414e575 [term 3 LEADER]: Becoming Leader. State: Replica: 39f9e4e2209748a69d253ea46414e575, State: Running, Role: LEADER
I20260709 13:04:19.532351 15228 consensus_queue.cc:237] T bb4dacc583b6453cb577a9e46c45c3af P 39f9e4e2209748a69d253ea46414e575 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:19.536209 14957 raft_consensus.cc:2468] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39f9e4e2209748a69d253ea46414e575 in term 4.
I20260709 13:04:19.539582 15231 raft_consensus.cc:993] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5: : Instructing follower 435d239f6f2441ac9d0e835010bfd2b5 to start an election
I20260709 13:04:19.540141 15221 raft_consensus.cc:1081] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 LEADER]: Signalling peer 435d239f6f2441ac9d0e835010bfd2b5 to start an election
I20260709 13:04:19.542277 15031 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "14bf28d1d32a48e7973484fb93521945"
dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5"
 from {username='slave'} at 127.0.0.1:58670
I20260709 13:04:19.542851 15031 raft_consensus.cc:493] T 14bf28d1d32a48e7973484fb93521945 P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:19.543211 15031 raft_consensus.cc:3060] T 14bf28d1d32a48e7973484fb93521945 P 435d239f6f2441ac9d0e835010bfd2b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:19.551479 15031 raft_consensus.cc:515] T 14bf28d1d32a48e7973484fb93521945 P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:19.553529 15031 leader_election.cc:290] T 14bf28d1d32a48e7973484fb93521945 P 435d239f6f2441ac9d0e835010bfd2b5 [CANDIDATE]: Term 2 election: Requested vote from peers 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371), 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2:33423)
I20260709 13:04:19.554569 14881 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bf28d1d32a48e7973484fb93521945" candidate_uuid: "435d239f6f2441ac9d0e835010bfd2b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "39f9e4e2209748a69d253ea46414e575"
I20260709 13:04:19.555222 14881 raft_consensus.cc:3060] T 14bf28d1d32a48e7973484fb93521945 P 39f9e4e2209748a69d253ea46414e575 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:19.555191 14957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14bf28d1d32a48e7973484fb93521945" candidate_uuid: "435d239f6f2441ac9d0e835010bfd2b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
I20260709 13:04:19.554919 14779 catalog_manager.cc:5697] T bb4dacc583b6453cb577a9e46c45c3af P 39f9e4e2209748a69d253ea46414e575 reported cstate change: term changed from 2 to 3, leader changed from 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2) to 39f9e4e2209748a69d253ea46414e575 (127.14.103.1). New cstate: current_term: 3 leader_uuid: "39f9e4e2209748a69d253ea46414e575" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:19.555888 14957 raft_consensus.cc:3055] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:19.556202 14957 raft_consensus.cc:740] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 09d3fa2b7ef942ae858e2c5727d99ce5, State: Running, Role: LEADER
I20260709 13:04:19.556468 14779 catalog_manager.cc:5697] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 39f9e4e2209748a69d253ea46414e575 reported cstate change: term changed from 3 to 4, leader changed from 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2) to 39f9e4e2209748a69d253ea46414e575 (127.14.103.1). New cstate: current_term: 4 leader_uuid: "39f9e4e2209748a69d253ea46414e575" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:19.556968 14957 consensus_queue.cc:260] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:19.558010 14957 raft_consensus.cc:3060] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:19.564539 14881 raft_consensus.cc:2468] T 14bf28d1d32a48e7973484fb93521945 P 39f9e4e2209748a69d253ea46414e575 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 435d239f6f2441ac9d0e835010bfd2b5 in term 2.
I20260709 13:04:19.566068 14996 leader_election.cc:304] T 14bf28d1d32a48e7973484fb93521945 P 435d239f6f2441ac9d0e835010bfd2b5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39f9e4e2209748a69d253ea46414e575, 435d239f6f2441ac9d0e835010bfd2b5; no voters: 
I20260709 13:04:19.566176 14957 raft_consensus.cc:2468] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 435d239f6f2441ac9d0e835010bfd2b5 in term 2.
I20260709 13:04:19.567055 15173 raft_consensus.cc:2804] T 14bf28d1d32a48e7973484fb93521945 P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:19.567544 15173 raft_consensus.cc:697] T 14bf28d1d32a48e7973484fb93521945 P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 LEADER]: Becoming Leader. State: Replica: 435d239f6f2441ac9d0e835010bfd2b5, State: Running, Role: LEADER
I20260709 13:04:19.568872 15173 consensus_queue.cc:237] T 14bf28d1d32a48e7973484fb93521945 P 435d239f6f2441ac9d0e835010bfd2b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } }
I20260709 13:04:19.577134 14778 catalog_manager.cc:5697] T 14bf28d1d32a48e7973484fb93521945 P 435d239f6f2441ac9d0e835010bfd2b5 reported cstate change: term changed from 1 to 2, leader changed from 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2) to 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3). New cstate: current_term: 2 leader_uuid: "435d239f6f2441ac9d0e835010bfd2b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:19.741046 15231 raft_consensus.cc:993] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5: : Instructing follower 39f9e4e2209748a69d253ea46414e575 to start an election
I20260709 13:04:19.741591 15221 raft_consensus.cc:1081] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 4 LEADER]: Signalling peer 39f9e4e2209748a69d253ea46414e575 to start an election
I20260709 13:04:19.743151 14881 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "810f60f24e33413985aa63f396a2b55a"
dest_uuid: "39f9e4e2209748a69d253ea46414e575"
 from {username='slave'} at 127.0.0.1:60528
I20260709 13:04:19.743638 14881 raft_consensus.cc:493] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:19.743976 14881 raft_consensus.cc:3060] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:19.748498 14881 raft_consensus.cc:515] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } }
I20260709 13:04:19.750211 14881 leader_election.cc:290] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [CANDIDATE]: Term 5 election: Requested vote from peers 435d239f6f2441ac9d0e835010bfd2b5 (127.14.103.3:45113), 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2:33423)
I20260709 13:04:19.751065 15031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "810f60f24e33413985aa63f396a2b55a" candidate_uuid: "39f9e4e2209748a69d253ea46414e575" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "435d239f6f2441ac9d0e835010bfd2b5"
I20260709 13:04:19.751331 14957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "810f60f24e33413985aa63f396a2b55a" candidate_uuid: "39f9e4e2209748a69d253ea46414e575" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5"
I20260709 13:04:19.751740 15031 raft_consensus.cc:3060] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:19.751899 14957 raft_consensus.cc:3055] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:19.752313 14957 raft_consensus.cc:740] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 09d3fa2b7ef942ae858e2c5727d99ce5, State: Running, Role: LEADER
I20260709 13:04:19.753006 14957 consensus_queue.cc:260] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } }
I20260709 13:04:19.753908 14957 raft_consensus.cc:3060] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:19.758762 14957 raft_consensus.cc:2468] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39f9e4e2209748a69d253ea46414e575 in term 5.
I20260709 13:04:19.759398 15031 raft_consensus.cc:2468] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39f9e4e2209748a69d253ea46414e575 in term 5.
I20260709 13:04:19.760291 14847 leader_election.cc:304] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d3fa2b7ef942ae858e2c5727d99ce5, 39f9e4e2209748a69d253ea46414e575; no voters: 
I20260709 13:04:19.761170 15229 raft_consensus.cc:2804] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:19.761718 15229 raft_consensus.cc:697] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [term 5 LEADER]: Becoming Leader. State: Replica: 39f9e4e2209748a69d253ea46414e575, State: Running, Role: LEADER
I20260709 13:04:19.762707 15229 consensus_queue.cc:237] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } } peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } }
I20260709 13:04:19.766151 14957 raft_consensus.cc:1275] T f54152e1e2cf41a3970778369876d656 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 3 FOLLOWER]: Refusing update from remote peer 435d239f6f2441ac9d0e835010bfd2b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:19.768879 15173 consensus_queue.cc:1048] T f54152e1e2cf41a3970778369876d656 P 435d239f6f2441ac9d0e835010bfd2b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:19.771286 14778 catalog_manager.cc:5697] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 reported cstate change: term changed from 4 to 5, leader changed from 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2) to 39f9e4e2209748a69d253ea46414e575 (127.14.103.1). New cstate: current_term: 5 leader_uuid: "39f9e4e2209748a69d253ea46414e575" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:19.790231 14881 raft_consensus.cc:1275] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575 [term 3 FOLLOWER]: Refusing update from remote peer 435d239f6f2441ac9d0e835010bfd2b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:19.792080 15222 consensus_queue.cc:1048] T f54152e1e2cf41a3970778369876d656 P 435d239f6f2441ac9d0e835010bfd2b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:19.977465 14957 raft_consensus.cc:1275] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 FOLLOWER]: Refusing update from remote peer 435d239f6f2441ac9d0e835010bfd2b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:19.978766 15239 consensus_queue.cc:1048] T 14bf28d1d32a48e7973484fb93521945 P 435d239f6f2441ac9d0e835010bfd2b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:19.993736 14881 raft_consensus.cc:1275] T 14bf28d1d32a48e7973484fb93521945 P 39f9e4e2209748a69d253ea46414e575 [term 2 FOLLOWER]: Refusing update from remote peer 435d239f6f2441ac9d0e835010bfd2b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:19.995205 15173 consensus_queue.cc:1048] T 14bf28d1d32a48e7973484fb93521945 P 435d239f6f2441ac9d0e835010bfd2b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39f9e4e2209748a69d253ea46414e575" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:20.022466 14957 raft_consensus.cc:1275] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 4 FOLLOWER]: Refusing update from remote peer 39f9e4e2209748a69d253ea46414e575: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:20.023551 15229 consensus_queue.cc:1048] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 39f9e4e2209748a69d253ea46414e575 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:20.026971 15031 raft_consensus.cc:1275] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5 [term 3 FOLLOWER]: Refusing update from remote peer 39f9e4e2209748a69d253ea46414e575: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:20.028983 15228 consensus_queue.cc:1048] T bb4dacc583b6453cb577a9e46c45c3af P 39f9e4e2209748a69d253ea46414e575 [LEADER]: Connected to new peer: Peer: permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:20.034960 15030 raft_consensus.cc:1275] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5 [term 4 FOLLOWER]: Refusing update from remote peer 39f9e4e2209748a69d253ea46414e575: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:20.036903 15228 consensus_queue.cc:1048] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 39f9e4e2209748a69d253ea46414e575 [LEADER]: Connected to new peer: Peer: permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:20.055724 14957 raft_consensus.cc:1275] T bb4dacc583b6453cb577a9e46c45c3af P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 3 FOLLOWER]: Refusing update from remote peer 39f9e4e2209748a69d253ea46414e575: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:20.057260 15229 consensus_queue.cc:1048] T bb4dacc583b6453cb577a9e46c45c3af P 39f9e4e2209748a69d253ea46414e575 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:20.174852 14957 raft_consensus.cc:1275] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 5 FOLLOWER]: Refusing update from remote peer 39f9e4e2209748a69d253ea46414e575: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:04:20.176211 15246 consensus_queue.cc:1048] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d3fa2b7ef942ae858e2c5727d99ce5" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 33423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:20.185765 15030 raft_consensus.cc:1275] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [term 5 FOLLOWER]: Refusing update from remote peer 39f9e4e2209748a69d253ea46414e575: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:04:20.187402 15229 consensus_queue.cc:1048] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [LEADER]: Connected to new peer: Peer: permanent_uuid: "435d239f6f2441ac9d0e835010bfd2b5" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 45113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:20.229751 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:20.236038 14748 tablet_server.cc:179] TabletServer@127.14.103.1:0 shutting down...
W20260709 13:04:20.255520 14922 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:20.270838 14922 consensus_peers.cc:597] T 32bc5a8acc774a1d9c7e99691a591567 P 09d3fa2b7ef942ae858e2c5727d99ce5 -> Peer 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371): Couldn't send request to peer 39f9e4e2209748a69d253ea46414e575. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:20.280644 14922 consensus_peers.cc:597] T be17f0536ffd4425a0db52b530c5f3ef P 09d3fa2b7ef942ae858e2c5727d99ce5 -> Peer 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371): Couldn't send request to peer 39f9e4e2209748a69d253ea46414e575. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:20.286499 14748 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:20.287472 14748 tablet_replica.cc:333] T be17f0536ffd4425a0db52b530c5f3ef P 39f9e4e2209748a69d253ea46414e575: stopping tablet replica
I20260709 13:04:20.288491 14748 raft_consensus.cc:2243] T be17f0536ffd4425a0db52b530c5f3ef P 39f9e4e2209748a69d253ea46414e575 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:20.289150 14748 raft_consensus.cc:2272] T be17f0536ffd4425a0db52b530c5f3ef P 39f9e4e2209748a69d253ea46414e575 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:20.290649 14922 consensus_peers.cc:597] T c7c38457995f4e988d3ccf6bd94e6c37 P 09d3fa2b7ef942ae858e2c5727d99ce5 -> Peer 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371): Couldn't send request to peer 39f9e4e2209748a69d253ea46414e575. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:20.291932 14748 tablet_replica.cc:333] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575: stopping tablet replica
I20260709 13:04:20.292537 14748 raft_consensus.cc:2243] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:04:20.293277 14748 raft_consensus.cc:2272] T 810f60f24e33413985aa63f396a2b55a P 39f9e4e2209748a69d253ea46414e575 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:20.295312 14748 tablet_replica.cc:333] T 32bc5a8acc774a1d9c7e99691a591567 P 39f9e4e2209748a69d253ea46414e575: stopping tablet replica
I20260709 13:04:20.295892 14748 raft_consensus.cc:2243] T 32bc5a8acc774a1d9c7e99691a591567 P 39f9e4e2209748a69d253ea46414e575 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:20.296350 14748 raft_consensus.cc:2272] T 32bc5a8acc774a1d9c7e99691a591567 P 39f9e4e2209748a69d253ea46414e575 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:20.297978 14748 tablet_replica.cc:333] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575: stopping tablet replica
I20260709 13:04:20.298465 14748 raft_consensus.cc:2243] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:20.298894 14748 raft_consensus.cc:2272] T f54152e1e2cf41a3970778369876d656 P 39f9e4e2209748a69d253ea46414e575 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:20.301026 14748 tablet_replica.cc:333] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 39f9e4e2209748a69d253ea46414e575: stopping tablet replica
I20260709 13:04:20.301499 14748 raft_consensus.cc:2243] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 39f9e4e2209748a69d253ea46414e575 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:04:20.302172 14748 raft_consensus.cc:2272] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 39f9e4e2209748a69d253ea46414e575 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:20.304075 14748 tablet_replica.cc:333] T 14bf28d1d32a48e7973484fb93521945 P 39f9e4e2209748a69d253ea46414e575: stopping tablet replica
I20260709 13:04:20.304565 14748 raft_consensus.cc:2243] T 14bf28d1d32a48e7973484fb93521945 P 39f9e4e2209748a69d253ea46414e575 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:20.304942 14748 raft_consensus.cc:2272] T 14bf28d1d32a48e7973484fb93521945 P 39f9e4e2209748a69d253ea46414e575 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:20.306856 14748 tablet_replica.cc:333] T bb4dacc583b6453cb577a9e46c45c3af P 39f9e4e2209748a69d253ea46414e575: stopping tablet replica
I20260709 13:04:20.307322 14748 raft_consensus.cc:2243] T bb4dacc583b6453cb577a9e46c45c3af P 39f9e4e2209748a69d253ea46414e575 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:04:20.308032 14748 raft_consensus.cc:2272] T bb4dacc583b6453cb577a9e46c45c3af P 39f9e4e2209748a69d253ea46414e575 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:20.309942 14748 tablet_replica.cc:333] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575: stopping tablet replica
I20260709 13:04:20.310407 14748 raft_consensus.cc:2243] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:20.310803 14748 raft_consensus.cc:2272] T c7c38457995f4e988d3ccf6bd94e6c37 P 39f9e4e2209748a69d253ea46414e575 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:20.312592 14748 tablet_replica.cc:333] T 90b851c6655d47c09fca18bbfc1a4d95 P 39f9e4e2209748a69d253ea46414e575: stopping tablet replica
I20260709 13:04:20.313030 14748 raft_consensus.cc:2243] T 90b851c6655d47c09fca18bbfc1a4d95 P 39f9e4e2209748a69d253ea46414e575 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:20.313405 14748 raft_consensus.cc:2272] T 90b851c6655d47c09fca18bbfc1a4d95 P 39f9e4e2209748a69d253ea46414e575 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:20.422030 14996 consensus_peers.cc:597] T f54152e1e2cf41a3970778369876d656 P 435d239f6f2441ac9d0e835010bfd2b5 -> Peer 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371): Couldn't send request to peer 39f9e4e2209748a69d253ea46414e575. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:33371: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:20.427182 14748 tablet_server.cc:196] TabletServer@127.14.103.1:0 shutdown complete.
I20260709 13:04:20.449115 14748 tablet_server.cc:179] TabletServer@127.14.103.2:0 shutting down...
W20260709 13:04:20.467622 14996 consensus_peers.cc:597] T 14bf28d1d32a48e7973484fb93521945 P 435d239f6f2441ac9d0e835010bfd2b5 -> Peer 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371): Couldn't send request to peer 39f9e4e2209748a69d253ea46414e575. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:33371: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:20.488476 14748 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:20.489264 14748 tablet_replica.cc:333] T bb4dacc583b6453cb577a9e46c45c3af P 09d3fa2b7ef942ae858e2c5727d99ce5: stopping tablet replica
I20260709 13:04:20.490065 14748 raft_consensus.cc:2243] T bb4dacc583b6453cb577a9e46c45c3af P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:20.490614 14748 raft_consensus.cc:2272] T bb4dacc583b6453cb577a9e46c45c3af P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:20.493564 14748 tablet_replica.cc:333] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5: stopping tablet replica
I20260709 13:04:20.494335 14748 raft_consensus.cc:2243] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:20.494879 14748 raft_consensus.cc:2272] T 14bf28d1d32a48e7973484fb93521945 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:20.497483 14748 tablet_replica.cc:333] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5: stopping tablet replica
I20260709 13:04:20.498208 14748 raft_consensus.cc:2243] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:20.498742 14748 raft_consensus.cc:2272] T 810f60f24e33413985aa63f396a2b55a P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:20.501295 14748 tablet_replica.cc:333] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5: stopping tablet replica
I20260709 13:04:20.501992 14748 raft_consensus.cc:2243] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:20.502499 14748 raft_consensus.cc:2272] T 90b851c6655d47c09fca18bbfc1a4d95 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:20.504849 14748 tablet_replica.cc:333] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5: stopping tablet replica
I20260709 13:04:20.505533 14748 raft_consensus.cc:2243] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:20.506057 14748 raft_consensus.cc:2272] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:20.508769 14748 tablet_replica.cc:333] T f54152e1e2cf41a3970778369876d656 P 09d3fa2b7ef942ae858e2c5727d99ce5: stopping tablet replica
I20260709 13:04:20.510164 14748 raft_consensus.cc:2243] T f54152e1e2cf41a3970778369876d656 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:20.510777 14748 raft_consensus.cc:2272] T f54152e1e2cf41a3970778369876d656 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:20.512308 14996 consensus_peers.cc:597] T 90b851c6655d47c09fca18bbfc1a4d95 P 435d239f6f2441ac9d0e835010bfd2b5 -> Peer 39f9e4e2209748a69d253ea46414e575 (127.14.103.1:33371): Couldn't send request to peer 39f9e4e2209748a69d253ea46414e575. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:33371: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:20.513862 14748 tablet_replica.cc:333] T be17f0536ffd4425a0db52b530c5f3ef P 09d3fa2b7ef942ae858e2c5727d99ce5: stopping tablet replica
I20260709 13:04:20.514595 14748 raft_consensus.cc:2243] T be17f0536ffd4425a0db52b530c5f3ef P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:20.515564 14748 raft_consensus.cc:2272] T be17f0536ffd4425a0db52b530c5f3ef P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:20.517844 14748 tablet_replica.cc:333] T 32bc5a8acc774a1d9c7e99691a591567 P 09d3fa2b7ef942ae858e2c5727d99ce5: stopping tablet replica
I20260709 13:04:20.518496 14748 raft_consensus.cc:2243] T 32bc5a8acc774a1d9c7e99691a591567 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:04:20.519542 14748 raft_consensus.cc:2272] T 32bc5a8acc774a1d9c7e99691a591567 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:20.521912 14748 tablet_replica.cc:333] T c7c38457995f4e988d3ccf6bd94e6c37 P 09d3fa2b7ef942ae858e2c5727d99ce5: stopping tablet replica
I20260709 13:04:20.522547 14748 raft_consensus.cc:2243] T c7c38457995f4e988d3ccf6bd94e6c37 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:20.523602 14748 raft_consensus.cc:2272] T c7c38457995f4e988d3ccf6bd94e6c37 P 09d3fa2b7ef942ae858e2c5727d99ce5 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:20.647642 14996 consensus_peers.cc:597] T f54152e1e2cf41a3970778369876d656 P 435d239f6f2441ac9d0e835010bfd2b5 -> Peer 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2:33423): Couldn't send request to peer 09d3fa2b7ef942ae858e2c5727d99ce5. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:33423: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:20.851363 14996 consensus_peers.cc:597] T 90b851c6655d47c09fca18bbfc1a4d95 P 435d239f6f2441ac9d0e835010bfd2b5 -> Peer 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2:33423): Couldn't send request to peer 09d3fa2b7ef942ae858e2c5727d99ce5. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:33423: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:20.918205 14996 consensus_peers.cc:597] T 14bf28d1d32a48e7973484fb93521945 P 435d239f6f2441ac9d0e835010bfd2b5 -> Peer 09d3fa2b7ef942ae858e2c5727d99ce5 (127.14.103.2:33423): Couldn't send request to peer 09d3fa2b7ef942ae858e2c5727d99ce5. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:33423: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:21.013175 14748 tablet_server.cc:196] TabletServer@127.14.103.2:0 shutdown complete.
I20260709 13:04:21.029910 14748 tablet_server.cc:179] TabletServer@127.14.103.3:0 shutting down...
I20260709 13:04:21.052692 14748 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:21.053359 14748 tablet_replica.cc:333] T 90b851c6655d47c09fca18bbfc1a4d95 P 435d239f6f2441ac9d0e835010bfd2b5: stopping tablet replica
I20260709 13:04:21.053942 14748 raft_consensus.cc:2243] T 90b851c6655d47c09fca18bbfc1a4d95 P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:21.054641 14748 raft_consensus.cc:2272] T 90b851c6655d47c09fca18bbfc1a4d95 P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:21.056353 14748 tablet_replica.cc:333] T 14bf28d1d32a48e7973484fb93521945 P 435d239f6f2441ac9d0e835010bfd2b5: stopping tablet replica
I20260709 13:04:21.056859 14748 raft_consensus.cc:2243] T 14bf28d1d32a48e7973484fb93521945 P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:21.057538 14748 raft_consensus.cc:2272] T 14bf28d1d32a48e7973484fb93521945 P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:21.059000 14748 tablet_replica.cc:333] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5: stopping tablet replica
I20260709 13:04:21.059463 14748 raft_consensus.cc:2243] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:21.059912 14748 raft_consensus.cc:2272] T bb4dacc583b6453cb577a9e46c45c3af P 435d239f6f2441ac9d0e835010bfd2b5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:21.061789 14748 tablet_replica.cc:333] T f54152e1e2cf41a3970778369876d656 P 435d239f6f2441ac9d0e835010bfd2b5: stopping tablet replica
I20260709 13:04:21.062275 14748 raft_consensus.cc:2243] T f54152e1e2cf41a3970778369876d656 P 435d239f6f2441ac9d0e835010bfd2b5 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:04:21.062924 14748 raft_consensus.cc:2272] T f54152e1e2cf41a3970778369876d656 P 435d239f6f2441ac9d0e835010bfd2b5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:21.064414 14748 tablet_replica.cc:333] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5: stopping tablet replica
I20260709 13:04:21.064889 14748 raft_consensus.cc:2243] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:21.065289 14748 raft_consensus.cc:2272] T 7481805a7cfe4c2c8da2b33cf58c7ea5 P 435d239f6f2441ac9d0e835010bfd2b5 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:21.066841 14748 tablet_replica.cc:333] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5: stopping tablet replica
I20260709 13:04:21.067317 14748 raft_consensus.cc:2243] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:21.067759 14748 raft_consensus.cc:2272] T be17f0536ffd4425a0db52b530c5f3ef P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:21.069233 14748 tablet_replica.cc:333] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5: stopping tablet replica
I20260709 13:04:21.069677 14748 raft_consensus.cc:2243] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:21.070092 14748 raft_consensus.cc:2272] T 32bc5a8acc774a1d9c7e99691a591567 P 435d239f6f2441ac9d0e835010bfd2b5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:21.071790 14748 tablet_replica.cc:333] T c7c38457995f4e988d3ccf6bd94e6c37 P 435d239f6f2441ac9d0e835010bfd2b5: stopping tablet replica
I20260709 13:04:21.072222 14748 raft_consensus.cc:2243] T c7c38457995f4e988d3ccf6bd94e6c37 P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:21.072606 14748 raft_consensus.cc:2272] T c7c38457995f4e988d3ccf6bd94e6c37 P 435d239f6f2441ac9d0e835010bfd2b5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:21.074157 14748 tablet_replica.cc:333] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5: stopping tablet replica
I20260709 13:04:21.074596 14748 raft_consensus.cc:2243] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:21.074997 14748 raft_consensus.cc:2272] T 810f60f24e33413985aa63f396a2b55a P 435d239f6f2441ac9d0e835010bfd2b5 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:21.101588 14748 tablet_server.cc:196] TabletServer@127.14.103.3:0 shutdown complete.
I20260709 13:04:21.117771 14748 master.cc:562] Master@127.14.103.62:38771 shutting down...
I20260709 13:04:21.328537 14748 raft_consensus.cc:2243] T 00000000000000000000000000000000 P d9948df4059e4dc0b63aebc1c6baa8e0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:21.329185 14748 raft_consensus.cc:2272] T 00000000000000000000000000000000 P d9948df4059e4dc0b63aebc1c6baa8e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:21.329577 14748 tablet_replica.cc:333] T 00000000000000000000000000000000 P d9948df4059e4dc0b63aebc1c6baa8e0: stopping tablet replica
I20260709 13:04:21.347994 14748 master.cc:584] Master@127.14.103.62:38771 shutdown complete.
09d3fa2b7ef942ae858e2c5727d99ce5  4
39f9e4e2209748a69d253ea46414e575  4
435d239f6f2441ac9d0e835010bfd2b5  1
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (13690 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 13:04:21.387926 14748 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.103.62:38869
I20260709 13:04:21.389057 14748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:21.394695 15256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:21.394894 15255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:21.395737 15258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:21.397146 14748 server_base.cc:1061] running on GCE node
I20260709 13:04:21.398012 14748 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:21.398187 14748 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:21.398309 14748 hybrid_clock.cc:648] HybridClock initialized: now 1783602261398298 us; error 0 us; skew 500 ppm
I20260709 13:04:21.398777 14748 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:21.401051 14748 webserver.cc:533] Webserver started at http://127.14.103.62:42461/ using document root <none> and password file <none>
I20260709 13:04:21.401486 14748 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:21.401638 14748 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:21.401860 14748 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:21.402890 14748 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602247531521-14748-0/minicluster-data/master-0-root/instance:
uuid: "fc08470892c143c685cfee2af1e4a7a5"
format_stamp: "Formatted at 2026-07-09 13:04:21 on dist-test-slave-fpp7"
I20260709 13:04:21.407032 14748 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.002s
I20260709 13:04:21.410104 15263 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:21.410887 14748 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:04:21.411199 14748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602247531521-14748-0/minicluster-data/master-0-root
uuid: "fc08470892c143c685cfee2af1e4a7a5"
format_stamp: "Formatted at 2026-07-09 13:04:21 on dist-test-slave-fpp7"
I20260709 13:04:21.411495 14748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602247531521-14748-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602247531521-14748-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602247531521-14748-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:21.430850 14748 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:21.432173 14748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:21.468705 14748 rpc_server.cc:307] RPC server started. Bound to: 127.14.103.62:38869
I20260709 13:04:21.468792 15314 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.103.62:38869 every 8 connection(s)
I20260709 13:04:21.472514 15315 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:21.481673 15315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc08470892c143c685cfee2af1e4a7a5: Bootstrap starting.
I20260709 13:04:21.486022 15315 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc08470892c143c685cfee2af1e4a7a5: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:21.489953 15315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc08470892c143c685cfee2af1e4a7a5: No bootstrap required, opened a new log
I20260709 13:04:21.492004 15315 raft_consensus.cc:359] T 00000000000000000000000000000000 P fc08470892c143c685cfee2af1e4a7a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc08470892c143c685cfee2af1e4a7a5" member_type: VOTER }
I20260709 13:04:21.492354 15315 raft_consensus.cc:385] T 00000000000000000000000000000000 P fc08470892c143c685cfee2af1e4a7a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:21.492527 15315 raft_consensus.cc:740] T 00000000000000000000000000000000 P fc08470892c143c685cfee2af1e4a7a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc08470892c143c685cfee2af1e4a7a5, State: Initialized, Role: FOLLOWER
I20260709 13:04:21.492998 15315 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc08470892c143c685cfee2af1e4a7a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc08470892c143c685cfee2af1e4a7a5" member_type: VOTER }
I20260709 13:04:21.493383 15315 raft_consensus.cc:399] T 00000000000000000000000000000000 P fc08470892c143c685cfee2af1e4a7a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:04:21.493566 15315 raft_consensus.cc:493] T 00000000000000000000000000000000 P fc08470892c143c685cfee2af1e4a7a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:04:21.493775 15315 raft_consensus.cc:3060] T 00000000000000000000000000000000 P fc08470892c143c685cfee2af1e4a7a5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:21.497570 15315 raft_consensus.cc:515] T 00000000000000000000000000000000 P fc08470892c143c685cfee2af1e4a7a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc08470892c143c685cfee2af1e4a7a5" member_type: VOTER }
I20260709 13:04:21.498039 15315 leader_election.cc:304] T 00000000000000000000000000000000 P fc08470892c143c685cfee2af1e4a7a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc08470892c143c685cfee2af1e4a7a5; no voters: 
I20260709 13:04:21.499074 15315 leader_election.cc:290] T 00000000000000000000000000000000 P fc08470892c143c685cfee2af1e4a7a5 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:04:21.499362 15318 raft_consensus.cc:2804] T 00000000000000000000000000000000 P fc08470892c143c685cfee2af1e4a7a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:21.500761 15318 raft_consensus.cc:697] T 00000000000000000000000000000000 P fc08470892c143c685cfee2af1e4a7a5 [term 1 LEADER]: Becoming Leader. State: Replica: fc08470892c143c685cfee2af1e4a7a5, State: Running, Role: LEADER
I20260709 13:04:21.501459 15318 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc08470892c143c685cfee2af1e4a7a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc08470892c143c685cfee2af1e4a7a5" member_type: VOTER }
I20260709 13:04:21.502040 15315 sys_catalog.cc:565] T 00000000000000000000000000000000 P fc08470892c143c685cfee2af1e4a7a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:04:21.506492 15320 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc08470892c143c685cfee2af1e4a7a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc08470892c143c685cfee2af1e4a7a5. Latest consensus state: current_term: 1 leader_uuid: "fc08470892c143c685cfee2af1e4a7a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc08470892c143c685cfee2af1e4a7a5" member_type: VOTER } }
I20260709 13:04:21.507170 15320 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc08470892c143c685cfee2af1e4a7a5 [sys.catalog]: This master's current role is: LEADER
I20260709 13:04:21.509415 15319 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc08470892c143c685cfee2af1e4a7a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc08470892c143c685cfee2af1e4a7a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc08470892c143c685cfee2af1e4a7a5" member_type: VOTER } }
I20260709 13:04:21.509920 15319 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc08470892c143c685cfee2af1e4a7a5 [sys.catalog]: This master's current role is: LEADER
I20260709 13:04:21.510921 15327 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:04:21.516187 15327 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:04:21.517143 14748 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:04:21.524405 15327 catalog_manager.cc:1383] Generated new cluster ID: 845187497b694868bdf59a9af1d3866c
I20260709 13:04:21.524701 15327 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:04:21.537994 15327 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:04:21.539173 15327 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:04:21.555522 15327 catalog_manager.cc:6070] T 00000000000000000000000000000000 P fc08470892c143c685cfee2af1e4a7a5: Generated new TSK 0
I20260709 13:04:21.556121 15327 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:04:21.583467 14748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:21.589279 15336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:21.590556 15337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:21.592103 15339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:21.592453 14748 server_base.cc:1061] running on GCE node
I20260709 13:04:21.593714 14748 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:21.594019 14748 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:21.594231 14748 hybrid_clock.cc:648] HybridClock initialized: now 1783602261594213 us; error 0 us; skew 500 ppm
I20260709 13:04:21.595039 14748 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:21.598028 14748 webserver.cc:533] Webserver started at http://127.14.103.1:44617/ using document root <none> and password file <none>
I20260709 13:04:21.598671 14748 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:21.598938 14748 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:21.599274 14748 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:21.600667 14748 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602247531521-14748-0/minicluster-data/ts-0-root/instance:
uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
format_stamp: "Formatted at 2026-07-09 13:04:21 on dist-test-slave-fpp7"
I20260709 13:04:21.605731 14748 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.005s
I20260709 13:04:21.608944 15344 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:21.609680 14748 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:04:21.609998 14748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602247531521-14748-0/minicluster-data/ts-0-root
uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
format_stamp: "Formatted at 2026-07-09 13:04:21 on dist-test-slave-fpp7"
I20260709 13:04:21.610306 14748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602247531521-14748-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602247531521-14748-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602247531521-14748-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:21.629458 14748 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:21.630714 14748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:21.632263 14748 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:21.634475 14748 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:21.634692 14748 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:21.634939 14748 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:21.635133 14748 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:21.673990 14748 rpc_server.cc:307] RPC server started. Bound to: 127.14.103.1:32793
I20260709 13:04:21.674098 15406 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.103.1:32793 every 8 connection(s)
I20260709 13:04:21.678498 14748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:21.686587 15412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:21.686610 15411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:21.689996 15415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:21.691931 15407 heartbeater.cc:344] Connected to a master server at 127.14.103.62:38869
I20260709 13:04:21.692395 15407 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:21.693176 15407 heartbeater.cc:507] Master 127.14.103.62:38869 requested a full tablet report, sending...
I20260709 13:04:21.693823 14748 server_base.cc:1061] running on GCE node
I20260709 13:04:21.694892 14748 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:21.695117 14748 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:21.695326 14748 hybrid_clock.cc:648] HybridClock initialized: now 1783602261695286 us; error 0 us; skew 500 ppm
I20260709 13:04:21.695648 15280 ts_manager.cc:194] Registered new tserver with Master: 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793)
I20260709 13:04:21.695986 14748 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:21.697786 15280 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36610
I20260709 13:04:21.698560 14748 webserver.cc:533] Webserver started at http://127.14.103.2:33587/ using document root <none> and password file <none>
I20260709 13:04:21.699219 14748 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:21.699456 14748 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:21.699803 14748 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:21.701045 14748 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602247531521-14748-0/minicluster-data/ts-1-root/instance:
uuid: "df338c374ed54ecbb74bda5e628cf85e"
format_stamp: "Formatted at 2026-07-09 13:04:21 on dist-test-slave-fpp7"
I20260709 13:04:21.705296 14748 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:04:21.708256 15419 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:21.708999 14748 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:04:21.709286 14748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602247531521-14748-0/minicluster-data/ts-1-root
uuid: "df338c374ed54ecbb74bda5e628cf85e"
format_stamp: "Formatted at 2026-07-09 13:04:21 on dist-test-slave-fpp7"
I20260709 13:04:21.709573 14748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602247531521-14748-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602247531521-14748-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602247531521-14748-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:21.723281 14748 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:21.724537 14748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:21.725996 14748 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:21.728199 14748 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:21.728431 14748 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:21.728668 14748 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:21.728843 14748 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:21.766826 14748 rpc_server.cc:307] RPC server started. Bound to: 127.14.103.2:45591
I20260709 13:04:21.766948 15481 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.103.2:45591 every 8 connection(s)
I20260709 13:04:21.771379 14748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:21.778429 15486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:21.779371 15485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:21.779983 15488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:21.781360 14748 server_base.cc:1061] running on GCE node
I20260709 13:04:21.782320 14748 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:21.782538 14748 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:21.782752 14748 hybrid_clock.cc:648] HybridClock initialized: now 1783602261782733 us; error 0 us; skew 500 ppm
I20260709 13:04:21.782943 15482 heartbeater.cc:344] Connected to a master server at 127.14.103.62:38869
I20260709 13:04:21.783335 15482 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:21.783344 14748 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:21.784246 15482 heartbeater.cc:507] Master 127.14.103.62:38869 requested a full tablet report, sending...
I20260709 13:04:21.786082 14748 webserver.cc:533] Webserver started at http://127.14.103.3:38171/ using document root <none> and password file <none>
I20260709 13:04:21.786504 15280 ts_manager.cc:194] Registered new tserver with Master: df338c374ed54ecbb74bda5e628cf85e (127.14.103.2:45591)
I20260709 13:04:21.786583 14748 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:21.786831 14748 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:21.787108 14748 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:21.787920 15280 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36622
I20260709 13:04:21.788290 14748 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602247531521-14748-0/minicluster-data/ts-2-root/instance:
uuid: "24398c56cc724c91b5116396e774e09c"
format_stamp: "Formatted at 2026-07-09 13:04:21 on dist-test-slave-fpp7"
I20260709 13:04:21.792505 14748 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 13:04:21.795552 15493 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:21.796291 14748 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:04:21.796593 14748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602247531521-14748-0/minicluster-data/ts-2-root
uuid: "24398c56cc724c91b5116396e774e09c"
format_stamp: "Formatted at 2026-07-09 13:04:21 on dist-test-slave-fpp7"
I20260709 13:04:21.796887 14748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602247531521-14748-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602247531521-14748-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602247531521-14748-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:21.808269 14748 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:21.809399 14748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:21.810913 14748 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:21.813328 14748 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:21.813532 14748 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:21.813819 14748 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:21.813982 14748 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:21.851198 14748 rpc_server.cc:307] RPC server started. Bound to: 127.14.103.3:42385
I20260709 13:04:21.851287 15555 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.103.3:42385 every 8 connection(s)
I20260709 13:04:21.864500 15556 heartbeater.cc:344] Connected to a master server at 127.14.103.62:38869
I20260709 13:04:21.864812 15556 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:21.865473 15556 heartbeater.cc:507] Master 127.14.103.62:38869 requested a full tablet report, sending...
I20260709 13:04:21.867242 15280 ts_manager.cc:194] Registered new tserver with Master: 24398c56cc724c91b5116396e774e09c (127.14.103.3:42385)
I20260709 13:04:21.867877 14748 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013371926s
I20260709 13:04:21.868527 15280 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36626
I20260709 13:04:21.891269 15280 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36632:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:04:21.894088 15280 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:04:21.978686 15447 tablet_service.cc:1511] Processing CreateTablet for tablet e7be94629c3c46638abeb602169bcda4 (DEFAULT_TABLE table=test-workload [id=15a4fe387f714f9f826f165044cf9d77]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:04:21.980175 15447 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7be94629c3c46638abeb602169bcda4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:21.980176 15443 tablet_service.cc:1511] Processing CreateTablet for tablet 70da92ca35b342a7b2182eaed60e3092 (DEFAULT_TABLE table=test-workload [id=15a4fe387f714f9f826f165044cf9d77]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:04:21.981598 15443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70da92ca35b342a7b2182eaed60e3092. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:21.984524 15446 tablet_service.cc:1511] Processing CreateTablet for tablet d3551b1f0a4e4d02af6cbe21a747f4ec (DEFAULT_TABLE table=test-workload [id=15a4fe387f714f9f826f165044cf9d77]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:04:21.985893 15446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3551b1f0a4e4d02af6cbe21a747f4ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:21.988835 15445 tablet_service.cc:1511] Processing CreateTablet for tablet 32f268907bbb4d1081fa0fccb595110e (DEFAULT_TABLE table=test-workload [id=15a4fe387f714f9f826f165044cf9d77]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:04:21.990172 15445 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32f268907bbb4d1081fa0fccb595110e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:21.992805 15444 tablet_service.cc:1511] Processing CreateTablet for tablet 42e07a8df5ac47eea3da5846963de3fa (DEFAULT_TABLE table=test-workload [id=15a4fe387f714f9f826f165044cf9d77]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:04:21.994123 15444 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42e07a8df5ac47eea3da5846963de3fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:21.999236 15571 tablet_bootstrap.cc:492] T e7be94629c3c46638abeb602169bcda4 P df338c374ed54ecbb74bda5e628cf85e: Bootstrap starting.
I20260709 13:04:22.002288 15519 tablet_service.cc:1511] Processing CreateTablet for tablet 32f268907bbb4d1081fa0fccb595110e (DEFAULT_TABLE table=test-workload [id=15a4fe387f714f9f826f165044cf9d77]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:04:22.004920 15518 tablet_service.cc:1511] Processing CreateTablet for tablet 42e07a8df5ac47eea3da5846963de3fa (DEFAULT_TABLE table=test-workload [id=15a4fe387f714f9f826f165044cf9d77]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:04:22.005617 15443 tablet_service.cc:1511] Processing CreateTablet for tablet 36e186173f7c42dbaa9840522867ce03 (DEFAULT_TABLE table=test-workload [id=15a4fe387f714f9f826f165044cf9d77]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:04:22.006875 15443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36e186173f7c42dbaa9840522867ce03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:22.007514 15571 tablet_bootstrap.cc:654] T e7be94629c3c46638abeb602169bcda4 P df338c374ed54ecbb74bda5e628cf85e: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:22.007175 15517 tablet_service.cc:1511] Processing CreateTablet for tablet 70da92ca35b342a7b2182eaed60e3092 (DEFAULT_TABLE table=test-workload [id=15a4fe387f714f9f826f165044cf9d77]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:04:22.009459 15516 tablet_service.cc:1511] Processing CreateTablet for tablet 844dc4a0feff4097a52882da0b69cef7 (DEFAULT_TABLE table=test-workload [id=15a4fe387f714f9f826f165044cf9d77]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:04:22.012094 15441 tablet_service.cc:1511] Processing CreateTablet for tablet 3590c79d23594f91a88f9646df62c52a (DEFAULT_TABLE table=test-workload [id=15a4fe387f714f9f826f165044cf9d77]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:04:22.013773 15441 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3590c79d23594f91a88f9646df62c52a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:22.014501 15447 tablet_service.cc:1511] Processing CreateTablet for tablet 844dc4a0feff4097a52882da0b69cef7 (DEFAULT_TABLE table=test-workload [id=15a4fe387f714f9f826f165044cf9d77]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:04:22.016016 15447 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 844dc4a0feff4097a52882da0b69cef7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:22.016831 15440 tablet_service.cc:1511] Processing CreateTablet for tablet deb69cd9c0414a348a3ee16058a36f00 (DEFAULT_TABLE table=test-workload [id=15a4fe387f714f9f826f165044cf9d77]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:04:22.016965 15513 tablet_service.cc:1511] Processing CreateTablet for tablet 3590c79d23594f91a88f9646df62c52a (DEFAULT_TABLE table=test-workload [id=15a4fe387f714f9f826f165044cf9d77]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:04:22.002437 15442 tablet_service.cc:1511] Processing CreateTablet for tablet b46d829f140440738e6d2d456f792c75 (DEFAULT_TABLE table=test-workload [id=15a4fe387f714f9f826f165044cf9d77]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:04:22.000762 15521 tablet_service.cc:1511] Processing CreateTablet for tablet e7be94629c3c46638abeb602169bcda4 (DEFAULT_TABLE table=test-workload [id=15a4fe387f714f9f826f165044cf9d77]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:04:22.021798 15514 tablet_service.cc:1511] Processing CreateTablet for tablet b46d829f140440738e6d2d456f792c75 (DEFAULT_TABLE table=test-workload [id=15a4fe387f714f9f826f165044cf9d77]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:04:22.022450 15520 tablet_service.cc:1511] Processing CreateTablet for tablet d3551b1f0a4e4d02af6cbe21a747f4ec (DEFAULT_TABLE table=test-workload [id=15a4fe387f714f9f826f165044cf9d77]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:04:22.024470 15515 tablet_service.cc:1511] Processing CreateTablet for tablet 36e186173f7c42dbaa9840522867ce03 (DEFAULT_TABLE table=test-workload [id=15a4fe387f714f9f826f165044cf9d77]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:04:22.024672 15512 tablet_service.cc:1511] Processing CreateTablet for tablet deb69cd9c0414a348a3ee16058a36f00 (DEFAULT_TABLE table=test-workload [id=15a4fe387f714f9f826f165044cf9d77]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:04:22.026726 15372 tablet_service.cc:1511] Processing CreateTablet for tablet e7be94629c3c46638abeb602169bcda4 (DEFAULT_TABLE table=test-workload [id=15a4fe387f714f9f826f165044cf9d77]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:04:22.027976 15372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7be94629c3c46638abeb602169bcda4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:22.028793 15371 tablet_service.cc:1511] Processing CreateTablet for tablet d3551b1f0a4e4d02af6cbe21a747f4ec (DEFAULT_TABLE table=test-workload [id=15a4fe387f714f9f826f165044cf9d77]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:04:22.030028 15371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3551b1f0a4e4d02af6cbe21a747f4ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:22.030062 15369 tablet_service.cc:1511] Processing CreateTablet for tablet 42e07a8df5ac47eea3da5846963de3fa (DEFAULT_TABLE table=test-workload [id=15a4fe387f714f9f826f165044cf9d77]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:04:22.031344 15369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42e07a8df5ac47eea3da5846963de3fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:22.032581 15442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b46d829f140440738e6d2d456f792c75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:22.032706 15370 tablet_service.cc:1511] Processing CreateTablet for tablet 32f268907bbb4d1081fa0fccb595110e (DEFAULT_TABLE table=test-workload [id=15a4fe387f714f9f826f165044cf9d77]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:04:22.034085 15370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32f268907bbb4d1081fa0fccb595110e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:22.033989 15367 tablet_service.cc:1511] Processing CreateTablet for tablet 844dc4a0feff4097a52882da0b69cef7 (DEFAULT_TABLE table=test-workload [id=15a4fe387f714f9f826f165044cf9d77]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:04:22.035254 15367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 844dc4a0feff4097a52882da0b69cef7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:22.036821 15366 tablet_service.cc:1511] Processing CreateTablet for tablet 36e186173f7c42dbaa9840522867ce03 (DEFAULT_TABLE table=test-workload [id=15a4fe387f714f9f826f165044cf9d77]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:04:22.038054 15366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36e186173f7c42dbaa9840522867ce03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:22.040185 15440 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet deb69cd9c0414a348a3ee16058a36f00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:22.019235 15519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32f268907bbb4d1081fa0fccb595110e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:22.045467 15365 tablet_service.cc:1511] Processing CreateTablet for tablet b46d829f140440738e6d2d456f792c75 (DEFAULT_TABLE table=test-workload [id=15a4fe387f714f9f826f165044cf9d77]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:04:22.046765 15365 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b46d829f140440738e6d2d456f792c75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:22.048878 15517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70da92ca35b342a7b2182eaed60e3092. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:22.049366 15364 tablet_service.cc:1511] Processing CreateTablet for tablet 3590c79d23594f91a88f9646df62c52a (DEFAULT_TABLE table=test-workload [id=15a4fe387f714f9f826f165044cf9d77]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:04:22.050626 15364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3590c79d23594f91a88f9646df62c52a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:22.052018 15518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42e07a8df5ac47eea3da5846963de3fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:22.055824 15363 tablet_service.cc:1511] Processing CreateTablet for tablet deb69cd9c0414a348a3ee16058a36f00 (DEFAULT_TABLE table=test-workload [id=15a4fe387f714f9f826f165044cf9d77]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:04:22.057133 15363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet deb69cd9c0414a348a3ee16058a36f00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:22.060707 15512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet deb69cd9c0414a348a3ee16058a36f00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:22.061367 15513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3590c79d23594f91a88f9646df62c52a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:22.061983 15514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b46d829f140440738e6d2d456f792c75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:22.062595 15515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36e186173f7c42dbaa9840522867ce03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:22.063961 15516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 844dc4a0feff4097a52882da0b69cef7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:22.064584 15521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7be94629c3c46638abeb602169bcda4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:22.032794 15368 tablet_service.cc:1511] Processing CreateTablet for tablet 70da92ca35b342a7b2182eaed60e3092 (DEFAULT_TABLE table=test-workload [id=15a4fe387f714f9f826f165044cf9d77]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:04:22.065583 15368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70da92ca35b342a7b2182eaed60e3092. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:22.068071 15520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3551b1f0a4e4d02af6cbe21a747f4ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:22.170217 15571 tablet_bootstrap.cc:492] T e7be94629c3c46638abeb602169bcda4 P df338c374ed54ecbb74bda5e628cf85e: No bootstrap required, opened a new log
I20260709 13:04:22.170694 15571 ts_tablet_manager.cc:1403] T e7be94629c3c46638abeb602169bcda4 P df338c374ed54ecbb74bda5e628cf85e: Time spent bootstrapping tablet: real 0.172s	user 0.018s	sys 0.034s
I20260709 13:04:22.173026 15574 tablet_bootstrap.cc:492] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0: Bootstrap starting.
I20260709 13:04:22.173249 15573 tablet_bootstrap.cc:492] T b46d829f140440738e6d2d456f792c75 P 24398c56cc724c91b5116396e774e09c: Bootstrap starting.
I20260709 13:04:22.173575 15571 raft_consensus.cc:359] T e7be94629c3c46638abeb602169bcda4 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } } peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } }
I20260709 13:04:22.174320 15571 raft_consensus.cc:385] T e7be94629c3c46638abeb602169bcda4 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:22.174608 15571 raft_consensus.cc:740] T e7be94629c3c46638abeb602169bcda4 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df338c374ed54ecbb74bda5e628cf85e, State: Initialized, Role: FOLLOWER
I20260709 13:04:22.175335 15571 consensus_queue.cc:260] T e7be94629c3c46638abeb602169bcda4 P df338c374ed54ecbb74bda5e628cf85e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } } peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } }
I20260709 13:04:22.179930 15573 tablet_bootstrap.cc:654] T b46d829f140440738e6d2d456f792c75 P 24398c56cc724c91b5116396e774e09c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:22.185544 15571 ts_tablet_manager.cc:1434] T e7be94629c3c46638abeb602169bcda4 P df338c374ed54ecbb74bda5e628cf85e: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.005s
I20260709 13:04:22.190078 15482 heartbeater.cc:499] Master 127.14.103.62:38869 was elected leader, sending a full tablet report...
I20260709 13:04:22.191978 15571 tablet_bootstrap.cc:492] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e: Bootstrap starting.
I20260709 13:04:22.193578 15573 tablet_bootstrap.cc:492] T b46d829f140440738e6d2d456f792c75 P 24398c56cc724c91b5116396e774e09c: No bootstrap required, opened a new log
I20260709 13:04:22.193883 15573 ts_tablet_manager.cc:1403] T b46d829f140440738e6d2d456f792c75 P 24398c56cc724c91b5116396e774e09c: Time spent bootstrapping tablet: real 0.021s	user 0.003s	sys 0.013s
I20260709 13:04:22.195791 15573 raft_consensus.cc:359] T b46d829f140440738e6d2d456f792c75 P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.196478 15573 raft_consensus.cc:385] T b46d829f140440738e6d2d456f792c75 P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:22.196748 15573 raft_consensus.cc:740] T b46d829f140440738e6d2d456f792c75 P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24398c56cc724c91b5116396e774e09c, State: Initialized, Role: FOLLOWER
I20260709 13:04:22.197427 15573 consensus_queue.cc:260] T b46d829f140440738e6d2d456f792c75 P 24398c56cc724c91b5116396e774e09c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.198163 15571 tablet_bootstrap.cc:654] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:22.208736 15574 tablet_bootstrap.cc:654] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:22.210518 15571 tablet_bootstrap.cc:492] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e: No bootstrap required, opened a new log
I20260709 13:04:22.211125 15571 ts_tablet_manager.cc:1403] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.009s
I20260709 13:04:22.214663 15571 raft_consensus.cc:359] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.214990 15556 heartbeater.cc:499] Master 127.14.103.62:38869 was elected leader, sending a full tablet report...
I20260709 13:04:22.216403 15573 ts_tablet_manager.cc:1434] T b46d829f140440738e6d2d456f792c75 P 24398c56cc724c91b5116396e774e09c: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.014s
I20260709 13:04:22.217669 15573 tablet_bootstrap.cc:492] T deb69cd9c0414a348a3ee16058a36f00 P 24398c56cc724c91b5116396e774e09c: Bootstrap starting.
I20260709 13:04:22.218652 15571 raft_consensus.cc:385] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:22.219074 15571 raft_consensus.cc:740] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df338c374ed54ecbb74bda5e628cf85e, State: Initialized, Role: FOLLOWER
I20260709 13:04:22.220047 15571 consensus_queue.cc:260] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.221720 15574 tablet_bootstrap.cc:492] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0: No bootstrap required, opened a new log
I20260709 13:04:22.222174 15574 ts_tablet_manager.cc:1403] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0: Time spent bootstrapping tablet: real 0.049s	user 0.004s	sys 0.012s
I20260709 13:04:22.222247 15571 ts_tablet_manager.cc:1434] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.002s
I20260709 13:04:22.223307 15571 tablet_bootstrap.cc:492] T d3551b1f0a4e4d02af6cbe21a747f4ec P df338c374ed54ecbb74bda5e628cf85e: Bootstrap starting.
I20260709 13:04:22.224258 15573 tablet_bootstrap.cc:654] T deb69cd9c0414a348a3ee16058a36f00 P 24398c56cc724c91b5116396e774e09c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:22.225167 15574 raft_consensus.cc:359] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.225937 15574 raft_consensus.cc:385] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:22.226243 15574 raft_consensus.cc:740] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ab6657817b84a1d94dba0a4eb2292d0, State: Initialized, Role: FOLLOWER
I20260709 13:04:22.226994 15574 consensus_queue.cc:260] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.230093 15571 tablet_bootstrap.cc:654] T d3551b1f0a4e4d02af6cbe21a747f4ec P df338c374ed54ecbb74bda5e628cf85e: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:22.233896 15407 heartbeater.cc:499] Master 127.14.103.62:38869 was elected leader, sending a full tablet report...
I20260709 13:04:22.234760 15574 ts_tablet_manager.cc:1434] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0: Time spent starting tablet: real 0.012s	user 0.013s	sys 0.000s
I20260709 13:04:22.235989 15574 tablet_bootstrap.cc:492] T 42e07a8df5ac47eea3da5846963de3fa P 3ab6657817b84a1d94dba0a4eb2292d0: Bootstrap starting.
I20260709 13:04:22.243597 15574 tablet_bootstrap.cc:654] T 42e07a8df5ac47eea3da5846963de3fa P 3ab6657817b84a1d94dba0a4eb2292d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:22.244653 15573 tablet_bootstrap.cc:492] T deb69cd9c0414a348a3ee16058a36f00 P 24398c56cc724c91b5116396e774e09c: No bootstrap required, opened a new log
I20260709 13:04:22.245185 15573 ts_tablet_manager.cc:1403] T deb69cd9c0414a348a3ee16058a36f00 P 24398c56cc724c91b5116396e774e09c: Time spent bootstrapping tablet: real 0.028s	user 0.022s	sys 0.004s
I20260709 13:04:22.247005 15571 tablet_bootstrap.cc:492] T d3551b1f0a4e4d02af6cbe21a747f4ec P df338c374ed54ecbb74bda5e628cf85e: No bootstrap required, opened a new log
I20260709 13:04:22.247562 15571 ts_tablet_manager.cc:1403] T d3551b1f0a4e4d02af6cbe21a747f4ec P df338c374ed54ecbb74bda5e628cf85e: Time spent bootstrapping tablet: real 0.024s	user 0.006s	sys 0.006s
I20260709 13:04:22.248725 15573 raft_consensus.cc:359] T deb69cd9c0414a348a3ee16058a36f00 P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.249558 15573 raft_consensus.cc:385] T deb69cd9c0414a348a3ee16058a36f00 P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:22.249841 15573 raft_consensus.cc:740] T deb69cd9c0414a348a3ee16058a36f00 P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24398c56cc724c91b5116396e774e09c, State: Initialized, Role: FOLLOWER
I20260709 13:04:22.250658 15571 raft_consensus.cc:359] T d3551b1f0a4e4d02af6cbe21a747f4ec P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.251449 15571 raft_consensus.cc:385] T d3551b1f0a4e4d02af6cbe21a747f4ec P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:22.250877 15573 consensus_queue.cc:260] T deb69cd9c0414a348a3ee16058a36f00 P 24398c56cc724c91b5116396e774e09c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.251500 15577 raft_consensus.cc:493] T b46d829f140440738e6d2d456f792c75 P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.251981 15571 raft_consensus.cc:740] T d3551b1f0a4e4d02af6cbe21a747f4ec P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df338c374ed54ecbb74bda5e628cf85e, State: Initialized, Role: FOLLOWER
I20260709 13:04:22.252193 15577 raft_consensus.cc:515] T b46d829f140440738e6d2d456f792c75 P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.253051 15571 consensus_queue.cc:260] T d3551b1f0a4e4d02af6cbe21a747f4ec P df338c374ed54ecbb74bda5e628cf85e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.255228 15571 ts_tablet_manager.cc:1434] T d3551b1f0a4e4d02af6cbe21a747f4ec P df338c374ed54ecbb74bda5e628cf85e: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:04:22.256253 15571 tablet_bootstrap.cc:492] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e: Bootstrap starting.
I20260709 13:04:22.261126 15577 leader_election.cc:290] T b46d829f140440738e6d2d456f792c75 P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793), df338c374ed54ecbb74bda5e628cf85e (127.14.103.2:45591)
I20260709 13:04:22.281337 15571 tablet_bootstrap.cc:654] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:22.287325 15573 ts_tablet_manager.cc:1434] T deb69cd9c0414a348a3ee16058a36f00 P 24398c56cc724c91b5116396e774e09c: Time spent starting tablet: real 0.042s	user 0.018s	sys 0.023s
I20260709 13:04:22.288641 15573 tablet_bootstrap.cc:492] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c: Bootstrap starting.
I20260709 13:04:22.295768 15457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b46d829f140440738e6d2d456f792c75" candidate_uuid: "24398c56cc724c91b5116396e774e09c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df338c374ed54ecbb74bda5e628cf85e" is_pre_election: true
I20260709 13:04:22.296226 15382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b46d829f140440738e6d2d456f792c75" candidate_uuid: "24398c56cc724c91b5116396e774e09c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" is_pre_election: true
W20260709 13:04:22.297933 15497 leader_election.cc:343] T b46d829f140440738e6d2d456f792c75 P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:22.298852 15497 leader_election.cc:343] T b46d829f140440738e6d2d456f792c75 P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer df338c374ed54ecbb74bda5e628cf85e (127.14.103.2:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:22.299702 15571 tablet_bootstrap.cc:492] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e: No bootstrap required, opened a new log
I20260709 13:04:22.300237 15571 ts_tablet_manager.cc:1403] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e: Time spent bootstrapping tablet: real 0.044s	user 0.006s	sys 0.012s
I20260709 13:04:22.302297 15573 tablet_bootstrap.cc:654] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:22.302834 15575 raft_consensus.cc:493] T e7be94629c3c46638abeb602169bcda4 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.299381 15497 leader_election.cc:304] T b46d829f140440738e6d2d456f792c75 P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24398c56cc724c91b5116396e774e09c; no voters: 3ab6657817b84a1d94dba0a4eb2292d0, df338c374ed54ecbb74bda5e628cf85e
I20260709 13:04:22.303310 15575 raft_consensus.cc:515] T e7be94629c3c46638abeb602169bcda4 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } } peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } }
I20260709 13:04:22.303841 15584 raft_consensus.cc:2749] T b46d829f140440738e6d2d456f792c75 P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:22.304970 15571 raft_consensus.cc:359] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.305747 15571 raft_consensus.cc:385] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:22.306120 15571 raft_consensus.cc:740] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df338c374ed54ecbb74bda5e628cf85e, State: Initialized, Role: FOLLOWER
I20260709 13:04:22.306944 15571 consensus_queue.cc:260] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.310582 15575 leader_election.cc:290] T e7be94629c3c46638abeb602169bcda4 P df338c374ed54ecbb74bda5e628cf85e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24398c56cc724c91b5116396e774e09c (127.14.103.3:42385), 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793)
I20260709 13:04:22.315231 15573 tablet_bootstrap.cc:492] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c: No bootstrap required, opened a new log
I20260709 13:04:22.315886 15573 ts_tablet_manager.cc:1403] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c: Time spent bootstrapping tablet: real 0.028s	user 0.019s	sys 0.002s
I20260709 13:04:22.319828 15573 raft_consensus.cc:359] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.320797 15573 raft_consensus.cc:385] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:22.321187 15573 raft_consensus.cc:740] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24398c56cc724c91b5116396e774e09c, State: Initialized, Role: FOLLOWER
I20260709 13:04:22.322290 15573 consensus_queue.cc:260] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.325528 15573 ts_tablet_manager.cc:1434] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:04:22.326804 15573 tablet_bootstrap.cc:492] T e7be94629c3c46638abeb602169bcda4 P 24398c56cc724c91b5116396e774e09c: Bootstrap starting.
I20260709 13:04:22.335232 15573 tablet_bootstrap.cc:654] T e7be94629c3c46638abeb602169bcda4 P 24398c56cc724c91b5116396e774e09c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:22.348544 15573 tablet_bootstrap.cc:492] T e7be94629c3c46638abeb602169bcda4 P 24398c56cc724c91b5116396e774e09c: No bootstrap required, opened a new log
I20260709 13:04:22.349191 15573 ts_tablet_manager.cc:1403] T e7be94629c3c46638abeb602169bcda4 P 24398c56cc724c91b5116396e774e09c: Time spent bootstrapping tablet: real 0.023s	user 0.011s	sys 0.009s
I20260709 13:04:22.353117 15573 raft_consensus.cc:359] T e7be94629c3c46638abeb602169bcda4 P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } } peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } }
I20260709 13:04:22.354115 15573 raft_consensus.cc:385] T e7be94629c3c46638abeb602169bcda4 P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:22.354482 15573 raft_consensus.cc:740] T e7be94629c3c46638abeb602169bcda4 P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24398c56cc724c91b5116396e774e09c, State: Initialized, Role: FOLLOWER
I20260709 13:04:22.355597 15573 consensus_queue.cc:260] T e7be94629c3c46638abeb602169bcda4 P 24398c56cc724c91b5116396e774e09c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } } peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } }
I20260709 13:04:22.361786 15573 ts_tablet_manager.cc:1434] T e7be94629c3c46638abeb602169bcda4 P 24398c56cc724c91b5116396e774e09c: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.002s
I20260709 13:04:22.363188 15573 tablet_bootstrap.cc:492] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c: Bootstrap starting.
I20260709 13:04:22.364279 15584 raft_consensus.cc:493] T deb69cd9c0414a348a3ee16058a36f00 P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.364894 15584 raft_consensus.cc:515] T deb69cd9c0414a348a3ee16058a36f00 P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.367417 15584 leader_election.cc:290] T deb69cd9c0414a348a3ee16058a36f00 P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793), df338c374ed54ecbb74bda5e628cf85e (127.14.103.2:45591)
I20260709 13:04:22.370198 15382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deb69cd9c0414a348a3ee16058a36f00" candidate_uuid: "24398c56cc724c91b5116396e774e09c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" is_pre_election: true
I20260709 13:04:22.371510 15573 tablet_bootstrap.cc:654] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:22.371440 15457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deb69cd9c0414a348a3ee16058a36f00" candidate_uuid: "24398c56cc724c91b5116396e774e09c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df338c374ed54ecbb74bda5e628cf85e" is_pre_election: true
W20260709 13:04:22.373385 15497 leader_election.cc:343] T deb69cd9c0414a348a3ee16058a36f00 P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer df338c374ed54ecbb74bda5e628cf85e (127.14.103.2:45591): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:22.374042 15497 leader_election.cc:343] T deb69cd9c0414a348a3ee16058a36f00 P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:22.374460 15497 leader_election.cc:304] T deb69cd9c0414a348a3ee16058a36f00 P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24398c56cc724c91b5116396e774e09c; no voters: 3ab6657817b84a1d94dba0a4eb2292d0, df338c374ed54ecbb74bda5e628cf85e
I20260709 13:04:22.375612 15584 raft_consensus.cc:2749] T deb69cd9c0414a348a3ee16058a36f00 P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:22.384164 15573 tablet_bootstrap.cc:492] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c: No bootstrap required, opened a new log
I20260709 13:04:22.384682 15573 ts_tablet_manager.cc:1403] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.009s
I20260709 13:04:22.386968 15571 ts_tablet_manager.cc:1434] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e: Time spent starting tablet: real 0.086s	user 0.031s	sys 0.038s
I20260709 13:04:22.387435 15573 raft_consensus.cc:359] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.387988 15571 tablet_bootstrap.cc:492] T 42e07a8df5ac47eea3da5846963de3fa P df338c374ed54ecbb74bda5e628cf85e: Bootstrap starting.
I20260709 13:04:22.388160 15573 raft_consensus.cc:385] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:22.388455 15573 raft_consensus.cc:740] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24398c56cc724c91b5116396e774e09c, State: Initialized, Role: FOLLOWER
I20260709 13:04:22.389158 15573 consensus_queue.cc:260] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.390849 15577 raft_consensus.cc:493] T e7be94629c3c46638abeb602169bcda4 P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.391471 15573 ts_tablet_manager.cc:1434] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.003s
I20260709 13:04:22.391289 15577 raft_consensus.cc:515] T e7be94629c3c46638abeb602169bcda4 P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } } peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } }
I20260709 13:04:22.392481 15573 tablet_bootstrap.cc:492] T 42e07a8df5ac47eea3da5846963de3fa P 24398c56cc724c91b5116396e774e09c: Bootstrap starting.
I20260709 13:04:22.393374 15577 leader_election.cc:290] T e7be94629c3c46638abeb602169bcda4 P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df338c374ed54ecbb74bda5e628cf85e (127.14.103.2:45591), 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793)
I20260709 13:04:22.394315 15571 tablet_bootstrap.cc:654] T 42e07a8df5ac47eea3da5846963de3fa P df338c374ed54ecbb74bda5e628cf85e: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:22.394810 15457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7be94629c3c46638abeb602169bcda4" candidate_uuid: "24398c56cc724c91b5116396e774e09c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df338c374ed54ecbb74bda5e628cf85e" is_pre_election: true
I20260709 13:04:22.395397 15457 raft_consensus.cc:2468] T e7be94629c3c46638abeb602169bcda4 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24398c56cc724c91b5116396e774e09c in term 0.
I20260709 13:04:22.396399 15382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7be94629c3c46638abeb602169bcda4" candidate_uuid: "24398c56cc724c91b5116396e774e09c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" is_pre_election: true
I20260709 13:04:22.398101 15497 leader_election.cc:304] T e7be94629c3c46638abeb602169bcda4 P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24398c56cc724c91b5116396e774e09c, df338c374ed54ecbb74bda5e628cf85e; no voters: 
I20260709 13:04:22.398583 15589 raft_consensus.cc:493] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.399040 15589 raft_consensus.cc:515] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.400131 15573 tablet_bootstrap.cc:654] T 42e07a8df5ac47eea3da5846963de3fa P 24398c56cc724c91b5116396e774e09c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:22.401119 15589 leader_election.cc:290] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793), 24398c56cc724c91b5116396e774e09c (127.14.103.3:42385)
I20260709 13:04:22.401451 15577 raft_consensus.cc:2804] T e7be94629c3c46638abeb602169bcda4 P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:22.401786 15577 raft_consensus.cc:493] T e7be94629c3c46638abeb602169bcda4 P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:22.399060 15497 leader_election.cc:343] T e7be94629c3c46638abeb602169bcda4 P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:22.402201 15577 raft_consensus.cc:3060] T e7be94629c3c46638abeb602169bcda4 P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:22.403656 15382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7be94629c3c46638abeb602169bcda4" candidate_uuid: "df338c374ed54ecbb74bda5e628cf85e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" is_pre_election: true
I20260709 13:04:22.404866 15381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36e186173f7c42dbaa9840522867ce03" candidate_uuid: "df338c374ed54ecbb74bda5e628cf85e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" is_pre_election: true
W20260709 13:04:22.405009 15423 leader_election.cc:343] T e7be94629c3c46638abeb602169bcda4 P df338c374ed54ecbb74bda5e628cf85e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:22.406170 15423 leader_election.cc:343] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:22.411573 15577 raft_consensus.cc:515] T e7be94629c3c46638abeb602169bcda4 P 24398c56cc724c91b5116396e774e09c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } } peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } }
I20260709 13:04:22.414472 15457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7be94629c3c46638abeb602169bcda4" candidate_uuid: "24398c56cc724c91b5116396e774e09c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df338c374ed54ecbb74bda5e628cf85e"
I20260709 13:04:22.415020 15457 raft_consensus.cc:3060] T e7be94629c3c46638abeb602169bcda4 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:22.419802 15531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7be94629c3c46638abeb602169bcda4" candidate_uuid: "df338c374ed54ecbb74bda5e628cf85e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24398c56cc724c91b5116396e774e09c" is_pre_election: true
I20260709 13:04:22.420938 15531 raft_consensus.cc:2393] T e7be94629c3c46638abeb602169bcda4 P 24398c56cc724c91b5116396e774e09c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate df338c374ed54ecbb74bda5e628cf85e in current term 1: Already voted for candidate 24398c56cc724c91b5116396e774e09c in this term.
I20260709 13:04:22.422399 15530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36e186173f7c42dbaa9840522867ce03" candidate_uuid: "df338c374ed54ecbb74bda5e628cf85e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24398c56cc724c91b5116396e774e09c" is_pre_election: true
I20260709 13:04:22.424152 15574 tablet_bootstrap.cc:492] T 42e07a8df5ac47eea3da5846963de3fa P 3ab6657817b84a1d94dba0a4eb2292d0: No bootstrap required, opened a new log
I20260709 13:04:22.424026 15381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7be94629c3c46638abeb602169bcda4" candidate_uuid: "24398c56cc724c91b5116396e774e09c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
I20260709 13:04:22.424612 15574 ts_tablet_manager.cc:1403] T 42e07a8df5ac47eea3da5846963de3fa P 3ab6657817b84a1d94dba0a4eb2292d0: Time spent bootstrapping tablet: real 0.189s	user 0.051s	sys 0.095s
W20260709 13:04:22.425537 15497 leader_election.cc:343] T e7be94629c3c46638abeb602169bcda4 P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:22.422425 15457 raft_consensus.cc:2468] T e7be94629c3c46638abeb602169bcda4 P df338c374ed54ecbb74bda5e628cf85e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24398c56cc724c91b5116396e774e09c in term 1.
I20260709 13:04:22.427374 15497 leader_election.cc:304] T e7be94629c3c46638abeb602169bcda4 P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24398c56cc724c91b5116396e774e09c, df338c374ed54ecbb74bda5e628cf85e; no voters: 3ab6657817b84a1d94dba0a4eb2292d0
I20260709 13:04:22.427565 15574 raft_consensus.cc:359] T 42e07a8df5ac47eea3da5846963de3fa P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.427927 15420 leader_election.cc:304] T e7be94629c3c46638abeb602169bcda4 P df338c374ed54ecbb74bda5e628cf85e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df338c374ed54ecbb74bda5e628cf85e; no voters: 24398c56cc724c91b5116396e774e09c, 3ab6657817b84a1d94dba0a4eb2292d0
I20260709 13:04:22.428285 15574 raft_consensus.cc:385] T 42e07a8df5ac47eea3da5846963de3fa P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:22.429551 15589 raft_consensus.cc:2749] T e7be94629c3c46638abeb602169bcda4 P df338c374ed54ecbb74bda5e628cf85e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:04:22.429508 15420 leader_election.cc:343] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24398c56cc724c91b5116396e774e09c (127.14.103.3:42385): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:22.428256 15584 raft_consensus.cc:2804] T e7be94629c3c46638abeb602169bcda4 P 24398c56cc724c91b5116396e774e09c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:22.429903 15574 raft_consensus.cc:740] T 42e07a8df5ac47eea3da5846963de3fa P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ab6657817b84a1d94dba0a4eb2292d0, State: Initialized, Role: FOLLOWER
I20260709 13:04:22.430152 15420 leader_election.cc:304] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df338c374ed54ecbb74bda5e628cf85e; no voters: 24398c56cc724c91b5116396e774e09c, 3ab6657817b84a1d94dba0a4eb2292d0
I20260709 13:04:22.431030 15589 raft_consensus.cc:2749] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:22.431640 15571 tablet_bootstrap.cc:492] T 42e07a8df5ac47eea3da5846963de3fa P df338c374ed54ecbb74bda5e628cf85e: No bootstrap required, opened a new log
I20260709 13:04:22.431217 15574 consensus_queue.cc:260] T 42e07a8df5ac47eea3da5846963de3fa P 3ab6657817b84a1d94dba0a4eb2292d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.432124 15571 ts_tablet_manager.cc:1403] T 42e07a8df5ac47eea3da5846963de3fa P df338c374ed54ecbb74bda5e628cf85e: Time spent bootstrapping tablet: real 0.044s	user 0.033s	sys 0.010s
I20260709 13:04:22.435055 15584 raft_consensus.cc:697] T e7be94629c3c46638abeb602169bcda4 P 24398c56cc724c91b5116396e774e09c [term 1 LEADER]: Becoming Leader. State: Replica: 24398c56cc724c91b5116396e774e09c, State: Running, Role: LEADER
I20260709 13:04:22.435034 15571 raft_consensus.cc:359] T 42e07a8df5ac47eea3da5846963de3fa P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.435914 15571 raft_consensus.cc:385] T 42e07a8df5ac47eea3da5846963de3fa P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:22.436239 15571 raft_consensus.cc:740] T 42e07a8df5ac47eea3da5846963de3fa P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df338c374ed54ecbb74bda5e628cf85e, State: Initialized, Role: FOLLOWER
I20260709 13:04:22.436074 15584 consensus_queue.cc:237] T e7be94629c3c46638abeb602169bcda4 P 24398c56cc724c91b5116396e774e09c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } } peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } }
I20260709 13:04:22.436998 15571 consensus_queue.cc:260] T 42e07a8df5ac47eea3da5846963de3fa P df338c374ed54ecbb74bda5e628cf85e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.439622 15571 ts_tablet_manager.cc:1434] T 42e07a8df5ac47eea3da5846963de3fa P df338c374ed54ecbb74bda5e628cf85e: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.002s
I20260709 13:04:22.440373 15574 ts_tablet_manager.cc:1434] T 42e07a8df5ac47eea3da5846963de3fa P 3ab6657817b84a1d94dba0a4eb2292d0: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:04:22.440567 15571 tablet_bootstrap.cc:492] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e: Bootstrap starting.
I20260709 13:04:22.440876 15577 leader_election.cc:290] T e7be94629c3c46638abeb602169bcda4 P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 1 election: Requested vote from peers df338c374ed54ecbb74bda5e628cf85e (127.14.103.2:45591), 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793)
I20260709 13:04:22.444885 15577 raft_consensus.cc:493] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.445398 15577 raft_consensus.cc:515] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.447285 15571 tablet_bootstrap.cc:654] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:22.442471 15573 tablet_bootstrap.cc:492] T 42e07a8df5ac47eea3da5846963de3fa P 24398c56cc724c91b5116396e774e09c: No bootstrap required, opened a new log
I20260709 13:04:22.448328 15573 ts_tablet_manager.cc:1403] T 42e07a8df5ac47eea3da5846963de3fa P 24398c56cc724c91b5116396e774e09c: Time spent bootstrapping tablet: real 0.056s	user 0.020s	sys 0.009s
I20260709 13:04:22.448231 15381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32f268907bbb4d1081fa0fccb595110e" candidate_uuid: "24398c56cc724c91b5116396e774e09c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" is_pre_election: true
W20260709 13:04:22.449743 15497 leader_election.cc:343] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:22.444150 15574 tablet_bootstrap.cc:492] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0: Bootstrap starting.
I20260709 13:04:22.451292 15573 raft_consensus.cc:359] T 42e07a8df5ac47eea3da5846963de3fa P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.456063 15573 raft_consensus.cc:385] T 42e07a8df5ac47eea3da5846963de3fa P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:22.456326 15573 raft_consensus.cc:740] T 42e07a8df5ac47eea3da5846963de3fa P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24398c56cc724c91b5116396e774e09c, State: Initialized, Role: FOLLOWER
I20260709 13:04:22.457005 15573 consensus_queue.cc:260] T 42e07a8df5ac47eea3da5846963de3fa P 24398c56cc724c91b5116396e774e09c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.458655 15573 ts_tablet_manager.cc:1434] T 42e07a8df5ac47eea3da5846963de3fa P 24398c56cc724c91b5116396e774e09c: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.000s
I20260709 13:04:22.459257 15574 tablet_bootstrap.cc:654] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:22.461951 15457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32f268907bbb4d1081fa0fccb595110e" candidate_uuid: "24398c56cc724c91b5116396e774e09c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df338c374ed54ecbb74bda5e628cf85e" is_pre_election: true
I20260709 13:04:22.459466 15573 tablet_bootstrap.cc:492] T 844dc4a0feff4097a52882da0b69cef7 P 24398c56cc724c91b5116396e774e09c: Bootstrap starting.
W20260709 13:04:22.463580 15497 leader_election.cc:343] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer df338c374ed54ecbb74bda5e628cf85e (127.14.103.2:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:22.464195 15497 leader_election.cc:304] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24398c56cc724c91b5116396e774e09c; no voters: 3ab6657817b84a1d94dba0a4eb2292d0, df338c374ed54ecbb74bda5e628cf85e
I20260709 13:04:22.465160 15584 raft_consensus.cc:2749] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:22.465391 15577 leader_election.cc:290] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793), df338c374ed54ecbb74bda5e628cf85e (127.14.103.2:45591)
I20260709 13:04:22.465988 15278 catalog_manager.cc:5697] T e7be94629c3c46638abeb602169bcda4 P 24398c56cc724c91b5116396e774e09c reported cstate change: term changed from 0 to 1, leader changed from <none> to 24398c56cc724c91b5116396e774e09c (127.14.103.3). New cstate: current_term: 1 leader_uuid: "24398c56cc724c91b5116396e774e09c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:22.470198 15573 tablet_bootstrap.cc:654] T 844dc4a0feff4097a52882da0b69cef7 P 24398c56cc724c91b5116396e774e09c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:22.472829 15571 tablet_bootstrap.cc:492] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e: No bootstrap required, opened a new log
I20260709 13:04:22.473327 15571 ts_tablet_manager.cc:1403] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e: Time spent bootstrapping tablet: real 0.033s	user 0.018s	sys 0.012s
I20260709 13:04:22.474313 15574 tablet_bootstrap.cc:492] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0: No bootstrap required, opened a new log
I20260709 13:04:22.474798 15574 ts_tablet_manager.cc:1403] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0: Time spent bootstrapping tablet: real 0.031s	user 0.015s	sys 0.000s
I20260709 13:04:22.476264 15571 raft_consensus.cc:359] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.476999 15571 raft_consensus.cc:385] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:22.477298 15571 raft_consensus.cc:740] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df338c374ed54ecbb74bda5e628cf85e, State: Initialized, Role: FOLLOWER
I20260709 13:04:22.478091 15574 raft_consensus.cc:359] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.478035 15571 consensus_queue.cc:260] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.478849 15574 raft_consensus.cc:385] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:22.479158 15574 raft_consensus.cc:740] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ab6657817b84a1d94dba0a4eb2292d0, State: Initialized, Role: FOLLOWER
I20260709 13:04:22.479871 15581 raft_consensus.cc:493] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.479869 15574 consensus_queue.cc:260] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.480394 15581 raft_consensus.cc:515] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.484225 15581 leader_election.cc:290] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df338c374ed54ecbb74bda5e628cf85e (127.14.103.2:45591), 24398c56cc724c91b5116396e774e09c (127.14.103.3:42385)
I20260709 13:04:22.485899 15571 ts_tablet_manager.cc:1434] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:04:22.486884 15571 tablet_bootstrap.cc:492] T 32f268907bbb4d1081fa0fccb595110e P df338c374ed54ecbb74bda5e628cf85e: Bootstrap starting.
I20260709 13:04:22.494570 15571 tablet_bootstrap.cc:654] T 32f268907bbb4d1081fa0fccb595110e P df338c374ed54ecbb74bda5e628cf85e: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:22.511199 15571 tablet_bootstrap.cc:492] T 32f268907bbb4d1081fa0fccb595110e P df338c374ed54ecbb74bda5e628cf85e: No bootstrap required, opened a new log
W20260709 13:04:22.511838 15328 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:22.512024 15571 ts_tablet_manager.cc:1403] T 32f268907bbb4d1081fa0fccb595110e P df338c374ed54ecbb74bda5e628cf85e: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.011s
I20260709 13:04:22.515852 15571 raft_consensus.cc:359] T 32f268907bbb4d1081fa0fccb595110e P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.516824 15571 raft_consensus.cc:385] T 32f268907bbb4d1081fa0fccb595110e P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:22.517207 15571 raft_consensus.cc:740] T 32f268907bbb4d1081fa0fccb595110e P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df338c374ed54ecbb74bda5e628cf85e, State: Initialized, Role: FOLLOWER
I20260709 13:04:22.517706 15573 tablet_bootstrap.cc:492] T 844dc4a0feff4097a52882da0b69cef7 P 24398c56cc724c91b5116396e774e09c: No bootstrap required, opened a new log
I20260709 13:04:22.518194 15573 ts_tablet_manager.cc:1403] T 844dc4a0feff4097a52882da0b69cef7 P 24398c56cc724c91b5116396e774e09c: Time spent bootstrapping tablet: real 0.059s	user 0.015s	sys 0.011s
I20260709 13:04:22.518097 15571 consensus_queue.cc:260] T 32f268907bbb4d1081fa0fccb595110e P df338c374ed54ecbb74bda5e628cf85e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.521157 15571 ts_tablet_manager.cc:1434] T 32f268907bbb4d1081fa0fccb595110e P df338c374ed54ecbb74bda5e628cf85e: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:04:22.522559 15571 tablet_bootstrap.cc:492] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e: Bootstrap starting.
I20260709 13:04:22.524698 15573 raft_consensus.cc:359] T 844dc4a0feff4097a52882da0b69cef7 P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.525405 15573 raft_consensus.cc:385] T 844dc4a0feff4097a52882da0b69cef7 P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:22.525681 15573 raft_consensus.cc:740] T 844dc4a0feff4097a52882da0b69cef7 P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24398c56cc724c91b5116396e774e09c, State: Initialized, Role: FOLLOWER
I20260709 13:04:22.528851 15457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844dc4a0feff4097a52882da0b69cef7" candidate_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df338c374ed54ecbb74bda5e628cf85e" is_pre_election: true
I20260709 13:04:22.529462 15457 raft_consensus.cc:2468] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ab6657817b84a1d94dba0a4eb2292d0 in term 0.
I20260709 13:04:22.530814 15348 leader_election.cc:304] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ab6657817b84a1d94dba0a4eb2292d0, df338c374ed54ecbb74bda5e628cf85e; no voters: 
I20260709 13:04:22.531713 15581 raft_consensus.cc:2804] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:22.532068 15581 raft_consensus.cc:493] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.532399 15581 raft_consensus.cc:3060] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:22.534564 15571 tablet_bootstrap.cc:654] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:22.534682 15530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844dc4a0feff4097a52882da0b69cef7" candidate_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24398c56cc724c91b5116396e774e09c" is_pre_election: true
I20260709 13:04:22.537441 15574 ts_tablet_manager.cc:1434] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0: Time spent starting tablet: real 0.062s	user 0.024s	sys 0.028s
I20260709 13:04:22.538479 15581 raft_consensus.cc:515] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.539986 15574 tablet_bootstrap.cc:492] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0: Bootstrap starting.
I20260709 13:04:22.540084 15571 tablet_bootstrap.cc:492] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e: No bootstrap required, opened a new log
I20260709 13:04:22.540488 15571 ts_tablet_manager.cc:1403] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e: Time spent bootstrapping tablet: real 0.018s	user 0.011s	sys 0.003s
I20260709 13:04:22.541072 15457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844dc4a0feff4097a52882da0b69cef7" candidate_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df338c374ed54ecbb74bda5e628cf85e"
I20260709 13:04:22.541514 15457 raft_consensus.cc:3060] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:22.542953 15571 raft_consensus.cc:359] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.543638 15571 raft_consensus.cc:385] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:22.543943 15571 raft_consensus.cc:740] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df338c374ed54ecbb74bda5e628cf85e, State: Initialized, Role: FOLLOWER
I20260709 13:04:22.544409 15601 raft_consensus.cc:493] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.544703 15571 consensus_queue.cc:260] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.544838 15601 raft_consensus.cc:515] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.546446 15574 tablet_bootstrap.cc:654] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:22.546976 15571 ts_tablet_manager.cc:1434] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:04:22.547014 15601 leader_election.cc:290] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df338c374ed54ecbb74bda5e628cf85e (127.14.103.2:45591), 24398c56cc724c91b5116396e774e09c (127.14.103.3:42385)
I20260709 13:04:22.547994 15571 tablet_bootstrap.cc:492] T 3590c79d23594f91a88f9646df62c52a P df338c374ed54ecbb74bda5e628cf85e: Bootstrap starting.
I20260709 13:04:22.547868 15456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36e186173f7c42dbaa9840522867ce03" candidate_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df338c374ed54ecbb74bda5e628cf85e" is_pre_election: true
I20260709 13:04:22.548519 15456 raft_consensus.cc:2468] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ab6657817b84a1d94dba0a4eb2292d0 in term 0.
I20260709 13:04:22.548697 15531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36e186173f7c42dbaa9840522867ce03" candidate_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24398c56cc724c91b5116396e774e09c" is_pre_election: true
I20260709 13:04:22.549748 15348 leader_election.cc:304] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ab6657817b84a1d94dba0a4eb2292d0, df338c374ed54ecbb74bda5e628cf85e; no voters: 
I20260709 13:04:22.550624 15581 leader_election.cc:290] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 1 election: Requested vote from peers df338c374ed54ecbb74bda5e628cf85e (127.14.103.2:45591), 24398c56cc724c91b5116396e774e09c (127.14.103.3:42385)
I20260709 13:04:22.551067 15601 raft_consensus.cc:2804] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:22.551421 15601 raft_consensus.cc:493] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.551790 15601 raft_consensus.cc:3060] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:22.552426 15573 consensus_queue.cc:260] T 844dc4a0feff4097a52882da0b69cef7 P 24398c56cc724c91b5116396e774e09c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.553498 15530 raft_consensus.cc:2468] T 844dc4a0feff4097a52882da0b69cef7 P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ab6657817b84a1d94dba0a4eb2292d0 in term 0.
I20260709 13:04:22.554963 15571 tablet_bootstrap.cc:654] T 3590c79d23594f91a88f9646df62c52a P df338c374ed54ecbb74bda5e628cf85e: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:22.550575 15345 leader_election.cc:343] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 24398c56cc724c91b5116396e774e09c (127.14.103.3:42385): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:22.557989 15530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844dc4a0feff4097a52882da0b69cef7" candidate_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24398c56cc724c91b5116396e774e09c"
I20260709 13:04:22.558550 15530 raft_consensus.cc:3060] T 844dc4a0feff4097a52882da0b69cef7 P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:22.558291 15573 ts_tablet_manager.cc:1434] T 844dc4a0feff4097a52882da0b69cef7 P 24398c56cc724c91b5116396e774e09c: Time spent starting tablet: real 0.040s	user 0.008s	sys 0.000s
I20260709 13:04:22.562654 15573 tablet_bootstrap.cc:492] T 70da92ca35b342a7b2182eaed60e3092 P 24398c56cc724c91b5116396e774e09c: Bootstrap starting.
I20260709 13:04:22.566844 15530 raft_consensus.cc:2468] T 844dc4a0feff4097a52882da0b69cef7 P 24398c56cc724c91b5116396e774e09c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ab6657817b84a1d94dba0a4eb2292d0 in term 1.
I20260709 13:04:22.568946 15574 tablet_bootstrap.cc:492] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0: No bootstrap required, opened a new log
I20260709 13:04:22.569051 15573 tablet_bootstrap.cc:654] T 70da92ca35b342a7b2182eaed60e3092 P 24398c56cc724c91b5116396e774e09c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:22.569476 15574 ts_tablet_manager.cc:1403] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0: Time spent bootstrapping tablet: real 0.030s	user 0.007s	sys 0.009s
I20260709 13:04:22.572659 15574 raft_consensus.cc:359] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.573335 15574 raft_consensus.cc:385] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:22.573630 15574 raft_consensus.cc:740] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ab6657817b84a1d94dba0a4eb2292d0, State: Initialized, Role: FOLLOWER
I20260709 13:04:22.573885 15571 tablet_bootstrap.cc:492] T 3590c79d23594f91a88f9646df62c52a P df338c374ed54ecbb74bda5e628cf85e: No bootstrap required, opened a new log
I20260709 13:04:22.574311 15571 ts_tablet_manager.cc:1403] T 3590c79d23594f91a88f9646df62c52a P df338c374ed54ecbb74bda5e628cf85e: Time spent bootstrapping tablet: real 0.027s	user 0.006s	sys 0.010s
I20260709 13:04:22.574448 15574 consensus_queue.cc:260] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.576802 15571 raft_consensus.cc:359] T 3590c79d23594f91a88f9646df62c52a P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.577463 15571 raft_consensus.cc:385] T 3590c79d23594f91a88f9646df62c52a P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:22.577728 15571 raft_consensus.cc:740] T 3590c79d23594f91a88f9646df62c52a P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df338c374ed54ecbb74bda5e628cf85e, State: Initialized, Role: FOLLOWER
I20260709 13:04:22.577907 15574 ts_tablet_manager.cc:1434] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.001s
I20260709 13:04:22.578601 15571 consensus_queue.cc:260] T 3590c79d23594f91a88f9646df62c52a P df338c374ed54ecbb74bda5e628cf85e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.579411 15574 tablet_bootstrap.cc:492] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0: Bootstrap starting.
I20260709 13:04:22.581569 15345 leader_election.cc:304] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24398c56cc724c91b5116396e774e09c, 3ab6657817b84a1d94dba0a4eb2292d0; no voters: 
I20260709 13:04:22.585080 15571 ts_tablet_manager.cc:1434] T 3590c79d23594f91a88f9646df62c52a P df338c374ed54ecbb74bda5e628cf85e: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:04:22.576151 15601 raft_consensus.cc:515] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.586017 15571 tablet_bootstrap.cc:492] T deb69cd9c0414a348a3ee16058a36f00 P df338c374ed54ecbb74bda5e628cf85e: Bootstrap starting.
I20260709 13:04:22.586884 15601 leader_election.cc:290] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 1 election: Requested vote from peers df338c374ed54ecbb74bda5e628cf85e (127.14.103.2:45591), 24398c56cc724c91b5116396e774e09c (127.14.103.3:42385)
I20260709 13:04:22.589927 15573 tablet_bootstrap.cc:492] T 70da92ca35b342a7b2182eaed60e3092 P 24398c56cc724c91b5116396e774e09c: No bootstrap required, opened a new log
I20260709 13:04:22.590391 15573 ts_tablet_manager.cc:1403] T 70da92ca35b342a7b2182eaed60e3092 P 24398c56cc724c91b5116396e774e09c: Time spent bootstrapping tablet: real 0.028s	user 0.012s	sys 0.001s
I20260709 13:04:22.590713 15574 tablet_bootstrap.cc:654] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:22.591795 15571 tablet_bootstrap.cc:654] T deb69cd9c0414a348a3ee16058a36f00 P df338c374ed54ecbb74bda5e628cf85e: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:22.592887 15573 raft_consensus.cc:359] T 70da92ca35b342a7b2182eaed60e3092 P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.593559 15573 raft_consensus.cc:385] T 70da92ca35b342a7b2182eaed60e3092 P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:22.593822 15573 raft_consensus.cc:740] T 70da92ca35b342a7b2182eaed60e3092 P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24398c56cc724c91b5116396e774e09c, State: Initialized, Role: FOLLOWER
I20260709 13:04:22.594754 15573 consensus_queue.cc:260] T 70da92ca35b342a7b2182eaed60e3092 P 24398c56cc724c91b5116396e774e09c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.596170 15581 raft_consensus.cc:2804] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:22.596800 15581 raft_consensus.cc:697] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 LEADER]: Becoming Leader. State: Replica: 3ab6657817b84a1d94dba0a4eb2292d0, State: Running, Role: LEADER
I20260709 13:04:22.597560 15573 ts_tablet_manager.cc:1434] T 70da92ca35b342a7b2182eaed60e3092 P 24398c56cc724c91b5116396e774e09c: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:04:22.597875 15581 consensus_queue.cc:237] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.598658 15573 tablet_bootstrap.cc:492] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c: Bootstrap starting.
I20260709 13:04:22.599859 15571 tablet_bootstrap.cc:492] T deb69cd9c0414a348a3ee16058a36f00 P df338c374ed54ecbb74bda5e628cf85e: No bootstrap required, opened a new log
I20260709 13:04:22.600279 15571 ts_tablet_manager.cc:1403] T deb69cd9c0414a348a3ee16058a36f00 P df338c374ed54ecbb74bda5e628cf85e: Time spent bootstrapping tablet: real 0.014s	user 0.009s	sys 0.004s
I20260709 13:04:22.601078 15530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36e186173f7c42dbaa9840522867ce03" candidate_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24398c56cc724c91b5116396e774e09c"
I20260709 13:04:22.603098 15571 raft_consensus.cc:359] T deb69cd9c0414a348a3ee16058a36f00 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.606055 15456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36e186173f7c42dbaa9840522867ce03" candidate_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df338c374ed54ecbb74bda5e628cf85e"
I20260709 13:04:22.611745 15571 raft_consensus.cc:385] T deb69cd9c0414a348a3ee16058a36f00 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:22.612043 15456 raft_consensus.cc:3060] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:22.612057 15571 raft_consensus.cc:740] T deb69cd9c0414a348a3ee16058a36f00 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df338c374ed54ecbb74bda5e628cf85e, State: Initialized, Role: FOLLOWER
W20260709 13:04:22.612370 15345 leader_election.cc:343] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 24398c56cc724c91b5116396e774e09c (127.14.103.3:42385): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:22.612574 15573 tablet_bootstrap.cc:654] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:22.612991 15571 consensus_queue.cc:260] T deb69cd9c0414a348a3ee16058a36f00 P df338c374ed54ecbb74bda5e628cf85e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.617276 15457 raft_consensus.cc:2468] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ab6657817b84a1d94dba0a4eb2292d0 in term 1.
I20260709 13:04:22.619594 15456 raft_consensus.cc:2468] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ab6657817b84a1d94dba0a4eb2292d0 in term 1.
I20260709 13:04:22.620751 15348 leader_election.cc:304] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ab6657817b84a1d94dba0a4eb2292d0, df338c374ed54ecbb74bda5e628cf85e; no voters: 24398c56cc724c91b5116396e774e09c
I20260709 13:04:22.621237 15571 ts_tablet_manager.cc:1434] T deb69cd9c0414a348a3ee16058a36f00 P df338c374ed54ecbb74bda5e628cf85e: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.004s
I20260709 13:04:22.621747 15601 raft_consensus.cc:2804] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:22.624974 15574 tablet_bootstrap.cc:492] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0: No bootstrap required, opened a new log
I20260709 13:04:22.625526 15574 ts_tablet_manager.cc:1403] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0: Time spent bootstrapping tablet: real 0.046s	user 0.022s	sys 0.003s
I20260709 13:04:22.628582 15574 raft_consensus.cc:359] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.629465 15574 raft_consensus.cc:385] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:22.629837 15574 raft_consensus.cc:740] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ab6657817b84a1d94dba0a4eb2292d0, State: Initialized, Role: FOLLOWER
I20260709 13:04:22.630402 15573 tablet_bootstrap.cc:492] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c: No bootstrap required, opened a new log
I20260709 13:04:22.630790 15574 consensus_queue.cc:260] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.632913 15574 ts_tablet_manager.cc:1434] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:04:22.630823 15573 ts_tablet_manager.cc:1403] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c: Time spent bootstrapping tablet: real 0.032s	user 0.005s	sys 0.005s
I20260709 13:04:22.633756 15574 tablet_bootstrap.cc:492] T d3551b1f0a4e4d02af6cbe21a747f4ec P 3ab6657817b84a1d94dba0a4eb2292d0: Bootstrap starting.
I20260709 13:04:22.636451 15279 catalog_manager.cc:5697] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:22.637526 15601 raft_consensus.cc:697] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 LEADER]: Becoming Leader. State: Replica: 3ab6657817b84a1d94dba0a4eb2292d0, State: Running, Role: LEADER
I20260709 13:04:22.637378 15573 raft_consensus.cc:359] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.638053 15573 raft_consensus.cc:385] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:22.638337 15573 raft_consensus.cc:740] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24398c56cc724c91b5116396e774e09c, State: Initialized, Role: FOLLOWER
I20260709 13:04:22.638401 15601 consensus_queue.cc:237] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.639173 15573 consensus_queue.cc:260] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.640461 15574 tablet_bootstrap.cc:654] T d3551b1f0a4e4d02af6cbe21a747f4ec P 3ab6657817b84a1d94dba0a4eb2292d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:22.650967 15573 ts_tablet_manager.cc:1434] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.000s
I20260709 13:04:22.652295 15589 raft_consensus.cc:493] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.652908 15589 raft_consensus.cc:515] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.657136 15530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70da92ca35b342a7b2182eaed60e3092" candidate_uuid: "df338c374ed54ecbb74bda5e628cf85e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24398c56cc724c91b5116396e774e09c" is_pre_election: true
I20260709 13:04:22.657621 15381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70da92ca35b342a7b2182eaed60e3092" candidate_uuid: "df338c374ed54ecbb74bda5e628cf85e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" is_pre_election: true
I20260709 13:04:22.657970 15530 raft_consensus.cc:2468] T 70da92ca35b342a7b2182eaed60e3092 P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df338c374ed54ecbb74bda5e628cf85e in term 0.
I20260709 13:04:22.658443 15381 raft_consensus.cc:2468] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df338c374ed54ecbb74bda5e628cf85e in term 0.
I20260709 13:04:22.659735 15420 leader_election.cc:304] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24398c56cc724c91b5116396e774e09c, df338c374ed54ecbb74bda5e628cf85e; no voters: 
I20260709 13:04:22.660845 15575 raft_consensus.cc:2804] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:22.661278 15575 raft_consensus.cc:493] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.661710 15575 raft_consensus.cc:3060] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:22.666476 15577 raft_consensus.cc:493] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.667097 15577 raft_consensus.cc:515] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.671962 15575 raft_consensus.cc:515] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.674093 15456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3551b1f0a4e4d02af6cbe21a747f4ec" candidate_uuid: "24398c56cc724c91b5116396e774e09c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df338c374ed54ecbb74bda5e628cf85e" is_pre_election: true
I20260709 13:04:22.674913 15456 raft_consensus.cc:2468] T d3551b1f0a4e4d02af6cbe21a747f4ec P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24398c56cc724c91b5116396e774e09c in term 0.
I20260709 13:04:22.676764 15530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70da92ca35b342a7b2182eaed60e3092" candidate_uuid: "df338c374ed54ecbb74bda5e628cf85e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24398c56cc724c91b5116396e774e09c"
I20260709 13:04:22.677561 15530 raft_consensus.cc:3060] T 70da92ca35b342a7b2182eaed60e3092 P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:22.678323 15382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70da92ca35b342a7b2182eaed60e3092" candidate_uuid: "df338c374ed54ecbb74bda5e628cf85e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
I20260709 13:04:22.679100 15382 raft_consensus.cc:3060] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:22.676690 15497 leader_election.cc:304] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24398c56cc724c91b5116396e774e09c, df338c374ed54ecbb74bda5e628cf85e; no voters: 
I20260709 13:04:22.684329 15584 raft_consensus.cc:2804] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:22.684792 15584 raft_consensus.cc:493] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.685209 15584 raft_consensus.cc:3060] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:22.686335 15530 raft_consensus.cc:2468] T 70da92ca35b342a7b2182eaed60e3092 P 24398c56cc724c91b5116396e774e09c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df338c374ed54ecbb74bda5e628cf85e in term 1.
I20260709 13:04:22.687847 15420 leader_election.cc:304] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24398c56cc724c91b5116396e774e09c, df338c374ed54ecbb74bda5e628cf85e; no voters: 
I20260709 13:04:22.688261 15381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3551b1f0a4e4d02af6cbe21a747f4ec" candidate_uuid: "24398c56cc724c91b5116396e774e09c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" is_pre_election: true
W20260709 13:04:22.691043 15497 leader_election.cc:343] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:22.691876 15589 leader_election.cc:290] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793), 24398c56cc724c91b5116396e774e09c (127.14.103.3:42385)
I20260709 13:04:22.692852 15589 raft_consensus.cc:2804] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:22.693112 15575 leader_election.cc:290] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e [CANDIDATE]: Term 1 election: Requested vote from peers 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793), 24398c56cc724c91b5116396e774e09c (127.14.103.3:42385)
I20260709 13:04:22.699038 15577 leader_election.cc:290] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793), df338c374ed54ecbb74bda5e628cf85e (127.14.103.2:45591)
I20260709 13:04:22.693504 15589 raft_consensus.cc:697] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e [term 1 LEADER]: Becoming Leader. State: Replica: df338c374ed54ecbb74bda5e628cf85e, State: Running, Role: LEADER
I20260709 13:04:22.701326 15589 consensus_queue.cc:237] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.710757 15382 raft_consensus.cc:2468] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df338c374ed54ecbb74bda5e628cf85e in term 1.
I20260709 13:04:22.710093 15584 raft_consensus.cc:515] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.711506 15573 tablet_bootstrap.cc:492] T 36e186173f7c42dbaa9840522867ce03 P 24398c56cc724c91b5116396e774e09c: Bootstrap starting.
I20260709 13:04:22.713384 15584 leader_election.cc:290] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 1 election: Requested vote from peers 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793), df338c374ed54ecbb74bda5e628cf85e (127.14.103.2:45591)
I20260709 13:04:22.715199 15456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3551b1f0a4e4d02af6cbe21a747f4ec" candidate_uuid: "24398c56cc724c91b5116396e774e09c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df338c374ed54ecbb74bda5e628cf85e"
I20260709 13:04:22.715950 15456 raft_consensus.cc:3060] T d3551b1f0a4e4d02af6cbe21a747f4ec P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:22.720647 15573 tablet_bootstrap.cc:654] T 36e186173f7c42dbaa9840522867ce03 P 24398c56cc724c91b5116396e774e09c: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:22.721211 15382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3551b1f0a4e4d02af6cbe21a747f4ec" candidate_uuid: "24398c56cc724c91b5116396e774e09c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
W20260709 13:04:22.724257 15497 leader_election.cc:343] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:22.727375 15456 raft_consensus.cc:2468] T d3551b1f0a4e4d02af6cbe21a747f4ec P df338c374ed54ecbb74bda5e628cf85e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24398c56cc724c91b5116396e774e09c in term 1.
I20260709 13:04:22.729946 15601 raft_consensus.cc:493] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.730566 15601 raft_consensus.cc:515] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.733223 15601 leader_election.cc:290] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df338c374ed54ecbb74bda5e628cf85e (127.14.103.2:45591), 24398c56cc724c91b5116396e774e09c (127.14.103.3:42385)
I20260709 13:04:22.735286 15457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32f268907bbb4d1081fa0fccb595110e" candidate_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df338c374ed54ecbb74bda5e628cf85e" is_pre_election: true
I20260709 13:04:22.736111 15457 raft_consensus.cc:2468] T 32f268907bbb4d1081fa0fccb595110e P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ab6657817b84a1d94dba0a4eb2292d0 in term 0.
I20260709 13:04:22.738539 15497 leader_election.cc:304] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24398c56cc724c91b5116396e774e09c, df338c374ed54ecbb74bda5e628cf85e; no voters: 3ab6657817b84a1d94dba0a4eb2292d0
I20260709 13:04:22.739708 15584 raft_consensus.cc:2804] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:22.739629 15348 leader_election.cc:304] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ab6657817b84a1d94dba0a4eb2292d0, df338c374ed54ecbb74bda5e628cf85e; no voters: 
I20260709 13:04:22.739979 15589 raft_consensus.cc:493] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.740242 15584 raft_consensus.cc:697] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [term 1 LEADER]: Becoming Leader. State: Replica: 24398c56cc724c91b5116396e774e09c, State: Running, Role: LEADER
I20260709 13:04:22.740453 15589 raft_consensus.cc:515] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.741051 15574 tablet_bootstrap.cc:492] T d3551b1f0a4e4d02af6cbe21a747f4ec P 3ab6657817b84a1d94dba0a4eb2292d0: No bootstrap required, opened a new log
I20260709 13:04:22.738082 15530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32f268907bbb4d1081fa0fccb595110e" candidate_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24398c56cc724c91b5116396e774e09c" is_pre_election: true
I20260709 13:04:22.741654 15530 raft_consensus.cc:2468] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ab6657817b84a1d94dba0a4eb2292d0 in term 0.
I20260709 13:04:22.741854 15601 raft_consensus.cc:2804] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:22.742241 15601 raft_consensus.cc:493] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.741638 15574 ts_tablet_manager.cc:1403] T d3551b1f0a4e4d02af6cbe21a747f4ec P 3ab6657817b84a1d94dba0a4eb2292d0: Time spent bootstrapping tablet: real 0.108s	user 0.031s	sys 0.071s
I20260709 13:04:22.742630 15601 raft_consensus.cc:3060] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:22.743261 15589 leader_election.cc:290] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793), 24398c56cc724c91b5116396e774e09c (127.14.103.3:42385)
I20260709 13:04:22.743721 15530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b46d829f140440738e6d2d456f792c75" candidate_uuid: "df338c374ed54ecbb74bda5e628cf85e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24398c56cc724c91b5116396e774e09c" is_pre_election: true
I20260709 13:04:22.744155 15530 raft_consensus.cc:2468] T b46d829f140440738e6d2d456f792c75 P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df338c374ed54ecbb74bda5e628cf85e in term 0.
I20260709 13:04:22.743556 15278 catalog_manager.cc:5697] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:22.745100 15420 leader_election.cc:304] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24398c56cc724c91b5116396e774e09c, df338c374ed54ecbb74bda5e628cf85e; no voters: 
I20260709 13:04:22.745887 15589 raft_consensus.cc:2804] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:22.746208 15589 raft_consensus.cc:493] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.746563 15589 raft_consensus.cc:3060] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:22.741076 15584 consensus_queue.cc:237] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.746860 15574 raft_consensus.cc:359] T d3551b1f0a4e4d02af6cbe21a747f4ec P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.747534 15574 raft_consensus.cc:385] T d3551b1f0a4e4d02af6cbe21a747f4ec P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:22.747893 15574 raft_consensus.cc:740] T d3551b1f0a4e4d02af6cbe21a747f4ec P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ab6657817b84a1d94dba0a4eb2292d0, State: Initialized, Role: FOLLOWER
I20260709 13:04:22.748633 15574 consensus_queue.cc:260] T d3551b1f0a4e4d02af6cbe21a747f4ec P 3ab6657817b84a1d94dba0a4eb2292d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.749043 15382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b46d829f140440738e6d2d456f792c75" candidate_uuid: "df338c374ed54ecbb74bda5e628cf85e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" is_pre_election: true
W20260709 13:04:22.750447 15423 leader_election.cc:343] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:22.751153 15574 ts_tablet_manager.cc:1434] T d3551b1f0a4e4d02af6cbe21a747f4ec P 3ab6657817b84a1d94dba0a4eb2292d0: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:04:22.752174 15574 tablet_bootstrap.cc:492] T 3590c79d23594f91a88f9646df62c52a P 3ab6657817b84a1d94dba0a4eb2292d0: Bootstrap starting.
I20260709 13:04:22.753654 15589 raft_consensus.cc:515] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.756534 15382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b46d829f140440738e6d2d456f792c75" candidate_uuid: "df338c374ed54ecbb74bda5e628cf85e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
I20260709 13:04:22.756795 15530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b46d829f140440738e6d2d456f792c75" candidate_uuid: "df338c374ed54ecbb74bda5e628cf85e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24398c56cc724c91b5116396e774e09c"
I20260709 13:04:22.757350 15530 raft_consensus.cc:3060] T b46d829f140440738e6d2d456f792c75 P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:22.757859 15423 leader_election.cc:343] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:22.758549 15574 tablet_bootstrap.cc:654] T 3590c79d23594f91a88f9646df62c52a P 3ab6657817b84a1d94dba0a4eb2292d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:22.759088 15601 raft_consensus.cc:515] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.760816 15279 catalog_manager.cc:5697] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e reported cstate change: term changed from 0 to 1, leader changed from <none> to df338c374ed54ecbb74bda5e628cf85e (127.14.103.2). New cstate: current_term: 1 leader_uuid: "df338c374ed54ecbb74bda5e628cf85e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:22.761831 15457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32f268907bbb4d1081fa0fccb595110e" candidate_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df338c374ed54ecbb74bda5e628cf85e"
I20260709 13:04:22.762364 15457 raft_consensus.cc:3060] T 32f268907bbb4d1081fa0fccb595110e P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:22.765291 15530 raft_consensus.cc:2468] T b46d829f140440738e6d2d456f792c75 P 24398c56cc724c91b5116396e774e09c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df338c374ed54ecbb74bda5e628cf85e in term 1.
I20260709 13:04:22.767408 15420 leader_election.cc:304] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24398c56cc724c91b5116396e774e09c, df338c374ed54ecbb74bda5e628cf85e; no voters: 3ab6657817b84a1d94dba0a4eb2292d0
I20260709 13:04:22.768659 15618 raft_consensus.cc:2804] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:22.768862 15457 raft_consensus.cc:2468] T 32f268907bbb4d1081fa0fccb595110e P df338c374ed54ecbb74bda5e628cf85e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ab6657817b84a1d94dba0a4eb2292d0 in term 1.
I20260709 13:04:22.769295 15618 raft_consensus.cc:697] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e [term 1 LEADER]: Becoming Leader. State: Replica: df338c374ed54ecbb74bda5e628cf85e, State: Running, Role: LEADER
I20260709 13:04:22.770047 15348 leader_election.cc:304] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ab6657817b84a1d94dba0a4eb2292d0, df338c374ed54ecbb74bda5e628cf85e; no voters: 
I20260709 13:04:22.771049 15615 raft_consensus.cc:2804] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:22.770452 15618 consensus_queue.cc:237] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.771528 15615 raft_consensus.cc:697] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 LEADER]: Becoming Leader. State: Replica: 3ab6657817b84a1d94dba0a4eb2292d0, State: Running, Role: LEADER
I20260709 13:04:22.772441 15615 consensus_queue.cc:237] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.765930 15589 leader_election.cc:290] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e [CANDIDATE]: Term 1 election: Requested vote from peers 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793), 24398c56cc724c91b5116396e774e09c (127.14.103.3:42385)
I20260709 13:04:22.778371 15573 tablet_bootstrap.cc:492] T 36e186173f7c42dbaa9840522867ce03 P 24398c56cc724c91b5116396e774e09c: No bootstrap required, opened a new log
I20260709 13:04:22.780916 15601 leader_election.cc:290] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 1 election: Requested vote from peers df338c374ed54ecbb74bda5e628cf85e (127.14.103.2:45591), 24398c56cc724c91b5116396e774e09c (127.14.103.3:42385)
I20260709 13:04:22.781265 15573 ts_tablet_manager.cc:1403] T 36e186173f7c42dbaa9840522867ce03 P 24398c56cc724c91b5116396e774e09c: Time spent bootstrapping tablet: real 0.070s	user 0.019s	sys 0.004s
I20260709 13:04:22.780952 15530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32f268907bbb4d1081fa0fccb595110e" candidate_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24398c56cc724c91b5116396e774e09c"
I20260709 13:04:22.782477 15530 raft_consensus.cc:3060] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:22.783427 15577 raft_consensus.cc:493] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.784024 15577 raft_consensus.cc:515] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.786514 15577 leader_election.cc:290] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793), df338c374ed54ecbb74bda5e628cf85e (127.14.103.2:45591)
I20260709 13:04:22.787295 15573 raft_consensus.cc:359] T 36e186173f7c42dbaa9840522867ce03 P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.787961 15573 raft_consensus.cc:385] T 36e186173f7c42dbaa9840522867ce03 P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:22.788213 15573 raft_consensus.cc:740] T 36e186173f7c42dbaa9840522867ce03 P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24398c56cc724c91b5116396e774e09c, State: Initialized, Role: FOLLOWER
I20260709 13:04:22.789000 15573 consensus_queue.cc:260] T 36e186173f7c42dbaa9840522867ce03 P 24398c56cc724c91b5116396e774e09c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.789400 15382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3590c79d23594f91a88f9646df62c52a" candidate_uuid: "24398c56cc724c91b5116396e774e09c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" is_pre_election: true
W20260709 13:04:22.790901 15497 leader_election.cc:343] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:22.793195 15457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3590c79d23594f91a88f9646df62c52a" candidate_uuid: "24398c56cc724c91b5116396e774e09c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df338c374ed54ecbb74bda5e628cf85e" is_pre_election: true
I20260709 13:04:22.793831 15457 raft_consensus.cc:2468] T 3590c79d23594f91a88f9646df62c52a P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24398c56cc724c91b5116396e774e09c in term 0.
I20260709 13:04:22.795146 15497 leader_election.cc:304] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24398c56cc724c91b5116396e774e09c, df338c374ed54ecbb74bda5e628cf85e; no voters: 3ab6657817b84a1d94dba0a4eb2292d0
I20260709 13:04:22.796502 15577 raft_consensus.cc:2804] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:22.796926 15577 raft_consensus.cc:493] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.797303 15577 raft_consensus.cc:3060] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:22.797677 15279 catalog_manager.cc:5697] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e reported cstate change: term changed from 0 to 1, leader changed from <none> to df338c374ed54ecbb74bda5e628cf85e (127.14.103.2). New cstate: current_term: 1 leader_uuid: "df338c374ed54ecbb74bda5e628cf85e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:22.804044 15278 catalog_manager.cc:5697] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c reported cstate change: term changed from 0 to 1, leader changed from <none> to 24398c56cc724c91b5116396e774e09c (127.14.103.3). New cstate: current_term: 1 leader_uuid: "24398c56cc724c91b5116396e774e09c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:22.807201 15577 raft_consensus.cc:515] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.810668 15573 ts_tablet_manager.cc:1434] T 36e186173f7c42dbaa9840522867ce03 P 24398c56cc724c91b5116396e774e09c: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.000s
I20260709 13:04:22.810619 15382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3590c79d23594f91a88f9646df62c52a" candidate_uuid: "24398c56cc724c91b5116396e774e09c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
I20260709 13:04:22.813503 15574 tablet_bootstrap.cc:492] T 3590c79d23594f91a88f9646df62c52a P 3ab6657817b84a1d94dba0a4eb2292d0: No bootstrap required, opened a new log
W20260709 13:04:22.813417 15497 leader_election.cc:343] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:22.814028 15574 ts_tablet_manager.cc:1403] T 3590c79d23594f91a88f9646df62c52a P 3ab6657817b84a1d94dba0a4eb2292d0: Time spent bootstrapping tablet: real 0.062s	user 0.012s	sys 0.038s
I20260709 13:04:22.815083 15530 raft_consensus.cc:2468] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ab6657817b84a1d94dba0a4eb2292d0 in term 1.
I20260709 13:04:22.816756 15574 raft_consensus.cc:359] T 3590c79d23594f91a88f9646df62c52a P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.817245 15457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3590c79d23594f91a88f9646df62c52a" candidate_uuid: "24398c56cc724c91b5116396e774e09c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df338c374ed54ecbb74bda5e628cf85e"
I20260709 13:04:22.817458 15574 raft_consensus.cc:385] T 3590c79d23594f91a88f9646df62c52a P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:22.817749 15574 raft_consensus.cc:740] T 3590c79d23594f91a88f9646df62c52a P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ab6657817b84a1d94dba0a4eb2292d0, State: Initialized, Role: FOLLOWER
I20260709 13:04:22.817875 15577 leader_election.cc:290] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 1 election: Requested vote from peers 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793), df338c374ed54ecbb74bda5e628cf85e (127.14.103.2:45591)
I20260709 13:04:22.817744 15280 catalog_manager.cc:5697] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:22.818708 15574 consensus_queue.cc:260] T 3590c79d23594f91a88f9646df62c52a P 3ab6657817b84a1d94dba0a4eb2292d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.820964 15574 ts_tablet_manager.cc:1434] T 3590c79d23594f91a88f9646df62c52a P 3ab6657817b84a1d94dba0a4eb2292d0: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260709 13:04:22.817780 15457 raft_consensus.cc:3060] T 3590c79d23594f91a88f9646df62c52a P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:22.821938 15574 tablet_bootstrap.cc:492] T e7be94629c3c46638abeb602169bcda4 P 3ab6657817b84a1d94dba0a4eb2292d0: Bootstrap starting.
I20260709 13:04:22.828187 15574 tablet_bootstrap.cc:654] T e7be94629c3c46638abeb602169bcda4 P 3ab6657817b84a1d94dba0a4eb2292d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:22.830971 15457 raft_consensus.cc:2468] T 3590c79d23594f91a88f9646df62c52a P df338c374ed54ecbb74bda5e628cf85e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24398c56cc724c91b5116396e774e09c in term 1.
I20260709 13:04:22.832105 15497 leader_election.cc:304] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24398c56cc724c91b5116396e774e09c, df338c374ed54ecbb74bda5e628cf85e; no voters: 3ab6657817b84a1d94dba0a4eb2292d0
I20260709 13:04:22.832860 15577 raft_consensus.cc:2804] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:22.833256 15577 raft_consensus.cc:697] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c [term 1 LEADER]: Becoming Leader. State: Replica: 24398c56cc724c91b5116396e774e09c, State: Running, Role: LEADER
I20260709 13:04:22.833988 15577 consensus_queue.cc:237] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.835075 15574 tablet_bootstrap.cc:492] T e7be94629c3c46638abeb602169bcda4 P 3ab6657817b84a1d94dba0a4eb2292d0: No bootstrap required, opened a new log
I20260709 13:04:22.835438 15574 ts_tablet_manager.cc:1403] T e7be94629c3c46638abeb602169bcda4 P 3ab6657817b84a1d94dba0a4eb2292d0: Time spent bootstrapping tablet: real 0.014s	user 0.008s	sys 0.001s
I20260709 13:04:22.837318 15574 raft_consensus.cc:359] T e7be94629c3c46638abeb602169bcda4 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } } peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } }
I20260709 13:04:22.838013 15574 raft_consensus.cc:385] T e7be94629c3c46638abeb602169bcda4 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:22.838270 15574 raft_consensus.cc:740] T e7be94629c3c46638abeb602169bcda4 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ab6657817b84a1d94dba0a4eb2292d0, State: Initialized, Role: FOLLOWER
I20260709 13:04:22.838927 15574 consensus_queue.cc:260] T e7be94629c3c46638abeb602169bcda4 P 3ab6657817b84a1d94dba0a4eb2292d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } } peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } }
I20260709 13:04:22.840799 15574 ts_tablet_manager.cc:1434] T e7be94629c3c46638abeb602169bcda4 P 3ab6657817b84a1d94dba0a4eb2292d0: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:04:22.841658 15574 tablet_bootstrap.cc:492] T deb69cd9c0414a348a3ee16058a36f00 P 3ab6657817b84a1d94dba0a4eb2292d0: Bootstrap starting.
I20260709 13:04:22.841882 15280 catalog_manager.cc:5697] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c reported cstate change: term changed from 0 to 1, leader changed from <none> to 24398c56cc724c91b5116396e774e09c (127.14.103.3). New cstate: current_term: 1 leader_uuid: "24398c56cc724c91b5116396e774e09c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:22.845577 15574 tablet_bootstrap.cc:654] T deb69cd9c0414a348a3ee16058a36f00 P 3ab6657817b84a1d94dba0a4eb2292d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:22.851322 15574 tablet_bootstrap.cc:492] T deb69cd9c0414a348a3ee16058a36f00 P 3ab6657817b84a1d94dba0a4eb2292d0: No bootstrap required, opened a new log
I20260709 13:04:22.851778 15574 ts_tablet_manager.cc:1403] T deb69cd9c0414a348a3ee16058a36f00 P 3ab6657817b84a1d94dba0a4eb2292d0: Time spent bootstrapping tablet: real 0.010s	user 0.006s	sys 0.003s
I20260709 13:04:22.853765 15574 raft_consensus.cc:359] T deb69cd9c0414a348a3ee16058a36f00 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.854171 15574 raft_consensus.cc:385] T deb69cd9c0414a348a3ee16058a36f00 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:22.854328 15574 raft_consensus.cc:740] T deb69cd9c0414a348a3ee16058a36f00 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ab6657817b84a1d94dba0a4eb2292d0, State: Initialized, Role: FOLLOWER
I20260709 13:04:22.854755 15574 consensus_queue.cc:260] T deb69cd9c0414a348a3ee16058a36f00 P 3ab6657817b84a1d94dba0a4eb2292d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.856179 15574 ts_tablet_manager.cc:1434] T deb69cd9c0414a348a3ee16058a36f00 P 3ab6657817b84a1d94dba0a4eb2292d0: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:04:22.856858 15574 tablet_bootstrap.cc:492] T b46d829f140440738e6d2d456f792c75 P 3ab6657817b84a1d94dba0a4eb2292d0: Bootstrap starting.
I20260709 13:04:22.862030 15574 tablet_bootstrap.cc:654] T b46d829f140440738e6d2d456f792c75 P 3ab6657817b84a1d94dba0a4eb2292d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:22.865914 15574 tablet_bootstrap.cc:492] T b46d829f140440738e6d2d456f792c75 P 3ab6657817b84a1d94dba0a4eb2292d0: No bootstrap required, opened a new log
I20260709 13:04:22.866326 15574 ts_tablet_manager.cc:1403] T b46d829f140440738e6d2d456f792c75 P 3ab6657817b84a1d94dba0a4eb2292d0: Time spent bootstrapping tablet: real 0.010s	user 0.000s	sys 0.007s
I20260709 13:04:22.868407 15574 raft_consensus.cc:359] T b46d829f140440738e6d2d456f792c75 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.868829 15574 raft_consensus.cc:385] T b46d829f140440738e6d2d456f792c75 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:22.868986 15574 raft_consensus.cc:740] T b46d829f140440738e6d2d456f792c75 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ab6657817b84a1d94dba0a4eb2292d0, State: Initialized, Role: FOLLOWER
I20260709 13:04:22.869427 15574 consensus_queue.cc:260] T b46d829f140440738e6d2d456f792c75 P 3ab6657817b84a1d94dba0a4eb2292d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.870810 15574 ts_tablet_manager.cc:1434] T b46d829f140440738e6d2d456f792c75 P 3ab6657817b84a1d94dba0a4eb2292d0: Time spent starting tablet: real 0.004s	user 0.003s	sys 0.000s
I20260709 13:04:22.884784 15618 raft_consensus.cc:493] T 42e07a8df5ac47eea3da5846963de3fa P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.885183 15618 raft_consensus.cc:515] T 42e07a8df5ac47eea3da5846963de3fa P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.886644 15618 leader_election.cc:290] T 42e07a8df5ac47eea3da5846963de3fa P df338c374ed54ecbb74bda5e628cf85e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793), 24398c56cc724c91b5116396e774e09c (127.14.103.3:42385)
I20260709 13:04:22.887379 15382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42e07a8df5ac47eea3da5846963de3fa" candidate_uuid: "df338c374ed54ecbb74bda5e628cf85e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" is_pre_election: true
I20260709 13:04:22.887616 15530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42e07a8df5ac47eea3da5846963de3fa" candidate_uuid: "df338c374ed54ecbb74bda5e628cf85e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24398c56cc724c91b5116396e774e09c" is_pre_election: true
I20260709 13:04:22.887904 15382 raft_consensus.cc:2468] T 42e07a8df5ac47eea3da5846963de3fa P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df338c374ed54ecbb74bda5e628cf85e in term 0.
I20260709 13:04:22.888207 15530 raft_consensus.cc:2468] T 42e07a8df5ac47eea3da5846963de3fa P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df338c374ed54ecbb74bda5e628cf85e in term 0.
I20260709 13:04:22.888844 15423 leader_election.cc:304] T 42e07a8df5ac47eea3da5846963de3fa P df338c374ed54ecbb74bda5e628cf85e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ab6657817b84a1d94dba0a4eb2292d0, df338c374ed54ecbb74bda5e628cf85e; no voters: 
I20260709 13:04:22.889673 15618 raft_consensus.cc:2804] T 42e07a8df5ac47eea3da5846963de3fa P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:22.890013 15618 raft_consensus.cc:493] T 42e07a8df5ac47eea3da5846963de3fa P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.890231 15618 raft_consensus.cc:3060] T 42e07a8df5ac47eea3da5846963de3fa P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:22.894634 15618 raft_consensus.cc:515] T 42e07a8df5ac47eea3da5846963de3fa P df338c374ed54ecbb74bda5e628cf85e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.896636 15618 leader_election.cc:290] T 42e07a8df5ac47eea3da5846963de3fa P df338c374ed54ecbb74bda5e628cf85e [CANDIDATE]: Term 1 election: Requested vote from peers 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793), 24398c56cc724c91b5116396e774e09c (127.14.103.3:42385)
I20260709 13:04:22.897285 15382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42e07a8df5ac47eea3da5846963de3fa" candidate_uuid: "df338c374ed54ecbb74bda5e628cf85e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
I20260709 13:04:22.897786 15382 raft_consensus.cc:3060] T 42e07a8df5ac47eea3da5846963de3fa P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:22.897579 15530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42e07a8df5ac47eea3da5846963de3fa" candidate_uuid: "df338c374ed54ecbb74bda5e628cf85e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24398c56cc724c91b5116396e774e09c"
I20260709 13:04:22.898288 15530 raft_consensus.cc:3060] T 42e07a8df5ac47eea3da5846963de3fa P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:22.904551 15382 raft_consensus.cc:2468] T 42e07a8df5ac47eea3da5846963de3fa P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df338c374ed54ecbb74bda5e628cf85e in term 1.
I20260709 13:04:22.904745 15530 raft_consensus.cc:2468] T 42e07a8df5ac47eea3da5846963de3fa P 24398c56cc724c91b5116396e774e09c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df338c374ed54ecbb74bda5e628cf85e in term 1.
I20260709 13:04:22.905712 15423 leader_election.cc:304] T 42e07a8df5ac47eea3da5846963de3fa P df338c374ed54ecbb74bda5e628cf85e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ab6657817b84a1d94dba0a4eb2292d0, df338c374ed54ecbb74bda5e628cf85e; no voters: 
I20260709 13:04:22.906342 15618 raft_consensus.cc:2804] T 42e07a8df5ac47eea3da5846963de3fa P df338c374ed54ecbb74bda5e628cf85e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:22.906764 15618 raft_consensus.cc:697] T 42e07a8df5ac47eea3da5846963de3fa P df338c374ed54ecbb74bda5e628cf85e [term 1 LEADER]: Becoming Leader. State: Replica: df338c374ed54ecbb74bda5e628cf85e, State: Running, Role: LEADER
I20260709 13:04:22.907444 15618 consensus_queue.cc:237] T 42e07a8df5ac47eea3da5846963de3fa P df338c374ed54ecbb74bda5e628cf85e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:22.912518 15382 raft_consensus.cc:3060] T e7be94629c3c46638abeb602169bcda4 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:22.915288 15278 catalog_manager.cc:5697] T 42e07a8df5ac47eea3da5846963de3fa P df338c374ed54ecbb74bda5e628cf85e reported cstate change: term changed from 0 to 1, leader changed from <none> to df338c374ed54ecbb74bda5e628cf85e (127.14.103.2). New cstate: current_term: 1 leader_uuid: "df338c374ed54ecbb74bda5e628cf85e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:22.919226 15615 raft_consensus.cc:493] T e7be94629c3c46638abeb602169bcda4 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.919723 15615 raft_consensus.cc:515] T e7be94629c3c46638abeb602169bcda4 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } } peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } }
I20260709 13:04:22.920106 15577 consensus_queue.cc:1048] T e7be94629c3c46638abeb602169bcda4 P 24398c56cc724c91b5116396e774e09c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:22.923221 15615 leader_election.cc:290] T e7be94629c3c46638abeb602169bcda4 P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers df338c374ed54ecbb74bda5e628cf85e (127.14.103.2:45591), 24398c56cc724c91b5116396e774e09c (127.14.103.3:42385)
I20260709 13:04:22.924636 15457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7be94629c3c46638abeb602169bcda4" candidate_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df338c374ed54ecbb74bda5e628cf85e" is_pre_election: true
I20260709 13:04:22.924636 15530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7be94629c3c46638abeb602169bcda4" candidate_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24398c56cc724c91b5116396e774e09c" is_pre_election: true
I20260709 13:04:22.925369 15457 raft_consensus.cc:2468] T e7be94629c3c46638abeb602169bcda4 P df338c374ed54ecbb74bda5e628cf85e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ab6657817b84a1d94dba0a4eb2292d0 in term 1.
I20260709 13:04:22.926484 15348 leader_election.cc:304] T e7be94629c3c46638abeb602169bcda4 P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ab6657817b84a1d94dba0a4eb2292d0, df338c374ed54ecbb74bda5e628cf85e; no voters: 24398c56cc724c91b5116396e774e09c
I20260709 13:04:22.927544 15615 raft_consensus.cc:2804] T e7be94629c3c46638abeb602169bcda4 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:22.927922 15615 raft_consensus.cc:493] T e7be94629c3c46638abeb602169bcda4 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 24398c56cc724c91b5116396e774e09c)
I20260709 13:04:22.928287 15615 raft_consensus.cc:3060] T e7be94629c3c46638abeb602169bcda4 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:22.939792 15595 consensus_queue.cc:1048] T e7be94629c3c46638abeb602169bcda4 P 24398c56cc724c91b5116396e774e09c [LEADER]: Connected to new peer: Peer: permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:22.942811 15615 raft_consensus.cc:515] T e7be94629c3c46638abeb602169bcda4 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } } peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } }
I20260709 13:04:22.943938 15382 raft_consensus.cc:1240] T e7be94629c3c46638abeb602169bcda4 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 FOLLOWER]: Rejecting Update request from peer 24398c56cc724c91b5116396e774e09c for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:22.945914 15456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7be94629c3c46638abeb602169bcda4" candidate_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "df338c374ed54ecbb74bda5e628cf85e"
I20260709 13:04:22.947834 15595 consensus_queue.cc:1059] T e7be94629c3c46638abeb602169bcda4 P 24398c56cc724c91b5116396e774e09c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:22.948616 15595 raft_consensus.cc:3055] T e7be94629c3c46638abeb602169bcda4 P 24398c56cc724c91b5116396e774e09c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:22.948939 15595 raft_consensus.cc:740] T e7be94629c3c46638abeb602169bcda4 P 24398c56cc724c91b5116396e774e09c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 24398c56cc724c91b5116396e774e09c, State: Running, Role: LEADER
I20260709 13:04:22.949326 15615 leader_election.cc:290] T e7be94629c3c46638abeb602169bcda4 P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 2 election: Requested vote from peers df338c374ed54ecbb74bda5e628cf85e (127.14.103.2:45591), 24398c56cc724c91b5116396e774e09c (127.14.103.3:42385)
I20260709 13:04:22.949880 15530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7be94629c3c46638abeb602169bcda4" candidate_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "24398c56cc724c91b5116396e774e09c"
I20260709 13:04:22.949713 15595 consensus_queue.cc:260] T e7be94629c3c46638abeb602169bcda4 P 24398c56cc724c91b5116396e774e09c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } } peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } }
I20260709 13:04:22.950850 15595 raft_consensus.cc:3060] T e7be94629c3c46638abeb602169bcda4 P 24398c56cc724c91b5116396e774e09c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:22.962442 15530 raft_consensus.cc:2468] T e7be94629c3c46638abeb602169bcda4 P 24398c56cc724c91b5116396e774e09c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ab6657817b84a1d94dba0a4eb2292d0 in term 2.
I20260709 13:04:22.963667 15345 leader_election.cc:304] T e7be94629c3c46638abeb602169bcda4 P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24398c56cc724c91b5116396e774e09c, 3ab6657817b84a1d94dba0a4eb2292d0; no voters: df338c374ed54ecbb74bda5e628cf85e
I20260709 13:04:22.964341 15615 raft_consensus.cc:2804] T e7be94629c3c46638abeb602169bcda4 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:22.964754 15615 raft_consensus.cc:697] T e7be94629c3c46638abeb602169bcda4 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 LEADER]: Becoming Leader. State: Replica: 3ab6657817b84a1d94dba0a4eb2292d0, State: Running, Role: LEADER
I20260709 13:04:22.965473 15615 consensus_queue.cc:237] T e7be94629c3c46638abeb602169bcda4 P 3ab6657817b84a1d94dba0a4eb2292d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } } peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } }
I20260709 13:04:22.971136 15278 catalog_manager.cc:5697] T e7be94629c3c46638abeb602169bcda4 P 3ab6657817b84a1d94dba0a4eb2292d0 reported cstate change: term changed from 1 to 2, leader changed from 24398c56cc724c91b5116396e774e09c (127.14.103.3) to 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:23.022902 15615 consensus_queue.cc:1048] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:23.033744 15581 consensus_queue.cc:1048] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:23.075192 15530 raft_consensus.cc:3060] T 36e186173f7c42dbaa9840522867ce03 P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:23.080956 15581 consensus_queue.cc:1048] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:23.091113 15615 consensus_queue.cc:1048] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:23.094810 15618 raft_consensus.cc:493] T deb69cd9c0414a348a3ee16058a36f00 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.095440 15618 raft_consensus.cc:515] T deb69cd9c0414a348a3ee16058a36f00 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:23.098351 15382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deb69cd9c0414a348a3ee16058a36f00" candidate_uuid: "df338c374ed54ecbb74bda5e628cf85e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" is_pre_election: true
I20260709 13:04:23.099020 15382 raft_consensus.cc:2468] T deb69cd9c0414a348a3ee16058a36f00 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df338c374ed54ecbb74bda5e628cf85e in term 0.
I20260709 13:04:23.099411 15618 leader_election.cc:290] T deb69cd9c0414a348a3ee16058a36f00 P df338c374ed54ecbb74bda5e628cf85e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793), 24398c56cc724c91b5116396e774e09c (127.14.103.3:42385)
I20260709 13:04:23.099989 15423 leader_election.cc:304] T deb69cd9c0414a348a3ee16058a36f00 P df338c374ed54ecbb74bda5e628cf85e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ab6657817b84a1d94dba0a4eb2292d0, df338c374ed54ecbb74bda5e628cf85e; no voters: 
I20260709 13:04:23.099030 15530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deb69cd9c0414a348a3ee16058a36f00" candidate_uuid: "df338c374ed54ecbb74bda5e628cf85e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24398c56cc724c91b5116396e774e09c" is_pre_election: true
I20260709 13:04:23.100692 15618 raft_consensus.cc:2804] T deb69cd9c0414a348a3ee16058a36f00 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:23.101097 15618 raft_consensus.cc:493] T deb69cd9c0414a348a3ee16058a36f00 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:23.101166 15530 raft_consensus.cc:2468] T deb69cd9c0414a348a3ee16058a36f00 P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df338c374ed54ecbb74bda5e628cf85e in term 0.
I20260709 13:04:23.101518 15618 raft_consensus.cc:3060] T deb69cd9c0414a348a3ee16058a36f00 P df338c374ed54ecbb74bda5e628cf85e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:23.107625 15618 raft_consensus.cc:515] T deb69cd9c0414a348a3ee16058a36f00 P df338c374ed54ecbb74bda5e628cf85e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:23.109278 15618 leader_election.cc:290] T deb69cd9c0414a348a3ee16058a36f00 P df338c374ed54ecbb74bda5e628cf85e [CANDIDATE]: Term 1 election: Requested vote from peers 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793), 24398c56cc724c91b5116396e774e09c (127.14.103.3:42385)
I20260709 13:04:23.109973 15382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deb69cd9c0414a348a3ee16058a36f00" candidate_uuid: "df338c374ed54ecbb74bda5e628cf85e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
I20260709 13:04:23.110388 15530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deb69cd9c0414a348a3ee16058a36f00" candidate_uuid: "df338c374ed54ecbb74bda5e628cf85e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24398c56cc724c91b5116396e774e09c"
I20260709 13:04:23.110528 15382 raft_consensus.cc:3060] T deb69cd9c0414a348a3ee16058a36f00 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:23.110970 15530 raft_consensus.cc:3060] T deb69cd9c0414a348a3ee16058a36f00 P 24398c56cc724c91b5116396e774e09c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:23.111477 15615 raft_consensus.cc:493] T d3551b1f0a4e4d02af6cbe21a747f4ec P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.112040 15615 raft_consensus.cc:515] T d3551b1f0a4e4d02af6cbe21a747f4ec P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:23.113724 15615 leader_election.cc:290] T d3551b1f0a4e4d02af6cbe21a747f4ec P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df338c374ed54ecbb74bda5e628cf85e (127.14.103.2:45591), 24398c56cc724c91b5116396e774e09c (127.14.103.3:42385)
I20260709 13:04:23.115265 15531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3551b1f0a4e4d02af6cbe21a747f4ec" candidate_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24398c56cc724c91b5116396e774e09c" is_pre_election: true
I20260709 13:04:23.117234 15530 raft_consensus.cc:2468] T deb69cd9c0414a348a3ee16058a36f00 P 24398c56cc724c91b5116396e774e09c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df338c374ed54ecbb74bda5e628cf85e in term 1.
I20260709 13:04:23.117220 15382 raft_consensus.cc:2468] T deb69cd9c0414a348a3ee16058a36f00 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df338c374ed54ecbb74bda5e628cf85e in term 1.
I20260709 13:04:23.117224 15457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3551b1f0a4e4d02af6cbe21a747f4ec" candidate_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df338c374ed54ecbb74bda5e628cf85e" is_pre_election: true
I20260709 13:04:23.118525 15457 raft_consensus.cc:2393] T d3551b1f0a4e4d02af6cbe21a747f4ec P df338c374ed54ecbb74bda5e628cf85e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ab6657817b84a1d94dba0a4eb2292d0 in current term 1: Already voted for candidate 24398c56cc724c91b5116396e774e09c in this term.
I20260709 13:04:23.118563 15420 leader_election.cc:304] T deb69cd9c0414a348a3ee16058a36f00 P df338c374ed54ecbb74bda5e628cf85e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24398c56cc724c91b5116396e774e09c, df338c374ed54ecbb74bda5e628cf85e; no voters: 
I20260709 13:04:23.119418 15618 raft_consensus.cc:2804] T deb69cd9c0414a348a3ee16058a36f00 P df338c374ed54ecbb74bda5e628cf85e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:23.119905 15618 raft_consensus.cc:697] T deb69cd9c0414a348a3ee16058a36f00 P df338c374ed54ecbb74bda5e628cf85e [term 1 LEADER]: Becoming Leader. State: Replica: df338c374ed54ecbb74bda5e628cf85e, State: Running, Role: LEADER
I20260709 13:04:23.119943 15348 leader_election.cc:304] T d3551b1f0a4e4d02af6cbe21a747f4ec P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ab6657817b84a1d94dba0a4eb2292d0; no voters: 24398c56cc724c91b5116396e774e09c, df338c374ed54ecbb74bda5e628cf85e
I20260709 13:04:23.120724 15615 raft_consensus.cc:3060] T d3551b1f0a4e4d02af6cbe21a747f4ec P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:23.120811 15618 consensus_queue.cc:237] T deb69cd9c0414a348a3ee16058a36f00 P df338c374ed54ecbb74bda5e628cf85e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:23.125751 15615 raft_consensus.cc:2749] T d3551b1f0a4e4d02af6cbe21a747f4ec P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:23.128441 15280 catalog_manager.cc:5697] T deb69cd9c0414a348a3ee16058a36f00 P df338c374ed54ecbb74bda5e628cf85e reported cstate change: term changed from 0 to 1, leader changed from <none> to df338c374ed54ecbb74bda5e628cf85e (127.14.103.2). New cstate: current_term: 1 leader_uuid: "df338c374ed54ecbb74bda5e628cf85e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:23.162479 15530 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d3551b1f0a4e4d02af6cbe21a747f4ec"
dest_uuid: "24398c56cc724c91b5116396e774e09c"
mode: GRACEFUL
new_leader_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
 from {username='slave'} at 127.0.0.1:57254
I20260709 13:04:23.162922 15530 raft_consensus.cc:606] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [term 1 LEADER]: Received request to transfer leadership to TS 3ab6657817b84a1d94dba0a4eb2292d0
I20260709 13:04:23.167905 15595 consensus_queue.cc:1048] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:23.168946 15615 consensus_queue.cc:1048] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:23.179051 15595 raft_consensus.cc:993] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c: : Instructing follower 3ab6657817b84a1d94dba0a4eb2292d0 to start an election
I20260709 13:04:23.179456 15595 raft_consensus.cc:1081] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [term 1 LEADER]: Signalling peer 3ab6657817b84a1d94dba0a4eb2292d0 to start an election
I20260709 13:04:23.182284 15615 raft_consensus.cc:493] T 3590c79d23594f91a88f9646df62c52a P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.183794 15381 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d3551b1f0a4e4d02af6cbe21a747f4ec"
dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
 from {username='slave'} at 127.0.0.1:56560
I20260709 13:04:23.184775 15381 raft_consensus.cc:493] T d3551b1f0a4e4d02af6cbe21a747f4ec P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:23.185137 15381 raft_consensus.cc:3060] T d3551b1f0a4e4d02af6cbe21a747f4ec P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.182741 15615 raft_consensus.cc:515] T 3590c79d23594f91a88f9646df62c52a P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:23.193540 15615 leader_election.cc:290] T 3590c79d23594f91a88f9646df62c52a P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df338c374ed54ecbb74bda5e628cf85e (127.14.103.2:45591), 24398c56cc724c91b5116396e774e09c (127.14.103.3:42385)
I20260709 13:04:23.194831 15456 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "42e07a8df5ac47eea3da5846963de3fa"
dest_uuid: "df338c374ed54ecbb74bda5e628cf85e"
mode: GRACEFUL
new_leader_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
 from {username='slave'} at 127.0.0.1:50220
I20260709 13:04:23.195595 15456 raft_consensus.cc:606] T 42e07a8df5ac47eea3da5846963de3fa P df338c374ed54ecbb74bda5e628cf85e [term 1 LEADER]: Received request to transfer leadership to TS 3ab6657817b84a1d94dba0a4eb2292d0
I20260709 13:04:23.196020 15455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3590c79d23594f91a88f9646df62c52a" candidate_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df338c374ed54ecbb74bda5e628cf85e" is_pre_election: true
I20260709 13:04:23.196857 15455 raft_consensus.cc:2393] T 3590c79d23594f91a88f9646df62c52a P df338c374ed54ecbb74bda5e628cf85e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ab6657817b84a1d94dba0a4eb2292d0 in current term 1: Already voted for candidate 24398c56cc724c91b5116396e774e09c in this term.
I20260709 13:04:23.198063 15584 consensus_queue.cc:1048] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [LEADER]: Connected to new peer: Peer: permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:23.201467 15618 consensus_queue.cc:1048] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:23.201798 15530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3590c79d23594f91a88f9646df62c52a" candidate_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24398c56cc724c91b5116396e774e09c" is_pre_election: true
I20260709 13:04:23.203236 15345 leader_election.cc:304] T 3590c79d23594f91a88f9646df62c52a P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ab6657817b84a1d94dba0a4eb2292d0; no voters: 24398c56cc724c91b5116396e774e09c, df338c374ed54ecbb74bda5e628cf85e
I20260709 13:04:23.204542 15457 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "70da92ca35b342a7b2182eaed60e3092"
dest_uuid: "df338c374ed54ecbb74bda5e628cf85e"
mode: GRACEFUL
new_leader_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
 from {username='slave'} at 127.0.0.1:50220
I20260709 13:04:23.204547 15645 consensus_queue.cc:1048] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:23.205262 15457 raft_consensus.cc:606] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e [term 1 LEADER]: Received request to transfer leadership to TS 3ab6657817b84a1d94dba0a4eb2292d0
I20260709 13:04:23.206111 15645 raft_consensus.cc:3060] T 3590c79d23594f91a88f9646df62c52a P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:23.224963 15645 raft_consensus.cc:2749] T 3590c79d23594f91a88f9646df62c52a P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:23.227692 15645 raft_consensus.cc:493] T b46d829f140440738e6d2d456f792c75 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.228236 15645 raft_consensus.cc:515] T b46d829f140440738e6d2d456f792c75 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:23.229103 15381 raft_consensus.cc:515] T d3551b1f0a4e4d02af6cbe21a747f4ec P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:23.231156 15381 leader_election.cc:290] T d3551b1f0a4e4d02af6cbe21a747f4ec P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 2 election: Requested vote from peers df338c374ed54ecbb74bda5e628cf85e (127.14.103.2:45591), 24398c56cc724c91b5116396e774e09c (127.14.103.3:42385)
I20260709 13:04:23.231884 15456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3551b1f0a4e4d02af6cbe21a747f4ec" candidate_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "df338c374ed54ecbb74bda5e628cf85e"
I20260709 13:04:23.232437 15456 raft_consensus.cc:3060] T d3551b1f0a4e4d02af6cbe21a747f4ec P df338c374ed54ecbb74bda5e628cf85e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.232682 15455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b46d829f140440738e6d2d456f792c75" candidate_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df338c374ed54ecbb74bda5e628cf85e" is_pre_election: true
I20260709 13:04:23.236604 15589 raft_consensus.cc:993] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e: : Instructing follower 3ab6657817b84a1d94dba0a4eb2292d0 to start an election
I20260709 13:04:23.237185 15575 raft_consensus.cc:1081] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e [term 1 LEADER]: Signalling peer 3ab6657817b84a1d94dba0a4eb2292d0 to start an election
I20260709 13:04:23.240079 15382 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "70da92ca35b342a7b2182eaed60e3092"
dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
 from {username='slave'} at 127.0.0.1:56574
I20260709 13:04:23.240686 15382 raft_consensus.cc:493] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:23.241106 15382 raft_consensus.cc:3060] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.241607 15530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3551b1f0a4e4d02af6cbe21a747f4ec" candidate_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "24398c56cc724c91b5116396e774e09c"
I20260709 13:04:23.242168 15530 raft_consensus.cc:3055] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:23.242455 15530 raft_consensus.cc:740] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 24398c56cc724c91b5116396e774e09c, State: Running, Role: LEADER
I20260709 13:04:23.243220 15530 consensus_queue.cc:260] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:23.244285 15530 raft_consensus.cc:3060] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.246089 15618 consensus_queue.cc:1048] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e [LEADER]: Connected to new peer: Peer: permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:23.248768 15382 raft_consensus.cc:515] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:23.249907 15381 raft_consensus.cc:1240] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 FOLLOWER]: Rejecting Update request from peer df338c374ed54ecbb74bda5e628cf85e for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:23.251703 15456 raft_consensus.cc:2468] T d3551b1f0a4e4d02af6cbe21a747f4ec P df338c374ed54ecbb74bda5e628cf85e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ab6657817b84a1d94dba0a4eb2292d0 in term 2.
I20260709 13:04:23.252168 15618 consensus_queue.cc:1059] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:23.252324 15381 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "844dc4a0feff4097a52882da0b69cef7"
dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
mode: GRACEFUL
new_leader_uuid: "df338c374ed54ecbb74bda5e628cf85e"
 from {username='slave'} at 127.0.0.1:56590
I20260709 13:04:23.252696 15530 raft_consensus.cc:2468] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ab6657817b84a1d94dba0a4eb2292d0 in term 2.
I20260709 13:04:23.252907 15618 raft_consensus.cc:3055] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:23.252993 15381 raft_consensus.cc:606] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 LEADER]: Received request to transfer leadership to TS df338c374ed54ecbb74bda5e628cf85e
I20260709 13:04:23.253382 15618 raft_consensus.cc:740] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: df338c374ed54ecbb74bda5e628cf85e, State: Running, Role: LEADER
I20260709 13:04:23.254150 15345 leader_election.cc:304] T d3551b1f0a4e4d02af6cbe21a747f4ec P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24398c56cc724c91b5116396e774e09c, 3ab6657817b84a1d94dba0a4eb2292d0; no voters: 
I20260709 13:04:23.254483 15618 consensus_queue.cc:260] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:23.255200 15601 raft_consensus.cc:2804] T d3551b1f0a4e4d02af6cbe21a747f4ec P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:23.255743 15601 raft_consensus.cc:697] T d3551b1f0a4e4d02af6cbe21a747f4ec P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 LEADER]: Becoming Leader. State: Replica: 3ab6657817b84a1d94dba0a4eb2292d0, State: Running, Role: LEADER
I20260709 13:04:23.255851 15645 leader_election.cc:290] T b46d829f140440738e6d2d456f792c75 P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df338c374ed54ecbb74bda5e628cf85e (127.14.103.2:45591), 24398c56cc724c91b5116396e774e09c (127.14.103.3:42385)
I20260709 13:04:23.256235 15618 raft_consensus.cc:3060] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.256237 15382 leader_election.cc:290] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 2 election: Requested vote from peers df338c374ed54ecbb74bda5e628cf85e (127.14.103.2:45591), 24398c56cc724c91b5116396e774e09c (127.14.103.3:42385)
I20260709 13:04:23.256655 15601 consensus_queue.cc:237] T d3551b1f0a4e4d02af6cbe21a747f4ec P 3ab6657817b84a1d94dba0a4eb2292d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:23.260890 15456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70da92ca35b342a7b2182eaed60e3092" candidate_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "df338c374ed54ecbb74bda5e628cf85e"
I20260709 13:04:23.264394 15279 catalog_manager.cc:5697] T d3551b1f0a4e4d02af6cbe21a747f4ec P 3ab6657817b84a1d94dba0a4eb2292d0 reported cstate change: term changed from 1 to 2, leader changed from 24398c56cc724c91b5116396e774e09c (127.14.103.3) to 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:23.265323 15584 consensus_queue.cc:1048] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c [LEADER]: Connected to new peer: Peer: permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:23.274194 15456 raft_consensus.cc:2468] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ab6657817b84a1d94dba0a4eb2292d0 in term 2.
I20260709 13:04:23.275954 15348 leader_election.cc:304] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ab6657817b84a1d94dba0a4eb2292d0, df338c374ed54ecbb74bda5e628cf85e; no voters: 
I20260709 13:04:23.280911 15601 raft_consensus.cc:2804] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:23.282114 15601 raft_consensus.cc:697] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 LEADER]: Becoming Leader. State: Replica: 3ab6657817b84a1d94dba0a4eb2292d0, State: Running, Role: LEADER
I20260709 13:04:23.282423 15531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b46d829f140440738e6d2d456f792c75" candidate_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24398c56cc724c91b5116396e774e09c" is_pre_election: true
I20260709 13:04:23.283232 15531 raft_consensus.cc:2393] T b46d829f140440738e6d2d456f792c75 P 24398c56cc724c91b5116396e774e09c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ab6657817b84a1d94dba0a4eb2292d0 in current term 1: Already voted for candidate df338c374ed54ecbb74bda5e628cf85e in this term.
I20260709 13:04:23.283133 15601 consensus_queue.cc:237] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:23.284648 15345 leader_election.cc:304] T b46d829f140440738e6d2d456f792c75 P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ab6657817b84a1d94dba0a4eb2292d0; no voters: 24398c56cc724c91b5116396e774e09c, df338c374ed54ecbb74bda5e628cf85e
I20260709 13:04:23.284977 15531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70da92ca35b342a7b2182eaed60e3092" candidate_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "24398c56cc724c91b5116396e774e09c"
I20260709 13:04:23.285565 15531 raft_consensus.cc:3060] T 70da92ca35b342a7b2182eaed60e3092 P 24398c56cc724c91b5116396e774e09c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.286401 15382 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "36e186173f7c42dbaa9840522867ce03"
dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
mode: GRACEFUL
new_leader_uuid: "df338c374ed54ecbb74bda5e628cf85e"
 from {username='slave'} at 127.0.0.1:56590
I20260709 13:04:23.287019 15382 raft_consensus.cc:606] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 LEADER]: Received request to transfer leadership to TS df338c374ed54ecbb74bda5e628cf85e
I20260709 13:04:23.287920 15615 raft_consensus.cc:3060] T b46d829f140440738e6d2d456f792c75 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:23.290611 15584 consensus_queue.cc:1048] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:23.292559 15531 raft_consensus.cc:2468] T 70da92ca35b342a7b2182eaed60e3092 P 24398c56cc724c91b5116396e774e09c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ab6657817b84a1d94dba0a4eb2292d0 in term 2.
I20260709 13:04:23.299762 15615 raft_consensus.cc:2749] T b46d829f140440738e6d2d456f792c75 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:23.301652 15531 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3590c79d23594f91a88f9646df62c52a"
dest_uuid: "24398c56cc724c91b5116396e774e09c"
mode: GRACEFUL
new_leader_uuid: "df338c374ed54ecbb74bda5e628cf85e"
 from {username='slave'} at 127.0.0.1:57254
I20260709 13:04:23.302335 15531 raft_consensus.cc:606] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c [term 1 LEADER]: Received request to transfer leadership to TS df338c374ed54ecbb74bda5e628cf85e
I20260709 13:04:23.303021 15278 catalog_manager.cc:5697] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 reported cstate change: term changed from 1 to 2, leader changed from df338c374ed54ecbb74bda5e628cf85e (127.14.103.2) to 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:23.307798 15456 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "deb69cd9c0414a348a3ee16058a36f00"
dest_uuid: "df338c374ed54ecbb74bda5e628cf85e"
mode: GRACEFUL
new_leader_uuid: "24398c56cc724c91b5116396e774e09c"
 from {username='slave'} at 127.0.0.1:50220
I20260709 13:04:23.308282 15456 raft_consensus.cc:606] T deb69cd9c0414a348a3ee16058a36f00 P df338c374ed54ecbb74bda5e628cf85e [term 1 LEADER]: Received request to transfer leadership to TS 24398c56cc724c91b5116396e774e09c
I20260709 13:04:23.328891 15618 consensus_queue.cc:1048] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e [LEADER]: Connected to new peer: Peer: permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:23.338800 15589 consensus_queue.cc:1048] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:23.418859 15589 consensus_queue.cc:1048] T 42e07a8df5ac47eea3da5846963de3fa P df338c374ed54ecbb74bda5e628cf85e [LEADER]: Connected to new peer: Peer: permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:23.428119 15589 consensus_queue.cc:1048] T 42e07a8df5ac47eea3da5846963de3fa P df338c374ed54ecbb74bda5e628cf85e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:23.435316 15575 raft_consensus.cc:993] T 42e07a8df5ac47eea3da5846963de3fa P df338c374ed54ecbb74bda5e628cf85e: : Instructing follower 3ab6657817b84a1d94dba0a4eb2292d0 to start an election
I20260709 13:04:23.435863 15589 raft_consensus.cc:1081] T 42e07a8df5ac47eea3da5846963de3fa P df338c374ed54ecbb74bda5e628cf85e [term 1 LEADER]: Signalling peer 3ab6657817b84a1d94dba0a4eb2292d0 to start an election
I20260709 13:04:23.437361 15382 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "42e07a8df5ac47eea3da5846963de3fa"
dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
 from {username='slave'} at 127.0.0.1:56574
I20260709 13:04:23.437891 15382 raft_consensus.cc:493] T 42e07a8df5ac47eea3da5846963de3fa P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:23.438273 15382 raft_consensus.cc:3060] T 42e07a8df5ac47eea3da5846963de3fa P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.443023 15382 raft_consensus.cc:515] T 42e07a8df5ac47eea3da5846963de3fa P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:23.444617 15382 leader_election.cc:290] T 42e07a8df5ac47eea3da5846963de3fa P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 2 election: Requested vote from peers df338c374ed54ecbb74bda5e628cf85e (127.14.103.2:45591), 24398c56cc724c91b5116396e774e09c (127.14.103.3:42385)
I20260709 13:04:23.445425 15456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42e07a8df5ac47eea3da5846963de3fa" candidate_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "df338c374ed54ecbb74bda5e628cf85e"
I20260709 13:04:23.445593 15531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42e07a8df5ac47eea3da5846963de3fa" candidate_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "24398c56cc724c91b5116396e774e09c"
I20260709 13:04:23.446040 15456 raft_consensus.cc:3055] T 42e07a8df5ac47eea3da5846963de3fa P df338c374ed54ecbb74bda5e628cf85e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:23.446089 15531 raft_consensus.cc:3060] T 42e07a8df5ac47eea3da5846963de3fa P 24398c56cc724c91b5116396e774e09c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.446377 15456 raft_consensus.cc:740] T 42e07a8df5ac47eea3da5846963de3fa P df338c374ed54ecbb74bda5e628cf85e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: df338c374ed54ecbb74bda5e628cf85e, State: Running, Role: LEADER
I20260709 13:04:23.447227 15456 consensus_queue.cc:260] T 42e07a8df5ac47eea3da5846963de3fa P df338c374ed54ecbb74bda5e628cf85e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:23.448395 15456 raft_consensus.cc:3060] T 42e07a8df5ac47eea3da5846963de3fa P df338c374ed54ecbb74bda5e628cf85e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.451586 15531 raft_consensus.cc:2468] T 42e07a8df5ac47eea3da5846963de3fa P 24398c56cc724c91b5116396e774e09c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ab6657817b84a1d94dba0a4eb2292d0 in term 2.
I20260709 13:04:23.452908 15345 leader_election.cc:304] T 42e07a8df5ac47eea3da5846963de3fa P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24398c56cc724c91b5116396e774e09c, 3ab6657817b84a1d94dba0a4eb2292d0; no voters: 
I20260709 13:04:23.453698 15531 raft_consensus.cc:1275] T e7be94629c3c46638abeb602169bcda4 P 24398c56cc724c91b5116396e774e09c [term 2 FOLLOWER]: Refusing update from remote peer 3ab6657817b84a1d94dba0a4eb2292d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:23.453663 15601 raft_consensus.cc:2804] T 42e07a8df5ac47eea3da5846963de3fa P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:23.454003 15456 raft_consensus.cc:2468] T 42e07a8df5ac47eea3da5846963de3fa P df338c374ed54ecbb74bda5e628cf85e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ab6657817b84a1d94dba0a4eb2292d0 in term 2.
I20260709 13:04:23.454293 15601 raft_consensus.cc:697] T 42e07a8df5ac47eea3da5846963de3fa P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 LEADER]: Becoming Leader. State: Replica: 3ab6657817b84a1d94dba0a4eb2292d0, State: Running, Role: LEADER
I20260709 13:04:23.455159 15601 consensus_queue.cc:237] T 42e07a8df5ac47eea3da5846963de3fa P 3ab6657817b84a1d94dba0a4eb2292d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:23.455756 15615 consensus_queue.cc:1048] T e7be94629c3c46638abeb602169bcda4 P 3ab6657817b84a1d94dba0a4eb2292d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:23.462268 15278 catalog_manager.cc:5697] T 42e07a8df5ac47eea3da5846963de3fa P 3ab6657817b84a1d94dba0a4eb2292d0 reported cstate change: term changed from 1 to 2, leader changed from df338c374ed54ecbb74bda5e628cf85e (127.14.103.2) to 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:23.463624 15456 raft_consensus.cc:3060] T e7be94629c3c46638abeb602169bcda4 P df338c374ed54ecbb74bda5e628cf85e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.473755 15456 raft_consensus.cc:1275] T e7be94629c3c46638abeb602169bcda4 P df338c374ed54ecbb74bda5e628cf85e [term 2 FOLLOWER]: Refusing update from remote peer 3ab6657817b84a1d94dba0a4eb2292d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:23.474958 15601 consensus_queue.cc:1048] T e7be94629c3c46638abeb602169bcda4 P 3ab6657817b84a1d94dba0a4eb2292d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:23.537761 15601 raft_consensus.cc:993] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0: : Instructing follower df338c374ed54ecbb74bda5e628cf85e to start an election
I20260709 13:04:23.538244 15615 raft_consensus.cc:1081] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 LEADER]: Signalling peer df338c374ed54ecbb74bda5e628cf85e to start an election
I20260709 13:04:23.539857 15456 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "36e186173f7c42dbaa9840522867ce03"
dest_uuid: "df338c374ed54ecbb74bda5e628cf85e"
 from {username='slave'} at 127.0.0.1:50216
I20260709 13:04:23.540412 15456 raft_consensus.cc:493] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:23.540783 15456 raft_consensus.cc:3060] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.545482 15456 raft_consensus.cc:515] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:23.547102 15456 leader_election.cc:290] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e [CANDIDATE]: Term 2 election: Requested vote from peers 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793), 24398c56cc724c91b5116396e774e09c (127.14.103.3:42385)
I20260709 13:04:23.548004 15382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36e186173f7c42dbaa9840522867ce03" candidate_uuid: "df338c374ed54ecbb74bda5e628cf85e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
I20260709 13:04:23.548331 15531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36e186173f7c42dbaa9840522867ce03" candidate_uuid: "df338c374ed54ecbb74bda5e628cf85e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "24398c56cc724c91b5116396e774e09c"
I20260709 13:04:23.548599 15382 raft_consensus.cc:3055] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:23.548866 15531 raft_consensus.cc:3060] T 36e186173f7c42dbaa9840522867ce03 P 24398c56cc724c91b5116396e774e09c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.548980 15382 raft_consensus.cc:740] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ab6657817b84a1d94dba0a4eb2292d0, State: Running, Role: LEADER
I20260709 13:04:23.549760 15382 consensus_queue.cc:260] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:23.550633 15382 raft_consensus.cc:3060] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.553496 15531 raft_consensus.cc:2468] T 36e186173f7c42dbaa9840522867ce03 P 24398c56cc724c91b5116396e774e09c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df338c374ed54ecbb74bda5e628cf85e in term 2.
I20260709 13:04:23.554884 15420 leader_election.cc:304] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24398c56cc724c91b5116396e774e09c, df338c374ed54ecbb74bda5e628cf85e; no voters: 
I20260709 13:04:23.555541 15589 raft_consensus.cc:2804] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:23.555858 15382 raft_consensus.cc:2468] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df338c374ed54ecbb74bda5e628cf85e in term 2.
I20260709 13:04:23.556039 15589 raft_consensus.cc:697] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e [term 2 LEADER]: Becoming Leader. State: Replica: df338c374ed54ecbb74bda5e628cf85e, State: Running, Role: LEADER
I20260709 13:04:23.556891 15589 consensus_queue.cc:237] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:23.562512 15280 catalog_manager.cc:5697] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e reported cstate change: term changed from 1 to 2, leader changed from 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1) to df338c374ed54ecbb74bda5e628cf85e (127.14.103.2). New cstate: current_term: 2 leader_uuid: "df338c374ed54ecbb74bda5e628cf85e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:23.596787 15589 consensus_queue.cc:1048] T deb69cd9c0414a348a3ee16058a36f00 P df338c374ed54ecbb74bda5e628cf85e [LEADER]: Connected to new peer: Peer: permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:23.603544 15575 raft_consensus.cc:993] T deb69cd9c0414a348a3ee16058a36f00 P df338c374ed54ecbb74bda5e628cf85e: : Instructing follower 24398c56cc724c91b5116396e774e09c to start an election
I20260709 13:04:23.604099 15618 raft_consensus.cc:1081] T deb69cd9c0414a348a3ee16058a36f00 P df338c374ed54ecbb74bda5e628cf85e [term 1 LEADER]: Signalling peer 24398c56cc724c91b5116396e774e09c to start an election
I20260709 13:04:23.605785 15530 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "deb69cd9c0414a348a3ee16058a36f00"
dest_uuid: "24398c56cc724c91b5116396e774e09c"
 from {username='slave'} at 127.0.0.1:57240
I20260709 13:04:23.606326 15530 raft_consensus.cc:493] T deb69cd9c0414a348a3ee16058a36f00 P 24398c56cc724c91b5116396e774e09c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:23.606722 15530 raft_consensus.cc:3060] T deb69cd9c0414a348a3ee16058a36f00 P 24398c56cc724c91b5116396e774e09c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.608526 15589 consensus_queue.cc:1048] T deb69cd9c0414a348a3ee16058a36f00 P df338c374ed54ecbb74bda5e628cf85e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:23.615236 15530 raft_consensus.cc:515] T deb69cd9c0414a348a3ee16058a36f00 P 24398c56cc724c91b5116396e774e09c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:23.618505 15382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deb69cd9c0414a348a3ee16058a36f00" candidate_uuid: "24398c56cc724c91b5116396e774e09c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
I20260709 13:04:23.619084 15382 raft_consensus.cc:3060] T deb69cd9c0414a348a3ee16058a36f00 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.620769 15456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deb69cd9c0414a348a3ee16058a36f00" candidate_uuid: "24398c56cc724c91b5116396e774e09c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "df338c374ed54ecbb74bda5e628cf85e"
I20260709 13:04:23.621335 15456 raft_consensus.cc:3055] T deb69cd9c0414a348a3ee16058a36f00 P df338c374ed54ecbb74bda5e628cf85e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:23.621675 15456 raft_consensus.cc:740] T deb69cd9c0414a348a3ee16058a36f00 P df338c374ed54ecbb74bda5e628cf85e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: df338c374ed54ecbb74bda5e628cf85e, State: Running, Role: LEADER
I20260709 13:04:23.622437 15456 consensus_queue.cc:260] T deb69cd9c0414a348a3ee16058a36f00 P df338c374ed54ecbb74bda5e628cf85e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:23.623525 15456 raft_consensus.cc:3060] T deb69cd9c0414a348a3ee16058a36f00 P df338c374ed54ecbb74bda5e628cf85e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.628618 15382 raft_consensus.cc:2468] T deb69cd9c0414a348a3ee16058a36f00 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24398c56cc724c91b5116396e774e09c in term 2.
I20260709 13:04:23.629654 15497 leader_election.cc:304] T deb69cd9c0414a348a3ee16058a36f00 P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24398c56cc724c91b5116396e774e09c, 3ab6657817b84a1d94dba0a4eb2292d0; no voters: 
I20260709 13:04:23.630348 15584 raft_consensus.cc:2804] T deb69cd9c0414a348a3ee16058a36f00 P 24398c56cc724c91b5116396e774e09c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:23.630777 15584 raft_consensus.cc:697] T deb69cd9c0414a348a3ee16058a36f00 P 24398c56cc724c91b5116396e774e09c [term 2 LEADER]: Becoming Leader. State: Replica: 24398c56cc724c91b5116396e774e09c, State: Running, Role: LEADER
I20260709 13:04:23.631438 15584 consensus_queue.cc:237] T deb69cd9c0414a348a3ee16058a36f00 P 24398c56cc724c91b5116396e774e09c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:23.632202 15456 raft_consensus.cc:2468] T deb69cd9c0414a348a3ee16058a36f00 P df338c374ed54ecbb74bda5e628cf85e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24398c56cc724c91b5116396e774e09c in term 2.
I20260709 13:04:23.633546 15530 leader_election.cc:290] T deb69cd9c0414a348a3ee16058a36f00 P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 2 election: Requested vote from peers 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793), df338c374ed54ecbb74bda5e628cf85e (127.14.103.2:45591)
I20260709 13:04:23.637636 15615 raft_consensus.cc:993] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0: : Instructing follower df338c374ed54ecbb74bda5e628cf85e to start an election
I20260709 13:04:23.638043 15645 raft_consensus.cc:1081] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 LEADER]: Signalling peer df338c374ed54ecbb74bda5e628cf85e to start an election
I20260709 13:04:23.640961 15280 catalog_manager.cc:5697] T deb69cd9c0414a348a3ee16058a36f00 P 24398c56cc724c91b5116396e774e09c reported cstate change: term changed from 1 to 2, leader changed from df338c374ed54ecbb74bda5e628cf85e (127.14.103.2) to 24398c56cc724c91b5116396e774e09c (127.14.103.3). New cstate: current_term: 2 leader_uuid: "24398c56cc724c91b5116396e774e09c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:23.639704 15456 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "844dc4a0feff4097a52882da0b69cef7"
dest_uuid: "df338c374ed54ecbb74bda5e628cf85e"
 from {username='slave'} at 127.0.0.1:50216
I20260709 13:04:23.642596 15456 raft_consensus.cc:493] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:23.642954 15456 raft_consensus.cc:3060] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.648593 15456 raft_consensus.cc:515] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:23.650429 15456 leader_election.cc:290] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e [CANDIDATE]: Term 2 election: Requested vote from peers 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793), 24398c56cc724c91b5116396e774e09c (127.14.103.3:42385)
I20260709 13:04:23.651129 15382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844dc4a0feff4097a52882da0b69cef7" candidate_uuid: "df338c374ed54ecbb74bda5e628cf85e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
I20260709 13:04:23.651386 15530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844dc4a0feff4097a52882da0b69cef7" candidate_uuid: "df338c374ed54ecbb74bda5e628cf85e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "24398c56cc724c91b5116396e774e09c"
I20260709 13:04:23.651754 15382 raft_consensus.cc:3055] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:23.651938 15530 raft_consensus.cc:3060] T 844dc4a0feff4097a52882da0b69cef7 P 24398c56cc724c91b5116396e774e09c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.652119 15382 raft_consensus.cc:740] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ab6657817b84a1d94dba0a4eb2292d0, State: Running, Role: LEADER
I20260709 13:04:23.652925 15382 consensus_queue.cc:260] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:23.654078 15382 raft_consensus.cc:3060] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.658622 15530 raft_consensus.cc:2468] T 844dc4a0feff4097a52882da0b69cef7 P 24398c56cc724c91b5116396e774e09c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df338c374ed54ecbb74bda5e628cf85e in term 2.
I20260709 13:04:23.659735 15420 leader_election.cc:304] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24398c56cc724c91b5116396e774e09c, df338c374ed54ecbb74bda5e628cf85e; no voters: 
I20260709 13:04:23.660485 15589 raft_consensus.cc:2804] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:23.660684 15382 raft_consensus.cc:2468] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df338c374ed54ecbb74bda5e628cf85e in term 2.
I20260709 13:04:23.660889 15589 raft_consensus.cc:697] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e [term 2 LEADER]: Becoming Leader. State: Replica: df338c374ed54ecbb74bda5e628cf85e, State: Running, Role: LEADER
I20260709 13:04:23.661666 15589 consensus_queue.cc:237] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:23.667541 15280 catalog_manager.cc:5697] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e reported cstate change: term changed from 1 to 2, leader changed from 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1) to df338c374ed54ecbb74bda5e628cf85e (127.14.103.2). New cstate: current_term: 2 leader_uuid: "df338c374ed54ecbb74bda5e628cf85e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:23.678238 15595 raft_consensus.cc:993] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c: : Instructing follower df338c374ed54ecbb74bda5e628cf85e to start an election
I20260709 13:04:23.678653 15584 raft_consensus.cc:1081] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c [term 1 LEADER]: Signalling peer df338c374ed54ecbb74bda5e628cf85e to start an election
I20260709 13:04:23.680056 15456 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3590c79d23594f91a88f9646df62c52a"
dest_uuid: "df338c374ed54ecbb74bda5e628cf85e"
 from {username='slave'} at 127.0.0.1:50208
I20260709 13:04:23.680496 15456 raft_consensus.cc:493] T 3590c79d23594f91a88f9646df62c52a P df338c374ed54ecbb74bda5e628cf85e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:23.680783 15456 raft_consensus.cc:3060] T 3590c79d23594f91a88f9646df62c52a P df338c374ed54ecbb74bda5e628cf85e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.685158 15456 raft_consensus.cc:515] T 3590c79d23594f91a88f9646df62c52a P df338c374ed54ecbb74bda5e628cf85e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:23.686782 15456 leader_election.cc:290] T 3590c79d23594f91a88f9646df62c52a P df338c374ed54ecbb74bda5e628cf85e [CANDIDATE]: Term 2 election: Requested vote from peers 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793), 24398c56cc724c91b5116396e774e09c (127.14.103.3:42385)
I20260709 13:04:23.687471 15382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3590c79d23594f91a88f9646df62c52a" candidate_uuid: "df338c374ed54ecbb74bda5e628cf85e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
I20260709 13:04:23.687840 15530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3590c79d23594f91a88f9646df62c52a" candidate_uuid: "df338c374ed54ecbb74bda5e628cf85e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "24398c56cc724c91b5116396e774e09c"
I20260709 13:04:23.688066 15382 raft_consensus.cc:3060] T 3590c79d23594f91a88f9646df62c52a P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.688385 15530 raft_consensus.cc:3055] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:23.688684 15530 raft_consensus.cc:740] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 24398c56cc724c91b5116396e774e09c, State: Running, Role: LEADER
I20260709 13:04:23.689349 15530 consensus_queue.cc:260] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:23.690346 15530 raft_consensus.cc:3060] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.692786 15382 raft_consensus.cc:2468] T 3590c79d23594f91a88f9646df62c52a P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df338c374ed54ecbb74bda5e628cf85e in term 2.
I20260709 13:04:23.693804 15423 leader_election.cc:304] T 3590c79d23594f91a88f9646df62c52a P df338c374ed54ecbb74bda5e628cf85e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ab6657817b84a1d94dba0a4eb2292d0, df338c374ed54ecbb74bda5e628cf85e; no voters: 
I20260709 13:04:23.694499 15589 raft_consensus.cc:2804] T 3590c79d23594f91a88f9646df62c52a P df338c374ed54ecbb74bda5e628cf85e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:23.694902 15589 raft_consensus.cc:697] T 3590c79d23594f91a88f9646df62c52a P df338c374ed54ecbb74bda5e628cf85e [term 2 LEADER]: Becoming Leader. State: Replica: df338c374ed54ecbb74bda5e628cf85e, State: Running, Role: LEADER
I20260709 13:04:23.695645 15589 consensus_queue.cc:237] T 3590c79d23594f91a88f9646df62c52a P df338c374ed54ecbb74bda5e628cf85e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:23.696149 15530 raft_consensus.cc:2468] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df338c374ed54ecbb74bda5e628cf85e in term 2.
I20260709 13:04:23.701615 15280 catalog_manager.cc:5697] T 3590c79d23594f91a88f9646df62c52a P df338c374ed54ecbb74bda5e628cf85e reported cstate change: term changed from 1 to 2, leader changed from 24398c56cc724c91b5116396e774e09c (127.14.103.3) to df338c374ed54ecbb74bda5e628cf85e (127.14.103.2). New cstate: current_term: 2 leader_uuid: "df338c374ed54ecbb74bda5e628cf85e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:23.736340 15456 raft_consensus.cc:1275] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e [term 2 FOLLOWER]: Refusing update from remote peer 3ab6657817b84a1d94dba0a4eb2292d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:23.737565 15645 consensus_queue.cc:1048] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:23.746793 15530 raft_consensus.cc:1275] T 70da92ca35b342a7b2182eaed60e3092 P 24398c56cc724c91b5116396e774e09c [term 2 FOLLOWER]: Refusing update from remote peer 3ab6657817b84a1d94dba0a4eb2292d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:23.748116 15645 consensus_queue.cc:1048] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:23.750960 15531 raft_consensus.cc:1275] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [term 2 FOLLOWER]: Refusing update from remote peer 3ab6657817b84a1d94dba0a4eb2292d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:23.752218 15645 consensus_queue.cc:1048] T d3551b1f0a4e4d02af6cbe21a747f4ec P 3ab6657817b84a1d94dba0a4eb2292d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:23.759390 15456 raft_consensus.cc:1275] T d3551b1f0a4e4d02af6cbe21a747f4ec P df338c374ed54ecbb74bda5e628cf85e [term 2 FOLLOWER]: Refusing update from remote peer 3ab6657817b84a1d94dba0a4eb2292d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:23.760649 15645 consensus_queue.cc:1048] T d3551b1f0a4e4d02af6cbe21a747f4ec P 3ab6657817b84a1d94dba0a4eb2292d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:23.999804 15456 raft_consensus.cc:1275] T 42e07a8df5ac47eea3da5846963de3fa P df338c374ed54ecbb74bda5e628cf85e [term 2 FOLLOWER]: Refusing update from remote peer 3ab6657817b84a1d94dba0a4eb2292d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:24.001191 15581 consensus_queue.cc:1048] T 42e07a8df5ac47eea3da5846963de3fa P 3ab6657817b84a1d94dba0a4eb2292d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:24.008751 15530 raft_consensus.cc:1275] T 42e07a8df5ac47eea3da5846963de3fa P 24398c56cc724c91b5116396e774e09c [term 2 FOLLOWER]: Refusing update from remote peer 3ab6657817b84a1d94dba0a4eb2292d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:24.010397 15581 consensus_queue.cc:1048] T 42e07a8df5ac47eea3da5846963de3fa P 3ab6657817b84a1d94dba0a4eb2292d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:24.019440 15382 raft_consensus.cc:1275] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 FOLLOWER]: Refusing update from remote peer df338c374ed54ecbb74bda5e628cf85e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:24.021044 15589 consensus_queue.cc:1048] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:24.028400 15530 raft_consensus.cc:1275] T 36e186173f7c42dbaa9840522867ce03 P 24398c56cc724c91b5116396e774e09c [term 2 FOLLOWER]: Refusing update from remote peer df338c374ed54ecbb74bda5e628cf85e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:24.029822 15575 consensus_queue.cc:1048] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e [LEADER]: Connected to new peer: Peer: permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:24.083276 15456 raft_consensus.cc:1275] T deb69cd9c0414a348a3ee16058a36f00 P df338c374ed54ecbb74bda5e628cf85e [term 2 FOLLOWER]: Refusing update from remote peer 24398c56cc724c91b5116396e774e09c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:24.084538 15584 consensus_queue.cc:1048] T deb69cd9c0414a348a3ee16058a36f00 P 24398c56cc724c91b5116396e774e09c [LEADER]: Connected to new peer: Peer: permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:24.091941 15382 raft_consensus.cc:1275] T deb69cd9c0414a348a3ee16058a36f00 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 FOLLOWER]: Refusing update from remote peer 24398c56cc724c91b5116396e774e09c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:24.094005 15595 consensus_queue.cc:1048] T deb69cd9c0414a348a3ee16058a36f00 P 24398c56cc724c91b5116396e774e09c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:24.113104 15530 raft_consensus.cc:1275] T 844dc4a0feff4097a52882da0b69cef7 P 24398c56cc724c91b5116396e774e09c [term 2 FOLLOWER]: Refusing update from remote peer df338c374ed54ecbb74bda5e628cf85e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:24.114249 15575 consensus_queue.cc:1048] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e [LEADER]: Connected to new peer: Peer: permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:24.121677 15382 raft_consensus.cc:1275] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 FOLLOWER]: Refusing update from remote peer df338c374ed54ecbb74bda5e628cf85e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:24.123517 15618 consensus_queue.cc:1048] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:24.134159 15382 raft_consensus.cc:1275] T 3590c79d23594f91a88f9646df62c52a P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 FOLLOWER]: Refusing update from remote peer df338c374ed54ecbb74bda5e628cf85e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:24.135378 15618 consensus_queue.cc:1048] T 3590c79d23594f91a88f9646df62c52a P df338c374ed54ecbb74bda5e628cf85e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:24.142341 15530 raft_consensus.cc:1275] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c [term 2 FOLLOWER]: Refusing update from remote peer df338c374ed54ecbb74bda5e628cf85e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:24.145747 15589 consensus_queue.cc:1048] T 3590c79d23594f91a88f9646df62c52a P df338c374ed54ecbb74bda5e628cf85e [LEADER]: Connected to new peer: Peer: permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:26.338634 14748 auto_leader_rebalancer-test.cc:348] The leader distribution is 
I20260709 13:04:26.339361 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:26.358780 15382 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "32f268907bbb4d1081fa0fccb595110e"
dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
mode: GRACEFUL
new_leader_uuid: "24398c56cc724c91b5116396e774e09c"
 from {username='slave'} at 127.0.0.1:56604
I20260709 13:04:26.359256 15382 raft_consensus.cc:606] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 LEADER]: Received request to transfer leadership to TS 24398c56cc724c91b5116396e774e09c
I20260709 13:04:26.361410 15382 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "70da92ca35b342a7b2182eaed60e3092"
dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
mode: GRACEFUL
new_leader_uuid: "24398c56cc724c91b5116396e774e09c"
 from {username='slave'} at 127.0.0.1:56604
I20260709 13:04:26.361789 15382 raft_consensus.cc:606] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 LEADER]: Received request to transfer leadership to TS 24398c56cc724c91b5116396e774e09c
I20260709 13:04:26.362679 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:04:26.363054 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:26.559290 15666 raft_consensus.cc:993] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0: : Instructing follower 24398c56cc724c91b5116396e774e09c to start an election
I20260709 13:04:26.559804 15615 raft_consensus.cc:1081] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 LEADER]: Signalling peer 24398c56cc724c91b5116396e774e09c to start an election
I20260709 13:04:26.561296 15531 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "32f268907bbb4d1081fa0fccb595110e"
dest_uuid: "24398c56cc724c91b5116396e774e09c"
 from {username='slave'} at 127.0.0.1:57242
I20260709 13:04:26.561877 15531 raft_consensus.cc:493] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:26.562270 15531 raft_consensus.cc:3060] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:26.567495 15531 raft_consensus.cc:515] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:26.569056 15531 leader_election.cc:290] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 2 election: Requested vote from peers 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793), df338c374ed54ecbb74bda5e628cf85e (127.14.103.2:45591)
I20260709 13:04:26.569875 15382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32f268907bbb4d1081fa0fccb595110e" candidate_uuid: "24398c56cc724c91b5116396e774e09c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
I20260709 13:04:26.570087 15456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32f268907bbb4d1081fa0fccb595110e" candidate_uuid: "24398c56cc724c91b5116396e774e09c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "df338c374ed54ecbb74bda5e628cf85e"
I20260709 13:04:26.570350 15382 raft_consensus.cc:3055] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:26.570591 15456 raft_consensus.cc:3060] T 32f268907bbb4d1081fa0fccb595110e P df338c374ed54ecbb74bda5e628cf85e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:26.570644 15382 raft_consensus.cc:740] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ab6657817b84a1d94dba0a4eb2292d0, State: Running, Role: LEADER
I20260709 13:04:26.571460 15382 consensus_queue.cc:260] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:26.572388 15382 raft_consensus.cc:3060] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:26.574822 15456 raft_consensus.cc:2468] T 32f268907bbb4d1081fa0fccb595110e P df338c374ed54ecbb74bda5e628cf85e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24398c56cc724c91b5116396e774e09c in term 2.
I20260709 13:04:26.575817 15497 leader_election.cc:304] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24398c56cc724c91b5116396e774e09c, df338c374ed54ecbb74bda5e628cf85e; no voters: 
I20260709 13:04:26.576447 15382 raft_consensus.cc:2468] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24398c56cc724c91b5116396e774e09c in term 2.
I20260709 13:04:26.576567 15663 raft_consensus.cc:2804] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:26.577699 15663 raft_consensus.cc:697] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c [term 2 LEADER]: Becoming Leader. State: Replica: 24398c56cc724c91b5116396e774e09c, State: Running, Role: LEADER
I20260709 13:04:26.578451 15663 consensus_queue.cc:237] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:26.584956 15280 catalog_manager.cc:5697] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c reported cstate change: term changed from 1 to 2, leader changed from 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1) to 24398c56cc724c91b5116396e774e09c (127.14.103.3). New cstate: current_term: 2 leader_uuid: "24398c56cc724c91b5116396e774e09c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:26.784705 15666 raft_consensus.cc:993] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0: : Instructing follower 24398c56cc724c91b5116396e774e09c to start an election
I20260709 13:04:26.785172 15615 raft_consensus.cc:1081] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 LEADER]: Signalling peer 24398c56cc724c91b5116396e774e09c to start an election
I20260709 13:04:26.786716 15531 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "70da92ca35b342a7b2182eaed60e3092"
dest_uuid: "24398c56cc724c91b5116396e774e09c"
 from {username='slave'} at 127.0.0.1:57242
I20260709 13:04:26.787250 15531 raft_consensus.cc:493] T 70da92ca35b342a7b2182eaed60e3092 P 24398c56cc724c91b5116396e774e09c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:26.787530 15531 raft_consensus.cc:3060] T 70da92ca35b342a7b2182eaed60e3092 P 24398c56cc724c91b5116396e774e09c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:26.793112 15531 raft_consensus.cc:515] T 70da92ca35b342a7b2182eaed60e3092 P 24398c56cc724c91b5116396e774e09c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:26.794700 15531 leader_election.cc:290] T 70da92ca35b342a7b2182eaed60e3092 P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 3 election: Requested vote from peers 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793), df338c374ed54ecbb74bda5e628cf85e (127.14.103.2:45591)
I20260709 13:04:26.795440 15382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70da92ca35b342a7b2182eaed60e3092" candidate_uuid: "24398c56cc724c91b5116396e774e09c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
I20260709 13:04:26.795595 15456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70da92ca35b342a7b2182eaed60e3092" candidate_uuid: "24398c56cc724c91b5116396e774e09c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "df338c374ed54ecbb74bda5e628cf85e"
I20260709 13:04:26.796059 15382 raft_consensus.cc:3055] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:26.796164 15456 raft_consensus.cc:3060] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:26.796355 15382 raft_consensus.cc:740] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ab6657817b84a1d94dba0a4eb2292d0, State: Running, Role: LEADER
I20260709 13:04:26.797472 15382 consensus_queue.cc:260] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:26.798352 15382 raft_consensus.cc:3060] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:26.800426 15456 raft_consensus.cc:2468] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24398c56cc724c91b5116396e774e09c in term 3.
I20260709 13:04:26.801298 15497 leader_election.cc:304] T 70da92ca35b342a7b2182eaed60e3092 P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24398c56cc724c91b5116396e774e09c, df338c374ed54ecbb74bda5e628cf85e; no voters: 
I20260709 13:04:26.801971 15663 raft_consensus.cc:2804] T 70da92ca35b342a7b2182eaed60e3092 P 24398c56cc724c91b5116396e774e09c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:26.802423 15663 raft_consensus.cc:697] T 70da92ca35b342a7b2182eaed60e3092 P 24398c56cc724c91b5116396e774e09c [term 3 LEADER]: Becoming Leader. State: Replica: 24398c56cc724c91b5116396e774e09c, State: Running, Role: LEADER
I20260709 13:04:26.803166 15382 raft_consensus.cc:2468] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24398c56cc724c91b5116396e774e09c in term 3.
I20260709 13:04:26.803195 15663 consensus_queue.cc:237] T 70da92ca35b342a7b2182eaed60e3092 P 24398c56cc724c91b5116396e774e09c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:26.809758 15280 catalog_manager.cc:5697] T 70da92ca35b342a7b2182eaed60e3092 P 24398c56cc724c91b5116396e774e09c reported cstate change: term changed from 2 to 3, leader changed from 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1) to 24398c56cc724c91b5116396e774e09c (127.14.103.3). New cstate: current_term: 3 leader_uuid: "24398c56cc724c91b5116396e774e09c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:26.959443 15456 raft_consensus.cc:1275] T 32f268907bbb4d1081fa0fccb595110e P df338c374ed54ecbb74bda5e628cf85e [term 2 FOLLOWER]: Refusing update from remote peer 24398c56cc724c91b5116396e774e09c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:26.960729 15663 consensus_queue.cc:1048] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c [LEADER]: Connected to new peer: Peer: permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:26.973111 15382 raft_consensus.cc:1275] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 FOLLOWER]: Refusing update from remote peer 24398c56cc724c91b5116396e774e09c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:26.974553 15676 consensus_queue.cc:1048] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:27.239364 15382 raft_consensus.cc:1275] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 3 FOLLOWER]: Refusing update from remote peer 24398c56cc724c91b5116396e774e09c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:27.240617 15676 consensus_queue.cc:1048] T 70da92ca35b342a7b2182eaed60e3092 P 24398c56cc724c91b5116396e774e09c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:27.247596 15456 raft_consensus.cc:1275] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e [term 3 FOLLOWER]: Refusing update from remote peer 24398c56cc724c91b5116396e774e09c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:27.250229 15667 consensus_queue.cc:1048] T 70da92ca35b342a7b2182eaed60e3092 P 24398c56cc724c91b5116396e774e09c [LEADER]: Connected to new peer: Peer: permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:27.363785 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:27.378945 15531 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "32f268907bbb4d1081fa0fccb595110e"
dest_uuid: "24398c56cc724c91b5116396e774e09c"
mode: GRACEFUL
new_leader_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
 from {username='slave'} at 127.0.0.1:57254
I20260709 13:04:27.379541 15531 raft_consensus.cc:606] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c [term 2 LEADER]: Received request to transfer leadership to TS 3ab6657817b84a1d94dba0a4eb2292d0
I20260709 13:04:27.387367 15531 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "70da92ca35b342a7b2182eaed60e3092"
dest_uuid: "24398c56cc724c91b5116396e774e09c"
mode: GRACEFUL
new_leader_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
 from {username='slave'} at 127.0.0.1:57254
I20260709 13:04:27.387950 15531 raft_consensus.cc:606] T 70da92ca35b342a7b2182eaed60e3092 P 24398c56cc724c91b5116396e774e09c [term 3 LEADER]: Received request to transfer leadership to TS 3ab6657817b84a1d94dba0a4eb2292d0
I20260709 13:04:27.392656 15456 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "844dc4a0feff4097a52882da0b69cef7"
dest_uuid: "df338c374ed54ecbb74bda5e628cf85e"
mode: GRACEFUL
new_leader_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
 from {username='slave'} at 127.0.0.1:50220
I20260709 13:04:27.393162 15456 raft_consensus.cc:606] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e [term 2 LEADER]: Received request to transfer leadership to TS 3ab6657817b84a1d94dba0a4eb2292d0
I20260709 13:04:27.397861 15456 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "36e186173f7c42dbaa9840522867ce03"
dest_uuid: "df338c374ed54ecbb74bda5e628cf85e"
mode: GRACEFUL
new_leader_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
 from {username='slave'} at 127.0.0.1:50220
I20260709 13:04:27.398342 15456 raft_consensus.cc:606] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e [term 2 LEADER]: Received request to transfer leadership to TS 3ab6657817b84a1d94dba0a4eb2292d0
I20260709 13:04:27.402426 15456 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b46d829f140440738e6d2d456f792c75"
dest_uuid: "df338c374ed54ecbb74bda5e628cf85e"
mode: GRACEFUL
new_leader_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
 from {username='slave'} at 127.0.0.1:50220
I20260709 13:04:27.402935 15456 raft_consensus.cc:606] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e [term 1 LEADER]: Received request to transfer leadership to TS 3ab6657817b84a1d94dba0a4eb2292d0
I20260709 13:04:27.405006 15663 raft_consensus.cc:993] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c: : Instructing follower 3ab6657817b84a1d94dba0a4eb2292d0 to start an election
I20260709 13:04:27.405433 15667 raft_consensus.cc:1081] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c [term 2 LEADER]: Signalling peer 3ab6657817b84a1d94dba0a4eb2292d0 to start an election
I20260709 13:04:27.406873 15382 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "32f268907bbb4d1081fa0fccb595110e"
dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
 from {username='slave'} at 127.0.0.1:56560
I20260709 13:04:27.407411 15382 raft_consensus.cc:493] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:27.407823 15382 raft_consensus.cc:3060] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:27.412127 15382 raft_consensus.cc:515] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:27.413683 15382 leader_election.cc:290] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 3 election: Requested vote from peers df338c374ed54ecbb74bda5e628cf85e (127.14.103.2:45591), 24398c56cc724c91b5116396e774e09c (127.14.103.3:42385)
I20260709 13:04:27.414310 15456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32f268907bbb4d1081fa0fccb595110e" candidate_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "df338c374ed54ecbb74bda5e628cf85e"
I20260709 13:04:27.414651 15531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32f268907bbb4d1081fa0fccb595110e" candidate_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "24398c56cc724c91b5116396e774e09c"
I20260709 13:04:27.414835 15456 raft_consensus.cc:3060] T 32f268907bbb4d1081fa0fccb595110e P df338c374ed54ecbb74bda5e628cf85e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:27.415165 15531 raft_consensus.cc:3055] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:27.415436 15531 raft_consensus.cc:740] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 24398c56cc724c91b5116396e774e09c, State: Running, Role: LEADER
I20260709 13:04:27.416139 15531 consensus_queue.cc:260] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:27.417012 15531 raft_consensus.cc:3060] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:27.418913 15456 raft_consensus.cc:2468] T 32f268907bbb4d1081fa0fccb595110e P df338c374ed54ecbb74bda5e628cf85e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ab6657817b84a1d94dba0a4eb2292d0 in term 3.
I20260709 13:04:27.419813 15348 leader_election.cc:304] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ab6657817b84a1d94dba0a4eb2292d0, df338c374ed54ecbb74bda5e628cf85e; no voters: 
I20260709 13:04:27.420447 15615 raft_consensus.cc:2804] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:27.421440 15615 raft_consensus.cc:697] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [term 3 LEADER]: Becoming Leader. State: Replica: 3ab6657817b84a1d94dba0a4eb2292d0, State: Running, Role: LEADER
I20260709 13:04:27.421514 15531 raft_consensus.cc:2468] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ab6657817b84a1d94dba0a4eb2292d0 in term 3.
I20260709 13:04:27.422230 15615 consensus_queue.cc:237] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:27.428047 15279 catalog_manager.cc:5697] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 reported cstate change: term changed from 2 to 3, leader changed from 24398c56cc724c91b5116396e774e09c (127.14.103.3) to 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1). New cstate: current_term: 3 leader_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:27.799955 15589 raft_consensus.cc:993] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e: : Instructing follower 3ab6657817b84a1d94dba0a4eb2292d0 to start an election
I20260709 13:04:27.801107 15682 raft_consensus.cc:1081] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e [term 2 LEADER]: Signalling peer 3ab6657817b84a1d94dba0a4eb2292d0 to start an election
I20260709 13:04:27.801800 15667 raft_consensus.cc:993] T 70da92ca35b342a7b2182eaed60e3092 P 24398c56cc724c91b5116396e774e09c: : Instructing follower 3ab6657817b84a1d94dba0a4eb2292d0 to start an election
I20260709 13:04:27.802353 15667 raft_consensus.cc:1081] T 70da92ca35b342a7b2182eaed60e3092 P 24398c56cc724c91b5116396e774e09c [term 3 LEADER]: Signalling peer 3ab6657817b84a1d94dba0a4eb2292d0 to start an election
I20260709 13:04:27.802640 15682 raft_consensus.cc:993] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e: : Instructing follower 3ab6657817b84a1d94dba0a4eb2292d0 to start an election
I20260709 13:04:27.803172 15589 raft_consensus.cc:1081] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e [term 1 LEADER]: Signalling peer 3ab6657817b84a1d94dba0a4eb2292d0 to start an election
I20260709 13:04:27.804528 15382 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "70da92ca35b342a7b2182eaed60e3092"
dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
 from {username='slave'} at 127.0.0.1:56560
I20260709 13:04:27.804605 15381 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "844dc4a0feff4097a52882da0b69cef7"
dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
 from {username='slave'} at 127.0.0.1:56574
I20260709 13:04:27.805022 15380 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b46d829f140440738e6d2d456f792c75"
dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
 from {username='slave'} at 127.0.0.1:56574
I20260709 13:04:27.805554 15381 raft_consensus.cc:493] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:27.805557 15382 raft_consensus.cc:493] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:27.805670 15380 raft_consensus.cc:493] T b46d829f140440738e6d2d456f792c75 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:27.806252 15381 raft_consensus.cc:3060] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:27.806308 15382 raft_consensus.cc:3060] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:27.806442 15380 raft_consensus.cc:3060] T b46d829f140440738e6d2d456f792c75 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:27.813942 15380 raft_consensus.cc:515] T b46d829f140440738e6d2d456f792c75 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:27.814792 15382 raft_consensus.cc:515] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:27.815136 15381 raft_consensus.cc:515] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:27.817067 15381 leader_election.cc:290] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 3 election: Requested vote from peers df338c374ed54ecbb74bda5e628cf85e (127.14.103.2:45591), 24398c56cc724c91b5116396e774e09c (127.14.103.3:42385)
I20260709 13:04:27.817778 15456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b46d829f140440738e6d2d456f792c75" candidate_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "df338c374ed54ecbb74bda5e628cf85e"
I20260709 13:04:27.818106 15531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b46d829f140440738e6d2d456f792c75" candidate_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "24398c56cc724c91b5116396e774e09c"
I20260709 13:04:27.818594 15456 raft_consensus.cc:3055] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:27.818809 15531 raft_consensus.cc:3060] T b46d829f140440738e6d2d456f792c75 P 24398c56cc724c91b5116396e774e09c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:27.818874 15456 raft_consensus.cc:740] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: df338c374ed54ecbb74bda5e628cf85e, State: Running, Role: LEADER
I20260709 13:04:27.819928 15456 consensus_queue.cc:260] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:27.820341 15529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844dc4a0feff4097a52882da0b69cef7" candidate_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "24398c56cc724c91b5116396e774e09c"
I20260709 13:04:27.821141 15456 raft_consensus.cc:3060] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:27.821177 15529 raft_consensus.cc:3060] T 844dc4a0feff4097a52882da0b69cef7 P 24398c56cc724c91b5116396e774e09c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:27.822566 15530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70da92ca35b342a7b2182eaed60e3092" candidate_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "24398c56cc724c91b5116396e774e09c"
I20260709 13:04:27.823082 15530 raft_consensus.cc:3055] T 70da92ca35b342a7b2182eaed60e3092 P 24398c56cc724c91b5116396e774e09c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:27.823356 15530 raft_consensus.cc:740] T 70da92ca35b342a7b2182eaed60e3092 P 24398c56cc724c91b5116396e774e09c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 24398c56cc724c91b5116396e774e09c, State: Running, Role: LEADER
I20260709 13:04:27.824133 15530 consensus_queue.cc:260] T 70da92ca35b342a7b2182eaed60e3092 P 24398c56cc724c91b5116396e774e09c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:27.825205 15530 raft_consensus.cc:3060] T 70da92ca35b342a7b2182eaed60e3092 P 24398c56cc724c91b5116396e774e09c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:27.826205 15456 raft_consensus.cc:2468] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ab6657817b84a1d94dba0a4eb2292d0 in term 2.
I20260709 13:04:27.826367 15531 raft_consensus.cc:2468] T b46d829f140440738e6d2d456f792c75 P 24398c56cc724c91b5116396e774e09c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ab6657817b84a1d94dba0a4eb2292d0 in term 2.
I20260709 13:04:27.816783 15382 leader_election.cc:290] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 4 election: Requested vote from peers df338c374ed54ecbb74bda5e628cf85e (127.14.103.2:45591), 24398c56cc724c91b5116396e774e09c (127.14.103.3:42385)
I20260709 13:04:27.828150 15456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70da92ca35b342a7b2182eaed60e3092" candidate_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "df338c374ed54ecbb74bda5e628cf85e"
I20260709 13:04:27.828171 15348 leader_election.cc:304] T b46d829f140440738e6d2d456f792c75 P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ab6657817b84a1d94dba0a4eb2292d0, df338c374ed54ecbb74bda5e628cf85e; no voters: 
I20260709 13:04:27.828939 15456 raft_consensus.cc:3060] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:27.829829 15615 raft_consensus.cc:2804] T b46d829f140440738e6d2d456f792c75 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:27.830359 15615 raft_consensus.cc:697] T b46d829f140440738e6d2d456f792c75 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 LEADER]: Becoming Leader. State: Replica: 3ab6657817b84a1d94dba0a4eb2292d0, State: Running, Role: LEADER
I20260709 13:04:27.831444 15457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844dc4a0feff4097a52882da0b69cef7" candidate_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "df338c374ed54ecbb74bda5e628cf85e"
I20260709 13:04:27.833200 15457 raft_consensus.cc:3055] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:27.833462 15457 raft_consensus.cc:740] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: df338c374ed54ecbb74bda5e628cf85e, State: Running, Role: LEADER
I20260709 13:04:27.834335 15457 consensus_queue.cc:260] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:27.835896 15530 raft_consensus.cc:2468] T 70da92ca35b342a7b2182eaed60e3092 P 24398c56cc724c91b5116396e774e09c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ab6657817b84a1d94dba0a4eb2292d0 in term 4.
I20260709 13:04:27.836789 15456 raft_consensus.cc:2468] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ab6657817b84a1d94dba0a4eb2292d0 in term 4.
I20260709 13:04:27.831426 15615 consensus_queue.cc:237] T b46d829f140440738e6d2d456f792c75 P 3ab6657817b84a1d94dba0a4eb2292d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:27.837635 15457 raft_consensus.cc:3060] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:27.836812 15380 leader_election.cc:290] T b46d829f140440738e6d2d456f792c75 P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 2 election: Requested vote from peers df338c374ed54ecbb74bda5e628cf85e (127.14.103.2:45591), 24398c56cc724c91b5116396e774e09c (127.14.103.3:42385)
I20260709 13:04:27.839704 15345 leader_election.cc:304] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24398c56cc724c91b5116396e774e09c, 3ab6657817b84a1d94dba0a4eb2292d0; no voters: 
I20260709 13:04:27.841198 15530 raft_consensus.cc:1275] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c [term 3 FOLLOWER]: Refusing update from remote peer 3ab6657817b84a1d94dba0a4eb2292d0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:27.840240 15529 raft_consensus.cc:2468] T 844dc4a0feff4097a52882da0b69cef7 P 24398c56cc724c91b5116396e774e09c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ab6657817b84a1d94dba0a4eb2292d0 in term 3.
I20260709 13:04:27.842170 15683 raft_consensus.cc:2804] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:27.843456 15345 leader_election.cc:304] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24398c56cc724c91b5116396e774e09c, 3ab6657817b84a1d94dba0a4eb2292d0; no voters: 
I20260709 13:04:27.846443 15683 raft_consensus.cc:697] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 4 LEADER]: Becoming Leader. State: Replica: 3ab6657817b84a1d94dba0a4eb2292d0, State: Running, Role: LEADER
I20260709 13:04:27.846560 15457 raft_consensus.cc:2468] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ab6657817b84a1d94dba0a4eb2292d0 in term 3.
I20260709 13:04:27.847090 15690 raft_consensus.cc:2804] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:27.848080 15690 raft_consensus.cc:697] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 3 LEADER]: Becoming Leader. State: Replica: 3ab6657817b84a1d94dba0a4eb2292d0, State: Running, Role: LEADER
I20260709 13:04:27.847855 15683 consensus_queue.cc:237] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:27.849081 15690 consensus_queue.cc:237] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:27.849674 15691 consensus_queue.cc:1048] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:27.862249 15279 catalog_manager.cc:5697] T b46d829f140440738e6d2d456f792c75 P 3ab6657817b84a1d94dba0a4eb2292d0 reported cstate change: term changed from 1 to 2, leader changed from df338c374ed54ecbb74bda5e628cf85e (127.14.103.2) to 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:27.869134 15457 raft_consensus.cc:1275] T 32f268907bbb4d1081fa0fccb595110e P df338c374ed54ecbb74bda5e628cf85e [term 3 FOLLOWER]: Refusing update from remote peer 3ab6657817b84a1d94dba0a4eb2292d0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:27.871558 15615 consensus_queue.cc:1048] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:27.884721 15280 catalog_manager.cc:5697] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 reported cstate change: term changed from 3 to 4, leader changed from 24398c56cc724c91b5116396e774e09c (127.14.103.3) to 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1). New cstate: current_term: 4 leader_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:27.886251 15280 catalog_manager.cc:5697] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 reported cstate change: term changed from 2 to 3, leader changed from df338c374ed54ecbb74bda5e628cf85e (127.14.103.2) to 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1). New cstate: current_term: 3 leader_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:27.905004 15589 raft_consensus.cc:993] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e: : Instructing follower 3ab6657817b84a1d94dba0a4eb2292d0 to start an election
I20260709 13:04:27.905491 15682 raft_consensus.cc:1081] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e [term 2 LEADER]: Signalling peer 3ab6657817b84a1d94dba0a4eb2292d0 to start an election
I20260709 13:04:27.906941 15380 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "36e186173f7c42dbaa9840522867ce03"
dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
 from {username='slave'} at 127.0.0.1:56574
I20260709 13:04:27.907454 15380 raft_consensus.cc:493] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:27.907829 15380 raft_consensus.cc:3060] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:27.913214 15380 raft_consensus.cc:515] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:27.915086 15380 leader_election.cc:290] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 3 election: Requested vote from peers df338c374ed54ecbb74bda5e628cf85e (127.14.103.2:45591), 24398c56cc724c91b5116396e774e09c (127.14.103.3:42385)
I20260709 13:04:27.915870 15457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36e186173f7c42dbaa9840522867ce03" candidate_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "df338c374ed54ecbb74bda5e628cf85e"
I20260709 13:04:27.916587 15457 raft_consensus.cc:3055] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:27.917079 15457 raft_consensus.cc:740] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: df338c374ed54ecbb74bda5e628cf85e, State: Running, Role: LEADER
I20260709 13:04:27.916909 15530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36e186173f7c42dbaa9840522867ce03" candidate_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "24398c56cc724c91b5116396e774e09c"
I20260709 13:04:27.917771 15530 raft_consensus.cc:3060] T 36e186173f7c42dbaa9840522867ce03 P 24398c56cc724c91b5116396e774e09c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:27.918005 15457 consensus_queue.cc:260] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:27.919771 15457 raft_consensus.cc:3060] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:27.924750 15530 raft_consensus.cc:2468] T 36e186173f7c42dbaa9840522867ce03 P 24398c56cc724c91b5116396e774e09c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ab6657817b84a1d94dba0a4eb2292d0 in term 3.
I20260709 13:04:27.925797 15345 leader_election.cc:304] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24398c56cc724c91b5116396e774e09c, 3ab6657817b84a1d94dba0a4eb2292d0; no voters: 
I20260709 13:04:27.926584 15615 raft_consensus.cc:2804] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:27.926597 15457 raft_consensus.cc:2468] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ab6657817b84a1d94dba0a4eb2292d0 in term 3.
I20260709 13:04:27.927165 15615 raft_consensus.cc:697] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 3 LEADER]: Becoming Leader. State: Replica: 3ab6657817b84a1d94dba0a4eb2292d0, State: Running, Role: LEADER
I20260709 13:04:27.928011 15615 consensus_queue.cc:237] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:27.934909 15279 catalog_manager.cc:5697] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 reported cstate change: term changed from 2 to 3, leader changed from df338c374ed54ecbb74bda5e628cf85e (127.14.103.2) to 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1). New cstate: current_term: 3 leader_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:28.225387 15530 raft_consensus.cc:1275] T b46d829f140440738e6d2d456f792c75 P 24398c56cc724c91b5116396e774e09c [term 2 FOLLOWER]: Refusing update from remote peer 3ab6657817b84a1d94dba0a4eb2292d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:28.226675 15615 consensus_queue.cc:1048] T b46d829f140440738e6d2d456f792c75 P 3ab6657817b84a1d94dba0a4eb2292d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:28.236121 15457 raft_consensus.cc:1275] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e [term 2 FOLLOWER]: Refusing update from remote peer 3ab6657817b84a1d94dba0a4eb2292d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:28.237545 15683 consensus_queue.cc:1048] T b46d829f140440738e6d2d456f792c75 P 3ab6657817b84a1d94dba0a4eb2292d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:28.349280 15455 raft_consensus.cc:1275] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e [term 3 FOLLOWER]: Refusing update from remote peer 3ab6657817b84a1d94dba0a4eb2292d0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:28.350688 15683 consensus_queue.cc:1048] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:28.362555 15529 raft_consensus.cc:1275] T 844dc4a0feff4097a52882da0b69cef7 P 24398c56cc724c91b5116396e774e09c [term 3 FOLLOWER]: Refusing update from remote peer 3ab6657817b84a1d94dba0a4eb2292d0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:28.366842 15691 consensus_queue.cc:1048] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:28.374959 15455 raft_consensus.cc:1275] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e [term 4 FOLLOWER]: Refusing update from remote peer 3ab6657817b84a1d94dba0a4eb2292d0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:28.376245 15691 consensus_queue.cc:1048] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:28.383232 15530 raft_consensus.cc:1275] T 36e186173f7c42dbaa9840522867ce03 P 24398c56cc724c91b5116396e774e09c [term 3 FOLLOWER]: Refusing update from remote peer 3ab6657817b84a1d94dba0a4eb2292d0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:28.385725 15690 consensus_queue.cc:1048] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:28.388074 15530 raft_consensus.cc:1275] T 70da92ca35b342a7b2182eaed60e3092 P 24398c56cc724c91b5116396e774e09c [term 4 FOLLOWER]: Refusing update from remote peer 3ab6657817b84a1d94dba0a4eb2292d0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:28.392874 15691 consensus_queue.cc:1048] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:28.398655 15455 raft_consensus.cc:1275] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e [term 3 FOLLOWER]: Refusing update from remote peer 3ab6657817b84a1d94dba0a4eb2292d0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:28.400646 15615 consensus_queue.cc:1048] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:30.410480 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:30.418473 15380 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "32f268907bbb4d1081fa0fccb595110e"
dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
mode: GRACEFUL
new_leader_uuid: "df338c374ed54ecbb74bda5e628cf85e"
 from {username='slave'} at 127.0.0.1:56604
I20260709 13:04:30.419039 15380 raft_consensus.cc:606] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [term 3 LEADER]: Received request to transfer leadership to TS df338c374ed54ecbb74bda5e628cf85e
I20260709 13:04:30.421358 15380 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "36e186173f7c42dbaa9840522867ce03"
dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
mode: GRACEFUL
new_leader_uuid: "24398c56cc724c91b5116396e774e09c"
 from {username='slave'} at 127.0.0.1:56604
I20260709 13:04:30.421818 15380 raft_consensus.cc:606] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 3 LEADER]: Received request to transfer leadership to TS 24398c56cc724c91b5116396e774e09c
I20260709 13:04:30.423941 15380 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "844dc4a0feff4097a52882da0b69cef7"
dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
mode: GRACEFUL
new_leader_uuid: "df338c374ed54ecbb74bda5e628cf85e"
 from {username='slave'} at 127.0.0.1:56604
I20260709 13:04:30.424396 15380 raft_consensus.cc:606] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 3 LEADER]: Received request to transfer leadership to TS df338c374ed54ecbb74bda5e628cf85e
I20260709 13:04:30.426668 15380 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d3551b1f0a4e4d02af6cbe21a747f4ec"
dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
mode: GRACEFUL
new_leader_uuid: "24398c56cc724c91b5116396e774e09c"
 from {username='slave'} at 127.0.0.1:56604
I20260709 13:04:30.427139 15380 raft_consensus.cc:606] T d3551b1f0a4e4d02af6cbe21a747f4ec P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 LEADER]: Received request to transfer leadership to TS 24398c56cc724c91b5116396e774e09c
I20260709 13:04:30.428258 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:04:30.428823 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:30.487995 15683 raft_consensus.cc:993] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0: : Instructing follower df338c374ed54ecbb74bda5e628cf85e to start an election
I20260709 13:04:30.488417 15709 raft_consensus.cc:1081] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [term 3 LEADER]: Signalling peer df338c374ed54ecbb74bda5e628cf85e to start an election
I20260709 13:04:30.489931 15455 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "32f268907bbb4d1081fa0fccb595110e"
dest_uuid: "df338c374ed54ecbb74bda5e628cf85e"
 from {username='slave'} at 127.0.0.1:50216
I20260709 13:04:30.490381 15455 raft_consensus.cc:493] T 32f268907bbb4d1081fa0fccb595110e P df338c374ed54ecbb74bda5e628cf85e [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:30.490620 15455 raft_consensus.cc:3060] T 32f268907bbb4d1081fa0fccb595110e P df338c374ed54ecbb74bda5e628cf85e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:30.496243 15455 raft_consensus.cc:515] T 32f268907bbb4d1081fa0fccb595110e P df338c374ed54ecbb74bda5e628cf85e [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:30.497833 15455 leader_election.cc:290] T 32f268907bbb4d1081fa0fccb595110e P df338c374ed54ecbb74bda5e628cf85e [CANDIDATE]: Term 4 election: Requested vote from peers 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793), 24398c56cc724c91b5116396e774e09c (127.14.103.3:42385)
I20260709 13:04:30.498679 15380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32f268907bbb4d1081fa0fccb595110e" candidate_uuid: "df338c374ed54ecbb74bda5e628cf85e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
I20260709 13:04:30.498939 15530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32f268907bbb4d1081fa0fccb595110e" candidate_uuid: "df338c374ed54ecbb74bda5e628cf85e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "24398c56cc724c91b5116396e774e09c"
I20260709 13:04:30.499351 15380 raft_consensus.cc:3055] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:30.499459 15530 raft_consensus.cc:3060] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:30.499804 15380 raft_consensus.cc:740] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3ab6657817b84a1d94dba0a4eb2292d0, State: Running, Role: LEADER
I20260709 13:04:30.500707 15380 consensus_queue.cc:260] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:30.502028 15380 raft_consensus.cc:3060] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:30.504180 15530 raft_consensus.cc:2468] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df338c374ed54ecbb74bda5e628cf85e in term 4.
I20260709 13:04:30.505224 15420 leader_election.cc:304] T 32f268907bbb4d1081fa0fccb595110e P df338c374ed54ecbb74bda5e628cf85e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24398c56cc724c91b5116396e774e09c, df338c374ed54ecbb74bda5e628cf85e; no voters: 
I20260709 13:04:30.505911 15708 raft_consensus.cc:2804] T 32f268907bbb4d1081fa0fccb595110e P df338c374ed54ecbb74bda5e628cf85e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:30.506426 15708 raft_consensus.cc:697] T 32f268907bbb4d1081fa0fccb595110e P df338c374ed54ecbb74bda5e628cf85e [term 4 LEADER]: Becoming Leader. State: Replica: df338c374ed54ecbb74bda5e628cf85e, State: Running, Role: LEADER
I20260709 13:04:30.507325 15708 consensus_queue.cc:237] T 32f268907bbb4d1081fa0fccb595110e P df338c374ed54ecbb74bda5e628cf85e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:30.508090 15380 raft_consensus.cc:2468] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df338c374ed54ecbb74bda5e628cf85e in term 4.
I20260709 13:04:30.515087 15280 catalog_manager.cc:5697] T 32f268907bbb4d1081fa0fccb595110e P df338c374ed54ecbb74bda5e628cf85e reported cstate change: term changed from 3 to 4, leader changed from 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1) to df338c374ed54ecbb74bda5e628cf85e (127.14.103.2). New cstate: current_term: 4 leader_uuid: "df338c374ed54ecbb74bda5e628cf85e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:30.548414 15683 raft_consensus.cc:993] T d3551b1f0a4e4d02af6cbe21a747f4ec P 3ab6657817b84a1d94dba0a4eb2292d0: : Instructing follower 24398c56cc724c91b5116396e774e09c to start an election
I20260709 13:04:30.548893 15709 raft_consensus.cc:1081] T d3551b1f0a4e4d02af6cbe21a747f4ec P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 LEADER]: Signalling peer 24398c56cc724c91b5116396e774e09c to start an election
I20260709 13:04:30.550329 15530 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d3551b1f0a4e4d02af6cbe21a747f4ec"
dest_uuid: "24398c56cc724c91b5116396e774e09c"
 from {username='slave'} at 127.0.0.1:57242
I20260709 13:04:30.550801 15530 raft_consensus.cc:493] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:30.551074 15530 raft_consensus.cc:3060] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:30.555015 15530 raft_consensus.cc:515] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:30.556386 15530 leader_election.cc:290] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 3 election: Requested vote from peers 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793), df338c374ed54ecbb74bda5e628cf85e (127.14.103.2:45591)
I20260709 13:04:30.557332 15380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3551b1f0a4e4d02af6cbe21a747f4ec" candidate_uuid: "24398c56cc724c91b5116396e774e09c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
I20260709 13:04:30.557699 15455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3551b1f0a4e4d02af6cbe21a747f4ec" candidate_uuid: "24398c56cc724c91b5116396e774e09c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "df338c374ed54ecbb74bda5e628cf85e"
I20260709 13:04:30.557845 15380 raft_consensus.cc:3055] T d3551b1f0a4e4d02af6cbe21a747f4ec P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:30.558218 15380 raft_consensus.cc:740] T d3551b1f0a4e4d02af6cbe21a747f4ec P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ab6657817b84a1d94dba0a4eb2292d0, State: Running, Role: LEADER
I20260709 13:04:30.558270 15455 raft_consensus.cc:3060] T d3551b1f0a4e4d02af6cbe21a747f4ec P df338c374ed54ecbb74bda5e628cf85e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:30.559010 15380 consensus_queue.cc:260] T d3551b1f0a4e4d02af6cbe21a747f4ec P 3ab6657817b84a1d94dba0a4eb2292d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:30.559959 15380 raft_consensus.cc:3060] T d3551b1f0a4e4d02af6cbe21a747f4ec P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:30.563458 15455 raft_consensus.cc:2468] T d3551b1f0a4e4d02af6cbe21a747f4ec P df338c374ed54ecbb74bda5e628cf85e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24398c56cc724c91b5116396e774e09c in term 3.
I20260709 13:04:30.564194 15380 raft_consensus.cc:2468] T d3551b1f0a4e4d02af6cbe21a747f4ec P 3ab6657817b84a1d94dba0a4eb2292d0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24398c56cc724c91b5116396e774e09c in term 3.
I20260709 13:04:30.564805 15497 leader_election.cc:304] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24398c56cc724c91b5116396e774e09c, df338c374ed54ecbb74bda5e628cf85e; no voters: 
I20260709 13:04:30.565537 15676 raft_consensus.cc:2804] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:30.566598 15676 raft_consensus.cc:697] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [term 3 LEADER]: Becoming Leader. State: Replica: 24398c56cc724c91b5116396e774e09c, State: Running, Role: LEADER
I20260709 13:04:30.567378 15676 consensus_queue.cc:237] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:30.573793 15280 catalog_manager.cc:5697] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c reported cstate change: term changed from 2 to 3, leader changed from 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1) to 24398c56cc724c91b5116396e774e09c (127.14.103.3). New cstate: current_term: 3 leader_uuid: "24398c56cc724c91b5116396e774e09c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:30.637230 15683 raft_consensus.cc:993] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0: : Instructing follower 24398c56cc724c91b5116396e774e09c to start an election
I20260709 13:04:30.637746 15709 raft_consensus.cc:1081] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 3 LEADER]: Signalling peer 24398c56cc724c91b5116396e774e09c to start an election
I20260709 13:04:30.639279 15530 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "36e186173f7c42dbaa9840522867ce03"
dest_uuid: "24398c56cc724c91b5116396e774e09c"
 from {username='slave'} at 127.0.0.1:57242
I20260709 13:04:30.639807 15530 raft_consensus.cc:493] T 36e186173f7c42dbaa9840522867ce03 P 24398c56cc724c91b5116396e774e09c [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:30.640149 15530 raft_consensus.cc:3060] T 36e186173f7c42dbaa9840522867ce03 P 24398c56cc724c91b5116396e774e09c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:30.644375 15530 raft_consensus.cc:515] T 36e186173f7c42dbaa9840522867ce03 P 24398c56cc724c91b5116396e774e09c [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:30.645984 15530 leader_election.cc:290] T 36e186173f7c42dbaa9840522867ce03 P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 4 election: Requested vote from peers 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793), df338c374ed54ecbb74bda5e628cf85e (127.14.103.2:45591)
I20260709 13:04:30.646694 15380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36e186173f7c42dbaa9840522867ce03" candidate_uuid: "24398c56cc724c91b5116396e774e09c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
I20260709 13:04:30.646996 15455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36e186173f7c42dbaa9840522867ce03" candidate_uuid: "24398c56cc724c91b5116396e774e09c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "df338c374ed54ecbb74bda5e628cf85e"
I20260709 13:04:30.647289 15380 raft_consensus.cc:3055] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:30.647531 15455 raft_consensus.cc:3060] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:30.647643 15380 raft_consensus.cc:740] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3ab6657817b84a1d94dba0a4eb2292d0, State: Running, Role: LEADER
I20260709 13:04:30.648550 15380 consensus_queue.cc:260] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:30.649739 15380 raft_consensus.cc:3060] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:30.654451 15455 raft_consensus.cc:2468] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24398c56cc724c91b5116396e774e09c in term 4.
I20260709 13:04:30.655596 15497 leader_election.cc:304] T 36e186173f7c42dbaa9840522867ce03 P 24398c56cc724c91b5116396e774e09c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24398c56cc724c91b5116396e774e09c, df338c374ed54ecbb74bda5e628cf85e; no voters: 
I20260709 13:04:30.656180 15380 raft_consensus.cc:2468] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24398c56cc724c91b5116396e774e09c in term 4.
I20260709 13:04:30.656355 15676 raft_consensus.cc:2804] T 36e186173f7c42dbaa9840522867ce03 P 24398c56cc724c91b5116396e774e09c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:30.656860 15676 raft_consensus.cc:697] T 36e186173f7c42dbaa9840522867ce03 P 24398c56cc724c91b5116396e774e09c [term 4 LEADER]: Becoming Leader. State: Replica: 24398c56cc724c91b5116396e774e09c, State: Running, Role: LEADER
I20260709 13:04:30.657606 15676 consensus_queue.cc:237] T 36e186173f7c42dbaa9840522867ce03 P 24398c56cc724c91b5116396e774e09c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:30.665132 15280 catalog_manager.cc:5697] T 36e186173f7c42dbaa9840522867ce03 P 24398c56cc724c91b5116396e774e09c reported cstate change: term changed from 3 to 4, leader changed from 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1) to 24398c56cc724c91b5116396e774e09c (127.14.103.3). New cstate: current_term: 4 leader_uuid: "24398c56cc724c91b5116396e774e09c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:30.821183 15683 raft_consensus.cc:993] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0: : Instructing follower df338c374ed54ecbb74bda5e628cf85e to start an election
I20260709 13:04:30.821630 15709 raft_consensus.cc:1081] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 3 LEADER]: Signalling peer df338c374ed54ecbb74bda5e628cf85e to start an election
I20260709 13:04:30.822842 15455 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "844dc4a0feff4097a52882da0b69cef7"
dest_uuid: "df338c374ed54ecbb74bda5e628cf85e"
 from {username='slave'} at 127.0.0.1:50216
I20260709 13:04:30.823288 15455 raft_consensus.cc:493] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:30.823590 15455 raft_consensus.cc:3060] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:30.828136 15455 raft_consensus.cc:515] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:30.829757 15455 leader_election.cc:290] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e [CANDIDATE]: Term 4 election: Requested vote from peers 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793), 24398c56cc724c91b5116396e774e09c (127.14.103.3:42385)
I20260709 13:04:30.830554 15380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844dc4a0feff4097a52882da0b69cef7" candidate_uuid: "df338c374ed54ecbb74bda5e628cf85e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3ab6657817b84a1d94dba0a4eb2292d0"
I20260709 13:04:30.830749 15530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844dc4a0feff4097a52882da0b69cef7" candidate_uuid: "df338c374ed54ecbb74bda5e628cf85e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "24398c56cc724c91b5116396e774e09c"
I20260709 13:04:30.831130 15380 raft_consensus.cc:3055] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:30.831183 15530 raft_consensus.cc:3060] T 844dc4a0feff4097a52882da0b69cef7 P 24398c56cc724c91b5116396e774e09c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:30.831594 15380 raft_consensus.cc:740] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3ab6657817b84a1d94dba0a4eb2292d0, State: Running, Role: LEADER
I20260709 13:04:30.832384 15380 consensus_queue.cc:260] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:30.833532 15380 raft_consensus.cc:3060] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:30.837342 15530 raft_consensus.cc:2468] T 844dc4a0feff4097a52882da0b69cef7 P 24398c56cc724c91b5116396e774e09c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df338c374ed54ecbb74bda5e628cf85e in term 4.
I20260709 13:04:30.838374 15420 leader_election.cc:304] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24398c56cc724c91b5116396e774e09c, df338c374ed54ecbb74bda5e628cf85e; no voters: 
I20260709 13:04:30.839130 15708 raft_consensus.cc:2804] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:30.839627 15708 raft_consensus.cc:697] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e [term 4 LEADER]: Becoming Leader. State: Replica: df338c374ed54ecbb74bda5e628cf85e, State: Running, Role: LEADER
I20260709 13:04:30.839910 15380 raft_consensus.cc:2468] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df338c374ed54ecbb74bda5e628cf85e in term 4.
I20260709 13:04:30.840492 15708 consensus_queue.cc:237] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } }
I20260709 13:04:30.847298 15280 catalog_manager.cc:5697] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e reported cstate change: term changed from 3 to 4, leader changed from 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1) to df338c374ed54ecbb74bda5e628cf85e (127.14.103.2). New cstate: current_term: 4 leader_uuid: "df338c374ed54ecbb74bda5e628cf85e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:31.052381 15530 raft_consensus.cc:1275] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c [term 4 FOLLOWER]: Refusing update from remote peer df338c374ed54ecbb74bda5e628cf85e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:31.053644 15708 consensus_queue.cc:1048] T 32f268907bbb4d1081fa0fccb595110e P df338c374ed54ecbb74bda5e628cf85e [LEADER]: Connected to new peer: Peer: permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:31.064604 15380 raft_consensus.cc:1275] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [term 4 FOLLOWER]: Refusing update from remote peer df338c374ed54ecbb74bda5e628cf85e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:31.065834 15707 consensus_queue.cc:1048] T 32f268907bbb4d1081fa0fccb595110e P df338c374ed54ecbb74bda5e628cf85e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:31.110453 15380 raft_consensus.cc:1275] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 4 FOLLOWER]: Refusing update from remote peer 24398c56cc724c91b5116396e774e09c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:31.111764 15676 consensus_queue.cc:1048] T 36e186173f7c42dbaa9840522867ce03 P 24398c56cc724c91b5116396e774e09c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:31.120296 15455 raft_consensus.cc:1275] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e [term 4 FOLLOWER]: Refusing update from remote peer 24398c56cc724c91b5116396e774e09c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:31.121547 15725 consensus_queue.cc:1048] T 36e186173f7c42dbaa9840522867ce03 P 24398c56cc724c91b5116396e774e09c [LEADER]: Connected to new peer: Peer: permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:31.122462 15382 raft_consensus.cc:1275] T d3551b1f0a4e4d02af6cbe21a747f4ec P 3ab6657817b84a1d94dba0a4eb2292d0 [term 3 FOLLOWER]: Refusing update from remote peer 24398c56cc724c91b5116396e774e09c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:31.126372 15676 consensus_queue.cc:1048] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:31.140836 15455 raft_consensus.cc:1275] T d3551b1f0a4e4d02af6cbe21a747f4ec P df338c374ed54ecbb74bda5e628cf85e [term 3 FOLLOWER]: Refusing update from remote peer 24398c56cc724c91b5116396e774e09c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:31.142503 15676 consensus_queue.cc:1048] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [LEADER]: Connected to new peer: Peer: permanent_uuid: "df338c374ed54ecbb74bda5e628cf85e" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:31.353477 15530 raft_consensus.cc:1275] T 844dc4a0feff4097a52882da0b69cef7 P 24398c56cc724c91b5116396e774e09c [term 4 FOLLOWER]: Refusing update from remote peer df338c374ed54ecbb74bda5e628cf85e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:31.355038 15707 consensus_queue.cc:1048] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e [LEADER]: Connected to new peer: Peer: permanent_uuid: "24398c56cc724c91b5116396e774e09c" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 42385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:31.363747 15382 raft_consensus.cc:1275] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 4 FOLLOWER]: Refusing update from remote peer df338c374ed54ecbb74bda5e628cf85e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:31.365537 15707 consensus_queue.cc:1048] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ab6657817b84a1d94dba0a4eb2292d0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 32793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:31.429715 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:31.435366 14748 tablet_server.cc:179] TabletServer@127.14.103.1:0 shutting down...
I20260709 13:04:31.459520 14748 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:31.460206 14748 tablet_replica.cc:333] T deb69cd9c0414a348a3ee16058a36f00 P 3ab6657817b84a1d94dba0a4eb2292d0: stopping tablet replica
I20260709 13:04:31.460807 14748 raft_consensus.cc:2243] T deb69cd9c0414a348a3ee16058a36f00 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.461262 14748 raft_consensus.cc:2272] T deb69cd9c0414a348a3ee16058a36f00 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.463272 14748 tablet_replica.cc:333] T e7be94629c3c46638abeb602169bcda4 P 3ab6657817b84a1d94dba0a4eb2292d0: stopping tablet replica
I20260709 13:04:31.463969 14748 raft_consensus.cc:2243] T e7be94629c3c46638abeb602169bcda4 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:04:31.464763 15423 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 11 similar messages]
I20260709 13:04:31.464883 14748 raft_consensus.cc:2272] T e7be94629c3c46638abeb602169bcda4 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.467422 14748 tablet_replica.cc:333] T 42e07a8df5ac47eea3da5846963de3fa P 3ab6657817b84a1d94dba0a4eb2292d0: stopping tablet replica
W20260709 13:04:31.467540 15423 consensus_peers.cc:597] T 32f268907bbb4d1081fa0fccb595110e P df338c374ed54ecbb74bda5e628cf85e -> Peer 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793): Couldn't send request to peer 3ab6657817b84a1d94dba0a4eb2292d0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:31.468327 14748 raft_consensus.cc:2243] T 42e07a8df5ac47eea3da5846963de3fa P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:31.469316 14748 raft_consensus.cc:2272] T 42e07a8df5ac47eea3da5846963de3fa P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.471084 14748 tablet_replica.cc:333] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0: stopping tablet replica
I20260709 13:04:31.471608 14748 raft_consensus.cc:2243] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.472086 14748 raft_consensus.cc:2272] T 32f268907bbb4d1081fa0fccb595110e P 3ab6657817b84a1d94dba0a4eb2292d0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.474232 14748 tablet_replica.cc:333] T 3590c79d23594f91a88f9646df62c52a P 3ab6657817b84a1d94dba0a4eb2292d0: stopping tablet replica
I20260709 13:04:31.474737 14748 raft_consensus.cc:2243] T 3590c79d23594f91a88f9646df62c52a P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.475106 14748 raft_consensus.cc:2272] T 3590c79d23594f91a88f9646df62c52a P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.477021 14748 tablet_replica.cc:333] T d3551b1f0a4e4d02af6cbe21a747f4ec P 3ab6657817b84a1d94dba0a4eb2292d0: stopping tablet replica
I20260709 13:04:31.477496 14748 raft_consensus.cc:2243] T d3551b1f0a4e4d02af6cbe21a747f4ec P 3ab6657817b84a1d94dba0a4eb2292d0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.477833 14748 raft_consensus.cc:2272] T d3551b1f0a4e4d02af6cbe21a747f4ec P 3ab6657817b84a1d94dba0a4eb2292d0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.480089 14748 tablet_replica.cc:333] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0: stopping tablet replica
I20260709 13:04:31.480551 14748 raft_consensus.cc:2243] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:04:31.481145 14748 raft_consensus.cc:2272] T 70da92ca35b342a7b2182eaed60e3092 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.482565 14748 tablet_replica.cc:333] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0: stopping tablet replica
I20260709 13:04:31.482963 14748 raft_consensus.cc:2243] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.483274 14748 raft_consensus.cc:2272] T 36e186173f7c42dbaa9840522867ce03 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.485122 14748 tablet_replica.cc:333] T b46d829f140440738e6d2d456f792c75 P 3ab6657817b84a1d94dba0a4eb2292d0: stopping tablet replica
I20260709 13:04:31.485597 14748 raft_consensus.cc:2243] T b46d829f140440738e6d2d456f792c75 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:31.486239 14748 raft_consensus.cc:2272] T b46d829f140440738e6d2d456f792c75 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.487866 14748 tablet_replica.cc:333] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0: stopping tablet replica
I20260709 13:04:31.488289 14748 raft_consensus.cc:2243] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.488667 14748 raft_consensus.cc:2272] T 844dc4a0feff4097a52882da0b69cef7 P 3ab6657817b84a1d94dba0a4eb2292d0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.518198 14748 tablet_server.cc:196] TabletServer@127.14.103.1:0 shutdown complete.
I20260709 13:04:31.536218 14748 tablet_server.cc:179] TabletServer@127.14.103.2:0 shutting down...
W20260709 13:04:31.546453 15497 consensus_peers.cc:597] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c -> Peer df338c374ed54ecbb74bda5e628cf85e (127.14.103.2:45591): Couldn't send request to peer df338c374ed54ecbb74bda5e628cf85e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:31.567515 14748 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:31.568220 14748 tablet_replica.cc:333] T deb69cd9c0414a348a3ee16058a36f00 P df338c374ed54ecbb74bda5e628cf85e: stopping tablet replica
I20260709 13:04:31.568902 14748 raft_consensus.cc:2243] T deb69cd9c0414a348a3ee16058a36f00 P df338c374ed54ecbb74bda5e628cf85e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.569455 14748 raft_consensus.cc:2272] T deb69cd9c0414a348a3ee16058a36f00 P df338c374ed54ecbb74bda5e628cf85e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.571231 14748 tablet_replica.cc:333] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e: stopping tablet replica
I20260709 13:04:31.571887 14748 raft_consensus.cc:2243] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.572283 14748 raft_consensus.cc:2272] T b46d829f140440738e6d2d456f792c75 P df338c374ed54ecbb74bda5e628cf85e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.573832 14748 tablet_replica.cc:333] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e: stopping tablet replica
I20260709 13:04:31.574297 14748 raft_consensus.cc:2243] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:04:31.574991 14748 raft_consensus.cc:2272] T 844dc4a0feff4097a52882da0b69cef7 P df338c374ed54ecbb74bda5e628cf85e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.577579 14748 tablet_replica.cc:333] T 3590c79d23594f91a88f9646df62c52a P df338c374ed54ecbb74bda5e628cf85e: stopping tablet replica
I20260709 13:04:31.578169 14748 raft_consensus.cc:2243] T 3590c79d23594f91a88f9646df62c52a P df338c374ed54ecbb74bda5e628cf85e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:31.578871 14748 raft_consensus.cc:2272] T 3590c79d23594f91a88f9646df62c52a P df338c374ed54ecbb74bda5e628cf85e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.580683 14748 tablet_replica.cc:333] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e: stopping tablet replica
W20260709 13:04:31.580915 15497 consensus_peers.cc:597] T deb69cd9c0414a348a3ee16058a36f00 P 24398c56cc724c91b5116396e774e09c -> Peer 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793): Couldn't send request to peer 3ab6657817b84a1d94dba0a4eb2292d0. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:32793: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:31.581262 14748 raft_consensus.cc:2243] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.581722 14748 raft_consensus.cc:2272] T 70da92ca35b342a7b2182eaed60e3092 P df338c374ed54ecbb74bda5e628cf85e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.583353 14748 tablet_replica.cc:333] T d3551b1f0a4e4d02af6cbe21a747f4ec P df338c374ed54ecbb74bda5e628cf85e: stopping tablet replica
I20260709 13:04:31.583911 14748 raft_consensus.cc:2243] T d3551b1f0a4e4d02af6cbe21a747f4ec P df338c374ed54ecbb74bda5e628cf85e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.584266 14748 raft_consensus.cc:2272] T d3551b1f0a4e4d02af6cbe21a747f4ec P df338c374ed54ecbb74bda5e628cf85e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.586102 14748 tablet_replica.cc:333] T 32f268907bbb4d1081fa0fccb595110e P df338c374ed54ecbb74bda5e628cf85e: stopping tablet replica
I20260709 13:04:31.586580 14748 raft_consensus.cc:2243] T 32f268907bbb4d1081fa0fccb595110e P df338c374ed54ecbb74bda5e628cf85e [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:04:31.587232 14748 raft_consensus.cc:2272] T 32f268907bbb4d1081fa0fccb595110e P df338c374ed54ecbb74bda5e628cf85e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.589010 14748 tablet_replica.cc:333] T 42e07a8df5ac47eea3da5846963de3fa P df338c374ed54ecbb74bda5e628cf85e: stopping tablet replica
I20260709 13:04:31.589496 14748 raft_consensus.cc:2243] T 42e07a8df5ac47eea3da5846963de3fa P df338c374ed54ecbb74bda5e628cf85e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.589888 14748 raft_consensus.cc:2272] T 42e07a8df5ac47eea3da5846963de3fa P df338c374ed54ecbb74bda5e628cf85e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.591451 14748 tablet_replica.cc:333] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e: stopping tablet replica
I20260709 13:04:31.592242 14748 raft_consensus.cc:2243] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.592748 14748 raft_consensus.cc:2272] T 36e186173f7c42dbaa9840522867ce03 P df338c374ed54ecbb74bda5e628cf85e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.595713 14748 tablet_replica.cc:333] T e7be94629c3c46638abeb602169bcda4 P df338c374ed54ecbb74bda5e628cf85e: stopping tablet replica
I20260709 13:04:31.596318 14748 raft_consensus.cc:2243] T e7be94629c3c46638abeb602169bcda4 P df338c374ed54ecbb74bda5e628cf85e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.596817 14748 raft_consensus.cc:2272] T e7be94629c3c46638abeb602169bcda4 P df338c374ed54ecbb74bda5e628cf85e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.626158 14748 tablet_server.cc:196] TabletServer@127.14.103.2:0 shutdown complete.
I20260709 13:04:31.643760 14748 tablet_server.cc:179] TabletServer@127.14.103.3:0 shutting down...
W20260709 13:04:31.649364 15497 consensus_peers.cc:597] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c -> Peer 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793): Couldn't send request to peer 3ab6657817b84a1d94dba0a4eb2292d0. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:32793: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:31.664980 15497 consensus_peers.cc:597] T 36e186173f7c42dbaa9840522867ce03 P 24398c56cc724c91b5116396e774e09c -> Peer 3ab6657817b84a1d94dba0a4eb2292d0 (127.14.103.1:32793): Couldn't send request to peer 3ab6657817b84a1d94dba0a4eb2292d0. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:32793: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:31.667958 14748 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:31.668578 14748 tablet_replica.cc:333] T 36e186173f7c42dbaa9840522867ce03 P 24398c56cc724c91b5116396e774e09c: stopping tablet replica
I20260709 13:04:31.669194 14748 raft_consensus.cc:2243] T 36e186173f7c42dbaa9840522867ce03 P 24398c56cc724c91b5116396e774e09c [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:04:31.669951 14748 raft_consensus.cc:2272] T 36e186173f7c42dbaa9840522867ce03 P 24398c56cc724c91b5116396e774e09c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.672202 14748 tablet_replica.cc:333] T e7be94629c3c46638abeb602169bcda4 P 24398c56cc724c91b5116396e774e09c: stopping tablet replica
I20260709 13:04:31.672709 14748 raft_consensus.cc:2243] T e7be94629c3c46638abeb602169bcda4 P 24398c56cc724c91b5116396e774e09c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.673151 14748 raft_consensus.cc:2272] T e7be94629c3c46638abeb602169bcda4 P 24398c56cc724c91b5116396e774e09c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.674741 14748 tablet_replica.cc:333] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c: stopping tablet replica
I20260709 13:04:31.675212 14748 raft_consensus.cc:2243] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.675594 14748 raft_consensus.cc:2272] T 3590c79d23594f91a88f9646df62c52a P 24398c56cc724c91b5116396e774e09c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.677084 14748 tablet_replica.cc:333] T 70da92ca35b342a7b2182eaed60e3092 P 24398c56cc724c91b5116396e774e09c: stopping tablet replica
I20260709 13:04:31.677599 14748 raft_consensus.cc:2243] T 70da92ca35b342a7b2182eaed60e3092 P 24398c56cc724c91b5116396e774e09c [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.677922 14748 raft_consensus.cc:2272] T 70da92ca35b342a7b2182eaed60e3092 P 24398c56cc724c91b5116396e774e09c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.679289 14748 tablet_replica.cc:333] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c: stopping tablet replica
I20260709 13:04:31.679814 14748 raft_consensus.cc:2243] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:04:31.680446 14748 raft_consensus.cc:2272] T d3551b1f0a4e4d02af6cbe21a747f4ec P 24398c56cc724c91b5116396e774e09c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.682718 14748 tablet_replica.cc:333] T deb69cd9c0414a348a3ee16058a36f00 P 24398c56cc724c91b5116396e774e09c: stopping tablet replica
I20260709 13:04:31.683131 14748 raft_consensus.cc:2243] T deb69cd9c0414a348a3ee16058a36f00 P 24398c56cc724c91b5116396e774e09c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:31.683856 14748 raft_consensus.cc:2272] T deb69cd9c0414a348a3ee16058a36f00 P 24398c56cc724c91b5116396e774e09c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.685293 14748 tablet_replica.cc:333] T 42e07a8df5ac47eea3da5846963de3fa P 24398c56cc724c91b5116396e774e09c: stopping tablet replica
I20260709 13:04:31.685676 14748 raft_consensus.cc:2243] T 42e07a8df5ac47eea3da5846963de3fa P 24398c56cc724c91b5116396e774e09c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.686002 14748 raft_consensus.cc:2272] T 42e07a8df5ac47eea3da5846963de3fa P 24398c56cc724c91b5116396e774e09c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.687493 14748 tablet_replica.cc:333] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c: stopping tablet replica
I20260709 13:04:31.687950 14748 raft_consensus.cc:2243] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.688277 14748 raft_consensus.cc:2272] T 32f268907bbb4d1081fa0fccb595110e P 24398c56cc724c91b5116396e774e09c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.689991 14748 tablet_replica.cc:333] T 844dc4a0feff4097a52882da0b69cef7 P 24398c56cc724c91b5116396e774e09c: stopping tablet replica
I20260709 13:04:31.690369 14748 raft_consensus.cc:2243] T 844dc4a0feff4097a52882da0b69cef7 P 24398c56cc724c91b5116396e774e09c [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.690697 14748 raft_consensus.cc:2272] T 844dc4a0feff4097a52882da0b69cef7 P 24398c56cc724c91b5116396e774e09c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.692394 14748 tablet_replica.cc:333] T b46d829f140440738e6d2d456f792c75 P 24398c56cc724c91b5116396e774e09c: stopping tablet replica
I20260709 13:04:31.692780 14748 raft_consensus.cc:2243] T b46d829f140440738e6d2d456f792c75 P 24398c56cc724c91b5116396e774e09c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:31.693097 14748 raft_consensus.cc:2272] T b46d829f140440738e6d2d456f792c75 P 24398c56cc724c91b5116396e774e09c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.718760 14748 tablet_server.cc:196] TabletServer@127.14.103.3:0 shutdown complete.
I20260709 13:04:31.735456 14748 master.cc:562] Master@127.14.103.62:38869 shutting down...
I20260709 13:04:31.754283 14748 raft_consensus.cc:2243] T 00000000000000000000000000000000 P fc08470892c143c685cfee2af1e4a7a5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:31.754812 14748 raft_consensus.cc:2272] T 00000000000000000000000000000000 P fc08470892c143c685cfee2af1e4a7a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:31.755141 14748 tablet_replica.cc:333] T 00000000000000000000000000000000 P fc08470892c143c685cfee2af1e4a7a5: stopping tablet replica
I20260709 13:04:31.775179 14748 master.cc:584] Master@127.14.103.62:38869 shutdown complete.
24398c56cc724c91b5116396e774e09c  1
3ab6657817b84a1d94dba0a4eb2292d0  5
df338c374ed54ecbb74bda5e628cf85e  4
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (10428 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 13:04:31.816488 14748 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.103.62:42985
I20260709 13:04:31.817521 14748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:31.823268 15735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:31.823273 15736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:31.823793 15738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:31.824369 14748 server_base.cc:1061] running on GCE node
I20260709 13:04:31.825076 14748 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:31.825287 14748 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:31.825443 14748 hybrid_clock.cc:648] HybridClock initialized: now 1783602271825424 us; error 0 us; skew 500 ppm
I20260709 13:04:31.825935 14748 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:31.828289 14748 webserver.cc:533] Webserver started at http://127.14.103.62:36211/ using document root <none> and password file <none>
I20260709 13:04:31.828744 14748 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:31.828943 14748 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:31.829217 14748 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:31.830289 14748 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/master-0-root/instance:
uuid: "d33cebb189b14a989836214fdbafb898"
format_stamp: "Formatted at 2026-07-09 13:04:31 on dist-test-slave-fpp7"
I20260709 13:04:31.834391 14748 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.002s
I20260709 13:04:31.837548 15743 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:31.838307 14748 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:04:31.838595 14748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/master-0-root
uuid: "d33cebb189b14a989836214fdbafb898"
format_stamp: "Formatted at 2026-07-09 13:04:31 on dist-test-slave-fpp7"
I20260709 13:04:31.838868 14748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:31.856143 14748 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:31.857299 14748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:31.893273 14748 rpc_server.cc:307] RPC server started. Bound to: 127.14.103.62:42985
I20260709 13:04:31.893368 15794 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.103.62:42985 every 8 connection(s)
I20260709 13:04:31.896939 15795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:31.906009 15795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d33cebb189b14a989836214fdbafb898: Bootstrap starting.
I20260709 13:04:31.910218 15795 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d33cebb189b14a989836214fdbafb898: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:31.914008 15795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d33cebb189b14a989836214fdbafb898: No bootstrap required, opened a new log
I20260709 13:04:31.915939 15795 raft_consensus.cc:359] T 00000000000000000000000000000000 P d33cebb189b14a989836214fdbafb898 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d33cebb189b14a989836214fdbafb898" member_type: VOTER }
I20260709 13:04:31.916273 15795 raft_consensus.cc:385] T 00000000000000000000000000000000 P d33cebb189b14a989836214fdbafb898 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:31.916453 15795 raft_consensus.cc:740] T 00000000000000000000000000000000 P d33cebb189b14a989836214fdbafb898 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d33cebb189b14a989836214fdbafb898, State: Initialized, Role: FOLLOWER
I20260709 13:04:31.916916 15795 consensus_queue.cc:260] T 00000000000000000000000000000000 P d33cebb189b14a989836214fdbafb898 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d33cebb189b14a989836214fdbafb898" member_type: VOTER }
I20260709 13:04:31.917281 15795 raft_consensus.cc:399] T 00000000000000000000000000000000 P d33cebb189b14a989836214fdbafb898 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:04:31.917461 15795 raft_consensus.cc:493] T 00000000000000000000000000000000 P d33cebb189b14a989836214fdbafb898 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:04:31.917666 15795 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d33cebb189b14a989836214fdbafb898 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:31.921593 15795 raft_consensus.cc:515] T 00000000000000000000000000000000 P d33cebb189b14a989836214fdbafb898 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d33cebb189b14a989836214fdbafb898" member_type: VOTER }
I20260709 13:04:31.922015 15795 leader_election.cc:304] T 00000000000000000000000000000000 P d33cebb189b14a989836214fdbafb898 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d33cebb189b14a989836214fdbafb898; no voters: 
I20260709 13:04:31.922983 15795 leader_election.cc:290] T 00000000000000000000000000000000 P d33cebb189b14a989836214fdbafb898 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:04:31.923298 15798 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d33cebb189b14a989836214fdbafb898 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:31.924654 15798 raft_consensus.cc:697] T 00000000000000000000000000000000 P d33cebb189b14a989836214fdbafb898 [term 1 LEADER]: Becoming Leader. State: Replica: d33cebb189b14a989836214fdbafb898, State: Running, Role: LEADER
I20260709 13:04:31.925323 15798 consensus_queue.cc:237] T 00000000000000000000000000000000 P d33cebb189b14a989836214fdbafb898 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d33cebb189b14a989836214fdbafb898" member_type: VOTER }
I20260709 13:04:31.926013 15795 sys_catalog.cc:565] T 00000000000000000000000000000000 P d33cebb189b14a989836214fdbafb898 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:04:31.930457 15800 sys_catalog.cc:455] T 00000000000000000000000000000000 P d33cebb189b14a989836214fdbafb898 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d33cebb189b14a989836214fdbafb898. Latest consensus state: current_term: 1 leader_uuid: "d33cebb189b14a989836214fdbafb898" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d33cebb189b14a989836214fdbafb898" member_type: VOTER } }
I20260709 13:04:31.930548 15799 sys_catalog.cc:455] T 00000000000000000000000000000000 P d33cebb189b14a989836214fdbafb898 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d33cebb189b14a989836214fdbafb898" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d33cebb189b14a989836214fdbafb898" member_type: VOTER } }
I20260709 13:04:31.931442 15800 sys_catalog.cc:458] T 00000000000000000000000000000000 P d33cebb189b14a989836214fdbafb898 [sys.catalog]: This master's current role is: LEADER
I20260709 13:04:31.931514 15799 sys_catalog.cc:458] T 00000000000000000000000000000000 P d33cebb189b14a989836214fdbafb898 [sys.catalog]: This master's current role is: LEADER
I20260709 13:04:31.936514 15805 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:04:31.941248 15805 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:04:31.943400 14748 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:04:31.950016 15805 catalog_manager.cc:1383] Generated new cluster ID: e973206cf0f74a0a907ef715cf0f7216
I20260709 13:04:31.950282 15805 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:04:31.970675 15805 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:04:31.971832 15805 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:04:31.985219 15805 catalog_manager.cc:6070] T 00000000000000000000000000000000 P d33cebb189b14a989836214fdbafb898: Generated new TSK 0
I20260709 13:04:31.985860 15805 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:04:32.012636 14748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:32.018081 15816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:32.019129 15817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:32.021874 14748 server_base.cc:1061] running on GCE node
W20260709 13:04:32.021880 15819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:32.023036 14748 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:32.023231 14748 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:32.023433 14748 hybrid_clock.cc:648] HybridClock initialized: now 1783602272023413 us; error 0 us; skew 500 ppm
I20260709 13:04:32.023976 14748 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:32.026134 14748 webserver.cc:533] Webserver started at http://127.14.103.1:46503/ using document root <none> and password file <none>
I20260709 13:04:32.026587 14748 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:32.026767 14748 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:32.027048 14748 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:32.028149 14748 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-0-root/instance:
uuid: "31882606e87447fb9a36bbf625129fe8"
format_stamp: "Formatted at 2026-07-09 13:04:32 on dist-test-slave-fpp7"
I20260709 13:04:32.032294 14748 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:04:32.035261 15824 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:32.035944 14748 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:04:32.036216 14748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-0-root
uuid: "31882606e87447fb9a36bbf625129fe8"
format_stamp: "Formatted at 2026-07-09 13:04:32 on dist-test-slave-fpp7"
I20260709 13:04:32.036475 14748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:32.072201 14748 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:32.073509 14748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:32.075006 14748 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:32.077237 14748 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:32.077450 14748 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:32.077692 14748 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:32.077870 14748 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:32.117527 14748 rpc_server.cc:307] RPC server started. Bound to: 127.14.103.1:38033
I20260709 13:04:32.117635 15886 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.103.1:38033 every 8 connection(s)
I20260709 13:04:32.122468 14748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:32.131059 15892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:32.131996 15891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:32.134341 15894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:32.134402 14748 server_base.cc:1061] running on GCE node
I20260709 13:04:32.135717 14748 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:32.135933 14748 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:32.136121 14748 hybrid_clock.cc:648] HybridClock initialized: now 1783602272136102 us; error 0 us; skew 500 ppm
I20260709 13:04:32.136646 14748 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:32.139195 14748 webserver.cc:533] Webserver started at http://127.14.103.2:34505/ using document root <none> and password file <none>
I20260709 13:04:32.139791 14748 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:32.140031 14748 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:32.140357 14748 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:32.142092 14748 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-1-root/instance:
uuid: "8430db80a39d4868bcf7641c16de35ea"
format_stamp: "Formatted at 2026-07-09 13:04:32 on dist-test-slave-fpp7"
I20260709 13:04:32.142326 15887 heartbeater.cc:344] Connected to a master server at 127.14.103.62:42985
I20260709 13:04:32.142768 15887 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:32.143420 15887 heartbeater.cc:507] Master 127.14.103.62:42985 requested a full tablet report, sending...
I20260709 13:04:32.145553 15760 ts_manager.cc:194] Registered new tserver with Master: 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
I20260709 13:04:32.147009 14748 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:04:32.147851 15760 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58426
I20260709 13:04:32.150878 15899 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:32.151592 14748 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:04:32.151878 14748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-1-root
uuid: "8430db80a39d4868bcf7641c16de35ea"
format_stamp: "Formatted at 2026-07-09 13:04:32 on dist-test-slave-fpp7"
I20260709 13:04:32.152099 14748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:32.164948 14748 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:32.166031 14748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:32.167279 14748 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:32.169348 14748 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:32.169519 14748 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:32.169819 14748 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:32.169981 14748 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:32.208009 14748 rpc_server.cc:307] RPC server started. Bound to: 127.14.103.2:43009
I20260709 13:04:32.208125 15961 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.103.2:43009 every 8 connection(s)
I20260709 13:04:32.212689 14748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:32.219550 15965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:32.220942 15966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:32.222659 14748 server_base.cc:1061] running on GCE node
W20260709 13:04:32.223435 15968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:32.224457 14748 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:32.224711 14748 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:32.224891 14748 hybrid_clock.cc:648] HybridClock initialized: now 1783602272224873 us; error 0 us; skew 500 ppm
I20260709 13:04:32.225342 15962 heartbeater.cc:344] Connected to a master server at 127.14.103.62:42985
I20260709 13:04:32.225493 14748 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:32.225692 15962 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:32.226436 15962 heartbeater.cc:507] Master 127.14.103.62:42985 requested a full tablet report, sending...
I20260709 13:04:32.228296 14748 webserver.cc:533] Webserver started at http://127.14.103.3:34329/ using document root <none> and password file <none>
I20260709 13:04:32.228631 15760 ts_manager.cc:194] Registered new tserver with Master: 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:32.228929 14748 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:32.229208 14748 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:32.229513 14748 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:32.230369 15760 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58434
I20260709 13:04:32.230969 14748 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-2-root/instance:
uuid: "85e997ba5003439781ec4f1f8db0c3ee"
format_stamp: "Formatted at 2026-07-09 13:04:32 on dist-test-slave-fpp7"
I20260709 13:04:32.235733 14748 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.007s	sys 0.000s
I20260709 13:04:32.238948 15973 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:32.239723 14748 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:04:32.240015 14748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-2-root
uuid: "85e997ba5003439781ec4f1f8db0c3ee"
format_stamp: "Formatted at 2026-07-09 13:04:32 on dist-test-slave-fpp7"
I20260709 13:04:32.240296 14748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:32.253415 14748 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:32.254518 14748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:32.255844 14748 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:32.257891 14748 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:32.258121 14748 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:32.258363 14748 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:32.258535 14748 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:32.297425 14748 rpc_server.cc:307] RPC server started. Bound to: 127.14.103.3:46251
I20260709 13:04:32.297545 16035 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.103.3:46251 every 8 connection(s)
I20260709 13:04:32.310882 16036 heartbeater.cc:344] Connected to a master server at 127.14.103.62:42985
I20260709 13:04:32.311220 16036 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:32.311949 16036 heartbeater.cc:507] Master 127.14.103.62:42985 requested a full tablet report, sending...
I20260709 13:04:32.313756 15760 ts_manager.cc:194] Registered new tserver with Master: 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:32.313854 14748 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013317472s
I20260709 13:04:32.315088 15760 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58448
I20260709 13:04:32.338749 15760 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58452:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:04:32.345054 15760 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:04:32.537302 16001 tablet_service.cc:1511] Processing CreateTablet for tablet e9577f76f40348e9b009255d441d012a (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:04:32.538730 16001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9577f76f40348e9b009255d441d012a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.540693 16000 tablet_service.cc:1511] Processing CreateTablet for tablet 1b76daed20584023b02d5f1bc7964f58 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:04:32.542001 16000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b76daed20584023b02d5f1bc7964f58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.544762 15999 tablet_service.cc:1511] Processing CreateTablet for tablet 40e7e6076e8d475da059aad457e0e8b1 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:04:32.546118 15999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40e7e6076e8d475da059aad457e0e8b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.546236 15998 tablet_service.cc:1511] Processing CreateTablet for tablet ffbfc0e667484157953a66ae2b697013 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:04:32.547600 15998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffbfc0e667484157953a66ae2b697013. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.548709 15927 tablet_service.cc:1511] Processing CreateTablet for tablet e9577f76f40348e9b009255d441d012a (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:04:32.549990 15997 tablet_service.cc:1511] Processing CreateTablet for tablet ca01a56f016b4291b8abfddb75b74dd9 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:04:32.551206 15926 tablet_service.cc:1511] Processing CreateTablet for tablet 1b76daed20584023b02d5f1bc7964f58 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:04:32.552211 15923 tablet_service.cc:1511] Processing CreateTablet for tablet ca01a56f016b4291b8abfddb75b74dd9 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:04:32.553427 15925 tablet_service.cc:1511] Processing CreateTablet for tablet 40e7e6076e8d475da059aad457e0e8b1 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:04:32.554525 15922 tablet_service.cc:1511] Processing CreateTablet for tablet 9f65380951b7478bad13f7cbe01eece2 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:04:32.555687 15924 tablet_service.cc:1511] Processing CreateTablet for tablet ffbfc0e667484157953a66ae2b697013 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:04:32.550031 15927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9577f76f40348e9b009255d441d012a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.556949 15924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffbfc0e667484157953a66ae2b697013. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.558233 15996 tablet_service.cc:1511] Processing CreateTablet for tablet 9f65380951b7478bad13f7cbe01eece2 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:04:32.559545 15926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b76daed20584023b02d5f1bc7964f58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.562791 15995 tablet_service.cc:1511] Processing CreateTablet for tablet 3a538795cdae403fae5941538575596b (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:04:32.564742 15921 tablet_service.cc:1511] Processing CreateTablet for tablet 3a538795cdae403fae5941538575596b (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:04:32.565985 15921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a538795cdae403fae5941538575596b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.568739 15920 tablet_service.cc:1511] Processing CreateTablet for tablet 2f401f200ca34a14a539f8a74675eab4 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:04:32.570019 15920 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f401f200ca34a14a539f8a74675eab4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.569885 15852 tablet_service.cc:1511] Processing CreateTablet for tablet e9577f76f40348e9b009255d441d012a (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:04:32.571152 15852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9577f76f40348e9b009255d441d012a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.572369 15993 tablet_service.cc:1511] Processing CreateTablet for tablet d50c1e8d433a407aa9fae9cb9866f184 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:04:32.572781 15851 tablet_service.cc:1511] Processing CreateTablet for tablet 1b76daed20584023b02d5f1bc7964f58 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:04:32.574061 15851 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b76daed20584023b02d5f1bc7964f58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.575102 15994 tablet_service.cc:1511] Processing CreateTablet for tablet 2f401f200ca34a14a539f8a74675eab4 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:04:32.576676 15850 tablet_service.cc:1511] Processing CreateTablet for tablet 40e7e6076e8d475da059aad457e0e8b1 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:04:32.577916 15850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40e7e6076e8d475da059aad457e0e8b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.578846 15992 tablet_service.cc:1511] Processing CreateTablet for tablet f9822eb87c7e4819af0dc0b0d2951def (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:04:32.580433 16051 tablet_bootstrap.cc:492] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:32.580734 15849 tablet_service.cc:1511] Processing CreateTablet for tablet ffbfc0e667484157953a66ae2b697013 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:04:32.581985 15849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffbfc0e667484157953a66ae2b697013. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.584385 15925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40e7e6076e8d475da059aad457e0e8b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.584848 15848 tablet_service.cc:1511] Processing CreateTablet for tablet ca01a56f016b4291b8abfddb75b74dd9 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:04:32.586095 15848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca01a56f016b4291b8abfddb75b74dd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.587177 16051 tablet_bootstrap.cc:654] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:32.588146 15923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca01a56f016b4291b8abfddb75b74dd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.588846 15847 tablet_service.cc:1511] Processing CreateTablet for tablet 9f65380951b7478bad13f7cbe01eece2 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:04:32.590134 15847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f65380951b7478bad13f7cbe01eece2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.592425 15922 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f65380951b7478bad13f7cbe01eece2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.593197 15846 tablet_service.cc:1511] Processing CreateTablet for tablet 3a538795cdae403fae5941538575596b (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:04:32.594449 15846 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a538795cdae403fae5941538575596b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.597195 15845 tablet_service.cc:1511] Processing CreateTablet for tablet 2f401f200ca34a14a539f8a74675eab4 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:04:32.598424 15845 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f401f200ca34a14a539f8a74675eab4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.598740 16051 tablet_bootstrap.cc:492] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:32.599237 16051 ts_tablet_manager.cc:1403] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.020s	user 0.018s	sys 0.000s
I20260709 13:04:32.604261 15924 tablet_service.cc:1511] Processing CreateTablet for tablet d50c1e8d433a407aa9fae9cb9866f184 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:04:32.605502 15924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d50c1e8d433a407aa9fae9cb9866f184. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.610924 15919 tablet_service.cc:1511] Processing CreateTablet for tablet f9822eb87c7e4819af0dc0b0d2951def (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:04:32.612370 15919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9822eb87c7e4819af0dc0b0d2951def. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.612452 15844 tablet_service.cc:1511] Processing CreateTablet for tablet d50c1e8d433a407aa9fae9cb9866f184 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:04:32.614318 15844 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d50c1e8d433a407aa9fae9cb9866f184. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.614895 15918 tablet_service.cc:1511] Processing CreateTablet for tablet c766f90082d54f299b03f3e9d75d2db0 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:04:32.616189 15918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c766f90082d54f299b03f3e9d75d2db0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.620749 15843 tablet_service.cc:1511] Processing CreateTablet for tablet f9822eb87c7e4819af0dc0b0d2951def (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:04:32.621731 15995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a538795cdae403fae5941538575596b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.622139 15843 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9822eb87c7e4819af0dc0b0d2951def. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.624131 15996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f65380951b7478bad13f7cbe01eece2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.628042 15997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca01a56f016b4291b8abfddb75b74dd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.632146 15992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9822eb87c7e4819af0dc0b0d2951def. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.636144 15993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d50c1e8d433a407aa9fae9cb9866f184. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.640748 15994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f401f200ca34a14a539f8a74675eab4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.640563 15924 tablet_service.cc:1511] Processing CreateTablet for tablet ae068fc0a8a84b148d068df2b4463b8c (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:04:32.602278 16051 raft_consensus.cc:359] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:32.643230 16051 raft_consensus.cc:385] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:32.645553 15924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae068fc0a8a84b148d068df2b4463b8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.646728 16051 raft_consensus.cc:740] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:32.652195 16051 consensus_queue.cc:260] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:32.681622 15927 tablet_service.cc:1511] Processing CreateTablet for tablet 626a74f434ad4e70bed42f8b404a6e81 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:04:32.687654 15845 tablet_service.cc:1511] Processing CreateTablet for tablet c766f90082d54f299b03f3e9d75d2db0 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:04:32.688900 15845 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c766f90082d54f299b03f3e9d75d2db0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.693575 16054 tablet_bootstrap.cc:492] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:32.695011 15852 tablet_service.cc:1511] Processing CreateTablet for tablet ae068fc0a8a84b148d068df2b4463b8c (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:04:32.696352 15852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae068fc0a8a84b148d068df2b4463b8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.699643 16054 tablet_bootstrap.cc:654] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:32.702484 15927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 626a74f434ad4e70bed42f8b404a6e81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.711961 16000 tablet_service.cc:1511] Processing CreateTablet for tablet c766f90082d54f299b03f3e9d75d2db0 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:04:32.713302 16000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c766f90082d54f299b03f3e9d75d2db0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.737599 15918 tablet_service.cc:1511] Processing CreateTablet for tablet ab61d516f4754d22b660c6fa5d052e3c (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:04:32.738970 15921 tablet_service.cc:1511] Processing CreateTablet for tablet c138dd864152446f83386a4cd43c6efc (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:04:32.738953 15918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab61d516f4754d22b660c6fa5d052e3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.747944 16051 ts_tablet_manager.cc:1434] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.148s	user 0.013s	sys 0.009s
I20260709 13:04:32.749675 15962 heartbeater.cc:499] Master 127.14.103.62:42985 was elected leader, sending a full tablet report...
I20260709 13:04:32.748624 16001 tablet_service.cc:1511] Processing CreateTablet for tablet ae068fc0a8a84b148d068df2b4463b8c (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:04:32.750731 16001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae068fc0a8a84b148d068df2b4463b8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.751024 16051 tablet_bootstrap.cc:492] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:32.758906 15997 tablet_service.cc:1511] Processing CreateTablet for tablet 626a74f434ad4e70bed42f8b404a6e81 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:04:32.759306 16051 tablet_bootstrap.cc:654] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:32.767053 15921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c138dd864152446f83386a4cd43c6efc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.768103 15993 tablet_service.cc:1511] Processing CreateTablet for tablet c138dd864152446f83386a4cd43c6efc (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:04:32.782492 16051 tablet_bootstrap.cc:492] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:32.783082 16051 ts_tablet_manager.cc:1403] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.033s	user 0.017s	sys 0.004s
I20260709 13:04:32.788076 15849 tablet_service.cc:1511] Processing CreateTablet for tablet 626a74f434ad4e70bed42f8b404a6e81 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:04:32.789443 15849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 626a74f434ad4e70bed42f8b404a6e81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.790096 15927 tablet_service.cc:1511] Processing CreateTablet for tablet 952bd0ea3bba4d38ad3383fdf572cd01 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:04:32.791289 15927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 952bd0ea3bba4d38ad3383fdf572cd01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.792050 15997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 626a74f434ad4e70bed42f8b404a6e81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.793414 15926 tablet_service.cc:1511] Processing CreateTablet for tablet f4908a0c5a714f5e8c8f2b20cc7f2840 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:04:32.794672 15926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4908a0c5a714f5e8c8f2b20cc7f2840. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.803252 15993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c138dd864152446f83386a4cd43c6efc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.788187 15920 tablet_service.cc:1511] Processing CreateTablet for tablet b5d68c78f0d94e0eb90543b642d1f1af (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:04:32.804325 16055 tablet_bootstrap.cc:492] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:32.794698 15847 tablet_service.cc:1511] Processing CreateTablet for tablet c138dd864152446f83386a4cd43c6efc (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:04:32.813838 16055 tablet_bootstrap.cc:654] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:32.814365 15920 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5d68c78f0d94e0eb90543b642d1f1af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.815652 15847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c138dd864152446f83386a4cd43c6efc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.821986 16051 raft_consensus.cc:359] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:32.825984 15999 tablet_service.cc:1511] Processing CreateTablet for tablet b5d68c78f0d94e0eb90543b642d1f1af (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:04:32.828048 15994 tablet_service.cc:1511] Processing CreateTablet for tablet ab61d516f4754d22b660c6fa5d052e3c (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:04:32.744837 15923 tablet_service.cc:1511] Processing CreateTablet for tablet dfeaf1f6660d48848ae69710739427a0 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:04:32.829870 15924 tablet_service.cc:1511] Processing CreateTablet for tablet 15301761b2ec47fe82ae2f431c8b34ee (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:04:32.840806 15848 tablet_service.cc:1511] Processing CreateTablet for tablet dfeaf1f6660d48848ae69710739427a0 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:04:32.842013 15848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfeaf1f6660d48848ae69710739427a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.842367 15846 tablet_service.cc:1511] Processing CreateTablet for tablet ab61d516f4754d22b660c6fa5d052e3c (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:04:32.817947 15995 tablet_service.cc:1511] Processing CreateTablet for tablet dfeaf1f6660d48848ae69710739427a0 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:04:32.843525 15846 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab61d516f4754d22b660c6fa5d052e3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.844056 15852 tablet_service.cc:1511] Processing CreateTablet for tablet b5d68c78f0d94e0eb90543b642d1f1af (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:04:32.845270 15852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5d68c78f0d94e0eb90543b642d1f1af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.846616 15850 tablet_service.cc:1511] Processing CreateTablet for tablet 952bd0ea3bba4d38ad3383fdf572cd01 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:04:32.847770 15850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 952bd0ea3bba4d38ad3383fdf572cd01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.850651 15922 tablet_service.cc:1511] Processing CreateTablet for tablet 0b3676041725488fb6ffad37889b386b (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:04:32.829972 15925 tablet_service.cc:1511] Processing CreateTablet for tablet d2be2d6f06bc47a39f26307ae9372194 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:04:32.860812 15844 tablet_service.cc:1511] Processing CreateTablet for tablet 15301761b2ec47fe82ae2f431c8b34ee (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:04:32.855252 15845 tablet_service.cc:1511] Processing CreateTablet for tablet f4908a0c5a714f5e8c8f2b20cc7f2840 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:04:32.862103 15844 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15301761b2ec47fe82ae2f431c8b34ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.862257 15923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfeaf1f6660d48848ae69710739427a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.832556 16051 raft_consensus.cc:385] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:32.862773 16051 raft_consensus.cc:740] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:32.863714 15925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2be2d6f06bc47a39f26307ae9372194. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.863562 16051 consensus_queue.cc:260] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:32.865736 16051 ts_tablet_manager.cc:1434] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.082s	user 0.002s	sys 0.002s
I20260709 13:04:32.866668 16051 tablet_bootstrap.cc:492] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:32.866488 15918 tablet_service.cc:1511] Processing CreateTablet for tablet 888cce851f074f3c969de10b02ff95ad (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:04:32.867751 15918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 888cce851f074f3c969de10b02ff95ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.871723 15998 tablet_service.cc:1511] Processing CreateTablet for tablet 952bd0ea3bba4d38ad3383fdf572cd01 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:04:32.872970 15998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 952bd0ea3bba4d38ad3383fdf572cd01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.873469 16001 tablet_service.cc:1511] Processing CreateTablet for tablet f4908a0c5a714f5e8c8f2b20cc7f2840 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:04:32.874691 16001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4908a0c5a714f5e8c8f2b20cc7f2840. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.875550 16053 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.876006 16053 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:32.881731 15843 tablet_service.cc:1511] Processing CreateTablet for tablet d2be2d6f06bc47a39f26307ae9372194 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:04:32.882968 16053 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:32.883486 15922 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b3676041725488fb6ffad37889b386b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.873469 16000 tablet_service.cc:1511] Processing CreateTablet for tablet 15301761b2ec47fe82ae2f431c8b34ee (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:04:32.888456 15994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab61d516f4754d22b660c6fa5d052e3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.891971 16000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15301761b2ec47fe82ae2f431c8b34ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.893960 15996 tablet_service.cc:1511] Processing CreateTablet for tablet d2be2d6f06bc47a39f26307ae9372194 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:04:32.895180 15996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2be2d6f06bc47a39f26307ae9372194. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.896027 15999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5d68c78f0d94e0eb90543b642d1f1af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.897081 15851 tablet_service.cc:1511] Processing CreateTablet for tablet ac847593d6724522bda12a2207f91c31 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:04:32.897240 15849 tablet_service.cc:1511] Processing CreateTablet for tablet 0b3676041725488fb6ffad37889b386b (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:04:32.898324 15851 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac847593d6724522bda12a2207f91c31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.898722 15995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfeaf1f6660d48848ae69710739427a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.902742 15924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15301761b2ec47fe82ae2f431c8b34ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.907375 15845 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4908a0c5a714f5e8c8f2b20cc7f2840. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.909703 15843 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2be2d6f06bc47a39f26307ae9372194. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.913056 15919 tablet_service.cc:1511] Processing CreateTablet for tablet ac847593d6724522bda12a2207f91c31 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:04:32.914393 15849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b3676041725488fb6ffad37889b386b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.914458 15919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac847593d6724522bda12a2207f91c31. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:04:32.945551 15744 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 45555.5us
I20260709 13:04:32.955241 15992 tablet_service.cc:1511] Processing CreateTablet for tablet 0b3676041725488fb6ffad37889b386b (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
W20260709 13:04:32.959111 15808 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:32.961627 15848 tablet_service.cc:1511] Processing CreateTablet for tablet 888cce851f074f3c969de10b02ff95ad (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:04:32.962919 15848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 888cce851f074f3c969de10b02ff95ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.965176 15927 tablet_service.cc:1511] Processing CreateTablet for tablet ecbdcd9cd6c944e7a9277f7c8f191e96 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:04:32.969233 15997 tablet_service.cc:1511] Processing CreateTablet for tablet ac847593d6724522bda12a2207f91c31 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:04:32.972043 16054 tablet_bootstrap.cc:492] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:32.977773 15926 tablet_service.cc:1511] Processing CreateTablet for tablet 9b72a30273dd49e883c237c7e5ad237b (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:04:32.980193 16054 ts_tablet_manager.cc:1403] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.287s	user 0.026s	sys 0.036s
I20260709 13:04:32.980923 15926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b72a30273dd49e883c237c7e5ad237b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.982458 15998 tablet_service.cc:1511] Processing CreateTablet for tablet 3a0a5e0b84774549a054f45adf4adf59 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:04:32.983474 16054 raft_consensus.cc:359] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:04:32.984300 16054 raft_consensus.cc:385] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:32.987048 15927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecbdcd9cd6c944e7a9277f7c8f191e96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:32.987365 15998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a0a5e0b84774549a054f45adf4adf59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.003697 15923 tablet_service.cc:1511] Processing CreateTablet for tablet 73f4b066dd3d464699e6b00ebf1cdc44 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:04:33.005002 15923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73f4b066dd3d464699e6b00ebf1cdc44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.005192 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:04:33.006731 15900 leader_election.cc:343] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:32.996147 16051 tablet_bootstrap.cc:654] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:32.984735 16054 raft_consensus.cc:740] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.014947 16054 consensus_queue.cc:260] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:04:32.955600 15921 tablet_service.cc:1511] Processing CreateTablet for tablet 3a0a5e0b84774549a054f45adf4adf59 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:04:33.016644 15843 tablet_service.cc:1511] Processing CreateTablet for tablet 3a0a5e0b84774549a054f45adf4adf59 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:04:33.027591 15844 tablet_service.cc:1511] Processing CreateTablet for tablet 9b72a30273dd49e883c237c7e5ad237b (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:04:33.043243 15844 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b72a30273dd49e883c237c7e5ad237b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.039201 15997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac847593d6724522bda12a2207f91c31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.051888 15992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b3676041725488fb6ffad37889b386b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.056365 15843 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a0a5e0b84774549a054f45adf4adf59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.062497 15918 tablet_service.cc:1511] Processing CreateTablet for tablet 850b6b8a2f54468fb87ea6a22c5c870f (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:04:32.996210 15993 tablet_service.cc:1511] Processing CreateTablet for tablet 888cce851f074f3c969de10b02ff95ad (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:04:33.072458 15850 tablet_service.cc:1511] Processing CreateTablet for tablet 73f4b066dd3d464699e6b00ebf1cdc44 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:04:33.073151 16055 tablet_bootstrap.cc:492] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:33.073657 16055 ts_tablet_manager.cc:1403] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.270s	user 0.008s	sys 0.010s
I20260709 13:04:33.074211 15850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73f4b066dd3d464699e6b00ebf1cdc44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.076426 16055 raft_consensus.cc:359] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:04:33.077150 16055 raft_consensus.cc:385] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.077431 16055 raft_consensus.cc:740] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.078107 16055 consensus_queue.cc:260] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:04:33.085881 15919 tablet_service.cc:1511] Processing CreateTablet for tablet 32207d388e0e4185898655b74ae976b1 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:04:33.088533 15852 tablet_service.cc:1511] Processing CreateTablet for tablet ecbdcd9cd6c944e7a9277f7c8f191e96 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:04:33.089983 15852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecbdcd9cd6c944e7a9277f7c8f191e96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.040686 15921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a0a5e0b84774549a054f45adf4adf59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.096285 15994 tablet_service.cc:1511] Processing CreateTablet for tablet ecbdcd9cd6c944e7a9277f7c8f191e96 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:04:33.114069 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:04:33.118391 15902 leader_election.cc:343] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:33.118822 15902 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:33.119589 16053 raft_consensus.cc:2749] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:33.120846 15925 tablet_service.cc:1511] Processing CreateTablet for tablet e71b32319bee4f979532b3ac6a16f2bd (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:04:33.121490 16051 tablet_bootstrap.cc:492] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:33.121949 16051 ts_tablet_manager.cc:1403] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.255s	user 0.015s	sys 0.010s
I20260709 13:04:33.124841 16051 raft_consensus.cc:359] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:04:33.125514 16051 raft_consensus.cc:385] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.111654 15918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 850b6b8a2f54468fb87ea6a22c5c870f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.133411 16054 ts_tablet_manager.cc:1434] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.153s	user 0.048s	sys 0.044s
I20260709 13:04:33.134557 15925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e71b32319bee4f979532b3ac6a16f2bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.134490 15887 heartbeater.cc:499] Master 127.14.103.62:42985 was elected leader, sending a full tablet report...
I20260709 13:04:33.138332 15919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32207d388e0e4185898655b74ae976b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.142169 15999 tablet_service.cc:1511] Processing CreateTablet for tablet 9b72a30273dd49e883c237c7e5ad237b (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:04:33.125785 16051 raft_consensus.cc:740] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.147101 15845 tablet_service.cc:1511] Processing CreateTablet for tablet e71b32319bee4f979532b3ac6a16f2bd (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:04:33.148505 15843 tablet_service.cc:1511] Processing CreateTablet for tablet 32207d388e0e4185898655b74ae976b1 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:04:33.149833 16051 consensus_queue.cc:260] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:04:33.154819 16051 ts_tablet_manager.cc:1434] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.033s	user 0.003s	sys 0.005s
I20260709 13:04:33.152334 15844 tablet_service.cc:1511] Processing CreateTablet for tablet 850b6b8a2f54468fb87ea6a22c5c870f (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:04:33.155855 16051 tablet_bootstrap.cc:492] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:33.162348 16036 heartbeater.cc:499] Master 127.14.103.62:42985 was elected leader, sending a full tablet report...
I20260709 13:04:33.162729 16055 ts_tablet_manager.cc:1434] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.089s	user 0.017s	sys 0.012s
I20260709 13:04:33.168275 16054 tablet_bootstrap.cc:492] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:33.175264 16053 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.177171 16053 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:33.177476 15924 tablet_service.cc:1511] Processing CreateTablet for tablet 679229ffe7114facacb8e3a2c4c6d226 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:04:33.179188 16053 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:33.181223 15843 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32207d388e0e4185898655b74ae976b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.181941 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:33.182698 15844 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 850b6b8a2f54468fb87ea6a22c5c870f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:04:33.183271 15900 leader_election.cc:343] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:33.176225 15849 tablet_service.cc:1511] Processing CreateTablet for tablet 0a10bba95cd1421aa3fc332639e97319 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:04:33.185209 15849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a10bba95cd1421aa3fc332639e97319. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.185776 15852 tablet_service.cc:1511] Processing CreateTablet for tablet 679229ffe7114facacb8e3a2c4c6d226 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:04:33.174714 16063 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.192250 16063 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:04:33.194882 16063 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:33.176532 15926 tablet_service.cc:1511] Processing CreateTablet for tablet 0a10bba95cd1421aa3fc332639e97319 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:04:33.196157 15852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 679229ffe7114facacb8e3a2c4c6d226. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.201759 15995 tablet_service.cc:1511] Processing CreateTablet for tablet 73f4b066dd3d464699e6b00ebf1cdc44 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:04:33.202287 15993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 888cce851f074f3c969de10b02ff95ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.202939 15996 tablet_service.cc:1511] Processing CreateTablet for tablet 32207d388e0e4185898655b74ae976b1 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:04:33.204600 16000 tablet_service.cc:1511] Processing CreateTablet for tablet e71b32319bee4f979532b3ac6a16f2bd (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:04:33.207080 16001 tablet_service.cc:1511] Processing CreateTablet for tablet 0a10bba95cd1421aa3fc332639e97319 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:04:33.209121 16055 tablet_bootstrap.cc:492] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:33.201759 15997 tablet_service.cc:1511] Processing CreateTablet for tablet 850b6b8a2f54468fb87ea6a22c5c870f (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:04:33.202996 15995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73f4b066dd3d464699e6b00ebf1cdc44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.190861 15924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 679229ffe7114facacb8e3a2c4c6d226. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.214864 15999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b72a30273dd49e883c237c7e5ad237b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.215953 15997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 850b6b8a2f54468fb87ea6a22c5c870f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.216176 15926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a10bba95cd1421aa3fc332639e97319. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.217960 15923 tablet_service.cc:1511] Processing CreateTablet for tablet 3b3e05ffd87b4bbcb04b315d3bfdf8ed (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:04:33.211071 16054 tablet_bootstrap.cc:654] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.220661 15994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecbdcd9cd6c944e7a9277f7c8f191e96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.222513 16061 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.222975 16061 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:04:33.225504 16061 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:33.239183 15998 tablet_service.cc:1511] Processing CreateTablet for tablet 679229ffe7114facacb8e3a2c4c6d226 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:04:33.190958 16051 tablet_bootstrap.cc:654] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.241724 16054 tablet_bootstrap.cc:492] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:33.243389 16054 ts_tablet_manager.cc:1403] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.075s	user 0.006s	sys 0.006s
I20260709 13:04:33.247186 16054 raft_consensus.cc:359] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:33.250226 15921 tablet_service.cc:1511] Processing CreateTablet for tablet fd8e1402dafb4300911e5b0189c8fe0a (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:04:33.253378 16054 raft_consensus.cc:385] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.268931 15923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b3e05ffd87b4bbcb04b315d3bfdf8ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.276597 16001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a10bba95cd1421aa3fc332639e97319. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.288020 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:33.291996 15846 tablet_service.cc:1511] Processing CreateTablet for tablet 3b3e05ffd87b4bbcb04b315d3bfdf8ed (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:04:33.277055 16054 raft_consensus.cc:740] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.306221 15921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd8e1402dafb4300911e5b0189c8fe0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.305979 15998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 679229ffe7114facacb8e3a2c4c6d226. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.312862 15845 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e71b32319bee4f979532b3ac6a16f2bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.324060 15846 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b3e05ffd87b4bbcb04b315d3bfdf8ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.328064 16000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e71b32319bee4f979532b3ac6a16f2bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.339303 15996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32207d388e0e4185898655b74ae976b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.313654 16054 consensus_queue.cc:260] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:33.342018 15919 tablet_service.cc:1511] Processing CreateTablet for tablet eabbcb31af804f2dac6af5731e18a4dd (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:04:33.347961 16053 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.348469 16053 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:04:33.348977 15919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eabbcb31af804f2dac6af5731e18a4dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.350461 16053 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
I20260709 13:04:33.352895 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:33.353713 15862 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 0.
I20260709 13:04:33.354647 16054 ts_tablet_manager.cc:1434] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.110s	user 0.003s	sys 0.004s
I20260709 13:04:33.354993 15900 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:33.355720 16054 tablet_bootstrap.cc:492] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:33.356665 16053 raft_consensus.cc:2804] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:33.357044 16053 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.357378 16053 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.360178 16051 tablet_bootstrap.cc:492] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:33.360658 16051 ts_tablet_manager.cc:1403] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.205s	user 0.016s	sys 0.007s
I20260709 13:04:33.364066 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:33.364722 16011 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 0.
I20260709 13:04:33.366779 16055 tablet_bootstrap.cc:654] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.366674 16053 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:04:33.370153 16053 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
I20260709 13:04:33.370076 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:33.370752 15862 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.370059 15827 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:33.376209 15994 tablet_service.cc:1511] Processing CreateTablet for tablet fd8e1402dafb4300911e5b0189c8fe0a (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:04:33.377452 15994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd8e1402dafb4300911e5b0189c8fe0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.382629 15862 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:04:33.383312 16061 raft_consensus.cc:2764] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:04:33.383355 15925 tablet_service.cc:1511] Processing CreateTablet for tablet 05f069356f3b4f72a756c3367d8713b7 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:04:33.389964 15850 tablet_service.cc:1511] Processing CreateTablet for tablet 05f069356f3b4f72a756c3367d8713b7 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:04:33.391723 15849 tablet_service.cc:1511] Processing CreateTablet for tablet fd8e1402dafb4300911e5b0189c8fe0a (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:04:33.403170 15849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd8e1402dafb4300911e5b0189c8fe0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.403450 15843 tablet_service.cc:1511] Processing CreateTablet for tablet eabbcb31af804f2dac6af5731e18a4dd (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:04:33.395139 15844 tablet_service.cc:1511] Processing CreateTablet for tablet c04fe1420dfe4602909deef4baa3f115 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:04:33.404012 15900 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:33.404831 15843 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eabbcb31af804f2dac6af5731e18a4dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.404742 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:33.405387 16053 raft_consensus.cc:2804] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:33.405447 16011 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 0.
I20260709 13:04:33.405527 15844 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c04fe1420dfe4602909deef4baa3f115. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.409606 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:33.410375 16011 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:33.408912 15902 leader_election.cc:343] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:33.403470 15993 tablet_service.cc:1511] Processing CreateTablet for tablet 05f069356f3b4f72a756c3367d8713b7 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:04:33.420485 15902 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:33.427292 15993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05f069356f3b4f72a756c3367d8713b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.439837 15992 tablet_service.cc:1511] Processing CreateTablet for tablet 3b3e05ffd87b4bbcb04b315d3bfdf8ed (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:04:33.441331 15992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b3e05ffd87b4bbcb04b315d3bfdf8ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.445372 15850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05f069356f3b4f72a756c3367d8713b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.363595 16051 raft_consensus.cc:359] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:33.447947 16051 raft_consensus.cc:385] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.448342 16051 raft_consensus.cc:740] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.449240 16051 consensus_queue.cc:260] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:33.460569 15995 tablet_service.cc:1511] Processing CreateTablet for tablet eabbcb31af804f2dac6af5731e18a4dd (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:04:33.461949 15995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eabbcb31af804f2dac6af5731e18a4dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.467967 15925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05f069356f3b4f72a756c3367d8713b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.482586 16053 raft_consensus.cc:697] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:04:33.483716 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 74813.8us
I20260709 13:04:33.489674 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:33.480207 16011 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:04:33.500928 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:33.501976 15862 raft_consensus.cc:2393] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 1: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:04:33.502610 15926 tablet_service.cc:1511] Processing CreateTablet for tablet c04fe1420dfe4602909deef4baa3f115 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:04:33.503859 15926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c04fe1420dfe4602909deef4baa3f115. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.519699 16054 tablet_bootstrap.cc:654] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.521293 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:33.531620 16074 raft_consensus.cc:2749] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:33.537200 16055 tablet_bootstrap.cc:492] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:33.537779 16055 ts_tablet_manager.cc:1403] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.329s	user 0.008s	sys 0.018s
I20260709 13:04:33.537597 15992 tablet_service.cc:1511] Processing CreateTablet for tablet c04fe1420dfe4602909deef4baa3f115 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:04:33.539081 15992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c04fe1420dfe4602909deef4baa3f115. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.540966 15974 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:04:33.541209 15845 tablet_service.cc:1511] Processing CreateTablet for tablet 4e1a3c90a16e428fb2b328aff66b4ea6 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:04:33.542438 15845 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e1a3c90a16e428fb2b328aff66b4ea6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.542824 15994 tablet_service.cc:1511] Processing CreateTablet for tablet 4e1a3c90a16e428fb2b328aff66b4ea6 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:04:33.540833 15923 tablet_service.cc:1511] Processing CreateTablet for tablet 4e1a3c90a16e428fb2b328aff66b4ea6 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:04:33.542459 16064 raft_consensus.cc:2749] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:33.544008 15994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e1a3c90a16e428fb2b328aff66b4ea6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.547077 16053 consensus_queue.cc:237] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:04:33.551059 15923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e1a3c90a16e428fb2b328aff66b4ea6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.562963 15921 tablet_service.cc:1511] Processing CreateTablet for tablet 6dc06813d5b7464fbdb39cde50f5ab48 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:04:33.563906 16073 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.575338 15921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6dc06813d5b7464fbdb39cde50f5ab48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.587610 16055 raft_consensus.cc:359] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:33.588316 16055 raft_consensus.cc:385] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.588587 16055 raft_consensus.cc:740] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.588734 16000 tablet_service.cc:1511] Processing CreateTablet for tablet 6dc06813d5b7464fbdb39cde50f5ab48 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:04:33.589272 16055 consensus_queue.cc:260] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:33.590049 16000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6dc06813d5b7464fbdb39cde50f5ab48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.590924 16051 ts_tablet_manager.cc:1434] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.230s	user 0.035s	sys 0.028s
I20260709 13:04:33.591260 16055 ts_tablet_manager.cc:1434] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.053s	user 0.006s	sys 0.000s
I20260709 13:04:33.584290 16073 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:33.599296 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:33.599309 15846 tablet_service.cc:1511] Processing CreateTablet for tablet 4fcbf678894c4a1eb80c68b5b53acaef (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:04:33.606387 15846 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fcbf678894c4a1eb80c68b5b53acaef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.606204 15919 tablet_service.cc:1511] Processing CreateTablet for tablet 4fcbf678894c4a1eb80c68b5b53acaef (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:04:33.600966 16073 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:33.600896 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:33.602813 15849 tablet_service.cc:1511] Processing CreateTablet for tablet 6dc06813d5b7464fbdb39cde50f5ab48 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:04:33.609905 16054 tablet_bootstrap.cc:492] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:33.591923 16051 tablet_bootstrap.cc:492] T c766f90082d54f299b03f3e9d75d2db0 P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:33.603606 16055 tablet_bootstrap.cc:492] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
W20260709 13:04:33.608435 15900 leader_election.cc:343] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:33.614213 15902 leader_election.cc:343] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:33.616863 15902 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:33.610378 16054 ts_tablet_manager.cc:1403] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.255s	user 0.019s	sys 0.006s
I20260709 13:04:33.610375 15849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6dc06813d5b7464fbdb39cde50f5ab48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.624181 15844 tablet_service.cc:1511] Processing CreateTablet for tablet e236a7bf118740c9b3d88227bed5670e (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:04:33.625125 15924 tablet_service.cc:1511] Processing CreateTablet for tablet e236a7bf118740c9b3d88227bed5670e (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:04:33.626287 16055 tablet_bootstrap.cc:654] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.618552 16073 raft_consensus.cc:2749] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:33.631040 16000 tablet_service.cc:1511] Processing CreateTablet for tablet 4fcbf678894c4a1eb80c68b5b53acaef (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:04:33.632282 16000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fcbf678894c4a1eb80c68b5b53acaef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.631875 16054 raft_consensus.cc:359] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:33.632632 16054 raft_consensus.cc:385] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.632920 16054 raft_consensus.cc:740] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.636751 15995 tablet_service.cc:1511] Processing CreateTablet for tablet e236a7bf118740c9b3d88227bed5670e (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:04:33.638091 15995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e236a7bf118740c9b3d88227bed5670e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.637303 15758 catalog_manager.cc:5697] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:33.644989 15927 tablet_service.cc:1511] Processing CreateTablet for tablet f8b7d73618ae489990e21e8cc312177d (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:04:33.650432 15999 tablet_service.cc:1511] Processing CreateTablet for tablet f8b7d73618ae489990e21e8cc312177d (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:04:33.651634 15999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8b7d73618ae489990e21e8cc312177d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.621555 15919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fcbf678894c4a1eb80c68b5b53acaef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.658634 15924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e236a7bf118740c9b3d88227bed5670e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.659695 16054 consensus_queue.cc:260] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:33.664067 15927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8b7d73618ae489990e21e8cc312177d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.664960 16054 ts_tablet_manager.cc:1434] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.000s
I20260709 13:04:33.665889 16054 tablet_bootstrap.cc:492] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:33.668087 15844 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e236a7bf118740c9b3d88227bed5670e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.673353 15843 tablet_service.cc:1511] Processing CreateTablet for tablet f8b7d73618ae489990e21e8cc312177d (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:04:33.674551 15843 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8b7d73618ae489990e21e8cc312177d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.677044 15850 tablet_service.cc:1511] Processing CreateTablet for tablet c65db3107b664e9c8ce9be2cd2cb36c6 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:04:33.677706 16055 tablet_bootstrap.cc:492] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:33.678188 16055 ts_tablet_manager.cc:1403] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.075s	user 0.010s	sys 0.000s
I20260709 13:04:33.678297 15850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c65db3107b664e9c8ce9be2cd2cb36c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.681838 16055 raft_consensus.cc:359] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:33.682561 16055 raft_consensus.cc:385] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.682850 16055 raft_consensus.cc:740] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.686348 15925 tablet_service.cc:1511] Processing CreateTablet for tablet c65db3107b664e9c8ce9be2cd2cb36c6 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:04:33.687608 15925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c65db3107b664e9c8ce9be2cd2cb36c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.688668 15993 tablet_service.cc:1511] Processing CreateTablet for tablet c65db3107b664e9c8ce9be2cd2cb36c6 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:04:33.689886 15993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c65db3107b664e9c8ce9be2cd2cb36c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.690948 16051 tablet_bootstrap.cc:654] T c766f90082d54f299b03f3e9d75d2db0 P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.691052 16054 tablet_bootstrap.cc:654] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.705982 15846 tablet_service.cc:1511] Processing CreateTablet for tablet bbd61c62d0f0489291627ee496b25051 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:04:33.707307 15846 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbd61c62d0f0489291627ee496b25051. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.707785 15921 tablet_service.cc:1511] Processing CreateTablet for tablet bbd61c62d0f0489291627ee496b25051 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:04:33.708993 15921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbd61c62d0f0489291627ee496b25051. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.717126 15995 tablet_service.cc:1511] Processing CreateTablet for tablet bbd61c62d0f0489291627ee496b25051 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:04:33.717744 16051 tablet_bootstrap.cc:492] T c766f90082d54f299b03f3e9d75d2db0 P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:33.718209 16051 ts_tablet_manager.cc:1403] T c766f90082d54f299b03f3e9d75d2db0 P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.126s	user 0.015s	sys 0.002s
I20260709 13:04:33.718370 15995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbd61c62d0f0489291627ee496b25051. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.721107 16051 raft_consensus.cc:359] T c766f90082d54f299b03f3e9d75d2db0 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:33.721778 16051 raft_consensus.cc:385] T c766f90082d54f299b03f3e9d75d2db0 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.722072 16051 raft_consensus.cc:740] T c766f90082d54f299b03f3e9d75d2db0 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.723059 16051 consensus_queue.cc:260] T c766f90082d54f299b03f3e9d75d2db0 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:33.724138 16061 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.724597 16061 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:33.725713 16051 ts_tablet_manager.cc:1434] T c766f90082d54f299b03f3e9d75d2db0 P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:04:33.726504 16061 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:33.726719 16051 tablet_bootstrap.cc:492] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:33.729393 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:33.729763 15845 tablet_service.cc:1511] Processing CreateTablet for tablet 260d22f804a345ed8f055536994fcb14 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
W20260709 13:04:33.730746 15827 leader_election.cc:343] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:33.730989 15845 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 260d22f804a345ed8f055536994fcb14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.727447 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:33.731590 15937 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 0.
I20260709 13:04:33.732723 16073 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.733129 16073 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:33.735044 16073 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:33.735095 15993 tablet_service.cc:1511] Processing CreateTablet for tablet 260d22f804a345ed8f055536994fcb14 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:04:33.736501 15993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 260d22f804a345ed8f055536994fcb14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.737305 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:33.738461 15825 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:04:33.743239 15900 leader_election.cc:343] T c766f90082d54f299b03f3e9d75d2db0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:33.741628 16061 raft_consensus.cc:2804] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:33.744216 16061 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.744629 16061 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.745182 15992 tablet_service.cc:1511] Processing CreateTablet for tablet 6e422bf1c2eb46fcb2fe33e4eb00f888 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:04:33.746428 15992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e422bf1c2eb46fcb2fe33e4eb00f888. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.748080 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:04:33.749327 15902 leader_election.cc:343] T c766f90082d54f299b03f3e9d75d2db0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:33.749734 15902 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:33.751503 16073 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:33.752197 15927 tablet_service.cc:1511] Processing CreateTablet for tablet 260d22f804a345ed8f055536994fcb14 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:04:33.752381 15848 tablet_service.cc:1511] Processing CreateTablet for tablet 1d9c19e253ce4d53bb9e2f2241d3cafc (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:04:33.753386 15927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 260d22f804a345ed8f055536994fcb14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.753697 15848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d9c19e253ce4d53bb9e2f2241d3cafc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.753847 15926 tablet_service.cc:1511] Processing CreateTablet for tablet 6e422bf1c2eb46fcb2fe33e4eb00f888 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:04:33.755143 15926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e422bf1c2eb46fcb2fe33e4eb00f888. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.760522 15846 tablet_service.cc:1511] Processing CreateTablet for tablet 6e422bf1c2eb46fcb2fe33e4eb00f888 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:04:33.761826 15846 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e422bf1c2eb46fcb2fe33e4eb00f888. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.774394 15923 tablet_service.cc:1511] Processing CreateTablet for tablet 1d9c19e253ce4d53bb9e2f2241d3cafc (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:04:33.776242 15923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d9c19e253ce4d53bb9e2f2241d3cafc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.683565 16055 consensus_queue.cc:260] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:33.787992 16055 ts_tablet_manager.cc:1434] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.109s	user 0.009s	sys 0.000s
I20260709 13:04:33.789623 16055 tablet_bootstrap.cc:492] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:33.790730 15848 tablet_service.cc:1511] Processing CreateTablet for tablet 471dbd2e16264fe5837e8b7ea3601f94 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:04:33.792138 15848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 471dbd2e16264fe5837e8b7ea3601f94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.796244 16061 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:33.798923 16061 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:33.799567 16055 tablet_bootstrap.cc:654] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.799768 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:33.800504 15937 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.799566 15918 tablet_service.cc:1511] Processing CreateTablet for tablet 471dbd2e16264fe5837e8b7ea3601f94 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:04:33.809532 15849 tablet_service.cc:1511] Processing CreateTablet for tablet 7c0fef632f9a4485838855b553a79163 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:04:33.810864 15849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c0fef632f9a4485838855b553a79163. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.813732 15918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 471dbd2e16264fe5837e8b7ea3601f94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.815936 16061 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.816452 16061 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:33.816674 15993 tablet_service.cc:1511] Processing CreateTablet for tablet 471dbd2e16264fe5837e8b7ea3601f94 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:04:33.818560 16061 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:33.821072 15937 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:04:33.822098 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:33.821960 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:04:33.823815 15827 leader_election.cc:343] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:33.824393 15825 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:33.825501 16061 raft_consensus.cc:2804] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:04:33.825714 15825 leader_election.cc:343] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:33.822016 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:04:33.835597 15827 leader_election.cc:343] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:33.835760 15924 tablet_service.cc:1511] Processing CreateTablet for tablet 7c0fef632f9a4485838855b553a79163 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:04:33.830933 15825 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:33.837155 15924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c0fef632f9a4485838855b553a79163. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.863788 15999 tablet_service.cc:1511] Processing CreateTablet for tablet 7c0fef632f9a4485838855b553a79163 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:04:33.863788 15843 tablet_service.cc:1511] Processing CreateTablet for tablet 91d0cc5018ee4a9fb3d196c783814c44 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:04:33.865298 15843 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91d0cc5018ee4a9fb3d196c783814c44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.868105 16000 tablet_service.cc:1511] Processing CreateTablet for tablet 1d9c19e253ce4d53bb9e2f2241d3cafc (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:04:33.860401 15993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 471dbd2e16264fe5837e8b7ea3601f94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.886996 15992 tablet_service.cc:1511] Processing CreateTablet for tablet 91d0cc5018ee4a9fb3d196c783814c44 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:04:33.887034 15999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c0fef632f9a4485838855b553a79163. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.892170 15992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91d0cc5018ee4a9fb3d196c783814c44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.902731 16054 tablet_bootstrap.cc:492] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:33.903119 16054 ts_tablet_manager.cc:1403] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.237s	user 0.033s	sys 0.012s
I20260709 13:04:33.876235 16061 raft_consensus.cc:697] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:04:33.906782 15918 tablet_service.cc:1511] Processing CreateTablet for tablet 91d0cc5018ee4a9fb3d196c783814c44 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:04:33.909099 15918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91d0cc5018ee4a9fb3d196c783814c44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.913041 16000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d9c19e253ce4d53bb9e2f2241d3cafc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.916023 16085 raft_consensus.cc:2749] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:33.915938 15849 tablet_service.cc:1511] Processing CreateTablet for tablet 8849099536af497bae91c72da893681c (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:04:33.919497 15849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8849099536af497bae91c72da893681c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.924909 16054 raft_consensus.cc:359] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:33.925446 16055 tablet_bootstrap.cc:492] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:33.925925 16055 ts_tablet_manager.cc:1403] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.137s	user 0.019s	sys 0.010s
I20260709 13:04:33.926905 16064 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.927377 16064 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:33.929570 16064 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:33.931432 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:33.932106 15936 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 0.
I20260709 13:04:33.932837 15994 tablet_service.cc:1511] Processing CreateTablet for tablet 8849099536af497bae91c72da893681c (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:04:33.933382 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:33.934067 15994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8849099536af497bae91c72da893681c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:04:33.936280 15974 leader_election.cc:343] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:33.936954 15974 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:04:33.938072 16064 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:33.938416 16064 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.938753 16064 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:33.925659 16054 raft_consensus.cc:385] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.944290 16054 raft_consensus.cc:740] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.944870 16054 consensus_queue.cc:260] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:33.946455 16054 ts_tablet_manager.cc:1434] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.043s	user 0.005s	sys 0.000s
I20260709 13:04:33.947402 16054 tablet_bootstrap.cc:492] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:33.950001 16073 consensus_queue.cc:1048] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:33.954669 15926 tablet_service.cc:1511] Processing CreateTablet for tablet 8849099536af497bae91c72da893681c (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:04:33.955869 16051 tablet_bootstrap.cc:654] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:33.956238 15926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8849099536af497bae91c72da893681c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.960731 15849 tablet_service.cc:1511] Processing CreateTablet for tablet 78530df4cdb742eda67781671b0431c6 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
W20260709 13:04:33.961444 15808 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:33.961776 15849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78530df4cdb742eda67781671b0431c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.968457 16061 consensus_queue.cc:237] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:33.972955 16055 raft_consensus.cc:359] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:33.973658 16055 raft_consensus.cc:385] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:33.973961 16055 raft_consensus.cc:740] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:33.974653 16055 consensus_queue.cc:260] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:33.952827 16064 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:33.987016 15924 tablet_service.cc:1511] Processing CreateTablet for tablet 78530df4cdb742eda67781671b0431c6 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:04:33.989576 15924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78530df4cdb742eda67781671b0431c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:33.997838 16064 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:34.000061 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:34.000667 15936 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.000798 16064 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.002046 16000 tablet_service.cc:1511] Processing CreateTablet for tablet 78530df4cdb742eda67781671b0431c6 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:04:34.002229 16064 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.003280 16000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78530df4cdb742eda67781671b0431c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:34.003726 15849 tablet_service.cc:1511] Processing CreateTablet for tablet dd0dfa7599d64160b42b750a38e5c239 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:04:34.005813 16055 ts_tablet_manager.cc:1434] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.080s	user 0.003s	sys 0.006s
I20260709 13:04:34.007201 16055 tablet_bootstrap.cc:492] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:34.012761 15936 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 1.
I20260709 13:04:33.996160 16054 tablet_bootstrap.cc:654] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:34.029727 15849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd0dfa7599d64160b42b750a38e5c239. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:34.045574 15974 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:34.046892 16088 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:34.047353 16088 raft_consensus.cc:697] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:04:34.050091 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:34.051990 15843 tablet_service.cc:1511] Processing CreateTablet for tablet 7b6e87097733428bb2167c6f308fe589 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:04:34.053220 15843 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b6e87097733428bb2167c6f308fe589. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:34.056252 16055 tablet_bootstrap.cc:654] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:34.065078 16064 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:34.057014 16088 consensus_queue.cc:237] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.057448 15992 tablet_service.cc:1511] Processing CreateTablet for tablet dd0dfa7599d64160b42b750a38e5c239 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:04:34.066515 15992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd0dfa7599d64160b42b750a38e5c239. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:34.069290 16051 tablet_bootstrap.cc:492] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:34.069787 16051 ts_tablet_manager.cc:1403] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.343s	user 0.019s	sys 0.017s
W20260709 13:04:34.058598 15974 leader_election.cc:343] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:34.073583 15924 tablet_service.cc:1511] Processing CreateTablet for tablet dd0dfa7599d64160b42b750a38e5c239 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:04:34.075338 15924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd0dfa7599d64160b42b750a38e5c239. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:34.085858 15994 tablet_service.cc:1511] Processing CreateTablet for tablet 7b6e87097733428bb2167c6f308fe589 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:04:34.088826 16000 tablet_service.cc:1511] Processing CreateTablet for tablet 975ecd3d27dd45ba9a554ac0ee591e0e (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:04:34.091713 16000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 975ecd3d27dd45ba9a554ac0ee591e0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:34.091441 15999 tablet_service.cc:1511] Processing CreateTablet for tablet 83a1b79cdcf046c78453695c798a14a5 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:04:34.093169 16061 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.096726 16061 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.098759 16061 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:34.100158 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:34.102077 16051 raft_consensus.cc:359] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.102754 16051 raft_consensus.cc:385] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:34.102370 15995 tablet_service.cc:1511] Processing CreateTablet for tablet 741afa89185645afa8b1c82d31a75748 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:04:34.103057 16051 raft_consensus.cc:740] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
W20260709 13:04:34.104063 15827 leader_election.cc:343] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:34.104399 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 45843.2us
I20260709 13:04:34.106297 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:34.106668 15993 tablet_service.cc:1511] Processing CreateTablet for tablet ceaf02d91e334d5eba077b59ccce3a38 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
W20260709 13:04:34.107790 15974 leader_election.cc:343] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:34.107841 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:04:34.109279 15974 leader_election.cc:343] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:34.109797 15974 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:04:34.114710 15843 tablet_service.cc:1511] Processing CreateTablet for tablet 975ecd3d27dd45ba9a554ac0ee591e0e (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:04:34.117296 15843 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 975ecd3d27dd45ba9a554ac0ee591e0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:34.119331 15849 tablet_service.cc:1511] Processing CreateTablet for tablet 741afa89185645afa8b1c82d31a75748 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:04:34.119803 15846 tablet_service.cc:1511] Processing CreateTablet for tablet 83a1b79cdcf046c78453695c798a14a5 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:04:34.115587 16051 consensus_queue.cc:260] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.113054 16092 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:34.122246 16051 ts_tablet_manager.cc:1434] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.052s	user 0.008s	sys 0.000s
I20260709 13:04:34.122354 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:34.133137 16051 tablet_bootstrap.cc:492] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:34.135267 15918 tablet_service.cc:1511] Processing CreateTablet for tablet 83a1b79cdcf046c78453695c798a14a5 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:04:34.136545 15918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83a1b79cdcf046c78453695c798a14a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:34.136655 15919 tablet_service.cc:1511] Processing CreateTablet for tablet 741afa89185645afa8b1c82d31a75748 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:04:34.138608 15923 tablet_service.cc:1511] Processing CreateTablet for tablet ceaf02d91e334d5eba077b59ccce3a38 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:04:34.131044 15758 catalog_manager.cc:5697] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:34.141225 15999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83a1b79cdcf046c78453695c798a14a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:34.141886 15994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b6e87097733428bb2167c6f308fe589. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:04:34.142712 15825 leader_election.cc:343] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:34.143153 15825 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:34.143424 15850 tablet_service.cc:1511] Processing CreateTablet for tablet ceaf02d91e334d5eba077b59ccce3a38 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:04:34.143884 16061 raft_consensus.cc:2749] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:34.135480 15926 tablet_service.cc:1511] Processing CreateTablet for tablet 7b6e87097733428bb2167c6f308fe589 (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:04:34.144665 15850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ceaf02d91e334d5eba077b59ccce3a38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:34.145439 15849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 741afa89185645afa8b1c82d31a75748. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:34.146287 15993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ceaf02d91e334d5eba077b59ccce3a38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:34.147331 15995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 741afa89185645afa8b1c82d31a75748. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:34.147466 15919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 741afa89185645afa8b1c82d31a75748. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:34.151046 16073 consensus_queue.cc:1048] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:34.151947 15923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ceaf02d91e334d5eba077b59ccce3a38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:34.152043 15846 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83a1b79cdcf046c78453695c798a14a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:34.160077 15926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b6e87097733428bb2167c6f308fe589. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:34.167598 15924 tablet_service.cc:1511] Processing CreateTablet for tablet 975ecd3d27dd45ba9a554ac0ee591e0e (DEFAULT_TABLE table=test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:04:34.169085 15924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 975ecd3d27dd45ba9a554ac0ee591e0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:34.180224 16051 tablet_bootstrap.cc:654] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:34.201951 16054 tablet_bootstrap.cc:492] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:34.209453 16088 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.210021 16088 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.211932 16088 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:34.206337 16054 ts_tablet_manager.cc:1403] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.259s	user 0.017s	sys 0.014s
I20260709 13:04:34.218721 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:34.220129 16054 raft_consensus.cc:359] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.220835 16054 raft_consensus.cc:385] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:04:34.220952 15974 leader_election.cc:343] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:34.221176 16054 raft_consensus.cc:740] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:34.214818 15759 catalog_manager.cc:5697] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:34.222015 16054 consensus_queue.cc:260] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.222399 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:34.216858 16055 tablet_bootstrap.cc:492] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:34.223007 15936 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 0.
I20260709 13:04:34.225010 15974 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:04:34.225805 16088 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:34.226150 16088 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.226486 16088 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.234372 16088 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.236977 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:04:34.238474 15974 leader_election.cc:343] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:34.240434 16088 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:34.240563 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:34.241552 15936 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.232152 16055 ts_tablet_manager.cc:1403] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.225s	user 0.009s	sys 0.022s
I20260709 13:04:34.247591 16055 raft_consensus.cc:359] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.248308 16055 raft_consensus.cc:385] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:34.248595 16055 raft_consensus.cc:740] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:34.249707 16055 consensus_queue.cc:260] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.250593 16054 ts_tablet_manager.cc:1434] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.033s	user 0.005s	sys 0.001s
I20260709 13:04:34.251248 15936 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 1.
I20260709 13:04:34.251561 16054 tablet_bootstrap.cc:492] T f9822eb87c7e4819af0dc0b0d2951def P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:34.252524 15974 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:04:34.256465 16051 tablet_bootstrap.cc:492] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:34.256968 16051 ts_tablet_manager.cc:1403] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.124s	user 0.014s	sys 0.000s
I20260709 13:04:34.257205 16064 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.262224 16064 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.264008 16064 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:34.256980 16092 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:34.264619 16092 raft_consensus.cc:697] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:04:34.264601 16051 raft_consensus.cc:359] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.265359 16051 raft_consensus.cc:385] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:34.265440 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:34.265719 16051 raft_consensus.cc:740] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:34.265657 16092 consensus_queue.cc:237] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.266121 15862 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 0.
I20260709 13:04:34.266525 16051 consensus_queue.cc:260] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.267303 15974 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:34.268589 16053 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.267097 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:34.261797 16054 tablet_bootstrap.cc:654] T f9822eb87c7e4819af0dc0b0d2951def P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:34.269086 16053 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.271243 16053 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:34.268879 16051 ts_tablet_manager.cc:1434] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:04:34.268832 16064 raft_consensus.cc:2804] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:34.272112 16064 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.272459 16064 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.272708 16051 tablet_bootstrap.cc:492] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:34.272518 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:34.273380 15936 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 0.
I20260709 13:04:34.275643 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:34.276234 15862 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 0.
I20260709 13:04:34.277452 15900 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:34.278354 16053 raft_consensus.cc:2804] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:34.278710 16053 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.279032 16053 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.280411 16051 tablet_bootstrap.cc:654] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:34.259310 16055 ts_tablet_manager.cc:1434] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 13:04:34.284586 16055 tablet_bootstrap.cc:492] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:34.286016 16053 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.289640 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:34.290194 15862 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.295166 16055 tablet_bootstrap.cc:654] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:34.301805 15862 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:04:34.302542 16053 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:34.304510 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:34.306452 16054 tablet_bootstrap.cc:492] T f9822eb87c7e4819af0dc0b0d2951def P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:34.307078 16054 ts_tablet_manager.cc:1403] T f9822eb87c7e4819af0dc0b0d2951def P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.056s	user 0.024s	sys 0.012s
I20260709 13:04:34.308101 15900 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:34.309676 16053 raft_consensus.cc:2804] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:34.309717 16064 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.311041 16053 raft_consensus.cc:697] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:04:34.311385 16011 raft_consensus.cc:2393] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 1: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:04:34.314107 16064 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:34.312047 16053 consensus_queue.cc:237] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.315917 16055 tablet_bootstrap.cc:492] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:34.316417 16055 ts_tablet_manager.cc:1403] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.032s	user 0.009s	sys 0.003s
I20260709 13:04:34.317880 16054 raft_consensus.cc:359] T f9822eb87c7e4819af0dc0b0d2951def P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.318729 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:34.318709 16054 raft_consensus.cc:385] T f9822eb87c7e4819af0dc0b0d2951def P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:34.319350 16055 raft_consensus.cc:359] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.320029 16054 raft_consensus.cc:740] T f9822eb87c7e4819af0dc0b0d2951def P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:34.320166 16055 raft_consensus.cc:385] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:34.320541 16055 raft_consensus.cc:740] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:34.320791 16054 consensus_queue.cc:260] T f9822eb87c7e4819af0dc0b0d2951def P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.321463 16055 consensus_queue.cc:260] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.323012 16054 ts_tablet_manager.cc:1434] T f9822eb87c7e4819af0dc0b0d2951def P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.016s	user 0.003s	sys 0.003s
I20260709 13:04:34.323781 16055 ts_tablet_manager.cc:1434] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:04:34.324926 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:34.325932 16054 tablet_bootstrap.cc:492] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:34.325378 16051 tablet_bootstrap.cc:492] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:34.326529 16051 ts_tablet_manager.cc:1403] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.054s	user 0.023s	sys 0.017s
I20260709 13:04:34.329258 16051 raft_consensus.cc:359] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.330013 16051 raft_consensus.cc:385] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:34.330331 16051 raft_consensus.cc:740] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:34.326007 15862 raft_consensus.cc:2393] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 1: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:04:34.331689 15974 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:04:34.331576 16051 consensus_queue.cc:260] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.332278 16054 tablet_bootstrap.cc:654] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:34.332607 16064 raft_consensus.cc:2749] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:04:34.332029 15759 catalog_manager.cc:5697] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:34.333745 16051 ts_tablet_manager.cc:1434] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:04:34.339614 16051 tablet_bootstrap.cc:492] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:34.334051 16061 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.334712 16055 tablet_bootstrap.cc:492] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:34.340375 16061 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.342399 16061 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:34.343921 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:34.344506 16011 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 0.
I20260709 13:04:34.346141 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:34.346653 16051 tablet_bootstrap.cc:654] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:34.347971 15827 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:34.348675 16054 tablet_bootstrap.cc:492] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
W20260709 13:04:34.348906 15825 leader_election.cc:343] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:34.349138 16054 ts_tablet_manager.cc:1403] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.023s	user 0.019s	sys 0.000s
I20260709 13:04:34.350410 15760 catalog_manager.cc:5697] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:34.353041 16054 raft_consensus.cc:359] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.353929 16054 raft_consensus.cc:385] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:34.354367 16054 raft_consensus.cc:740] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:34.354743 16055 tablet_bootstrap.cc:654] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:34.355295 16054 consensus_queue.cc:260] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.360949 16054 ts_tablet_manager.cc:1434] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:04:34.362337 16054 tablet_bootstrap.cc:492] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:34.370604 16054 tablet_bootstrap.cc:654] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:34.353870 16061 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:34.382174 16061 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.383242 16061 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.391628 16054 tablet_bootstrap.cc:492] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:34.392256 16054 ts_tablet_manager.cc:1403] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.030s	user 0.022s	sys 0.007s
I20260709 13:04:34.393759 16061 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.395529 16054 raft_consensus.cc:359] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.396407 16054 raft_consensus.cc:385] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:34.396727 16054 raft_consensus.cc:740] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:34.397481 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:34.397540 16054 consensus_queue.cc:260] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:04:34.399561 15825 leader_election.cc:343] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:34.402052 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:34.402761 16011 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.404944 16054 ts_tablet_manager.cc:1434] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.002s
I20260709 13:04:34.406121 16054 tablet_bootstrap.cc:492] T ae068fc0a8a84b148d068df2b4463b8c P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:34.413394 16054 tablet_bootstrap.cc:654] T ae068fc0a8a84b148d068df2b4463b8c P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:34.421962 16064 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.422557 16064 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.425200 16064 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:34.425465 16054 tablet_bootstrap.cc:492] T ae068fc0a8a84b148d068df2b4463b8c P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:34.426026 16054 ts_tablet_manager.cc:1403] T ae068fc0a8a84b148d068df2b4463b8c P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.020s	user 0.017s	sys 0.000s
I20260709 13:04:34.428047 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:34.429183 16054 raft_consensus.cc:359] T ae068fc0a8a84b148d068df2b4463b8c P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:04:34.429859 15974 leader_election.cc:343] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:34.430060 16054 raft_consensus.cc:385] T ae068fc0a8a84b148d068df2b4463b8c P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:34.430590 16054 raft_consensus.cc:740] T ae068fc0a8a84b148d068df2b4463b8c P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:34.431054 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:34.431870 15936 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 0.
I20260709 13:04:34.431618 16054 consensus_queue.cc:260] T ae068fc0a8a84b148d068df2b4463b8c P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.433547 15974 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:04:34.434624 16054 ts_tablet_manager.cc:1434] T ae068fc0a8a84b148d068df2b4463b8c P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:04:34.435027 16064 raft_consensus.cc:2804] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:34.435580 16064 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.435813 16054 tablet_bootstrap.cc:492] T 3a538795cdae403fae5941538575596b P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:34.436192 16064 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.444154 16054 tablet_bootstrap.cc:654] T 3a538795cdae403fae5941538575596b P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:34.453507 16061 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:34.421312 16011 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:04:34.455983 15827 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:04:34.457387 16061 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:34.458195 16054 tablet_bootstrap.cc:492] T 3a538795cdae403fae5941538575596b P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:34.458550 16061 raft_consensus.cc:697] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:04:34.458678 16054 ts_tablet_manager.cc:1403] T 3a538795cdae403fae5941538575596b P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.008s
I20260709 13:04:34.459455 16061 consensus_queue.cc:237] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.460191 16055 tablet_bootstrap.cc:492] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:34.460729 16055 ts_tablet_manager.cc:1403] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.126s	user 0.033s	sys 0.054s
I20260709 13:04:34.461551 16054 raft_consensus.cc:359] T 3a538795cdae403fae5941538575596b P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.462236 16054 raft_consensus.cc:385] T 3a538795cdae403fae5941538575596b P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:34.462533 16054 raft_consensus.cc:740] T 3a538795cdae403fae5941538575596b P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:34.462885 16064 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.463896 16055 raft_consensus.cc:359] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.464687 16055 raft_consensus.cc:385] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:34.465032 16055 raft_consensus.cc:740] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:34.464993 16054 consensus_queue.cc:260] T 3a538795cdae403fae5941538575596b P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.462695 16051 tablet_bootstrap.cc:492] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:34.465602 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:34.466133 16051 ts_tablet_manager.cc:1403] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.127s	user 0.045s	sys 0.073s
I20260709 13:04:34.466172 15862 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.466331 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:34.466943 15936 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.467484 16054 ts_tablet_manager.cc:1434] T 3a538795cdae403fae5941538575596b P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 13:04:34.467446 16055 consensus_queue.cc:260] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.470496 16054 tablet_bootstrap.cc:492] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:34.472033 16055 ts_tablet_manager.cc:1434] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.004s
I20260709 13:04:34.473698 15862 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 1.
I20260709 13:04:34.474856 15974 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:34.475641 16092 raft_consensus.cc:2804] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:34.476094 15936 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 1.
I20260709 13:04:34.476154 16092 raft_consensus.cc:697] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:04:34.469339 16051 raft_consensus.cc:359] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.476313 16054 tablet_bootstrap.cc:654] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:34.477351 16055 tablet_bootstrap.cc:492] T ffbfc0e667484157953a66ae2b697013 P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:34.477156 16092 consensus_queue.cc:237] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.477229 16064 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:34.477216 16051 raft_consensus.cc:385] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:34.481366 16051 raft_consensus.cc:740] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:34.482167 16051 consensus_queue.cc:260] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.486728 16051 ts_tablet_manager.cc:1434] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.020s	user 0.001s	sys 0.005s
I20260709 13:04:34.488379 16055 tablet_bootstrap.cc:654] T ffbfc0e667484157953a66ae2b697013 P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:34.497044 15760 catalog_manager.cc:5697] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:34.499858 16054 tablet_bootstrap.cc:492] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:34.499992 16055 tablet_bootstrap.cc:492] T ffbfc0e667484157953a66ae2b697013 P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:34.500401 16054 ts_tablet_manager.cc:1403] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.030s	user 0.009s	sys 0.014s
I20260709 13:04:34.500540 16055 ts_tablet_manager.cc:1403] T ffbfc0e667484157953a66ae2b697013 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.028s	user 0.012s	sys 0.003s
I20260709 13:04:34.503098 16054 raft_consensus.cc:359] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.503808 16054 raft_consensus.cc:385] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:34.504140 16054 raft_consensus.cc:740] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:34.503412 16055 raft_consensus.cc:359] T ffbfc0e667484157953a66ae2b697013 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.504896 16051 tablet_bootstrap.cc:492] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:34.504402 15759 catalog_manager.cc:5697] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:34.505369 16054 consensus_queue.cc:260] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.504849 16055 raft_consensus.cc:385] T ffbfc0e667484157953a66ae2b697013 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:34.506353 16055 raft_consensus.cc:740] T ffbfc0e667484157953a66ae2b697013 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:34.507229 16055 consensus_queue.cc:260] T ffbfc0e667484157953a66ae2b697013 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.508463 16054 ts_tablet_manager.cc:1434] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.001s
I20260709 13:04:34.509327 16054 tablet_bootstrap.cc:492] T d50c1e8d433a407aa9fae9cb9866f184 P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:34.509346 16055 ts_tablet_manager.cc:1434] T ffbfc0e667484157953a66ae2b697013 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:04:34.510561 16055 tablet_bootstrap.cc:492] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:34.513353 16051 tablet_bootstrap.cc:654] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:34.515339 16054 tablet_bootstrap.cc:654] T d50c1e8d433a407aa9fae9cb9866f184 P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:34.515938 16055 tablet_bootstrap.cc:654] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:34.519373 16061 consensus_queue.cc:1048] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:34.541517 16055 tablet_bootstrap.cc:492] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:34.542698 16061 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.543439 16061 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.545395 16061 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:34.546873 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:04:34.548231 15827 leader_election.cc:343] T ae068fc0a8a84b148d068df2b4463b8c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:34.549960 16055 ts_tablet_manager.cc:1403] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.040s	user 0.014s	sys 0.000s
I20260709 13:04:34.550068 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:04:34.551712 15825 leader_election.cc:343] T ae068fc0a8a84b148d068df2b4463b8c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:34.552150 15825 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:34.553188 16061 raft_consensus.cc:2749] T ae068fc0a8a84b148d068df2b4463b8c P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:34.562564 16055 raft_consensus.cc:359] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.564113 16055 raft_consensus.cc:385] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:34.564453 16055 raft_consensus.cc:740] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:34.566550 16011 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.567157 16051 tablet_bootstrap.cc:492] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:34.567566 16051 ts_tablet_manager.cc:1403] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.063s	user 0.017s	sys 0.038s
I20260709 13:04:34.570380 16054 tablet_bootstrap.cc:492] T d50c1e8d433a407aa9fae9cb9866f184 P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:34.570072 16051 raft_consensus.cc:359] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.570659 16051 raft_consensus.cc:385] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:34.570784 16054 ts_tablet_manager.cc:1403] T d50c1e8d433a407aa9fae9cb9866f184 P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.062s	user 0.020s	sys 0.027s
I20260709 13:04:34.570935 16051 raft_consensus.cc:740] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:34.571532 16051 consensus_queue.cc:260] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.573381 16051 ts_tablet_manager.cc:1434] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.001s
I20260709 13:04:34.573194 16054 raft_consensus.cc:359] T d50c1e8d433a407aa9fae9cb9866f184 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.574190 16054 raft_consensus.cc:385] T d50c1e8d433a407aa9fae9cb9866f184 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:34.574690 16054 raft_consensus.cc:740] T d50c1e8d433a407aa9fae9cb9866f184 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:34.574406 16051 tablet_bootstrap.cc:492] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
W20260709 13:04:34.577235 15974 consensus_peers.cc:597] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:34.565181 16055 consensus_queue.cc:260] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.578428 16084 consensus_queue.cc:1048] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:34.580791 16054 consensus_queue.cc:260] T d50c1e8d433a407aa9fae9cb9866f184 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.580310 16055 ts_tablet_manager.cc:1434] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.026s	user 0.009s	sys 0.003s
I20260709 13:04:34.582547 16055 tablet_bootstrap.cc:492] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:34.582810 16054 ts_tablet_manager.cc:1434] T d50c1e8d433a407aa9fae9cb9866f184 P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.001s
I20260709 13:04:34.586627 16054 tablet_bootstrap.cc:492] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:34.588765 16051 tablet_bootstrap.cc:654] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:34.588842 16084 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.589329 16084 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.590183 16073 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.590651 16073 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.592429 16054 tablet_bootstrap.cc:654] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:34.592777 16073 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:34.592280 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:34.594581 15936 raft_consensus.cc:2393] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 1: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:04:34.594758 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:34.595649 16010 raft_consensus.cc:2393] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 1: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:04:34.602039 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:34.605033 15900 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:34.611632 16073 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.616672 16055 tablet_bootstrap.cc:654] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:34.618911 16084 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:34.620954 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:34.622972 15827 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:34.626029 16054 tablet_bootstrap.cc:492] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:34.626502 16054 ts_tablet_manager.cc:1403] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.040s	user 0.018s	sys 0.019s
I20260709 13:04:34.628441 16055 tablet_bootstrap.cc:492] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:34.628855 16055 ts_tablet_manager.cc:1403] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.046s	user 0.012s	sys 0.001s
I20260709 13:04:34.629271 16054 raft_consensus.cc:359] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.629958 16054 raft_consensus.cc:385] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:34.630219 16054 raft_consensus.cc:740] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:34.631032 16054 consensus_queue.cc:260] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.631910 16084 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.633049 16054 ts_tablet_manager.cc:1434] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:04:34.634791 16054 tablet_bootstrap.cc:492] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:34.640822 16054 tablet_bootstrap.cc:654] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:34.649363 16122 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.649893 16092 consensus_queue.cc:1048] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:34.649859 16122 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.652720 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:04:34.654083 15825 leader_election.cc:343] T f9822eb87c7e4819af0dc0b0d2951def P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:34.655164 16122 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:34.656962 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:04:34.659871 15827 leader_election.cc:343] T f9822eb87c7e4819af0dc0b0d2951def P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:34.660456 15827 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:34.661531 16122 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:34.633699 16055 raft_consensus.cc:359] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.662973 16055 raft_consensus.cc:385] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:34.663336 16055 raft_consensus.cc:740] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:34.664299 16055 consensus_queue.cc:260] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.667135 16055 ts_tablet_manager.cc:1434] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.000s
I20260709 13:04:34.668243 16055 tablet_bootstrap.cc:492] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:34.676105 16084 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:34.679909 16122 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.680363 16122 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.680382 16054 tablet_bootstrap.cc:492] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:34.682471 16122 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:34.685776 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:34.685882 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:34.686411 15937 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 0.
I20260709 13:04:34.686681 16088 consensus_queue.cc:1048] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:34.686416 16011 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 0.
I20260709 13:04:34.682700 16054 ts_tablet_manager.cc:1403] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.048s	user 0.014s	sys 0.008s
I20260709 13:04:34.687837 15825 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:34.689191 16122 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:34.689639 16122 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.690044 16122 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.693189 16088 consensus_queue.cc:1048] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:34.695055 16053 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.695516 16053 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.683797 16073 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:34.700470 16054 raft_consensus.cc:359] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.701030 16053 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:34.701162 16054 raft_consensus.cc:385] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:34.701308 16051 tablet_bootstrap.cc:492] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:34.701494 16054 raft_consensus.cc:740] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:34.701697 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:34.702356 15861 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 0.
I20260709 13:04:34.702365 16054 consensus_queue.cc:260] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.703387 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:34.703500 15900 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:34.702384 16088 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.704963 16053 raft_consensus.cc:2804] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:04:34.704815 15902 leader_election.cc:343] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:34.705480 16053 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.705420 16088 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.705976 16053 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.704787 16054 ts_tablet_manager.cc:1434] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:04:34.705247 16055 tablet_bootstrap.cc:654] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:34.702239 16051 ts_tablet_manager.cc:1403] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.128s	user 0.032s	sys 0.056s
I20260709 13:04:34.709091 16054 tablet_bootstrap.cc:492] T b5d68c78f0d94e0eb90543b642d1f1af P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:34.712105 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:34.712368 16088 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:34.712724 15861 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 0.
I20260709 13:04:34.714174 15974 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:34.715168 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:04:34.716385 15974 leader_election.cc:343] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:34.716177 16130 consensus_queue.cc:1048] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:34.717458 16088 raft_consensus.cc:2804] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:34.717437 16053 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.718441 16088 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.718824 16088 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.719606 16053 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:34.721395 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:34.722455 15860 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.724184 16051 raft_consensus.cc:359] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.724817 16051 raft_consensus.cc:385] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:34.725092 16051 raft_consensus.cc:740] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:34.726060 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:04:34.727283 15902 leader_election.cc:343] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:34.729775 15860 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:04:34.740079 15900 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:34.741180 16053 raft_consensus.cc:2804] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:34.720572 16122 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.743427 16053 raft_consensus.cc:697] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:04:34.743749 16064 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.744170 16064 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.744330 16053 consensus_queue.cc:237] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.745625 16088 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.747300 16054 tablet_bootstrap.cc:654] T b5d68c78f0d94e0eb90543b642d1f1af P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:34.746152 16064 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:34.748078 16051 consensus_queue.cc:260] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.752321 16051 ts_tablet_manager.cc:1434] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.044s	user 0.005s	sys 0.001s
I20260709 13:04:34.748075 16088 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:34.754122 16051 tablet_bootstrap.cc:492] T 40e7e6076e8d475da059aad457e0e8b1 P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:34.752336 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:34.753786 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:34.756997 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:34.757319 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:34.757804 15859 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:34.760504 15974 leader_election.cc:343] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:34.763909 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:34.764438 15935 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.757036 15936 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 0.
I20260709 13:04:34.757743 15860 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 0.
I20260709 13:04:34.769556 15974 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:34.770462 16073 consensus_queue.cc:1048] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:34.771195 16088 raft_consensus.cc:2804] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:34.771574 16088 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.771955 16088 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.783645 16122 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:34.786491 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:34.787084 16011 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.788853 16088 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.789870 15859 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 1.
I20260709 13:04:34.790967 16088 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:34.791574 15935 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:04:34.793665 15974 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:04:34.794883 16088 raft_consensus.cc:2804] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:34.796293 16088 raft_consensus.cc:697] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:04:34.797309 16088 consensus_queue.cc:237] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.797930 16011 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:04:34.799114 15827 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:34.785003 16130 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.799988 16122 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:34.800758 16122 raft_consensus.cc:697] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:04:34.801887 16122 consensus_queue.cc:237] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.800503 16130 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.804775 16130 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:34.805969 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:34.806826 15861 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 0.
I20260709 13:04:34.810478 16051 tablet_bootstrap.cc:654] T 40e7e6076e8d475da059aad457e0e8b1 P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:34.811327 15900 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:34.813364 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:34.814550 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:34.818940 15859 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.821363 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:34.822202 15935 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.815824 16130 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:34.831750 16055 tablet_bootstrap.cc:492] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:34.832396 16055 ts_tablet_manager.cc:1403] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.164s	user 0.012s	sys 0.010s
I20260709 13:04:34.837528 16055 raft_consensus.cc:359] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.838903 16055 raft_consensus.cc:385] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:04:34.843780 15902 leader_election.cc:343] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:34.845039 16055 raft_consensus.cc:740] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:34.829908 16130 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.846488 16130 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.852877 16055 consensus_queue.cc:260] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.855482 16055 ts_tablet_manager.cc:1434] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.001s
I20260709 13:04:34.856316 16055 tablet_bootstrap.cc:492] T 626a74f434ad4e70bed42f8b404a6e81 P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:34.861605 15935 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 1.
I20260709 13:04:34.865700 15974 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:34.867498 16092 consensus_queue.cc:1048] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:34.870167 15759 catalog_manager.cc:5697] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:34.870893 15758 catalog_manager.cc:5697] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:34.874857 16088 raft_consensus.cc:2804] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:34.876227 16088 raft_consensus.cc:697] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:04:34.877969 16088 consensus_queue.cc:237] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.884819 16054 tablet_bootstrap.cc:492] T b5d68c78f0d94e0eb90543b642d1f1af P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:34.885290 16054 ts_tablet_manager.cc:1403] T b5d68c78f0d94e0eb90543b642d1f1af P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.177s	user 0.022s	sys 0.024s
I20260709 13:04:34.889730 16054 raft_consensus.cc:359] T b5d68c78f0d94e0eb90543b642d1f1af P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.890475 16054 raft_consensus.cc:385] T b5d68c78f0d94e0eb90543b642d1f1af P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:34.890714 16054 raft_consensus.cc:740] T b5d68c78f0d94e0eb90543b642d1f1af P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:34.895948 16130 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.901135 15859 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 1.
I20260709 13:04:34.900673 16054 consensus_queue.cc:260] T b5d68c78f0d94e0eb90543b642d1f1af P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.905283 15760 catalog_manager.cc:5697] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:34.908047 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:34.908710 15862 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:34.910995 16055 tablet_bootstrap.cc:654] T 626a74f434ad4e70bed42f8b404a6e81 P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:34.922128 16064 consensus_queue.cc:1048] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:34.930379 16054 ts_tablet_manager.cc:1434] T b5d68c78f0d94e0eb90543b642d1f1af P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.045s	user 0.005s	sys 0.003s
I20260709 13:04:34.931607 16054 tablet_bootstrap.cc:492] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:34.937711 16130 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:34.939497 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:04:34.940976 15902 leader_election.cc:343] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:34.942970 16122 consensus_queue.cc:1048] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:34.952837 16054 tablet_bootstrap.cc:654] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:34.958077 16051 tablet_bootstrap.cc:492] T 40e7e6076e8d475da059aad457e0e8b1 P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:34.958460 16051 ts_tablet_manager.cc:1403] T 40e7e6076e8d475da059aad457e0e8b1 P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.205s	user 0.024s	sys 0.028s
I20260709 13:04:34.959064 15862 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:04:34.959188 15760 catalog_manager.cc:5697] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:34.965660 16051 raft_consensus.cc:359] T 40e7e6076e8d475da059aad457e0e8b1 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.966157 16051 raft_consensus.cc:385] T 40e7e6076e8d475da059aad457e0e8b1 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:34.966370 16051 raft_consensus.cc:740] T 40e7e6076e8d475da059aad457e0e8b1 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:34.966955 16051 consensus_queue.cc:260] T 40e7e6076e8d475da059aad457e0e8b1 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.968747 16051 ts_tablet_manager.cc:1434] T 40e7e6076e8d475da059aad457e0e8b1 P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.003s
I20260709 13:04:34.969702 15900 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:34.970925 16053 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:34.971378 16053 raft_consensus.cc:697] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:04:34.972206 16053 consensus_queue.cc:237] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:34.975046 16061 consensus_queue.cc:1048] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:34.976735 16051 tablet_bootstrap.cc:492] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
W20260709 13:04:34.984179 15808 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:34.994226 16054 tablet_bootstrap.cc:492] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:34.994704 16054 ts_tablet_manager.cc:1403] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.063s	user 0.014s	sys 0.019s
I20260709 13:04:34.999567 16054 raft_consensus.cc:359] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.000128 16051 tablet_bootstrap.cc:654] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:35.000329 16054 raft_consensus.cc:385] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:35.000720 16054 raft_consensus.cc:740] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:35.001535 16054 consensus_queue.cc:260] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.003782 16054 ts_tablet_manager.cc:1434] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 13:04:35.004556 16054 tablet_bootstrap.cc:492] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:35.006008 16055 tablet_bootstrap.cc:492] T 626a74f434ad4e70bed42f8b404a6e81 P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:35.006458 16055 ts_tablet_manager.cc:1403] T 626a74f434ad4e70bed42f8b404a6e81 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.150s	user 0.010s	sys 0.006s
I20260709 13:04:35.008966 16054 tablet_bootstrap.cc:654] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:35.011857 16061 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.012288 16061 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.012003 15758 catalog_manager.cc:5697] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:35.013715 16054 tablet_bootstrap.cc:492] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:35.014191 16054 ts_tablet_manager.cc:1403] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.010s	user 0.008s	sys 0.001s
I20260709 13:04:35.014362 16061 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:35.015112 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:35.015342 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:04:35.016465 15825 leader_election.cc:343] T b5d68c78f0d94e0eb90543b642d1f1af P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:35.016896 16054 raft_consensus.cc:359] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.017592 16054 raft_consensus.cc:385] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:35.017890 16054 raft_consensus.cc:740] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:35.019032 16051 tablet_bootstrap.cc:492] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:35.018679 16054 consensus_queue.cc:260] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.019526 16051 ts_tablet_manager.cc:1403] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.050s	user 0.008s	sys 0.004s
I20260709 13:04:35.020763 16054 ts_tablet_manager.cc:1434] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:04:35.021610 16054 tablet_bootstrap.cc:492] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
W20260709 13:04:35.022622 15827 leader_election.cc:343] T b5d68c78f0d94e0eb90543b642d1f1af P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:35.023006 16084 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.023062 15827 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:35.023463 16084 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.026171 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:35.026767 16084 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:35.027468 16084 raft_consensus.cc:2749] T b5d68c78f0d94e0eb90543b642d1f1af P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:35.022208 16051 raft_consensus.cc:359] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.028119 16054 tablet_bootstrap.cc:654] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:35.028426 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:35.030018 15827 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:35.013727 16055 raft_consensus.cc:359] T 626a74f434ad4e70bed42f8b404a6e81 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.031078 16084 raft_consensus.cc:2749] T d50c1e8d433a407aa9fae9cb9866f184 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:35.031571 16055 raft_consensus.cc:385] T 626a74f434ad4e70bed42f8b404a6e81 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:35.031960 16055 raft_consensus.cc:740] T 626a74f434ad4e70bed42f8b404a6e81 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:35.028448 16051 raft_consensus.cc:385] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:35.032488 16051 raft_consensus.cc:740] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:35.032841 16055 consensus_queue.cc:260] T 626a74f434ad4e70bed42f8b404a6e81 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.033489 16051 consensus_queue.cc:260] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.035233 16055 ts_tablet_manager.cc:1434] T 626a74f434ad4e70bed42f8b404a6e81 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.001s
I20260709 13:04:35.036315 16055 tablet_bootstrap.cc:492] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:35.036617 16051 ts_tablet_manager.cc:1434] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.001s
I20260709 13:04:35.037632 16051 tablet_bootstrap.cc:492] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:35.042220 16055 tablet_bootstrap.cc:654] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:35.044070 16051 tablet_bootstrap.cc:654] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:35.060038 16054 tablet_bootstrap.cc:492] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:35.060421 16054 ts_tablet_manager.cc:1403] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.039s	user 0.024s	sys 0.006s
I20260709 13:04:35.061209 16055 tablet_bootstrap.cc:492] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:35.061630 16055 ts_tablet_manager.cc:1403] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.025s	user 0.008s	sys 0.005s
I20260709 13:04:35.062328 16054 raft_consensus.cc:359] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.062804 16054 raft_consensus.cc:385] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:35.063007 16054 raft_consensus.cc:740] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:35.063562 16054 consensus_queue.cc:260] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.063619 16055 raft_consensus.cc:359] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.064314 16055 raft_consensus.cc:385] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:35.064603 16055 raft_consensus.cc:740] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:35.065269 16055 consensus_queue.cc:260] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.065433 16054 ts_tablet_manager.cc:1434] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:04:35.066629 16054 tablet_bootstrap.cc:492] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:35.066731 16055 ts_tablet_manager.cc:1434] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20260709 13:04:35.067480 16055 tablet_bootstrap.cc:492] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:35.072301 16054 tablet_bootstrap.cc:654] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:35.073483 16055 tablet_bootstrap.cc:654] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:35.078505 16051 tablet_bootstrap.cc:492] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:35.079140 16051 ts_tablet_manager.cc:1403] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.042s	user 0.013s	sys 0.011s
I20260709 13:04:35.081694 16055 tablet_bootstrap.cc:492] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:35.082150 16055 ts_tablet_manager.cc:1403] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.015s	user 0.012s	sys 0.000s
I20260709 13:04:35.082043 16051 raft_consensus.cc:359] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.082831 16051 raft_consensus.cc:385] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:35.083129 16051 raft_consensus.cc:740] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:35.083988 16051 consensus_queue.cc:260] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.084738 16055 raft_consensus.cc:359] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.085355 16055 raft_consensus.cc:385] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:35.085594 16055 raft_consensus.cc:740] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:35.086167 16055 consensus_queue.cc:260] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.089087 16051 ts_tablet_manager.cc:1434] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:04:35.089241 16054 tablet_bootstrap.cc:492] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:35.089696 16054 ts_tablet_manager.cc:1403] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.008s
I20260709 13:04:35.090076 16051 tablet_bootstrap.cc:492] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:35.092135 16054 raft_consensus.cc:359] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.092772 16054 raft_consensus.cc:385] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:35.093034 16054 raft_consensus.cc:740] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:35.093653 16054 consensus_queue.cc:260] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.095620 16054 ts_tablet_manager.cc:1434] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:04:35.096544 16054 tablet_bootstrap.cc:492] T 0b3676041725488fb6ffad37889b386b P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:35.096660 16051 tablet_bootstrap.cc:654] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:35.098253 16055 ts_tablet_manager.cc:1434] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:04:35.099215 16055 tablet_bootstrap.cc:492] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:35.102118 16054 tablet_bootstrap.cc:654] T 0b3676041725488fb6ffad37889b386b P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:35.106535 16055 tablet_bootstrap.cc:654] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:35.114876 15862 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:35.119494 16051 tablet_bootstrap.cc:492] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:35.120251 16051 ts_tablet_manager.cc:1403] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.030s	user 0.011s	sys 0.007s
I20260709 13:04:35.123708 16051 raft_consensus.cc:359] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.124732 16051 raft_consensus.cc:385] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:35.125111 16051 raft_consensus.cc:740] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:35.126117 16051 consensus_queue.cc:260] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.132261 16051 ts_tablet_manager.cc:1434] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:04:35.133471 16051 tablet_bootstrap.cc:492] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:35.130043 16054 tablet_bootstrap.cc:492] T 0b3676041725488fb6ffad37889b386b P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:35.137883 16054 ts_tablet_manager.cc:1403] T 0b3676041725488fb6ffad37889b386b P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.042s	user 0.009s	sys 0.013s
I20260709 13:04:35.140717 16084 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.141360 16084 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.143760 16055 tablet_bootstrap.cc:492] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:35.144320 16055 ts_tablet_manager.cc:1403] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.045s	user 0.029s	sys 0.015s
I20260709 13:04:35.143369 16054 raft_consensus.cc:359] T 0b3676041725488fb6ffad37889b386b P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.145054 16054 raft_consensus.cc:385] T 0b3676041725488fb6ffad37889b386b P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:35.145385 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:35.146926 16051 tablet_bootstrap.cc:654] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:35.147192 15825 leader_election.cc:343] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:35.147526 16055 raft_consensus.cc:359] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.148356 16055 raft_consensus.cc:385] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:35.148753 16055 raft_consensus.cc:740] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:35.149722 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:35.149458 16055 consensus_queue.cc:260] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:04:35.151611 15827 leader_election.cc:343] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:35.152195 15827 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:35.153235 16061 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:35.154737 16084 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:35.155977 16061 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.156581 16061 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.159129 16061 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:35.161173 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:35.165272 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:35.173789 16053 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.176326 16051 tablet_bootstrap.cc:492] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:35.175872 16053 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.176831 16051 ts_tablet_manager.cc:1403] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.044s	user 0.015s	sys 0.012s
W20260709 13:04:35.176672 15827 leader_election.cc:343] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:35.179198 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:35.179383 16051 raft_consensus.cc:359] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.180059 16051 raft_consensus.cc:385] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:35.180230 16053 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:35.180325 16051 raft_consensus.cc:740] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
W20260709 13:04:35.180677 15900 leader_election.cc:343] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:35.181772 15825 leader_election.cc:343] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:35.181655 16051 consensus_queue.cc:260] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.182022 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:35.182337 15825 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:04:35.183459 15902 leader_election.cc:343] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:35.145413 16054 raft_consensus.cc:740] T 0b3676041725488fb6ffad37889b386b P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:35.183980 16051 ts_tablet_manager.cc:1434] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:04:35.183881 15902 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:35.184881 16051 tablet_bootstrap.cc:492] T 952bd0ea3bba4d38ad3383fdf572cd01 P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:35.184582 16054 consensus_queue.cc:260] T 0b3676041725488fb6ffad37889b386b P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.185132 16053 raft_consensus.cc:2749] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:35.162464 16055 ts_tablet_manager.cc:1434] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.001s
I20260709 13:04:35.188295 16055 tablet_bootstrap.cc:492] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:35.190109 16051 tablet_bootstrap.cc:654] T 952bd0ea3bba4d38ad3383fdf572cd01 P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:35.195924 16084 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:35.198130 16054 ts_tablet_manager.cc:1434] T 0b3676041725488fb6ffad37889b386b P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.060s	user 0.007s	sys 0.000s
I20260709 13:04:35.198995 16054 tablet_bootstrap.cc:492] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:35.199795 16122 consensus_queue.cc:1048] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:35.201349 16051 tablet_bootstrap.cc:492] T 952bd0ea3bba4d38ad3383fdf572cd01 P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:35.201783 16051 ts_tablet_manager.cc:1403] T 952bd0ea3bba4d38ad3383fdf572cd01 P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.017s	user 0.008s	sys 0.006s
I20260709 13:04:35.202060 16122 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.202502 16122 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.203892 16051 raft_consensus.cc:359] T 952bd0ea3bba4d38ad3383fdf572cd01 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.204591 16051 raft_consensus.cc:385] T 952bd0ea3bba4d38ad3383fdf572cd01 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:35.204895 16051 raft_consensus.cc:740] T 952bd0ea3bba4d38ad3383fdf572cd01 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:35.205457 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:35.205721 16051 consensus_queue.cc:260] T 952bd0ea3bba4d38ad3383fdf572cd01 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.206370 16122 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:35.206550 15937 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 0.
I20260709 13:04:35.207434 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:35.207823 15825 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:35.208031 16011 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 0.
I20260709 13:04:35.208649 16122 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:35.209005 16122 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:35.209334 16122 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:35.212571 16055 tablet_bootstrap.cc:654] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:35.216099 16061 consensus_queue.cc:1048] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:35.216169 16122 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.217718 16162 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.218319 16122 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:35.218158 16162 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.219563 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:35.220130 16051 ts_tablet_manager.cc:1434] T 952bd0ea3bba4d38ad3383fdf572cd01 P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.018s	user 0.002s	sys 0.005s
I20260709 13:04:35.220117 16162 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:35.220266 15935 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:35.221191 16051 tablet_bootstrap.cc:492] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:35.222436 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:35.223032 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:04:35.223848 15827 leader_election.cc:343] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:35.224355 15825 leader_election.cc:343] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:35.224772 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:35.224782 15825 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:35.225477 16010 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:35.225948 16162 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:35.227105 16051 tablet_bootstrap.cc:654] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:35.227175 15935 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:04:35.228587 15825 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:35.229462 16162 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:35.229902 16162 raft_consensus.cc:697] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:04:35.230674 16162 consensus_queue.cc:237] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.231554 16054 tablet_bootstrap.cc:654] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:35.245698 16055 tablet_bootstrap.cc:492] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:35.246156 16055 ts_tablet_manager.cc:1403] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.058s	user 0.008s	sys 0.002s
I20260709 13:04:35.248755 16055 raft_consensus.cc:359] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.249672 16055 raft_consensus.cc:385] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:35.250000 16055 raft_consensus.cc:740] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:35.250857 16055 consensus_queue.cc:260] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.256266 16055 ts_tablet_manager.cc:1434] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:04:35.257009 16055 tablet_bootstrap.cc:492] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:35.262940 16055 tablet_bootstrap.cc:654] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:35.263844 16092 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.264521 16092 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.267088 16092 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:35.269461 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:35.270295 15937 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 0.
I20260709 13:04:35.272058 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:35.272322 15974 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:04:35.273878 15974 leader_election.cc:343] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:35.275516 16092 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:35.276026 16092 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:35.276463 16092 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:35.275547 16055 tablet_bootstrap.cc:492] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:35.280565 16055 ts_tablet_manager.cc:1403] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.024s	user 0.005s	sys 0.012s
I20260709 13:04:35.282838 16162 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.283422 16162 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.234067 16010 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:04:35.291320 16064 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.291910 16064 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.293902 16064 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:35.293486 16055 raft_consensus.cc:359] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.294377 16055 raft_consensus.cc:385] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:35.294723 16055 raft_consensus.cc:740] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:35.295717 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:35.295526 16055 consensus_queue.cc:260] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.297281 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:35.298329 16055 ts_tablet_manager.cc:1434] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
W20260709 13:04:35.299098 15825 leader_election.cc:343] T 0b3676041725488fb6ffad37889b386b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:35.300798 16162 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:04:35.301342 15827 leader_election.cc:343] T 0b3676041725488fb6ffad37889b386b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:35.302968 16054 tablet_bootstrap.cc:492] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:35.303252 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:35.304342 15937 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 0.
I20260709 13:04:35.302966 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:35.306394 15862 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 0.
I20260709 13:04:35.307016 15974 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:35.307407 16051 tablet_bootstrap.cc:492] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:35.307891 16051 ts_tablet_manager.cc:1403] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.087s	user 0.045s	sys 0.037s
I20260709 13:04:35.308221 15827 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:35.305462 16092 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.312076 16122 raft_consensus.cc:2749] T 0b3676041725488fb6ffad37889b386b P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:35.305598 16010 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:35.303573 16054 ts_tablet_manager.cc:1403] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.105s	user 0.032s	sys 0.012s
I20260709 13:04:35.312629 16122 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.313033 16122 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.313362 16092 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:35.308924 16088 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.314050 16088 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.314985 16122 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:35.308415 16064 raft_consensus.cc:2804] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:35.314783 16054 raft_consensus.cc:359] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.299453 16055 tablet_bootstrap.cc:492] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:35.316843 16054 raft_consensus.cc:385] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:35.317361 16064 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:35.318419 16054 raft_consensus.cc:740] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:35.310499 16051 raft_consensus.cc:359] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.315953 16088 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:35.319244 16051 raft_consensus.cc:385] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:35.318634 16064 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:35.319751 16054 consensus_queue.cc:260] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.320336 16073 consensus_queue.cc:1048] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:35.320725 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:04:35.323132 15825 leader_election.cc:343] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:35.323748 16051 raft_consensus.cc:740] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:35.324406 16051 consensus_queue.cc:260] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.325130 16054 ts_tablet_manager.cc:1434] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:04:35.325853 16054 tablet_bootstrap.cc:492] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:35.328733 16162 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.329152 16162 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.331784 16162 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:35.332799 16051 ts_tablet_manager.cc:1434] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.025s	user 0.009s	sys 0.003s
I20260709 13:04:35.333238 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:35.333194 16073 consensus_queue.cc:1048] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:35.333650 16051 tablet_bootstrap.cc:492] T 888cce851f074f3c969de10b02ff95ad P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:35.341357 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:35.341493 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:35.342523 15935 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:35.343045 15825 leader_election.cc:343] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:35.347736 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:35.348438 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:35.348803 16051 tablet_bootstrap.cc:654] T 888cce851f074f3c969de10b02ff95ad P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:35.348156 15759 catalog_manager.cc:5697] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:35.349787 15862 raft_consensus.cc:2393] T f9822eb87c7e4819af0dc0b0d2951def P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 1: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:04:35.345520 16064 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.352954 15935 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 1.
I20260709 13:04:35.353356 16054 tablet_bootstrap.cc:654] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:35.352181 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:35.359017 16055 tablet_bootstrap.cc:654] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:35.362275 15827 leader_election.cc:343] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:35.362737 15827 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:35.366430 15974 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:35.367188 16092 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:04:35.367297 15974 leader_election.cc:343] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:35.367657 16092 raft_consensus.cc:697] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:04:35.368610 15827 leader_election.cc:343] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:35.368455 16092 consensus_queue.cc:237] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.368979 16088 consensus_queue.cc:1048] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:35.369277 16162 raft_consensus.cc:2749] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:35.363739 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:35.369061 15827 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:35.371318 16122 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:35.378646 16064 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:35.380132 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:35.380761 15859 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:35.380522 15935 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:35.389083 16054 tablet_bootstrap.cc:492] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:35.389822 16054 ts_tablet_manager.cc:1403] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.064s	user 0.009s	sys 0.003s
I20260709 13:04:35.398540 16054 raft_consensus.cc:359] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.399587 16054 raft_consensus.cc:385] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:35.400012 16054 raft_consensus.cc:740] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:35.400857 16054 consensus_queue.cc:260] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.405774 16054 ts_tablet_manager.cc:1434] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.002s
I20260709 13:04:35.407178 16054 tablet_bootstrap.cc:492] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:35.408548 16130 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.409000 16130 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.414160 16130 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:35.416208 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:04:35.418107 15902 leader_election.cc:343] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:35.419091 15974 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:04:35.419473 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:35.420857 15937 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:35.424902 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:35.423075 16064 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:35.426882 15860 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 0.
I20260709 13:04:35.433794 15937 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 1.
I20260709 13:04:35.422796 15859 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 1.
I20260709 13:04:35.437669 16053 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.440164 16053 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.442721 16054 tablet_bootstrap.cc:654] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:35.449604 15900 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:35.450428 16073 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:35.450789 16073 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:35.443686 16092 consensus_queue.cc:1048] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:04:35.451143 16073 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:35.463426 16051 tablet_bootstrap.cc:492] T 888cce851f074f3c969de10b02ff95ad P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:35.463954 16051 ts_tablet_manager.cc:1403] T 888cce851f074f3c969de10b02ff95ad P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.130s	user 0.019s	sys 0.040s
I20260709 13:04:35.465903 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:35.464890 16053 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:35.467754 16051 raft_consensus.cc:359] T 888cce851f074f3c969de10b02ff95ad P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.468624 16051 raft_consensus.cc:385] T 888cce851f074f3c969de10b02ff95ad P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:35.469007 16051 raft_consensus.cc:740] T 888cce851f074f3c969de10b02ff95ad P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:35.472472 16051 consensus_queue.cc:260] T 888cce851f074f3c969de10b02ff95ad P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.477001 16051 ts_tablet_manager.cc:1434] T 888cce851f074f3c969de10b02ff95ad P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.002s
I20260709 13:04:35.472872 16073 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.483088 16073 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:35.483596 16051 tablet_bootstrap.cc:492] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:35.490085 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:35.492295 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:35.497635 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:35.498178 15859 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:35.498740 15900 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:35.499771 16053 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:04:35.500968 15902 leader_election.cc:343] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:35.502394 16064 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:35.489045 16073 consensus_queue.cc:1048] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 13:04:35.511767 16055 tablet_bootstrap.cc:492] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:35.512291 16055 ts_tablet_manager.cc:1403] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.213s	user 0.008s	sys 0.007s
I20260709 13:04:35.514694 15859 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:04:35.515406 16055 raft_consensus.cc:359] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.515800 15900 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:35.516202 16055 raft_consensus.cc:385] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:35.516613 16055 raft_consensus.cc:740] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:35.517303 16055 consensus_queue.cc:260] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.518111 16073 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:35.518654 16073 raft_consensus.cc:697] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:04:35.519383 16073 consensus_queue.cc:237] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.519878 16055 ts_tablet_manager.cc:1434] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:04:35.520524 16051 tablet_bootstrap.cc:654] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:35.521571 16055 tablet_bootstrap.cc:492] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:35.521781 16054 tablet_bootstrap.cc:492] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:35.522233 16054 ts_tablet_manager.cc:1403] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.115s	user 0.015s	sys 0.022s
W20260709 13:04:35.528874 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 94795.8us
I20260709 13:04:35.529487 16055 tablet_bootstrap.cc:654] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:35.529866 15974 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:35.530772 16178 raft_consensus.cc:2804] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:35.531260 16178 raft_consensus.cc:697] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:04:35.532102 16178 consensus_queue.cc:237] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.527819 16054 raft_consensus.cc:359] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.533325 16054 raft_consensus.cc:385] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:35.533671 16054 raft_consensus.cc:740] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:35.534480 16054 consensus_queue.cc:260] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.536760 16054 ts_tablet_manager.cc:1434] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:04:35.538061 16092 consensus_queue.cc:1048] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:35.541733 15758 catalog_manager.cc:5697] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:35.542083 16176 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.548236 16176 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.550824 16176 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:35.551646 16053 consensus_queue.cc:1048] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:35.585505 16054 tablet_bootstrap.cc:492] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:35.605450 16088 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.611954 16088 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.615540 16181 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.616192 16181 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.614466 16088 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:35.631428 16181 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:35.632182 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:35.633098 15934 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 0.
I20260709 13:04:35.608273 16092 consensus_queue.cc:1048] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260709 13:04:35.639217 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:35.639914 15860 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 0.
I20260709 13:04:35.637991 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:35.649760 16051 tablet_bootstrap.cc:492] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:35.650240 16051 ts_tablet_manager.cc:1403] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.172s	user 0.007s	sys 0.006s
I20260709 13:04:35.656767 16051 raft_consensus.cc:359] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.659641 16051 raft_consensus.cc:385] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:35.659967 16051 raft_consensus.cc:740] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:35.660550 16051 consensus_queue.cc:260] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.662891 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:35.663538 15933 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 0.
I20260709 13:04:35.663928 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:35.667243 15860 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 0.
I20260709 13:04:35.670980 16051 ts_tablet_manager.cc:1434] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.020s	user 0.003s	sys 0.003s
I20260709 13:04:35.669425 15974 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:04:35.673262 15974 leader_election.cc:343] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:35.675786 16181 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:35.675925 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:35.676187 16181 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:35.676548 16181 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:35.728746 16188 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.740085 16051 tablet_bootstrap.cc:492] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:35.740309 16188 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.722096 15974 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:04:35.713527 15757 catalog_manager.cc:5697] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:35.759418 16181 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.761035 16122 consensus_queue.cc:1048] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:04:35.761986 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 46526.6us
I20260709 13:04:35.763418 15974 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:35.764747 16088 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:35.765092 16088 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:35.765409 16088 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:35.768539 15974 leader_election.cc:343] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:35.753497 16176 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:35.783262 16188 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:35.784339 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:04:35.785837 15900 leader_election.cc:343] T 888cce851f074f3c969de10b02ff95ad P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:35.778366 16176 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:35.788070 16176 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:35.788756 16181 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:35.793313 16055 tablet_bootstrap.cc:492] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:35.794289 16055 ts_tablet_manager.cc:1403] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.274s	user 0.050s	sys 0.043s
I20260709 13:04:35.798629 16053 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.799399 16053 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.800218 16055 raft_consensus.cc:359] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.800737 16054 tablet_bootstrap.cc:654] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:35.800875 16055 raft_consensus.cc:385] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:35.801148 16055 raft_consensus.cc:740] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:35.801239 16053 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:35.802727 16055 consensus_queue.cc:260] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.808775 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:04:35.811872 15902 leader_election.cc:343] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:35.812940 16051 tablet_bootstrap.cc:654] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:35.814006 16055 ts_tablet_manager.cc:1434] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.001s
I20260709 13:04:35.814672 15759 catalog_manager.cc:5697] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:35.816838 16122 consensus_queue.cc:1048] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:35.819662 16055 tablet_bootstrap.cc:492] T d2be2d6f06bc47a39f26307ae9372194 P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:35.820425 16088 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.823809 16088 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:35.806284 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:35.825182 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:35.826062 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:35.833696 15861 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:35.827366 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:35.834411 15933 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:35.829337 15974 consensus_peers.cc:597] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:35.842077 15902 leader_election.cc:343] T 888cce851f074f3c969de10b02ff95ad P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:35.842566 15902 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:35.845111 16130 raft_consensus.cc:2749] T 888cce851f074f3c969de10b02ff95ad P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:35.836690 15860 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 0.
I20260709 13:04:35.870172 15900 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:35.871699 16188 raft_consensus.cc:2804] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:35.872171 16188 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:35.872628 16188 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:35.884039 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:35.900792 16051 tablet_bootstrap.cc:492] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:35.901294 16051 ts_tablet_manager.cc:1403] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.162s	user 0.006s	sys 0.014s
I20260709 13:04:35.902272 16054 tablet_bootstrap.cc:492] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:35.902706 16054 ts_tablet_manager.cc:1403] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.317s	user 0.009s	sys 0.004s
I20260709 13:04:35.903087 16055 tablet_bootstrap.cc:654] T d2be2d6f06bc47a39f26307ae9372194 P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:35.903993 16051 raft_consensus.cc:359] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.904644 16051 raft_consensus.cc:385] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:35.904893 16051 raft_consensus.cc:740] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:35.905032 16054 raft_consensus.cc:359] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.905628 16054 raft_consensus.cc:385] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:35.905882 16054 raft_consensus.cc:740] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:35.906083 15861 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 1.
I20260709 13:04:35.906234 16051 consensus_queue.cc:260] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.906481 16054 consensus_queue.cc:260] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.907198 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:35.907811 15935 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:35.908604 16051 ts_tablet_manager.cc:1434] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:04:35.909463 16051 tablet_bootstrap.cc:492] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:35.912971 15933 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 1.
I20260709 13:04:35.913740 16130 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.913924 16092 consensus_queue.cc:1048] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:35.914201 16130 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:04:35.915544 15974 leader_election.cc:343] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:35.916138 16130 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:35.916954 16176 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.917801 16054 ts_tablet_manager.cc:1434] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.002s
I20260709 13:04:35.918378 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:35.918932 16011 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 0.
I20260709 13:04:35.919907 15902 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:35.920660 16130 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:35.920981 16130 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:35.921315 16130 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:35.922345 15935 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 1.
I20260709 13:04:35.926115 15974 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:35.927376 16176 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:35.928087 16092 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:35.928223 16130 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.928650 16092 raft_consensus.cc:697] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:04:35.929507 16092 consensus_queue.cc:237] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.930920 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:35.931697 15859 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:35.932169 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:35.932701 16011 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:35.933611 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:35.935046 15974 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:04:35.935922 16178 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:35.936611 16178 raft_consensus.cc:697] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:04:35.937834 16178 consensus_queue.cc:237] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.938325 15859 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:04:35.939563 16054 tablet_bootstrap.cc:492] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:35.939491 15900 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:35.940356 16053 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:35.940827 16053 raft_consensus.cc:697] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:04:35.941581 16053 consensus_queue.cc:237] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.944773 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:35.945786 16011 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:04:35.951763 16130 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:35.955351 16051 tablet_bootstrap.cc:654] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:35.956442 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:35.957067 15860 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:35.955291 15974 leader_election.cc:343] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:35.958249 16188 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.961107 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:35.961654 15859 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:35.966112 16188 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:35.966964 16122 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.967407 16122 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.969726 16122 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:35.972069 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:35.972457 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:35.972585 15758 catalog_manager.cc:5697] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:35.973165 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:04:35.974304 15902 leader_election.cc:343] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:35.974304 15825 leader_election.cc:343] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:35.975392 15827 leader_election.cc:343] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:35.975842 15827 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:35.976349 16055 tablet_bootstrap.cc:492] T d2be2d6f06bc47a39f26307ae9372194 P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:35.976683 16122 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:35.977195 16051 tablet_bootstrap.cc:492] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:35.977604 16051 ts_tablet_manager.cc:1403] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.068s	user 0.009s	sys 0.000s
I20260709 13:04:35.978514 16176 consensus_queue.cc:1048] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:35.974612 15758 catalog_manager.cc:5697] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:35.980127 16051 raft_consensus.cc:359] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.980727 16051 raft_consensus.cc:385] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:35.976799 16055 ts_tablet_manager.cc:1403] T d2be2d6f06bc47a39f26307ae9372194 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.162s	user 0.019s	sys 0.024s
I20260709 13:04:35.980990 16051 raft_consensus.cc:740] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:35.983424 16055 raft_consensus.cc:359] T d2be2d6f06bc47a39f26307ae9372194 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.984702 16054 tablet_bootstrap.cc:654] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:35.985190 16051 consensus_queue.cc:260] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.985563 16055 raft_consensus.cc:385] T d2be2d6f06bc47a39f26307ae9372194 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:35.994186 16055 raft_consensus.cc:740] T d2be2d6f06bc47a39f26307ae9372194 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:35.994773 16055 consensus_queue.cc:260] T d2be2d6f06bc47a39f26307ae9372194 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:35.986775 15759 catalog_manager.cc:5697] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:35.995867 15859 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:04:35.996671 16055 ts_tablet_manager.cc:1434] T d2be2d6f06bc47a39f26307ae9372194 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 13:04:35.996723 15860 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 1.
I20260709 13:04:35.997601 16055 tablet_bootstrap.cc:492] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:36.012792 16051 ts_tablet_manager.cc:1434] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
I20260709 13:04:36.015437 15974 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:04:36.011788 15900 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:04:36.016862 15902 consensus_peers.cc:597] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea -> Peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Couldn't send request to peer 85e997ba5003439781ec4f1f8db0c3ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:36.017026 16092 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:36.017535 16092 raft_consensus.cc:697] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:04:36.017858 16053 raft_consensus.cc:2804] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:36.017489 16176 consensus_queue.cc:1048] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:36.018427 16053 raft_consensus.cc:697] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:04:36.018395 16092 consensus_queue.cc:237] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.020346 16051 tablet_bootstrap.cc:492] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:36.019618 16053 consensus_queue.cc:237] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.040874 16055 tablet_bootstrap.cc:654] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:36.050379 16054 tablet_bootstrap.cc:492] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:36.050931 16054 ts_tablet_manager.cc:1403] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.112s	user 0.016s	sys 0.014s
I20260709 13:04:36.051751 16051 tablet_bootstrap.cc:654] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:36.054528 15808 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:36.059720 16055 tablet_bootstrap.cc:492] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:36.060218 16055 ts_tablet_manager.cc:1403] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.063s	user 0.008s	sys 0.005s
I20260709 13:04:36.060055 16054 raft_consensus.cc:359] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.060778 16054 raft_consensus.cc:385] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:36.061008 16054 raft_consensus.cc:740] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:36.061599 16054 consensus_queue.cc:260] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.062628 16055 raft_consensus.cc:359] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.063043 16054 ts_tablet_manager.cc:1434] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.012s	user 0.002s	sys 0.000s
I20260709 13:04:36.063280 16055 raft_consensus.cc:385] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:36.063587 16055 raft_consensus.cc:740] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:36.064075 16054 tablet_bootstrap.cc:492] T c138dd864152446f83386a4cd43c6efc P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:36.064344 16055 consensus_queue.cc:260] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.066519 16055 ts_tablet_manager.cc:1434] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:04:36.065306 16051 tablet_bootstrap.cc:492] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:36.067179 16051 ts_tablet_manager.cc:1403] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.050s	user 0.006s	sys 0.005s
I20260709 13:04:36.067384 16055 tablet_bootstrap.cc:492] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:36.069756 16051 raft_consensus.cc:359] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.070430 16051 raft_consensus.cc:385] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:36.070736 16051 raft_consensus.cc:740] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:36.071396 16051 consensus_queue.cc:260] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.073376 16055 tablet_bootstrap.cc:654] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:36.073489 16051 ts_tablet_manager.cc:1434] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.002s
I20260709 13:04:36.074971 16051 tablet_bootstrap.cc:492] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:36.078953 16054 tablet_bootstrap.cc:654] T c138dd864152446f83386a4cd43c6efc P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:36.080432 16051 tablet_bootstrap.cc:654] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:36.082535 16130 consensus_queue.cc:1048] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:36.084082 15758 catalog_manager.cc:5697] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:36.091887 16051 tablet_bootstrap.cc:492] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:36.099431 16051 ts_tablet_manager.cc:1403] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.000s
I20260709 13:04:36.096891 16055 tablet_bootstrap.cc:492] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:36.100090 16055 ts_tablet_manager.cc:1403] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.033s	user 0.010s	sys 0.015s
I20260709 13:04:36.102995 16055 raft_consensus.cc:359] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.103696 16055 raft_consensus.cc:385] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:36.103956 16055 raft_consensus.cc:740] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:36.102823 16051 raft_consensus.cc:359] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.104580 16055 consensus_queue.cc:260] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.105049 16051 raft_consensus.cc:385] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:36.105399 16051 raft_consensus.cc:740] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:36.106243 16092 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.106144 16051 consensus_queue.cc:260] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.106765 16055 ts_tablet_manager.cc:1434] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260709 13:04:36.106698 16092 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.107244 16054 tablet_bootstrap.cc:492] T c138dd864152446f83386a4cd43c6efc P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:36.107695 16055 tablet_bootstrap.cc:492] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:36.107797 16054 ts_tablet_manager.cc:1403] T c138dd864152446f83386a4cd43c6efc P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.044s	user 0.011s	sys 0.011s
I20260709 13:04:36.108673 16051 ts_tablet_manager.cc:1434] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:04:36.109191 16092 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:36.109510 16051 tablet_bootstrap.cc:492] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:36.110474 16054 raft_consensus.cc:359] T c138dd864152446f83386a4cd43c6efc P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.110883 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:36.111369 16054 raft_consensus.cc:385] T c138dd864152446f83386a4cd43c6efc P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:36.111366 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:36.112170 15936 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 0.
I20260709 13:04:36.113626 16055 tablet_bootstrap.cc:654] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:36.113441 15974 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:36.111528 15860 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 0.
I20260709 13:04:36.114251 16092 raft_consensus.cc:2804] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:36.114758 16054 raft_consensus.cc:740] T c138dd864152446f83386a4cd43c6efc P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:36.115206 16092 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.115618 16092 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:36.115630 16054 consensus_queue.cc:260] T c138dd864152446f83386a4cd43c6efc P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.117040 16051 tablet_bootstrap.cc:654] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:36.116884 15759 catalog_manager.cc:5697] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:36.118373 16054 ts_tablet_manager.cc:1434] T c138dd864152446f83386a4cd43c6efc P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:04:36.119277 16054 tablet_bootstrap.cc:492] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:36.122254 16092 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.124842 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:36.125363 15860 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:36.126648 16054 tablet_bootstrap.cc:654] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:36.132890 15860 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 1.
I20260709 13:04:36.134039 15974 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:36.134792 16176 raft_consensus.cc:2804] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:36.135403 16176 raft_consensus.cc:697] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:04:36.136469 16176 consensus_queue.cc:237] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.139822 16092 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:36.140010 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:36.140564 15936 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:36.141088 16051 tablet_bootstrap.cc:492] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:36.141510 16051 ts_tablet_manager.cc:1403] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.032s	user 0.012s	sys 0.005s
I20260709 13:04:36.144985 16051 raft_consensus.cc:359] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.145594 16051 raft_consensus.cc:385] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:36.145845 16051 raft_consensus.cc:740] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:36.146462 16051 consensus_queue.cc:260] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.147140 15936 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 1.
I20260709 13:04:36.158612 16051 ts_tablet_manager.cc:1434] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:04:36.162539 16051 tablet_bootstrap.cc:492] T 73f4b066dd3d464699e6b00ebf1cdc44 P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:36.163866 15757 catalog_manager.cc:5697] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:36.169878 16051 tablet_bootstrap.cc:654] T 73f4b066dd3d464699e6b00ebf1cdc44 P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:36.173040 16055 tablet_bootstrap.cc:492] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:36.173485 16055 ts_tablet_manager.cc:1403] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.066s	user 0.022s	sys 0.017s
I20260709 13:04:36.174117 16054 tablet_bootstrap.cc:492] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:36.174723 16054 ts_tablet_manager.cc:1403] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.056s	user 0.031s	sys 0.013s
I20260709 13:04:36.176016 16055 raft_consensus.cc:359] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.176671 16055 raft_consensus.cc:385] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:36.176927 16055 raft_consensus.cc:740] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:36.177541 16055 consensus_queue.cc:260] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.178562 16054 raft_consensus.cc:359] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.179389 16055 ts_tablet_manager.cc:1434] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:04:36.179478 16054 raft_consensus.cc:385] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:36.180047 16054 raft_consensus.cc:740] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:36.180337 16055 tablet_bootstrap.cc:492] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:36.180919 16054 consensus_queue.cc:260] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.182111 16051 tablet_bootstrap.cc:492] T 73f4b066dd3d464699e6b00ebf1cdc44 P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:36.183537 16054 ts_tablet_manager.cc:1434] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:04:36.182957 16051 ts_tablet_manager.cc:1403] T 73f4b066dd3d464699e6b00ebf1cdc44 P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.005s
I20260709 13:04:36.186444 16055 tablet_bootstrap.cc:654] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:36.186648 16051 raft_consensus.cc:359] T 73f4b066dd3d464699e6b00ebf1cdc44 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.187315 16051 raft_consensus.cc:385] T 73f4b066dd3d464699e6b00ebf1cdc44 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:36.187573 16051 raft_consensus.cc:740] T 73f4b066dd3d464699e6b00ebf1cdc44 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:36.188042 16054 tablet_bootstrap.cc:492] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:36.188292 16051 consensus_queue.cc:260] T 73f4b066dd3d464699e6b00ebf1cdc44 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.190387 16051 ts_tablet_manager.cc:1434] T 73f4b066dd3d464699e6b00ebf1cdc44 P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:04:36.194382 16054 tablet_bootstrap.cc:654] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:36.196834 16176 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.197649 16051 tablet_bootstrap.cc:492] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:36.197275 16176 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.199379 16176 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:36.200717 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:36.201905 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:36.202548 15860 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 0.
W20260709 13:04:36.204074 15974 leader_election.cc:343] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:36.204735 15974 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:04:36.205029 16051 tablet_bootstrap.cc:654] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:36.205765 16176 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:36.206139 16176 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.206445 16176 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:36.216287 16176 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.219630 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:36.220263 15860 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:36.221433 16051 tablet_bootstrap.cc:492] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:36.221880 16051 ts_tablet_manager.cc:1403] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.030s	user 0.011s	sys 0.003s
I20260709 13:04:36.224287 16051 raft_consensus.cc:359] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.224937 16051 raft_consensus.cc:385] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:36.225222 16051 raft_consensus.cc:740] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:36.225850 16051 consensus_queue.cc:260] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.227784 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:36.228338 16051 ts_tablet_manager.cc:1434] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 13:04:36.228338 15936 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:36.229347 16051 tablet_bootstrap.cc:492] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:36.232410 16073 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.232860 16073 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.235025 15936 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 1.
I20260709 13:04:36.239432 15974 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:36.240309 16092 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:36.240783 16092 raft_consensus.cc:697] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:04:36.240439 16176 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:36.241659 16092 consensus_queue.cc:237] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.243006 15860 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 1.
I20260709 13:04:36.246444 16073 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:36.247776 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:36.250067 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:36.251058 15860 raft_consensus.cc:2393] T b5d68c78f0d94e0eb90543b642d1f1af P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 1: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:04:36.252446 15900 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:36.253036 16051 tablet_bootstrap.cc:654] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:36.254046 16073 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:36.279503 16051 tablet_bootstrap.cc:492] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:36.280176 16051 ts_tablet_manager.cc:1403] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.051s	user 0.008s	sys 0.009s
I20260709 13:04:36.283138 16051 raft_consensus.cc:359] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.283807 16051 raft_consensus.cc:385] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:36.284054 16051 raft_consensus.cc:740] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:36.284842 16051 consensus_queue.cc:260] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.287575 16051 ts_tablet_manager.cc:1434] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:04:36.288695 16051 tablet_bootstrap.cc:492] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:36.290745 16073 raft_consensus.cc:2749] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:36.294291 16054 tablet_bootstrap.cc:492] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:36.295109 16054 ts_tablet_manager.cc:1403] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.107s	user 0.016s	sys 0.027s
I20260709 13:04:36.297617 16054 raft_consensus.cc:359] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.298293 16054 raft_consensus.cc:385] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:36.298672 16051 tablet_bootstrap.cc:654] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:36.298635 16054 raft_consensus.cc:740] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:36.298403 15757 catalog_manager.cc:5697] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:36.299757 16054 consensus_queue.cc:260] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.304504 16055 tablet_bootstrap.cc:492] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:36.304955 16055 ts_tablet_manager.cc:1403] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.125s	user 0.045s	sys 0.057s
I20260709 13:04:36.307596 16055 raft_consensus.cc:359] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.308234 16055 raft_consensus.cc:385] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:36.308503 16055 raft_consensus.cc:740] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:36.309213 16055 consensus_queue.cc:260] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.311223 16055 ts_tablet_manager.cc:1434] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:04:36.312083 16055 tablet_bootstrap.cc:492] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:36.317603 16055 tablet_bootstrap.cc:654] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:36.320324 16054 ts_tablet_manager.cc:1434] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 13:04:36.320205 16073 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.322156 16073 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.325161 16073 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:36.326336 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:04:36.327715 15900 leader_election.cc:343] T 73f4b066dd3d464699e6b00ebf1cdc44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:36.329409 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:04:36.330730 15902 leader_election.cc:343] T 73f4b066dd3d464699e6b00ebf1cdc44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:36.331123 15902 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:36.332587 16053 raft_consensus.cc:2749] T 73f4b066dd3d464699e6b00ebf1cdc44 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:36.333304 16055 tablet_bootstrap.cc:492] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:36.333676 16055 ts_tablet_manager.cc:1403] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.022s	user 0.018s	sys 0.002s
I20260709 13:04:36.336074 16054 tablet_bootstrap.cc:492] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:36.336139 16055 raft_consensus.cc:359] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.340798 16053 consensus_queue.cc:1048] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:04:36.342926 16055 raft_consensus.cc:385] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:36.343241 16055 raft_consensus.cc:740] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:36.343890 16055 consensus_queue.cc:260] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.345966 16055 ts_tablet_manager.cc:1434] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.004s
I20260709 13:04:36.347285 16055 tablet_bootstrap.cc:492] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:36.352654 16055 tablet_bootstrap.cc:654] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:36.361562 16054 tablet_bootstrap.cc:654] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:36.362887 16061 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.362987 16092 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.363404 16061 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.363534 16092 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.364499 16051 tablet_bootstrap.cc:492] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:36.364923 16051 ts_tablet_manager.cc:1403] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.076s	user 0.024s	sys 0.030s
I20260709 13:04:36.365923 16055 tablet_bootstrap.cc:492] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:36.366034 16162 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.366485 16055 ts_tablet_manager.cc:1403] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.009s
I20260709 13:04:36.366554 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:36.367398 15860 raft_consensus.cc:2393] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 1: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:04:36.367465 16051 raft_consensus.cc:359] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.368185 16051 raft_consensus.cc:385] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:36.368456 16051 raft_consensus.cc:740] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:36.368930 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:36.369542 16055 raft_consensus.cc:359] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.370257 16055 raft_consensus.cc:385] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:36.369954 16051 consensus_queue.cc:260] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.370580 16055 raft_consensus.cc:740] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:36.370793 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:36.371179 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:36.371520 16055 consensus_queue.cc:260] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.372059 16051 ts_tablet_manager.cc:1434] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:04:36.372618 15974 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:04:36.372934 16051 tablet_bootstrap.cc:492] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:36.373683 16176 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:36.374226 16055 ts_tablet_manager.cc:1434] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:04:36.375182 16055 tablet_bootstrap.cc:492] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:36.366508 16162 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.382478 16051 tablet_bootstrap.cc:654] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:36.382771 16162 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:36.382948 15860 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:36.383788 16055 tablet_bootstrap.cc:654] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:36.386080 15827 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:36.387624 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:36.387856 16162 raft_consensus.cc:2749] T c138dd864152446f83386a4cd43c6efc P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:36.388641 15936 raft_consensus.cc:2393] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 1: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:04:36.387638 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:36.391911 15827 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:36.391968 16051 tablet_bootstrap.cc:492] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:36.393476 16051 ts_tablet_manager.cc:1403] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.001s
I20260709 13:04:36.395911 16051 raft_consensus.cc:359] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.396533 16051 raft_consensus.cc:385] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:36.396799 16051 raft_consensus.cc:740] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:36.397421 16051 consensus_queue.cc:260] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.369864 16092 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:36.365924 16061 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:36.403048 16055 tablet_bootstrap.cc:492] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:36.403156 16176 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:36.403530 16055 ts_tablet_manager.cc:1403] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.029s	user 0.009s	sys 0.012s
I20260709 13:04:36.405825 16122 raft_consensus.cc:2749] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:36.406006 16055 raft_consensus.cc:359] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.406647 15861 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:36.408143 16064 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:36.406698 16055 raft_consensus.cc:385] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:36.413864 16055 raft_consensus.cc:740] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:36.414568 16055 consensus_queue.cc:260] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.416870 16055 ts_tablet_manager.cc:1434] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 13:04:36.418360 16176 consensus_queue.cc:1048] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:36.421846 16055 tablet_bootstrap.cc:492] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:36.422134 16188 consensus_queue.cc:1048] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:36.434770 16055 tablet_bootstrap.cc:654] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:36.435141 16176 consensus_queue.cc:1048] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:36.450423 16176 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:36.473322 16054 tablet_bootstrap.cc:492] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:36.473796 16054 ts_tablet_manager.cc:1403] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.138s	user 0.032s	sys 0.055s
I20260709 13:04:36.467490 16051 ts_tablet_manager.cc:1434] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.074s	user 0.005s	sys 0.000s
I20260709 13:04:36.476580 16054 raft_consensus.cc:359] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.477346 16054 raft_consensus.cc:385] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:36.477612 16054 raft_consensus.cc:740] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:36.477888 16051 tablet_bootstrap.cc:492] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:36.478323 16054 consensus_queue.cc:260] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.480680 16054 ts_tablet_manager.cc:1434] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:04:36.482673 16054 tablet_bootstrap.cc:492] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:36.488214 16054 tablet_bootstrap.cc:654] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:36.488703 16122 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.489540 16122 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.494578 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:36.496804 16122 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:36.498572 16176 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.499207 16176 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.505390 16188 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.509184 16188 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.509451 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:04:36.511209 15827 leader_election.cc:343] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:36.511482 16188 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:36.522492 16055 tablet_bootstrap.cc:492] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:36.522974 16055 ts_tablet_manager.cc:1403] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.101s	user 0.026s	sys 0.017s
I20260709 13:04:36.524187 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:04:36.521821 15825 leader_election.cc:343] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:36.524802 15859 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 0.
I20260709 13:04:36.525100 15825 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:36.525851 16122 raft_consensus.cc:2749] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:36.525987 15900 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:36.526824 16188 raft_consensus.cc:2804] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:36.526878 16176 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:36.527257 16188 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.528400 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:04:36.529872 15902 leader_election.cc:343] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:36.531764 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:36.532557 15935 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 0.
I20260709 13:04:36.532645 16055 raft_consensus.cc:359] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.533373 16055 raft_consensus.cc:385] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:36.533658 16055 raft_consensus.cc:740] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:36.534482 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:36.527608 16188 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:36.534420 16055 consensus_queue.cc:260] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.535296 15859 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 0.
I20260709 13:04:36.536541 15974 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:36.541646 16176 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:36.541996 16176 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.542356 16176 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:36.547811 16073 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.548923 16051 tablet_bootstrap.cc:654] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:36.548408 16073 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.555907 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:36.556607 16055 ts_tablet_manager.cc:1434] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.001s
I20260709 13:04:36.557523 16055 tablet_bootstrap.cc:492] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
W20260709 13:04:36.557828 15902 leader_election.cc:343] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:36.553196 16073 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:36.564800 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:36.566387 16055 tablet_bootstrap.cc:654] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:36.566946 16064 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:04:36.568280 15900 leader_election.cc:343] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:36.572467 16064 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.572985 16064 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.572677 15900 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:36.574610 16073 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:36.576177 16064 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:04:36.576918 16236 log.cc:927] Time spent T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:04:36.582371 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:36.583076 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:36.586280 16176 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.592471 16176 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:36.593855 16055 tablet_bootstrap.cc:492] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:36.594352 16055 ts_tablet_manager.cc:1403] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.037s	user 0.002s	sys 0.009s
I20260709 13:04:36.595770 16073 consensus_queue.cc:1048] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:36.596169 16054 tablet_bootstrap.cc:492] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:36.596632 16054 ts_tablet_manager.cc:1403] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.114s	user 0.025s	sys 0.051s
I20260709 13:04:36.596339 16064 consensus_queue.cc:1048] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:36.597627 16051 tablet_bootstrap.cc:492] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:36.598147 16051 ts_tablet_manager.cc:1403] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.120s	user 0.011s	sys 0.001s
I20260709 13:04:36.599121 16054 raft_consensus.cc:359] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.599813 16054 raft_consensus.cc:385] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:36.600073 16054 raft_consensus.cc:740] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:36.600699 16054 consensus_queue.cc:260] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.600786 16051 raft_consensus.cc:359] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.601605 16051 raft_consensus.cc:385] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:36.601934 16051 raft_consensus.cc:740] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:36.602218 16054 ts_tablet_manager.cc:1434] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:04:36.603081 16054 tablet_bootstrap.cc:492] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:36.603595 16051 consensus_queue.cc:260] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.604449 16055 raft_consensus.cc:359] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.605178 16055 raft_consensus.cc:385] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:36.605436 16055 raft_consensus.cc:740] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:36.606002 16051 ts_tablet_manager.cc:1434] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:04:36.607116 16051 tablet_bootstrap.cc:492] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:36.606029 16055 consensus_queue.cc:260] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.610189 16053 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.610854 16055 ts_tablet_manager.cc:1434] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:04:36.610656 16053 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.611768 16055 tablet_bootstrap.cc:492] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:36.612182 16054 tablet_bootstrap.cc:654] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:36.612907 16053 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:36.614297 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:36.615010 16011 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 0.
I20260709 13:04:36.615770 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:36.616526 15902 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:36.617336 16073 raft_consensus.cc:2804] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:36.617611 16055 tablet_bootstrap.cc:654] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:36.618319 16073 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.618659 16073 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:36.617301 16053 consensus_queue.cc:1048] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:36.616642 15861 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 0.
I20260709 13:04:36.631140 16073 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.632874 16051 tablet_bootstrap.cc:654] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:36.641211 16073 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:36.646870 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:36.647447 16009 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:36.648794 16010 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:36.643844 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:36.653010 15861 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:36.655611 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:36.656560 15935 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:36.656646 16162 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.657100 16162 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.655818 16188 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.659137 16162 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:36.659822 16176 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.660246 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:36.660259 16176 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.661836 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:36.662395 16176 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:36.663038 15974 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:04:36.663103 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:36.663785 15860 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:36.665652 16054 tablet_bootstrap.cc:492] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:36.666079 16054 ts_tablet_manager.cc:1403] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.063s	user 0.016s	sys 0.008s
I20260709 13:04:36.679543 16064 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:04:36.659171 16188 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:04:36.681351 15902 leader_election.cc:343] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:36.693718 16051 tablet_bootstrap.cc:492] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:36.694159 16051 ts_tablet_manager.cc:1403] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.087s	user 0.007s	sys 0.013s
W20260709 13:04:36.681262 15825 leader_election.cc:343] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:36.709743 16054 raft_consensus.cc:359] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.710661 16054 raft_consensus.cc:385] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:36.711002 16054 raft_consensus.cc:740] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:36.693881 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:36.708952 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:36.713030 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:36.714849 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:36.722015 15862 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 0.
I20260709 13:04:36.722829 16009 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:04:36.723587 15859 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:36.724035 15902 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:36.725188 16188 raft_consensus.cc:2804] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:36.725646 16188 raft_consensus.cc:697] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:04:36.727905 15934 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 0.
I20260709 13:04:36.726418 16188 consensus_queue.cc:237] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.728747 16054 consensus_queue.cc:260] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.706312 15861 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 1.
W20260709 13:04:36.720955 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 47187.4us
I20260709 13:04:36.734954 15860 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:04:36.735780 16176 raft_consensus.cc:2749] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:36.729705 16008 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 0.
I20260709 13:04:36.738713 16054 ts_tablet_manager.cc:1434] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.072s	user 0.008s	sys 0.000s
I20260709 13:04:36.739356 16054 tablet_bootstrap.cc:492] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:36.741976 16055 tablet_bootstrap.cc:492] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:36.742403 16055 ts_tablet_manager.cc:1403] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.131s	user 0.020s	sys 0.030s
I20260709 13:04:36.745857 15827 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:04:36.747017 16122 raft_consensus.cc:2804] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:36.746768 16051 raft_consensus.cc:359] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.748050 16051 raft_consensus.cc:385] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:36.748313 16051 raft_consensus.cc:740] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:36.747370 16122 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.748915 16055 raft_consensus.cc:359] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.749595 16055 raft_consensus.cc:385] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:36.749871 16055 raft_consensus.cc:740] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:36.750398 16122 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:36.750476 16055 consensus_queue.cc:260] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.752256 15974 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:36.752519 16055 ts_tablet_manager.cc:1434] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:04:36.753383 16055 tablet_bootstrap.cc:492] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:36.754411 16176 raft_consensus.cc:2804] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:36.754812 16176 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.754858 15935 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 1.
I20260709 13:04:36.755223 16176 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:36.755898 16073 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.757750 16051 consensus_queue.cc:260] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.761186 16073 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.763836 16073 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:36.765244 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:36.765812 16009 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 0.
I20260709 13:04:36.766917 15902 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:36.766862 16122 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.769474 16051 ts_tablet_manager.cc:1434] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.075s	user 0.004s	sys 0.004s
I20260709 13:04:36.770354 16051 tablet_bootstrap.cc:492] T 0a10bba95cd1421aa3fc332639e97319 P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:36.770089 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:36.771206 16008 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:36.772913 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:36.773555 15933 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:36.774925 15859 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:04:36.771611 16188 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:36.780501 15974 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:36.778321 16188 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.778308 16122 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:36.784937 16092 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:36.785674 16178 consensus_queue.cc:1048] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:04:36.787988 16092 raft_consensus.cc:697] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:04:36.790397 16092 consensus_queue.cc:237] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.795197 16051 tablet_bootstrap.cc:654] T 0a10bba95cd1421aa3fc332639e97319 P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:36.785192 16188 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:36.797072 16064 consensus_queue.cc:1048] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:36.798885 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:36.801571 16064 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.799168 16176 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.808084 15859 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 0.
I20260709 13:04:36.818933 15933 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:04:36.808558 16064 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.809463 15900 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:36.822543 16073 raft_consensus.cc:2804] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:36.827253 16055 tablet_bootstrap.cc:654] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:36.824600 16064 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:36.825830 16073 raft_consensus.cc:697] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:04:36.841701 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:36.843210 15858 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 0.
I20260709 13:04:36.838630 15825 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:36.847658 15974 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:36.849184 15760 catalog_manager.cc:5697] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:36.850872 16054 tablet_bootstrap.cc:654] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:36.850417 16122 raft_consensus.cc:2804] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:36.854624 16122 raft_consensus.cc:697] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:04:36.855413 16122 consensus_queue.cc:237] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.856813 16073 consensus_queue.cc:237] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.858239 15757 catalog_manager.cc:5697] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:36.852738 16251 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:36.872850 16251 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.873317 16251 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:36.873662 16176 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:36.850603 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:36.872725 16008 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:04:36.874473 15933 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 0.
I20260709 13:04:36.900931 16187 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.906109 16187 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.906306 16188 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.910434 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:36.911082 15861 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:36.908353 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:36.918695 15934 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:36.921780 16054 tablet_bootstrap.cc:492] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:36.922289 16054 ts_tablet_manager.cc:1403] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.183s	user 0.019s	sys 0.003s
I20260709 13:04:36.927757 15934 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 1.
I20260709 13:04:36.928117 16055 tablet_bootstrap.cc:492] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:36.928587 16055 ts_tablet_manager.cc:1403] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.175s	user 0.007s	sys 0.018s
I20260709 13:04:36.932888 16051 tablet_bootstrap.cc:492] T 0a10bba95cd1421aa3fc332639e97319 P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:36.933382 16051 ts_tablet_manager.cc:1403] T 0a10bba95cd1421aa3fc332639e97319 P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.163s	user 0.018s	sys 0.028s
I20260709 13:04:36.933846 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:36.942137 16055 raft_consensus.cc:359] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.942269 15974 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:36.942715 16055 raft_consensus.cc:385] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:36.943017 16055 raft_consensus.cc:740] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:36.943482 16055 consensus_queue.cc:260] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.941902 16187 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:36.946691 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:36.947297 16008 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 0.
I20260709 13:04:36.948026 16251 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.950274 16251 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:36.951591 15902 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:36.951759 15861 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 1.
I20260709 13:04:36.953991 16064 raft_consensus.cc:2804] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:36.953157 16055 ts_tablet_manager.cc:1434] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.002s
I20260709 13:04:36.956239 16055 tablet_bootstrap.cc:492] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:36.955108 16092 consensus_queue.cc:1048] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:04:36.956743 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:36.957306 15858 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:36.960093 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:36.960650 15934 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:36.961829 16051 raft_consensus.cc:359] T 0a10bba95cd1421aa3fc332639e97319 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.962325 16051 raft_consensus.cc:385] T 0a10bba95cd1421aa3fc332639e97319 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:36.962543 16051 raft_consensus.cc:740] T 0a10bba95cd1421aa3fc332639e97319 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:36.963079 16051 consensus_queue.cc:260] T 0a10bba95cd1421aa3fc332639e97319 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:04:36.970068 15900 leader_election.cc:343] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:36.971164 15934 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 1.
I20260709 13:04:36.972004 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:36.972374 16260 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:36.972771 15859 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:36.972913 16260 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.973464 16260 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:36.975485 16064 raft_consensus.cc:697] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:04:36.986505 16064 consensus_queue.cc:237] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:36.999599 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:37.000300 16009 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:37.039945 16009 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:04:37.042579 15902 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:37.043812 16187 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:37.044387 16187 raft_consensus.cc:697] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:04:37.045430 16187 consensus_queue.cc:237] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.047346 16055 tablet_bootstrap.cc:654] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:37.008996 16188 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:37.048115 16051 ts_tablet_manager.cc:1434] T 0a10bba95cd1421aa3fc332639e97319 P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.114s	user 0.031s	sys 0.059s
I20260709 13:04:37.049739 16051 tablet_bootstrap.cc:492] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:37.042251 15858 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 1.
W20260709 13:04:37.053767 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 76871.8us
I20260709 13:04:37.052027 15859 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:04:37.054405 15974 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:37.055343 16054 raft_consensus.cc:359] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.056566 16054 raft_consensus.cc:385] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:37.056831 16054 raft_consensus.cc:740] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:37.057471 16054 consensus_queue.cc:260] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.059695 16251 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:37.065517 16054 ts_tablet_manager.cc:1434] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.143s	user 0.037s	sys 0.043s
I20260709 13:04:37.065933 16251 raft_consensus.cc:697] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:04:37.066556 16054 tablet_bootstrap.cc:492] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:37.066780 16251 consensus_queue.cc:237] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.069039 16260 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.076463 16265 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.082710 16054 tablet_bootstrap.cc:654] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:37.084070 15759 catalog_manager.cc:5697] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:37.083575 16265 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.090859 16055 tablet_bootstrap.cc:492] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:37.091291 16055 ts_tablet_manager.cc:1403] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.135s	user 0.017s	sys 0.007s
I20260709 13:04:37.092418 16265 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:37.106724 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:37.102667 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:37.107312 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:37.107402 15933 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 0.
I20260709 13:04:37.107656 15858 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 0.
I20260709 13:04:37.107915 15862 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:37.109725 16051 tablet_bootstrap.cc:654] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:37.111169 16260 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:37.113122 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:37.113734 16009 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:37.115223 15862 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:04:37.116348 15900 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:37.122676 16051 tablet_bootstrap.cc:492] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:37.123134 16051 ts_tablet_manager.cc:1403] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.074s	user 0.011s	sys 0.001s
I20260709 13:04:37.124370 15974 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:37.129204 15757 catalog_manager.cc:5697] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:37.131263 15757 catalog_manager.cc:5697] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:37.134469 16055 raft_consensus.cc:359] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.135178 16055 raft_consensus.cc:385] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:37.135492 16055 raft_consensus.cc:740] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:37.137398 16009 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:04:37.140595 16187 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:37.142721 16187 raft_consensus.cc:697] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:04:37.140897 16064 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:37.146032 16064 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.145129 16051 raft_consensus.cc:359] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.146598 16051 raft_consensus.cc:385] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:37.146974 16051 raft_consensus.cc:740] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:37.147758 16064 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:37.148062 16055 consensus_queue.cc:260] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.150717 16055 ts_tablet_manager.cc:1434] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.059s	user 0.007s	sys 0.002s
I20260709 13:04:37.151849 16055 tablet_bootstrap.cc:492] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:37.147760 16051 consensus_queue.cc:260] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.146284 16187 consensus_queue.cc:237] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.159735 16054 tablet_bootstrap.cc:492] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:37.160154 16054 ts_tablet_manager.cc:1403] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.094s	user 0.021s	sys 0.016s
I20260709 13:04:37.161376 16051 ts_tablet_manager.cc:1434] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.038s	user 0.008s	sys 0.000s
I20260709 13:04:37.161510 16064 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.162323 16051 tablet_bootstrap.cc:492] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:37.162712 16054 raft_consensus.cc:359] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.163364 16054 raft_consensus.cc:385] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:37.163610 16054 raft_consensus.cc:740] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:37.164428 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:37.171059 15862 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:37.171901 16064 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:37.167723 15759 catalog_manager.cc:5697] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:37.164240 16054 consensus_queue.cc:260] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.172351 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:37.182772 15934 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:37.184159 16054 ts_tablet_manager.cc:1434] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 13:04:37.185117 16054 tablet_bootstrap.cc:492] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:37.189891 15934 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 1.
I20260709 13:04:37.190929 16054 tablet_bootstrap.cc:654] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:37.190883 15974 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:37.191764 16064 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:37.192251 16064 raft_consensus.cc:697] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:04:37.192623 15759 catalog_manager.cc:5697] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:37.193028 16064 consensus_queue.cc:237] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.176038 16055 tablet_bootstrap.cc:654] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:37.180217 15862 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 1.
W20260709 13:04:37.200155 15808 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:37.206480 16053 consensus_queue.cc:1048] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:37.208422 16051 tablet_bootstrap.cc:654] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:37.210466 16054 tablet_bootstrap.cc:492] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:37.214426 16054 ts_tablet_manager.cc:1403] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.029s	user 0.018s	sys 0.007s
I20260709 13:04:37.217131 16054 raft_consensus.cc:359] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.218287 16064 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.218712 16064 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.220795 16064 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:37.221863 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:37.222592 15859 raft_consensus.cc:2393] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 1: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:04:37.217083 15757 catalog_manager.cc:5697] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:37.217856 16054 raft_consensus.cc:385] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:37.225193 16054 raft_consensus.cc:740] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:37.226001 16054 consensus_queue.cc:260] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.228814 16055 tablet_bootstrap.cc:492] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:37.228262 16053 consensus_queue.cc:1048] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:37.229347 16055 ts_tablet_manager.cc:1403] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.078s	user 0.014s	sys 0.000s
I20260709 13:04:37.227648 15759 catalog_manager.cc:5697] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:37.228557 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:37.228510 16054 ts_tablet_manager.cc:1434] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:04:37.231585 15974 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:04:37.232620 16054 tablet_bootstrap.cc:492] T fd8e1402dafb4300911e5b0189c8fe0a P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:37.233286 16064 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:37.235651 16055 raft_consensus.cc:359] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.238718 16055 raft_consensus.cc:385] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:37.239364 16055 raft_consensus.cc:740] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:37.242141 16054 tablet_bootstrap.cc:654] T fd8e1402dafb4300911e5b0189c8fe0a P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:37.243443 16061 consensus_queue.cc:1048] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:37.248519 16064 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:37.253453 16054 tablet_bootstrap.cc:492] T fd8e1402dafb4300911e5b0189c8fe0a P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:37.253969 16054 ts_tablet_manager.cc:1403] T fd8e1402dafb4300911e5b0189c8fe0a P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.005s
I20260709 13:04:37.240886 16055 consensus_queue.cc:260] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.271128 16055 ts_tablet_manager.cc:1434] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.041s	user 0.005s	sys 0.000s
I20260709 13:04:37.271310 16073 consensus_queue.cc:1048] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:37.282712 16055 tablet_bootstrap.cc:492] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:37.289065 16055 tablet_bootstrap.cc:654] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:37.281213 16061 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.256763 16054 raft_consensus.cc:359] T fd8e1402dafb4300911e5b0189c8fe0a P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.292265 16054 raft_consensus.cc:385] T fd8e1402dafb4300911e5b0189c8fe0a P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:37.292773 16054 raft_consensus.cc:740] T fd8e1402dafb4300911e5b0189c8fe0a P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:37.293411 16054 consensus_queue.cc:260] T fd8e1402dafb4300911e5b0189c8fe0a P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.299778 16054 ts_tablet_manager.cc:1434] T fd8e1402dafb4300911e5b0189c8fe0a P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.000s
I20260709 13:04:37.347250 16054 tablet_bootstrap.cc:492] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:37.349226 16122 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.349722 16122 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.351274 16251 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.351758 16251 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.354172 16064 consensus_queue.cc:1048] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:37.356465 16251 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:37.358304 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:37.358947 15934 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 0.
I20260709 13:04:37.360594 16051 tablet_bootstrap.cc:492] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:37.295828 16061 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.361071 16051 ts_tablet_manager.cc:1403] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.199s	user 0.050s	sys 0.025s
I20260709 13:04:37.360916 15974 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:37.387053 16178 raft_consensus.cc:2804] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:37.387435 16178 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.387856 16178 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:37.390396 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:37.379935 16122 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:37.364974 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:37.391341 16122 consensus_queue.cc:1048] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:04:37.391829 15827 leader_election.cc:343] T fd8e1402dafb4300911e5b0189c8fe0a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:37.380498 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:37.365921 16283 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.398092 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:37.402135 15935 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 0.
I20260709 13:04:37.406759 15825 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:37.397877 16283 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.416842 16284 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:37.417807 16054 tablet_bootstrap.cc:654] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:37.403015 16178 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.417240 16284 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.425808 16284 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:37.443799 16262 consensus_queue.cc:1048] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260709 13:04:37.424758 15862 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 0.
I20260709 13:04:37.445075 16055 tablet_bootstrap.cc:492] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:37.505988 16051 raft_consensus.cc:359] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.507781 16051 raft_consensus.cc:385] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:37.507961 16055 ts_tablet_manager.cc:1403] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.225s	user 0.021s	sys 0.030s
I20260709 13:04:37.508095 16051 raft_consensus.cc:740] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:37.509250 16051 consensus_queue.cc:260] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.427028 16061 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:04:37.439345 15825 leader_election.cc:343] T fd8e1402dafb4300911e5b0189c8fe0a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:37.429834 16265 consensus_queue.cc:1048] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.082s
I20260709 13:04:37.447383 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:37.524637 16051 ts_tablet_manager.cc:1434] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.139s	user 0.004s	sys 0.004s
I20260709 13:04:37.525715 16051 tablet_bootstrap.cc:492] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:37.533509 16178 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:37.539366 16055 raft_consensus.cc:359] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.540488 16055 raft_consensus.cc:385] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:37.540778 16055 raft_consensus.cc:740] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:37.541368 16055 consensus_queue.cc:260] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.543651 16055 ts_tablet_manager.cc:1434] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.035s	user 0.005s	sys 0.000s
I20260709 13:04:37.544502 16055 tablet_bootstrap.cc:492] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:37.524132 15825 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:37.549803 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:37.550369 15932 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:37.553651 16122 raft_consensus.cc:2749] T fd8e1402dafb4300911e5b0189c8fe0a P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:04:37.545591 15827 leader_election.cc:343] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:37.558168 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:37.558460 16283 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:04:37.558954 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 121105us
I20260709 13:04:37.552856 16265 consensus_queue.cc:1048] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:04:37.578199 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:37.581836 16092 consensus_queue.cc:1048] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:37.592846 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:37.596728 16010 raft_consensus.cc:2393] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 1: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:04:37.599066 16054 tablet_bootstrap.cc:492] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:37.599511 16054 ts_tablet_manager.cc:1403] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.254s	user 0.021s	sys 0.008s
I20260709 13:04:37.602999 16054 raft_consensus.cc:359] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.605360 16284 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.612305 16260 consensus_queue.cc:1048] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:37.612587 15827 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:37.603617 16054 raft_consensus.cc:385] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:37.614578 16054 raft_consensus.cc:740] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:37.616546 16054 consensus_queue.cc:260] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.621048 16054 ts_tablet_manager.cc:1434] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.000s
I20260709 13:04:37.614928 16053 consensus_queue.cc:1048] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:04:37.625092 16054 tablet_bootstrap.cc:492] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:37.619890 16122 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:37.632649 16055 tablet_bootstrap.cc:654] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:37.640959 16051 tablet_bootstrap.cc:654] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:37.628294 16284 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:37.644737 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:37.645309 15934 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:37.652629 16122 raft_consensus.cc:2749] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:37.655074 15932 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 1.
I20260709 13:04:37.684752 16176 consensus_queue.cc:1048] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:04:37.691443 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:37.693783 15974 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
W20260709 13:04:37.694921 15827 leader_election.cc:343] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:37.697362 16176 raft_consensus.cc:2804] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:37.697834 16176 raft_consensus.cc:697] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:04:37.698598 16176 consensus_queue.cc:237] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.714280 16051 tablet_bootstrap.cc:492] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:37.718886 16260 consensus_queue.cc:1048] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:04:37.728381 16251 consensus_queue.cc:1048] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:37.714920 16051 ts_tablet_manager.cc:1403] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.190s	user 0.016s	sys 0.004s
I20260709 13:04:37.725150 16054 tablet_bootstrap.cc:654] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:37.734134 16055 tablet_bootstrap.cc:492] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:37.871832 16055 ts_tablet_manager.cc:1403] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.328s	user 0.033s	sys 0.037s
W20260709 13:04:37.816519 16295 log.cc:927] Time spent T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:04:37.719565 16290 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.722591 15934 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:04:37.715072 16187 consensus_queue.cc:1048] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.166s
I20260709 13:04:37.853120 16073 consensus_queue.cc:1048] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.031s
I20260709 13:04:37.759781 16265 consensus_queue.cc:1048] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.124s
I20260709 13:04:37.889518 15825 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:37.891391 16284 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:37.893790 16284 raft_consensus.cc:697] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:04:37.894917 16284 consensus_queue.cc:237] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.907366 16051 raft_consensus.cc:359] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.908187 16051 raft_consensus.cc:385] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:37.908532 16051 raft_consensus.cc:740] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:37.909338 16051 consensus_queue.cc:260] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.883536 16290 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.915748 16051 ts_tablet_manager.cc:1434] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.185s	user 0.004s	sys 0.012s
I20260709 13:04:37.920589 16054 tablet_bootstrap.cc:492] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:37.921151 16054 ts_tablet_manager.cc:1403] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.296s	user 0.019s	sys 0.064s
I20260709 13:04:37.917575 16290 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:37.924650 16055 raft_consensus.cc:359] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.925310 16055 raft_consensus.cc:385] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:37.925570 16055 raft_consensus.cc:740] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:37.926074 16055 consensus_queue.cc:260] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.926717 16054 raft_consensus.cc:359] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.927256 16054 raft_consensus.cc:385] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:37.927456 16054 raft_consensus.cc:740] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:37.927523 16055 ts_tablet_manager.cc:1434] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.049s	user 0.002s	sys 0.000s
I20260709 13:04:37.928117 16054 consensus_queue.cc:260] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:37.929488 16055 tablet_bootstrap.cc:492] T eabbcb31af804f2dac6af5731e18a4dd P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:37.972234 16054 ts_tablet_manager.cc:1434] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.051s	user 0.004s	sys 0.003s
I20260709 13:04:37.969916 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:37.978163 15856 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 0.
I20260709 13:04:37.980589 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:37.981861 15936 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 0.
I20260709 13:04:37.989740 16162 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.990908 16162 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.001431 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:38.003731 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:04:38.004029 15825 leader_election.cc:343] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:37.996197 16054 tablet_bootstrap.cc:492] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:37.994181 16162 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:38.006552 15974 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:04:38.009670 15827 leader_election.cc:343] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:38.010445 15827 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:38.014472 16283 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:38.023867 16262 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:38.024407 16262 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:38.025197 16262 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.029956 16051 tablet_bootstrap.cc:492] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
W20260709 13:04:38.044457 16301 log.cc:927] Time spent T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.187s	user 0.003s	sys 0.000s
I20260709 13:04:38.070384 16051 tablet_bootstrap.cc:654] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.085296 16265 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.085995 16265 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.086862 16054 tablet_bootstrap.cc:654] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.099901 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:04:38.101171 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 45574.3us
I20260709 13:04:38.105026 16055 tablet_bootstrap.cc:654] T eabbcb31af804f2dac6af5731e18a4dd P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.093348 16262 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.099342 16265 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:38.126116 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:38.129853 16054 tablet_bootstrap.cc:492] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:38.131760 16054 ts_tablet_manager.cc:1403] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.157s	user 0.020s	sys 0.000s
I20260709 13:04:38.127123 15856 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.129849 16051 tablet_bootstrap.cc:492] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:38.133133 16051 ts_tablet_manager.cc:1403] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.104s	user 0.012s	sys 0.006s
I20260709 13:04:38.135159 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:38.135571 16262 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:38.140475 15937 raft_consensus.cc:2393] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 1: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:04:38.150554 16051 raft_consensus.cc:359] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.151306 16051 raft_consensus.cc:385] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.151608 16051 raft_consensus.cc:740] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.153964 16051 consensus_queue.cc:260] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.157133 15856 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 1.
I20260709 13:04:38.169274 16051 ts_tablet_manager.cc:1434] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.036s	user 0.006s	sys 0.002s
I20260709 13:04:38.169919 16055 tablet_bootstrap.cc:492] T eabbcb31af804f2dac6af5731e18a4dd P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:38.163849 15974 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:04:38.169620 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:38.170517 16051 tablet_bootstrap.cc:492] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:38.170562 15934 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.171770 15974 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:38.172622 16251 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.145320 16054 raft_consensus.cc:359] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.174264 16054 raft_consensus.cc:385] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.174571 16054 raft_consensus.cc:740] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.175320 16054 consensus_queue.cc:260] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.176429 16092 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:38.176908 16092 raft_consensus.cc:697] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:04:38.177461 16054 ts_tablet_manager.cc:1434] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.045s	user 0.006s	sys 0.000s
I20260709 13:04:38.177856 16092 consensus_queue.cc:237] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.178476 16054 tablet_bootstrap.cc:492] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:38.179144 15934 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 1.
I20260709 13:04:38.180408 16251 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:38.181715 16055 ts_tablet_manager.cc:1403] T eabbcb31af804f2dac6af5731e18a4dd P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.252s	user 0.019s	sys 0.024s
I20260709 13:04:38.184587 16055 raft_consensus.cc:359] T eabbcb31af804f2dac6af5731e18a4dd P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.185349 16055 raft_consensus.cc:385] T eabbcb31af804f2dac6af5731e18a4dd P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.185604 16055 raft_consensus.cc:740] T eabbcb31af804f2dac6af5731e18a4dd P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.186285 16055 consensus_queue.cc:260] T eabbcb31af804f2dac6af5731e18a4dd P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.188441 16055 ts_tablet_manager.cc:1434] T eabbcb31af804f2dac6af5731e18a4dd P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:04:38.188764 16260 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.189512 16055 tablet_bootstrap.cc:492] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:38.189334 16260 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.192433 16260 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:38.193430 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:04:38.194440 15902 leader_election.cc:343] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:38.195499 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:04:38.197618 15900 leader_election.cc:343] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:38.198122 15900 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:38.198455 16051 tablet_bootstrap.cc:654] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.197723 15757 catalog_manager.cc:5697] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:38.199329 16260 raft_consensus.cc:2749] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:38.203094 16054 tablet_bootstrap.cc:654] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.205372 16055 tablet_bootstrap.cc:654] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.206200 16051 tablet_bootstrap.cc:492] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:38.206712 16051 ts_tablet_manager.cc:1403] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.037s	user 0.004s	sys 0.009s
I20260709 13:04:38.208832 16054 tablet_bootstrap.cc:492] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:38.209439 16054 ts_tablet_manager.cc:1403] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.031s	user 0.010s	sys 0.000s
I20260709 13:04:38.211257 16051 raft_consensus.cc:359] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.211984 16051 raft_consensus.cc:385] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.212327 16051 raft_consensus.cc:740] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.213042 16051 consensus_queue.cc:260] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.213194 16054 raft_consensus.cc:359] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.214072 16054 raft_consensus.cc:385] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.214421 16054 raft_consensus.cc:740] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.216300 16051 ts_tablet_manager.cc:1434] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:04:38.217187 16051 tablet_bootstrap.cc:492] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:38.221907 16051 tablet_bootstrap.cc:654] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.234084 16051 tablet_bootstrap.cc:492] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:38.234688 16051 ts_tablet_manager.cc:1403] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.018s	user 0.010s	sys 0.005s
I20260709 13:04:38.238186 16054 consensus_queue.cc:260] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.240108 16054 ts_tablet_manager.cc:1434] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.000s
I20260709 13:04:38.241111 16054 tablet_bootstrap.cc:492] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:38.242012 16051 raft_consensus.cc:359] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.242892 16051 raft_consensus.cc:385] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.243232 16051 raft_consensus.cc:740] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.244442 16051 consensus_queue.cc:260] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.246505 16051 ts_tablet_manager.cc:1434] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:04:38.247769 16051 tablet_bootstrap.cc:492] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:38.249369 16260 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.249881 16260 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.252846 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:38.253495 15856 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 0.
I20260709 13:04:38.255785 16260 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:38.257329 15900 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:38.257484 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:38.258158 16260 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:38.258508 16260 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:38.258846 16260 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:38.258945 15902 leader_election.cc:343] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:38.262821 16054 tablet_bootstrap.cc:654] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.265017 16051 tablet_bootstrap.cc:654] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.267802 16055 tablet_bootstrap.cc:492] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:38.268236 16055 ts_tablet_manager.cc:1403] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.079s	user 0.036s	sys 0.023s
I20260709 13:04:38.270964 16055 raft_consensus.cc:359] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.271602 16055 raft_consensus.cc:385] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.271898 16055 raft_consensus.cc:740] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.272552 16055 consensus_queue.cc:260] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.274749 16055 ts_tablet_manager.cc:1434] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:04:38.275898 16055 tablet_bootstrap.cc:492] T c04fe1420dfe4602909deef4baa3f115 P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:38.277093 16064 consensus_queue.cc:1048] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:38.302577 16055 tablet_bootstrap.cc:654] T c04fe1420dfe4602909deef4baa3f115 P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.306053 16162 consensus_queue.cc:1048] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:38.316681 16092 consensus_queue.cc:1048] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:04:38.301879 16260 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.332653 16260 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:04:38.340608 15808 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:38.339246 15759 catalog_manager.cc:5697] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:38.350127 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:38.350749 15862 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.354962 16054 tablet_bootstrap.cc:492] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:38.355397 16054 ts_tablet_manager.cc:1403] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.115s	user 0.024s	sys 0.012s
I20260709 13:04:38.355944 16055 tablet_bootstrap.cc:492] T c04fe1420dfe4602909deef4baa3f115 P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:38.357251 16055 ts_tablet_manager.cc:1403] T c04fe1420dfe4602909deef4baa3f115 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.082s	user 0.009s	sys 0.008s
I20260709 13:04:38.357954 16054 raft_consensus.cc:359] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.358680 16054 raft_consensus.cc:385] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.358960 16054 raft_consensus.cc:740] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.359767 16055 raft_consensus.cc:359] T c04fe1420dfe4602909deef4baa3f115 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.362052 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:38.363899 16055 raft_consensus.cc:385] T c04fe1420dfe4602909deef4baa3f115 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.364198 16055 raft_consensus.cc:740] T c04fe1420dfe4602909deef4baa3f115 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.364794 16055 consensus_queue.cc:260] T c04fe1420dfe4602909deef4baa3f115 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.365609 16010 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.365934 16054 consensus_queue.cc:260] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.366398 15862 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:04:38.367575 15900 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:38.368680 16187 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:38.369208 16187 raft_consensus.cc:697] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:04:38.370064 16187 consensus_queue.cc:237] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.370740 16054 ts_tablet_manager.cc:1434] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.003s
I20260709 13:04:38.371747 16054 tablet_bootstrap.cc:492] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:38.372329 16055 ts_tablet_manager.cc:1434] T c04fe1420dfe4602909deef4baa3f115 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:04:38.373263 16055 tablet_bootstrap.cc:492] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:38.374898 16010 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:04:38.379338 16055 tablet_bootstrap.cc:654] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.384212 16283 consensus_queue.cc:1048] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:38.389322 16054 tablet_bootstrap.cc:654] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.394575 16051 tablet_bootstrap.cc:492] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:38.394985 16051 ts_tablet_manager.cc:1403] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.148s	user 0.025s	sys 0.025s
I20260709 13:04:38.398023 16051 raft_consensus.cc:359] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.398654 16051 raft_consensus.cc:385] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.398916 16051 raft_consensus.cc:740] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.399535 16051 consensus_queue.cc:260] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.401608 16051 ts_tablet_manager.cc:1434] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20260709 13:04:38.402562 16051 tablet_bootstrap.cc:492] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:38.416941 16051 tablet_bootstrap.cc:654] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.424225 16055 tablet_bootstrap.cc:492] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:38.424702 16055 ts_tablet_manager.cc:1403] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.052s	user 0.024s	sys 0.007s
I20260709 13:04:38.428323 16055 raft_consensus.cc:359] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.429101 16055 raft_consensus.cc:385] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.429412 16055 raft_consensus.cc:740] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.430151 16055 consensus_queue.cc:260] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.431664 16055 ts_tablet_manager.cc:1434] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.000s
I20260709 13:04:38.432801 16055 tablet_bootstrap.cc:492] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:38.433825 16054 tablet_bootstrap.cc:492] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:38.434705 16054 ts_tablet_manager.cc:1403] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.063s	user 0.010s	sys 0.007s
I20260709 13:04:38.438647 16055 tablet_bootstrap.cc:654] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.441273 16054 raft_consensus.cc:359] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.442191 16054 raft_consensus.cc:385] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.442515 16054 raft_consensus.cc:740] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.443356 16054 consensus_queue.cc:260] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.446195 16054 ts_tablet_manager.cc:1434] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.004s
I20260709 13:04:38.447350 16054 tablet_bootstrap.cc:492] T f8b7d73618ae489990e21e8cc312177d P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:38.448001 16283 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.448447 16283 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.450492 16283 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:38.453122 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:38.453805 15934 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 0.
I20260709 13:04:38.455623 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:38.456257 16010 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 0.
I20260709 13:04:38.453068 16051 tablet_bootstrap.cc:492] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:38.458041 15825 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:38.458951 16051 ts_tablet_manager.cc:1403] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.057s	user 0.002s	sys 0.011s
I20260709 13:04:38.464723 16051 raft_consensus.cc:359] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.469177 16051 raft_consensus.cc:385] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.469482 16051 raft_consensus.cc:740] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.470086 16051 consensus_queue.cc:260] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.472276 16051 ts_tablet_manager.cc:1434] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:04:38.473193 16051 tablet_bootstrap.cc:492] T e236a7bf118740c9b3d88227bed5670e P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:38.460757 16054 tablet_bootstrap.cc:654] T f8b7d73618ae489990e21e8cc312177d P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.460906 16122 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:38.473930 16122 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:38.474277 16122 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.476943 16055 tablet_bootstrap.cc:492] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:38.477372 16055 ts_tablet_manager.cc:1403] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.045s	user 0.013s	sys 0.029s
I20260709 13:04:38.479995 16055 raft_consensus.cc:359] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.480609 16055 raft_consensus.cc:385] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.480966 16051 tablet_bootstrap.cc:654] T e236a7bf118740c9b3d88227bed5670e P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.460960 15759 catalog_manager.cc:5697] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:38.480924 16055 raft_consensus.cc:740] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.484687 16054 tablet_bootstrap.cc:492] T f8b7d73618ae489990e21e8cc312177d P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:38.484423 16055 consensus_queue.cc:260] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.485322 16054 ts_tablet_manager.cc:1403] T f8b7d73618ae489990e21e8cc312177d P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.038s	user 0.013s	sys 0.000s
I20260709 13:04:38.486830 16055 ts_tablet_manager.cc:1434] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:04:38.487895 16055 tablet_bootstrap.cc:492] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:38.487869 16054 raft_consensus.cc:359] T f8b7d73618ae489990e21e8cc312177d P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.490494 16054 raft_consensus.cc:385] T f8b7d73618ae489990e21e8cc312177d P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.490780 16054 raft_consensus.cc:740] T f8b7d73618ae489990e21e8cc312177d P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.492645 16051 tablet_bootstrap.cc:492] T e236a7bf118740c9b3d88227bed5670e P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:38.491927 16054 consensus_queue.cc:260] T f8b7d73618ae489990e21e8cc312177d P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.494413 16054 ts_tablet_manager.cc:1434] T f8b7d73618ae489990e21e8cc312177d P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.002s
I20260709 13:04:38.493172 16051 ts_tablet_manager.cc:1403] T e236a7bf118740c9b3d88227bed5670e P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.004s
I20260709 13:04:38.496366 15757 catalog_manager.cc:5697] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:38.498209 16055 tablet_bootstrap.cc:654] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.499367 16054 tablet_bootstrap.cc:492] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:38.503017 16122 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.505059 16122 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:38.505317 16051 raft_consensus.cc:359] T e236a7bf118740c9b3d88227bed5670e P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.506063 16051 raft_consensus.cc:385] T e236a7bf118740c9b3d88227bed5670e P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.506371 16051 raft_consensus.cc:740] T e236a7bf118740c9b3d88227bed5670e P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.511091 16051 consensus_queue.cc:260] T e236a7bf118740c9b3d88227bed5670e P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.508558 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:38.512203 16010 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.513311 16051 ts_tablet_manager.cc:1434] T e236a7bf118740c9b3d88227bed5670e P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.018s	user 0.000s	sys 0.005s
I20260709 13:04:38.514038 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:38.514384 16051 tablet_bootstrap.cc:492] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:38.514601 15932 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.510519 16054 tablet_bootstrap.cc:654] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.523914 16051 tablet_bootstrap.cc:654] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.533172 16051 tablet_bootstrap.cc:492] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:38.534561 16051 ts_tablet_manager.cc:1403] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.020s	user 0.005s	sys 0.005s
I20260709 13:04:38.535221 16010 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:04:38.536815 15827 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:38.538039 16122 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:38.538645 16122 raft_consensus.cc:697] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:04:38.540297 16051 raft_consensus.cc:359] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.539472 16122 consensus_queue.cc:237] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.542645 16051 raft_consensus.cc:385] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.543246 16051 raft_consensus.cc:740] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.544275 16051 consensus_queue.cc:260] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.547091 16051 ts_tablet_manager.cc:1434] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.012s	user 0.002s	sys 0.003s
I20260709 13:04:38.549016 16051 tablet_bootstrap.cc:492] T 471dbd2e16264fe5837e8b7ea3601f94 P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:38.553817 16055 tablet_bootstrap.cc:492] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:38.554236 16055 ts_tablet_manager.cc:1403] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.067s	user 0.029s	sys 0.013s
I20260709 13:04:38.557178 16055 raft_consensus.cc:359] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.557782 16055 raft_consensus.cc:385] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.558032 16055 raft_consensus.cc:740] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.558597 16055 consensus_queue.cc:260] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.560491 16055 ts_tablet_manager.cc:1434] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.007s
I20260709 13:04:38.561353 16055 tablet_bootstrap.cc:492] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:38.562157 16051 tablet_bootstrap.cc:654] T 471dbd2e16264fe5837e8b7ea3601f94 P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.567107 16055 tablet_bootstrap.cc:654] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.566658 15760 catalog_manager.cc:5697] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:38.569597 16290 consensus_queue.cc:1048] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:38.573701 16051 tablet_bootstrap.cc:492] T 471dbd2e16264fe5837e8b7ea3601f94 P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:38.574259 16051 ts_tablet_manager.cc:1403] T 471dbd2e16264fe5837e8b7ea3601f94 P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.025s	user 0.007s	sys 0.004s
I20260709 13:04:38.582599 16051 raft_consensus.cc:359] T 471dbd2e16264fe5837e8b7ea3601f94 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.583273 16051 raft_consensus.cc:385] T 471dbd2e16264fe5837e8b7ea3601f94 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.583525 16051 raft_consensus.cc:740] T 471dbd2e16264fe5837e8b7ea3601f94 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.586704 16051 consensus_queue.cc:260] T 471dbd2e16264fe5837e8b7ea3601f94 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.588265 16051 ts_tablet_manager.cc:1434] T 471dbd2e16264fe5837e8b7ea3601f94 P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:04:38.590325 16064 consensus_queue.cc:1048] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:38.590955 16260 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.591400 16260 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.592818 16055 tablet_bootstrap.cc:492] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:38.593245 16055 ts_tablet_manager.cc:1403] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.032s	user 0.008s	sys 0.021s
I20260709 13:04:38.593822 16260 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:38.595885 16055 raft_consensus.cc:359] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.596480 16055 raft_consensus.cc:385] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.596714 16055 raft_consensus.cc:740] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.597294 16055 consensus_queue.cc:260] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.597784 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:38.598592 16010 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 0.
I20260709 13:04:38.599153 16055 ts_tablet_manager.cc:1434] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.002s
I20260709 13:04:38.600117 16055 tablet_bootstrap.cc:492] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:38.600742 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:38.601280 15862 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 0.
I20260709 13:04:38.601486 16251 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.602072 16251 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.602471 15900 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:38.603732 16073 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:38.604166 16073 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:38.604578 16073 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.605840 16055 tablet_bootstrap.cc:654] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.612346 16251 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:38.615593 16073 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.619179 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:38.622210 16051 tablet_bootstrap.cc:492] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:38.623234 16260 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.623723 16260 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.620014 15862 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.628400 16051 tablet_bootstrap.cc:654] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.632187 15932 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:04:38.634407 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:38.638691 16260 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:38.640458 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:38.641224 16010 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 0.
I20260709 13:04:38.641894 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:38.642804 15902 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:38.643639 16260 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:04:38.643816 15900 leader_election.cc:343] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:38.645332 16051 tablet_bootstrap.cc:492] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:38.645744 16051 ts_tablet_manager.cc:1403] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.024s	user 0.015s	sys 0.000s
I20260709 13:04:38.648185 16051 raft_consensus.cc:359] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:04:38.648470 15974 leader_election.cc:343] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:38.644052 16260 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:38.648800 16051 raft_consensus.cc:385] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.649643 16051 raft_consensus.cc:740] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.651149 16260 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.654026 16051 consensus_queue.cc:260] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.655565 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:38.656272 16010 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.657953 16051 ts_tablet_manager.cc:1434] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:04:38.658941 16051 tablet_bootstrap.cc:492] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:38.659183 16251 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.659929 16251 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.665988 16122 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.668030 16122 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.670789 16051 tablet_bootstrap.cc:654] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.672796 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:38.673445 15936 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 0.
I20260709 13:04:38.673739 16122 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:38.675458 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:38.676051 15825 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:04:38.677402 15827 leader_election.cc:343] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:38.679665 16283 raft_consensus.cc:2804] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:38.680261 16283 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:38.680704 16283 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.681533 16073 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:38.680392 16051 tablet_bootstrap.cc:492] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:38.682500 16051 ts_tablet_manager.cc:1403] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.024s	user 0.008s	sys 0.002s
I20260709 13:04:38.648658 15862 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:04:38.679706 16187 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.684381 15900 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:38.684676 16187 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.685954 16010 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:04:38.687963 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:38.685196 16073 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:38.690681 16073 raft_consensus.cc:697] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:04:38.691493 16312 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.692103 16312 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.691870 16073 consensus_queue.cc:237] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.690686 16051 raft_consensus.cc:359] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.703317 16051 raft_consensus.cc:385] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.703573 16051 raft_consensus.cc:740] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
W20260709 13:04:38.711146 15974 leader_election.cc:343] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:38.711851 15974 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:04:38.715356 16051 consensus_queue.cc:260] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.708196 16162 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.715821 16262 raft_consensus.cc:2749] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:38.714003 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:38.726369 16051 ts_tablet_manager.cc:1434] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.044s	user 0.006s	sys 0.000s
I20260709 13:04:38.723194 16162 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.729516 16162 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:38.730567 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:38.731850 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:38.732483 15937 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 0.
I20260709 13:04:38.734696 16312 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:38.736495 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:38.740262 15857 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 0.
W20260709 13:04:38.731936 15827 leader_election.cc:343] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:38.741325 15902 leader_election.cc:343] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:38.742660 16051 tablet_bootstrap.cc:492] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:38.743523 16312 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.744570 15900 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:38.749019 16251 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:38.749567 16306 raft_consensus.cc:2804] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:38.749907 15825 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:38.750366 16260 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.750634 16187 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:38.749933 16306 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:38.751622 16306 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.737983 16283 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.752228 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:38.753809 16312 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.754627 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:38.755188 16283 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:04:38.755900 15827 leader_election.cc:343] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:38.756129 15902 leader_election.cc:343] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:38.756865 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:38.757664 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:38.758324 15937 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.758255 16283 raft_consensus.cc:2804] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:04:38.758612 15974 leader_election.cc:343] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:38.758924 16283 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:38.760260 16283 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.761006 16055 tablet_bootstrap.cc:492] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:38.761508 16055 ts_tablet_manager.cc:1403] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.162s	user 0.028s	sys 0.052s
I20260709 13:04:38.761490 16312 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:38.765040 16055 raft_consensus.cc:359] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.765645 16055 raft_consensus.cc:385] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.765915 16055 raft_consensus.cc:740] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.766124 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:38.766614 16055 consensus_queue.cc:260] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.768649 16260 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:38.768914 16055 ts_tablet_manager.cc:1434] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:04:38.769335 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:38.769915 15932 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 0.
I20260709 13:04:38.770404 16055 tablet_bootstrap.cc:492] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:38.770533 16054 tablet_bootstrap.cc:492] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:38.770942 16054 ts_tablet_manager.cc:1403] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.272s	user 0.048s	sys 0.074s
I20260709 13:04:38.771234 15974 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:04:38.771953 16290 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:38.772287 16290 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:38.772607 16290 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.773567 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:38.774271 16010 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:38.776747 15902 leader_election.cc:343] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:38.780714 16290 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.782660 16283 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.784037 16290 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:38.781304 16051 tablet_bootstrap.cc:654] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.785260 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:38.785476 16283 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:38.785908 15932 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.787362 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:04:38.789116 15974 leader_election.cc:343] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:38.790223 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:38.790891 16008 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.790951 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:04:38.791460 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 47373.9us
I20260709 13:04:38.791599 15934 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.791009 16353 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.793923 16353 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.789173 16054 raft_consensus.cc:359] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.796070 16054 raft_consensus.cc:385] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:38.796334 16054 raft_consensus.cc:740] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:38.796948 16054 consensus_queue.cc:260] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.799005 16054 ts_tablet_manager.cc:1434] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.028s	user 0.004s	sys 0.000s
I20260709 13:04:38.800527 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:38.800591 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:38.801293 15862 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.801317 16353 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:38.801908 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:38.802040 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:38.802604 16009 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 0.
I20260709 13:04:38.804972 16054 tablet_bootstrap.cc:492] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:38.806695 15902 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:38.798254 15934 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 1.
I20260709 13:04:38.807770 16353 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:38.808024 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:38.808184 16353 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:38.808629 16353 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.808831 15856 raft_consensus.cc:2393] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 1: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:04:38.814235 15862 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:04:38.815869 15974 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:04:38.816661 16251 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:38.817086 16251 raft_consensus.cc:697] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:04:38.817232 15937 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:04:38.817867 16251 consensus_queue.cc:237] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.818411 15825 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:38.819330 16283 raft_consensus.cc:2804] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:04:38.823901 15900 leader_election.cc:343] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:38.824280 15900 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:04:38.825176 15900 leader_election.cc:343] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:38.825806 15900 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:38.828259 16260 consensus_queue.cc:1048] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:38.829524 16260 raft_consensus.cc:2749] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:38.830498 15932 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:04:38.827908 16353 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.832005 15825 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:38.832929 16162 raft_consensus.cc:2804] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:38.833415 16162 raft_consensus.cc:697] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:04:38.766152 16306 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.834234 16162 consensus_queue.cc:237] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.835487 16306 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:38.833072 16010 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:04:38.835861 16353 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:38.837334 15760 catalog_manager.cc:5697] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:38.840564 15900 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:38.844254 16353 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.844679 16353 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.846736 16073 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:38.846972 16008 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:04:38.847220 16073 raft_consensus.cc:697] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:04:38.848474 16122 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.848901 16122 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.848158 15758 catalog_manager.cc:5697] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:38.848017 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:38.851167 16352 raft_consensus.cc:2749] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:38.851239 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:38.851877 15860 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:38.851958 16353 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:38.852026 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:38.853375 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:04:38.853406 15900 leader_election.cc:343] T 471dbd2e16264fe5837e8b7ea3601f94 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:38.851265 16011 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:38.854648 15900 leader_election.cc:343] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:38.848105 16073 consensus_queue.cc:237] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.860764 16122 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:38.824435 16283 raft_consensus.cc:697] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:04:38.861824 16283 consensus_queue.cc:237] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.865221 16011 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:04:38.871821 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:38.875481 16055 tablet_bootstrap.cc:654] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:38.876116 15902 leader_election.cc:343] T 471dbd2e16264fe5837e8b7ea3601f94 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:38.876432 15902 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:38.864786 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:38.881965 15902 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:04:38.881532 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:38.880545 16353 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:38.883366 16352 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:38.884251 16352 raft_consensus.cc:697] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:04:38.883572 15827 leader_election.cc:343] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:38.875391 16290 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.882579 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:38.888303 16290 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:04:38.889258 15825 leader_election.cc:343] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:38.890094 15825 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:38.891106 16290 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:38.891136 16122 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:38.885090 16352 consensus_queue.cc:237] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.893196 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:04:38.893810 15902 leader_election.cc:343] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:38.897042 15974 leader_election.cc:343] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:38.901118 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:38.906988 15860 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
W20260709 13:04:38.907840 15974 leader_election.cc:343] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:38.908281 15974 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:04:38.908633 15900 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:38.910126 16290 raft_consensus.cc:2749] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:38.909682 16353 raft_consensus.cc:2804] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:38.911230 16353 raft_consensus.cc:697] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:04:38.911131 16306 consensus_queue.cc:1048] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:38.913573 16054 tablet_bootstrap.cc:654] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:38.913218 16353 consensus_queue.cc:237] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:38.923491 16051 tablet_bootstrap.cc:492] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:38.923964 16051 ts_tablet_manager.cc:1403] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.181s	user 0.030s	sys 0.016s
I20260709 13:04:38.928043 15758 catalog_manager.cc:5697] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:38.930282 15758 catalog_manager.cc:5697] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:38.988057 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 44740us
I20260709 13:04:38.994139 15760 catalog_manager.cc:5697] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:39.007360 16051 raft_consensus.cc:359] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.008152 16051 raft_consensus.cc:385] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:39.008440 16051 raft_consensus.cc:740] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:39.009145 16051 consensus_queue.cc:260] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.013710 16122 consensus_queue.cc:1048] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:39.017508 16051 ts_tablet_manager.cc:1434] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.093s	user 0.006s	sys 0.001s
I20260709 13:04:39.027520 16051 tablet_bootstrap.cc:492] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:39.036891 16162 consensus_queue.cc:1048] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:39.054104 16055 tablet_bootstrap.cc:492] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:39.054780 16055 ts_tablet_manager.cc:1403] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.285s	user 0.021s	sys 0.019s
I20260709 13:04:39.058111 16055 raft_consensus.cc:359] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.061563 16055 raft_consensus.cc:385] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:39.062043 16055 raft_consensus.cc:740] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:39.062726 16054 tablet_bootstrap.cc:492] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:39.059758 16051 tablet_bootstrap.cc:654] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:39.063231 16054 ts_tablet_manager.cc:1403] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.258s	user 0.024s	sys 0.032s
I20260709 13:04:39.066102 16054 raft_consensus.cc:359] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.066745 16054 raft_consensus.cc:385] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:39.066994 16054 raft_consensus.cc:740] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:39.067615 16054 consensus_queue.cc:260] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.069609 16054 ts_tablet_manager.cc:1434] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.004s
I20260709 13:04:39.071086 16054 tablet_bootstrap.cc:492] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:39.078248 16054 tablet_bootstrap.cc:654] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:39.085860 16054 tablet_bootstrap.cc:492] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:39.086287 16054 ts_tablet_manager.cc:1403] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.016s	user 0.011s	sys 0.000s
I20260709 13:04:39.062785 16055 consensus_queue.cc:260] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.090231 16055 ts_tablet_manager.cc:1434] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.035s	user 0.003s	sys 0.003s
I20260709 13:04:39.090781 16054 raft_consensus.cc:359] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.091337 16055 tablet_bootstrap.cc:492] T 260d22f804a345ed8f055536994fcb14 P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:39.091389 16054 raft_consensus.cc:385] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:39.092029 16054 raft_consensus.cc:740] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:39.092772 16054 consensus_queue.cc:260] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.094738 16054 ts_tablet_manager.cc:1434] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:04:39.096369 16054 tablet_bootstrap.cc:492] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:39.120105 16054 tablet_bootstrap.cc:654] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:39.122903 16073 consensus_queue.cc:1048] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:39.142379 15758 catalog_manager.cc:5697] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:39.144058 15758 catalog_manager.cc:5697] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:39.146950 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 47016.5us
I20260709 13:04:39.149197 16260 consensus_queue.cc:1048] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:39.151338 16073 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.151808 16073 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.155270 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:04:39.156934 15900 leader_election.cc:343] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:39.159281 16054 tablet_bootstrap.cc:492] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:39.159916 16054 ts_tablet_manager.cc:1403] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.064s	user 0.006s	sys 0.033s
I20260709 13:04:39.164403 16073 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:39.166091 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:39.166764 16011 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 0.
I20260709 13:04:39.162988 16054 raft_consensus.cc:359] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.167340 16054 raft_consensus.cc:385] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:39.167718 16054 raft_consensus.cc:740] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:39.168089 15902 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:04:39.168671 16054 consensus_queue.cc:260] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.170077 16122 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.170447 16122 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.171010 16054 ts_tablet_manager.cc:1434] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.000s
I20260709 13:04:39.171936 16054 tablet_bootstrap.cc:492] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:39.172184 16122 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:39.173390 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:39.173554 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:04:39.174834 15827 leader_election.cc:343] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:39.175293 16073 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:04:39.175477 15825 leader_election.cc:343] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:39.175690 16073 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:39.175928 15825 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:39.176128 16073 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:39.176569 16055 tablet_bootstrap.cc:654] T 260d22f804a345ed8f055536994fcb14 P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:39.176587 16122 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:39.177548 16054 tablet_bootstrap.cc:654] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:39.192832 16054 tablet_bootstrap.cc:492] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:39.193243 16054 ts_tablet_manager.cc:1403] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.022s	user 0.008s	sys 0.003s
I20260709 13:04:39.202767 16073 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.205155 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:04:39.206457 15900 leader_election.cc:343] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:39.205766 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:39.212464 16011 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:39.213848 16051 tablet_bootstrap.cc:492] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:39.214282 16051 ts_tablet_manager.cc:1403] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.187s	user 0.006s	sys 0.018s
I20260709 13:04:39.216274 16054 raft_consensus.cc:359] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.216905 16054 raft_consensus.cc:385] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:39.217142 16054 raft_consensus.cc:740] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:39.217692 16054 consensus_queue.cc:260] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.218420 16011 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:04:39.219455 15902 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:04:39.219648 16054 ts_tablet_manager.cc:1434] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.002s
I20260709 13:04:39.220209 16260 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:39.220644 16054 tablet_bootstrap.cc:492] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:39.220685 16260 raft_consensus.cc:697] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:04:39.221568 16260 consensus_queue.cc:237] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.214975 16073 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:39.230247 16055 tablet_bootstrap.cc:492] T 260d22f804a345ed8f055536994fcb14 P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:39.230815 16055 ts_tablet_manager.cc:1403] T 260d22f804a345ed8f055536994fcb14 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.140s	user 0.012s	sys 0.011s
I20260709 13:04:39.231562 16051 raft_consensus.cc:359] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.232254 16051 raft_consensus.cc:385] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:39.232512 16051 raft_consensus.cc:740] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:39.233206 16051 consensus_queue.cc:260] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.233917 16055 raft_consensus.cc:359] T 260d22f804a345ed8f055536994fcb14 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.234663 16055 raft_consensus.cc:385] T 260d22f804a345ed8f055536994fcb14 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:39.235000 16055 raft_consensus.cc:740] T 260d22f804a345ed8f055536994fcb14 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:39.235289 16051 ts_tablet_manager.cc:1434] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.002s
I20260709 13:04:39.235934 16055 consensus_queue.cc:260] T 260d22f804a345ed8f055536994fcb14 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.238358 16054 tablet_bootstrap.cc:654] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:39.239255 16051 tablet_bootstrap.cc:492] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:39.245239 16055 ts_tablet_manager.cc:1434] T 260d22f804a345ed8f055536994fcb14 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.004s
I20260709 13:04:39.246271 16055 tablet_bootstrap.cc:492] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:39.249619 16051 tablet_bootstrap.cc:654] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:39.250964 15758 catalog_manager.cc:5697] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:39.254539 16284 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.254972 16284 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.257417 16284 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:39.258348 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:04:39.259758 15827 leader_election.cc:343] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:39.259837 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:39.260483 16055 tablet_bootstrap.cc:654] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:39.260430 15932 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 0.
I20260709 13:04:39.260900 16054 tablet_bootstrap.cc:492] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:39.261330 16054 ts_tablet_manager.cc:1403] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.041s	user 0.013s	sys 0.009s
I20260709 13:04:39.261668 15825 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:39.264217 16054 raft_consensus.cc:359] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.264840 16054 raft_consensus.cc:385] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:39.265071 16054 raft_consensus.cc:740] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:39.266630 16054 consensus_queue.cc:260] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.269059 16054 ts_tablet_manager.cc:1434] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:04:39.270447 16054 tablet_bootstrap.cc:492] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:39.274792 16122 consensus_queue.cc:1048] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:39.262634 16284 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:39.277697 16284 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:39.279872 16284 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:39.280550 16368 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:39.282058 16054 tablet_bootstrap.cc:654] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:39.282675 16073 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.285867 16092 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.286469 16092 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.283088 16073 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.290144 16092 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:39.291039 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:39.291862 15860 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 0.
I20260709 13:04:39.292979 15900 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:39.293690 16306 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:39.293763 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:39.294571 15862 raft_consensus.cc:2468] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 0.
I20260709 13:04:39.294059 16306 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:39.295560 16306 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:39.300339 16073 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:39.301288 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:04:39.302503 15902 leader_election.cc:343] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:39.304659 16284 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.291905 16260 consensus_queue.cc:1048] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:04:39.315008 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:39.307060 16284 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:39.307457 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:04:39.320429 15827 leader_election.cc:343] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:39.321393 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:39.322175 15937 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:39.309119 16051 tablet_bootstrap.cc:492] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:39.312577 15974 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:04:39.323588 15974 leader_election.cc:343] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:39.324111 16092 raft_consensus.cc:2804] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:39.324460 16092 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:39.324828 16092 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:39.325872 15860 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:39.328881 16055 tablet_bootstrap.cc:492] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:39.329348 16055 ts_tablet_manager.cc:1403] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.083s	user 0.004s	sys 0.012s
I20260709 13:04:39.331018 16284 consensus_queue.cc:1048] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:39.335186 16051 ts_tablet_manager.cc:1403] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.097s	user 0.010s	sys 0.000s
I20260709 13:04:39.337795 16092 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.340034 16055 raft_consensus.cc:359] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.340143 16051 raft_consensus.cc:359] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.340718 16055 raft_consensus.cc:385] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:39.340852 16051 raft_consensus.cc:385] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:39.341040 16055 raft_consensus.cc:740] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:39.341158 16051 raft_consensus.cc:740] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:39.341754 16055 consensus_queue.cc:260] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.343187 16055 ts_tablet_manager.cc:1434] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.014s	user 0.002s	sys 0.002s
I20260709 13:04:39.343427 16054 tablet_bootstrap.cc:492] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:39.311206 16306 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.343878 16054 ts_tablet_manager.cc:1403] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.074s	user 0.021s	sys 0.019s
I20260709 13:04:39.346009 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:39.346551 16260 consensus_queue.cc:1048] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:39.347013 16054 raft_consensus.cc:359] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.347939 16051 consensus_queue.cc:260] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.348393 16306 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:39.346601 15856 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:39.352210 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:39.352433 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:39.352784 15857 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:39.351305 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:04:39.353739 15902 leader_election.cc:343] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:39.353997 15934 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:39.358584 15856 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:04:39.350075 16051 ts_tablet_manager.cc:1434] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.015s	user 0.002s	sys 0.002s
I20260709 13:04:39.360268 16051 tablet_bootstrap.cc:492] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:39.360131 15900 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:39.361622 16260 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:39.362068 16260 raft_consensus.cc:697] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:04:39.362838 16260 consensus_queue.cc:237] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.339594 16092 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:39.365796 16055 tablet_bootstrap.cc:492] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:39.370091 16051 tablet_bootstrap.cc:654] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:39.370143 15934 raft_consensus.cc:2468] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 1.
I20260709 13:04:39.347638 16054 raft_consensus.cc:385] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:39.371495 16054 raft_consensus.cc:740] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:39.372131 16054 consensus_queue.cc:260] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.383455 15937 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:04:39.386256 16054 ts_tablet_manager.cc:1434] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.042s	user 0.007s	sys 0.000s
I20260709 13:04:39.387545 15825 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:39.389434 16122 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:39.389844 16122 raft_consensus.cc:697] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:04:39.388562 15974 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:39.391304 16092 raft_consensus.cc:2804] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:39.391764 16092 raft_consensus.cc:697] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:04:39.392410 16092 consensus_queue.cc:237] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.390478 16122 consensus_queue.cc:237] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.371867 15857 raft_consensus.cc:2468] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 1.
I20260709 13:04:39.401180 16073 consensus_queue.cc:1048] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:39.401593 16369 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:39.402573 16055 tablet_bootstrap.cc:654] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:39.424606 16054 tablet_bootstrap.cc:492] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:39.437366 15760 catalog_manager.cc:5697] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:39.462044 15808 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:39.469939 15862 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:39.497932 16064 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.498566 16064 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.506206 16073 consensus_queue.cc:1048] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:39.510070 16064 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:39.518273 16055 tablet_bootstrap.cc:492] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:39.518316 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:39.518918 16055 ts_tablet_manager.cc:1403] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.153s	user 0.022s	sys 0.007s
I20260709 13:04:39.521072 16051 tablet_bootstrap.cc:492] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:39.522387 16051 ts_tablet_manager.cc:1403] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.162s	user 0.023s	sys 0.034s
I20260709 13:04:39.528009 16162 consensus_queue.cc:1048] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:39.509922 16306 consensus_queue.cc:1048] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 13:04:39.540311 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:39.540915 16054 tablet_bootstrap.cc:654] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:39.541144 15857 raft_consensus.cc:2393] T f8b7d73618ae489990e21e8cc312177d P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 1: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:04:39.542459 16051 raft_consensus.cc:359] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.544286 16051 raft_consensus.cc:385] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:39.544533 16051 raft_consensus.cc:740] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:39.545157 16051 consensus_queue.cc:260] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.546213 15974 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:04:39.550674 16368 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:39.553292 15827 consensus_peers.cc:597] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 -> Peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Couldn't send request to peer 85e997ba5003439781ec4f1f8db0c3ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:39.555586 16051 ts_tablet_manager.cc:1434] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.000s
I20260709 13:04:39.563967 15759 catalog_manager.cc:5697] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:39.566524 16051 tablet_bootstrap.cc:492] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:39.568557 16251 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.569020 16251 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.574579 16251 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:39.608628 16122 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.609077 16122 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.625453 16122 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:39.615590 16055 raft_consensus.cc:359] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.609774 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:39.627655 16055 raft_consensus.cc:385] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:39.628021 16055 raft_consensus.cc:740] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:39.631750 16055 consensus_queue.cc:260] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.637760 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:39.638415 15861 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 0.
I20260709 13:04:39.639443 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:39.640339 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:39.640938 16010 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 0.
I20260709 13:04:39.631276 15937 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 0.
I20260709 13:04:39.643944 15827 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:04:39.645087 16284 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:39.645444 16284 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:39.645579 16051 tablet_bootstrap.cc:654] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:39.645814 16284 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:39.650043 16055 ts_tablet_manager.cc:1434] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.127s	user 0.007s	sys 0.000s
I20260709 13:04:39.651029 16055 tablet_bootstrap.cc:492] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:39.652179 16368 raft_consensus.cc:2749] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:39.660121 15974 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:39.672315 16262 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:39.672698 16262 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:39.673024 16262 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:39.675026 16284 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.676530 15758 catalog_manager.cc:5697] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:39.677791 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:39.678309 15937 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:39.679031 16284 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:39.681082 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:39.683849 16054 tablet_bootstrap.cc:492] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:39.684269 16054 ts_tablet_manager.cc:1403] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.285s	user 0.016s	sys 0.020s
I20260709 13:04:39.685972 16369 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.690604 16369 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.691776 16260 consensus_queue.cc:1048] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:04:39.698652 16369 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:39.700531 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:39.702312 16262 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.704212 16009 raft_consensus.cc:2393] T 260d22f804a345ed8f055536994fcb14 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 1: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:04:39.707135 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:39.707901 15932 raft_consensus.cc:2393] T 471dbd2e16264fe5837e8b7ea3601f94 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 1: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:04:39.716015 16284 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.716476 16284 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.718307 15974 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:04:39.719478 16368 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:39.722322 16284 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:39.723563 16055 tablet_bootstrap.cc:654] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:39.723812 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:04:39.725329 15827 leader_election.cc:343] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:39.725646 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:39.726222 15936 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 0.
I20260709 13:04:39.727300 16073 consensus_queue.cc:1048] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:39.727841 15825 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:39.730466 16051 tablet_bootstrap.cc:492] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:39.730865 16051 ts_tablet_manager.cc:1403] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.169s	user 0.013s	sys 0.001s
I20260709 13:04:39.729555 16283 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:39.731746 16283 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:39.732074 16283 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:39.732605 16368 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:39.738694 15937 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:04:39.740382 15825 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:39.741449 16162 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:39.741931 16162 raft_consensus.cc:697] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:04:39.742707 16162 consensus_queue.cc:237] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.738747 16283 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:04:39.748454 15900 consensus_peers.cc:597] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:39.749095 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:39.749871 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:39.755815 15937 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:39.751519 16262 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:04:39.756088 15827 leader_election.cc:343] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:39.750392 16283 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:39.760126 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:39.765899 16054 raft_consensus.cc:359] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.767166 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:39.768272 16054 raft_consensus.cc:385] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:39.768674 15936 raft_consensus.cc:2393] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 1: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:04:39.770956 16054 raft_consensus.cc:740] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:39.774425 15937 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:04:39.775002 16054 consensus_queue.cc:260] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.775449 15825 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:39.777213 16054 ts_tablet_manager.cc:1434] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.093s	user 0.006s	sys 0.000s
I20260709 13:04:39.777184 16162 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:39.777787 16162 raft_consensus.cc:697] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:04:39.778441 16054 tablet_bootstrap.cc:492] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:39.778633 16162 consensus_queue.cc:237] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.786360 16054 tablet_bootstrap.cc:654] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:39.792665 16051 raft_consensus.cc:359] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.793318 16051 raft_consensus.cc:385] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:39.793560 16051 raft_consensus.cc:740] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:39.794109 16051 consensus_queue.cc:260] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.796048 16051 ts_tablet_manager.cc:1434] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.065s	user 0.005s	sys 0.001s
I20260709 13:04:39.796964 16051 tablet_bootstrap.cc:492] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:39.802802 15974 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:04:39.804399 16092 raft_consensus.cc:2749] T 260d22f804a345ed8f055536994fcb14 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:04:39.814975 16054 tablet_bootstrap.cc:492] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:39.815467 16054 ts_tablet_manager.cc:1403] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.037s	user 0.014s	sys 0.003s
I20260709 13:04:39.817958 16054 raft_consensus.cc:359] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.818542 16054 raft_consensus.cc:385] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:39.818957 16054 raft_consensus.cc:740] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:39.818506 15758 catalog_manager.cc:5697] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:39.819391 16283 consensus_queue.cc:1048] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:39.819533 16054 consensus_queue.cc:260] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.820137 15758 catalog_manager.cc:5697] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:39.821364 16054 ts_tablet_manager.cc:1434] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:04:39.822647 16054 tablet_bootstrap.cc:492] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:39.827991 16054 tablet_bootstrap.cc:654] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:39.833274 16055 tablet_bootstrap.cc:492] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:39.833680 16055 ts_tablet_manager.cc:1403] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.183s	user 0.018s	sys 0.026s
I20260709 13:04:39.842347 16162 consensus_queue.cc:1048] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:39.846688 16051 tablet_bootstrap.cc:654] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:39.848086 16352 consensus_queue.cc:1048] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:39.854908 16055 raft_consensus.cc:359] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.855584 16055 raft_consensus.cc:385] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:39.856222 16055 raft_consensus.cc:740] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:39.856936 16055 consensus_queue.cc:260] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.858875 16055 ts_tablet_manager.cc:1434] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
W20260709 13:04:39.869457 15902 consensus_peers.cc:597] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea -> Peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Couldn't send request to peer 85e997ba5003439781ec4f1f8db0c3ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:39.871649 16055 tablet_bootstrap.cc:492] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:39.874125 16054 tablet_bootstrap.cc:492] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:39.874549 16054 ts_tablet_manager.cc:1403] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.052s	user 0.003s	sys 0.005s
I20260709 13:04:39.876977 16054 raft_consensus.cc:359] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.877576 16054 raft_consensus.cc:385] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:39.877851 16054 raft_consensus.cc:740] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:39.878399 16054 consensus_queue.cc:260] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.880400 16054 ts_tablet_manager.cc:1434] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 13:04:39.881242 16054 tablet_bootstrap.cc:492] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:39.886663 16054 tablet_bootstrap.cc:654] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:39.892683 16092 consensus_queue.cc:1048] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:39.897121 16092 consensus_queue.cc:1048] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:39.901903 16051 tablet_bootstrap.cc:492] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:39.902357 16051 ts_tablet_manager.cc:1403] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.106s	user 0.013s	sys 0.018s
I20260709 13:04:39.904675 16055 tablet_bootstrap.cc:654] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:39.905661 16051 raft_consensus.cc:359] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.906301 16051 raft_consensus.cc:385] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:39.906533 16051 raft_consensus.cc:740] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:39.907207 16051 consensus_queue.cc:260] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.911625 16051 ts_tablet_manager.cc:1434] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:04:39.912559 16051 tablet_bootstrap.cc:492] T dd0dfa7599d64160b42b750a38e5c239 P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:39.918064 16051 tablet_bootstrap.cc:654] T dd0dfa7599d64160b42b750a38e5c239 P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:39.929195 16054 tablet_bootstrap.cc:492] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:39.929646 16054 ts_tablet_manager.cc:1403] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.049s	user 0.007s	sys 0.005s
I20260709 13:04:39.932190 16054 raft_consensus.cc:359] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.932816 16054 raft_consensus.cc:385] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:39.933059 16054 raft_consensus.cc:740] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:39.933605 16054 consensus_queue.cc:260] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:39.935566 16054 ts_tablet_manager.cc:1434] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 13:04:39.937620 16054 tablet_bootstrap.cc:492] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:39.944442 16054 tablet_bootstrap.cc:654] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:40.012513 16055 tablet_bootstrap.cc:492] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:40.012987 16055 ts_tablet_manager.cc:1403] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.141s	user 0.011s	sys 0.028s
I20260709 13:04:40.016219 16051 tablet_bootstrap.cc:492] T dd0dfa7599d64160b42b750a38e5c239 P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:40.016475 16054 tablet_bootstrap.cc:492] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:40.016682 16051 ts_tablet_manager.cc:1403] T dd0dfa7599d64160b42b750a38e5c239 P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.104s	user 0.003s	sys 0.029s
I20260709 13:04:40.016836 16054 ts_tablet_manager.cc:1403] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.079s	user 0.009s	sys 0.000s
I20260709 13:04:40.020452 16051 raft_consensus.cc:359] T dd0dfa7599d64160b42b750a38e5c239 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.021126 16051 raft_consensus.cc:385] T dd0dfa7599d64160b42b750a38e5c239 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:40.021399 16051 raft_consensus.cc:740] T dd0dfa7599d64160b42b750a38e5c239 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:40.022006 16051 consensus_queue.cc:260] T dd0dfa7599d64160b42b750a38e5c239 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.023375 16051 ts_tablet_manager.cc:1434] T dd0dfa7599d64160b42b750a38e5c239 P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.000s
I20260709 13:04:40.023326 16054 raft_consensus.cc:359] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.023981 16054 raft_consensus.cc:385] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:40.024219 16054 raft_consensus.cc:740] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:40.024326 16051 tablet_bootstrap.cc:492] T 8849099536af497bae91c72da893681c P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:40.024724 16054 consensus_queue.cc:260] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.026175 16054 ts_tablet_manager.cc:1434] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.000s
I20260709 13:04:40.028445 16054 tablet_bootstrap.cc:492] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:40.027782 16055 raft_consensus.cc:359] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.032191 16055 raft_consensus.cc:385] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:40.032466 16055 raft_consensus.cc:740] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:40.033067 16055 consensus_queue.cc:260] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.039633 16054 tablet_bootstrap.cc:654] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:40.046155 16009 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:40.050348 16055 ts_tablet_manager.cc:1434] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.037s	user 0.005s	sys 0.001s
I20260709 13:04:40.064677 16055 tablet_bootstrap.cc:492] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:40.068990 16051 tablet_bootstrap.cc:654] T 8849099536af497bae91c72da893681c P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:40.084121 16055 tablet_bootstrap.cc:654] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:40.091147 16054 tablet_bootstrap.cc:492] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:40.091766 16054 ts_tablet_manager.cc:1403] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.064s	user 0.017s	sys 0.011s
I20260709 13:04:40.095196 16054 raft_consensus.cc:359] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.096035 16054 raft_consensus.cc:385] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:40.096555 16055 tablet_bootstrap.cc:492] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:40.096750 16054 raft_consensus.cc:740] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:40.097018 16055 ts_tablet_manager.cc:1403] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.033s	user 0.004s	sys 0.011s
I20260709 13:04:40.098199 16054 consensus_queue.cc:260] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.100772 16054 ts_tablet_manager.cc:1434] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:04:40.107213 16055 raft_consensus.cc:359] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.107959 16055 raft_consensus.cc:385] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:40.108232 16055 raft_consensus.cc:740] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:40.108428 16054 tablet_bootstrap.cc:492] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:40.109323 16055 consensus_queue.cc:260] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.114131 16054 tablet_bootstrap.cc:654] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:40.115410 16055 ts_tablet_manager.cc:1434] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 13:04:40.116668 16055 tablet_bootstrap.cc:492] T 78530df4cdb742eda67781671b0431c6 P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:40.119236 16051 tablet_bootstrap.cc:492] T 8849099536af497bae91c72da893681c P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:40.119994 16051 ts_tablet_manager.cc:1403] T 8849099536af497bae91c72da893681c P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.096s	user 0.032s	sys 0.009s
I20260709 13:04:40.122645 16162 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.124845 16122 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.125025 16162 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.125326 16122 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.127032 16162 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:40.127099 16051 raft_consensus.cc:359] T 8849099536af497bae91c72da893681c P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.127892 16051 raft_consensus.cc:385] T 8849099536af497bae91c72da893681c P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:40.128159 16051 raft_consensus.cc:740] T 8849099536af497bae91c72da893681c P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:40.129168 16122 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:40.129835 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:40.130443 15936 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 0.
I20260709 13:04:40.128939 16051 consensus_queue.cc:260] T 8849099536af497bae91c72da893681c P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.132040 15825 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:40.132493 16051 ts_tablet_manager.cc:1434] T 8849099536af497bae91c72da893681c P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.001s
I20260709 13:04:40.124640 16055 tablet_bootstrap.cc:654] T 78530df4cdb742eda67781671b0431c6 P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:40.129844 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:40.136008 16051 tablet_bootstrap.cc:492] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:40.136408 16162 raft_consensus.cc:2804] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:40.136813 16162 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:40.137194 16162 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:40.142880 16055 tablet_bootstrap.cc:492] T 78530df4cdb742eda67781671b0431c6 P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:40.143330 16055 ts_tablet_manager.cc:1403] T 78530df4cdb742eda67781671b0431c6 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.027s	user 0.009s	sys 0.000s
I20260709 13:04:40.145799 16162 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.148339 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:40.148813 15937 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:40.150094 16055 raft_consensus.cc:359] T 78530df4cdb742eda67781671b0431c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.151031 16055 raft_consensus.cc:385] T 78530df4cdb742eda67781671b0431c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:40.151355 16055 raft_consensus.cc:740] T 78530df4cdb742eda67781671b0431c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:40.152185 16055 consensus_queue.cc:260] T 78530df4cdb742eda67781671b0431c6 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.155045 15937 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:04:40.156289 15825 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:40.156863 16051 tablet_bootstrap.cc:654] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:40.157056 16122 raft_consensus.cc:2804] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:40.158327 16122 raft_consensus.cc:697] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:04:40.159107 16122 consensus_queue.cc:237] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.163981 16162 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:40.166285 16092 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.166671 16092 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.168443 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:40.168629 16054 tablet_bootstrap.cc:492] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:40.169184 16054 ts_tablet_manager.cc:1403] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.061s	user 0.020s	sys 0.031s
I20260709 13:04:40.170392 16092 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:40.171046 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:40.171582 16010 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 0.
I20260709 13:04:40.171974 16054 raft_consensus.cc:359] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.173127 16054 raft_consensus.cc:385] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:40.173436 16054 raft_consensus.cc:740] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:40.174165 16054 consensus_queue.cc:260] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.175662 16054 ts_tablet_manager.cc:1434] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.001s
I20260709 13:04:40.176663 16054 tablet_bootstrap.cc:492] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:40.183054 15760 catalog_manager.cc:5697] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:40.188422 16054 tablet_bootstrap.cc:654] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:40.189507 16051 tablet_bootstrap.cc:492] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:40.189935 16051 ts_tablet_manager.cc:1403] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.054s	user 0.021s	sys 0.015s
I20260709 13:04:40.192582 16051 raft_consensus.cc:359] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.193183 16051 raft_consensus.cc:385] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:40.193456 16051 raft_consensus.cc:740] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:40.194132 16051 consensus_queue.cc:260] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.196286 16051 ts_tablet_manager.cc:1434] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 13:04:40.159107 16055 ts_tablet_manager.cc:1434] T 78530df4cdb742eda67781671b0431c6 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:04:40.203768 16054 tablet_bootstrap.cc:492] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:40.204018 16055 tablet_bootstrap.cc:492] T 91d0cc5018ee4a9fb3d196c783814c44 P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:40.204325 16054 ts_tablet_manager.cc:1403] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.003s
I20260709 13:04:40.205904 16051 tablet_bootstrap.cc:492] T 7b6e87097733428bb2167c6f308fe589 P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:40.207041 16054 raft_consensus.cc:359] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.207862 16054 raft_consensus.cc:385] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:40.208178 16054 raft_consensus.cc:740] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:40.208925 16054 consensus_queue.cc:260] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.209587 16162 consensus_queue.cc:1048] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:40.210489 16054 ts_tablet_manager.cc:1434] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:04:40.211556 16054 tablet_bootstrap.cc:492] T ceaf02d91e334d5eba077b59ccce3a38 P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:40.216100 16051 tablet_bootstrap.cc:654] T 7b6e87097733428bb2167c6f308fe589 P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:40.217831 16054 tablet_bootstrap.cc:654] T ceaf02d91e334d5eba077b59ccce3a38 P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:40.220912 16055 tablet_bootstrap.cc:654] T 91d0cc5018ee4a9fb3d196c783814c44 P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:40.227756 16051 tablet_bootstrap.cc:492] T 7b6e87097733428bb2167c6f308fe589 P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:40.228253 16051 ts_tablet_manager.cc:1403] T 7b6e87097733428bb2167c6f308fe589 P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.023s	user 0.006s	sys 0.010s
I20260709 13:04:40.230861 16051 raft_consensus.cc:359] T 7b6e87097733428bb2167c6f308fe589 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.231521 16051 raft_consensus.cc:385] T 7b6e87097733428bb2167c6f308fe589 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:40.231801 16051 raft_consensus.cc:740] T 7b6e87097733428bb2167c6f308fe589 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:40.232134 16054 tablet_bootstrap.cc:492] T ceaf02d91e334d5eba077b59ccce3a38 P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:40.232564 16054 ts_tablet_manager.cc:1403] T ceaf02d91e334d5eba077b59ccce3a38 P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.006s
I20260709 13:04:40.232939 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:40.232686 16051 consensus_queue.cc:260] T 7b6e87097733428bb2167c6f308fe589 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.233412 16010 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:40.235196 16051 ts_tablet_manager.cc:1434] T 7b6e87097733428bb2167c6f308fe589 P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:04:40.236188 16051 tablet_bootstrap.cc:492] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:40.235716 16054 raft_consensus.cc:359] T ceaf02d91e334d5eba077b59ccce3a38 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.237932 16054 raft_consensus.cc:385] T ceaf02d91e334d5eba077b59ccce3a38 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:40.238188 16054 raft_consensus.cc:740] T ceaf02d91e334d5eba077b59ccce3a38 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:40.239702 16010 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:04:40.240031 16054 consensus_queue.cc:260] T ceaf02d91e334d5eba077b59ccce3a38 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.242347 16051 tablet_bootstrap.cc:654] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:40.242705 16054 ts_tablet_manager.cc:1434] T ceaf02d91e334d5eba077b59ccce3a38 P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:04:40.245157 16054 tablet_bootstrap.cc:492] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8: Bootstrap starting.
I20260709 13:04:40.247840 16162 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.248449 16162 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.251603 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:40.252269 15936 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 0.
I20260709 13:04:40.252473 16054 tablet_bootstrap.cc:654] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:40.253059 16051 tablet_bootstrap.cc:492] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:40.253490 16051 ts_tablet_manager.cc:1403] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.004s
I20260709 13:04:40.253427 15825 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:40.254103 16122 raft_consensus.cc:2804] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:40.254436 16122 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:40.254709 16122 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:40.256736 16051 raft_consensus.cc:359] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.257336 16051 raft_consensus.cc:385] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:40.260387 15861 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:40.257575 16051 raft_consensus.cc:740] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:40.264094 16051 consensus_queue.cc:260] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.266372 16051 ts_tablet_manager.cc:1434] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.003s
I20260709 13:04:40.267294 16051 tablet_bootstrap.cc:492] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:40.270803 16122 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.272264 16312 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.272827 16312 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.274674 16312 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:40.276333 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:04:40.278409 15900 leader_election.cc:343] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:40.280346 16051 tablet_bootstrap.cc:654] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:40.284762 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 92448.2us
I20260709 13:04:40.290160 16283 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.290804 16283 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.294230 16283 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:40.296000 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:40.297129 15936 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 0.
I20260709 13:04:40.298144 16283 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.298804 16283 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.302484 16283 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:40.303474 15825 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:40.307873 16283 raft_consensus.cc:2804] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:40.308436 16283 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:40.308944 16283 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:40.306905 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:40.314193 15936 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 0.
I20260709 13:04:40.315616 15825 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:40.316517 16284 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:40.316867 16284 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:40.317235 16284 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:40.323086 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:40.328408 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:40.330590 16284 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.333086 16284 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:40.330665 15936 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:40.334919 16162 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:40.338136 16122 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:40.339774 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:40.340554 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:04:40.342402 15902 leader_election.cc:343] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:40.342775 15902 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:40.343657 16352 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:40.346385 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:40.348618 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:40.350894 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:40.351293 15936 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:04:40.374297 15825 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:40.376366 16434 raft_consensus.cc:2804] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:40.376955 16434 raft_consensus.cc:697] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:04:40.377772 16434 consensus_queue.cc:237] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.378784 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:40.405710 15937 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:40.381006 16162 consensus_queue.cc:1048] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:40.391706 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:40.393229 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:40.394796 16006 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:40.394881 15827 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:40.411929 15937 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:04:40.419615 16284 raft_consensus.cc:2749] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:40.422979 16009 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:40.432159 16284 consensus_queue.cc:1048] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:40.441687 16352 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.442351 16352 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.446508 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:04:40.448324 15902 leader_election.cc:343] T dd0dfa7599d64160b42b750a38e5c239 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:40.451375 16352 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:40.456728 16055 tablet_bootstrap.cc:492] T 91d0cc5018ee4a9fb3d196c783814c44 P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:40.464926 16055 ts_tablet_manager.cc:1403] T 91d0cc5018ee4a9fb3d196c783814c44 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.261s	user 0.067s	sys 0.095s
W20260709 13:04:40.476671 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 97311us
I20260709 13:04:40.477952 15825 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:40.480270 16434 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:40.480723 16434 raft_consensus.cc:697] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:04:40.480773 16055 raft_consensus.cc:359] T 91d0cc5018ee4a9fb3d196c783814c44 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.481470 16055 raft_consensus.cc:385] T 91d0cc5018ee4a9fb3d196c783814c44 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:40.481788 16055 raft_consensus.cc:740] T 91d0cc5018ee4a9fb3d196c783814c44 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:40.481510 16434 consensus_queue.cc:237] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.482578 16352 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.482424 16055 consensus_queue.cc:260] T 91d0cc5018ee4a9fb3d196c783814c44 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.483027 16352 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.484555 16055 ts_tablet_manager.cc:1434] T 91d0cc5018ee4a9fb3d196c783814c44 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:04:40.484961 16352 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:40.485415 16055 tablet_bootstrap.cc:492] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:40.488214 16368 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.488564 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:04:40.490159 15902 leader_election.cc:343] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:40.488925 16368 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.492439 16055 tablet_bootstrap.cc:654] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:40.495201 16368 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:04:40.496258 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 81977us
W20260709 13:04:40.498190 15827 leader_election.cc:343] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:40.507187 16284 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.507854 16284 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.510319 16284 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:04:40.511459 15827 leader_election.cc:343] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:40.512642 15827 leader_election.cc:343] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:40.513669 15827 leader_election.cc:343] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:40.514653 15827 leader_election.cc:343] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:40.522977 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:40.524446 15937 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 0.
W20260709 13:04:40.524583 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 119607us
I20260709 13:04:40.533326 16054 tablet_bootstrap.cc:492] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8: No bootstrap required, opened a new log
I20260709 13:04:40.533773 16054 ts_tablet_manager.cc:1403] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8: Time spent bootstrapping tablet: real 0.289s	user 0.066s	sys 0.116s
I20260709 13:04:40.533947 16051 tablet_bootstrap.cc:492] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:40.534389 16051 ts_tablet_manager.cc:1403] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.267s	user 0.039s	sys 0.060s
I20260709 13:04:40.460408 16283 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.537606 16055 tablet_bootstrap.cc:492] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:40.538084 16055 ts_tablet_manager.cc:1403] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.053s	user 0.025s	sys 0.016s
I20260709 13:04:40.536722 16051 raft_consensus.cc:359] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.539565 16051 raft_consensus.cc:385] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:40.539537 15974 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:04:40.540040 16051 raft_consensus.cc:740] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:40.540625 16441 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:40.540552 16055 raft_consensus.cc:359] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.540889 16051 consensus_queue.cc:260] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.541428 16055 raft_consensus.cc:385] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:40.541728 16055 raft_consensus.cc:740] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:40.541116 16283 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:40.542460 16055 consensus_queue.cc:260] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.544250 15825 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:40.545218 16055 ts_tablet_manager.cc:1434] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:04:40.545399 16054 raft_consensus.cc:359] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.546129 16054 raft_consensus.cc:385] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:40.546324 16055 tablet_bootstrap.cc:492] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:40.546377 16054 raft_consensus.cc:740] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Initialized, Role: FOLLOWER
I20260709 13:04:40.545214 16435 raft_consensus.cc:2804] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:40.546942 16435 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:40.547279 16435 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:40.547081 16054 consensus_queue.cc:260] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.553371 16055 tablet_bootstrap.cc:654] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:40.553191 16054 ts_tablet_manager.cc:1434] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.000s
I20260709 13:04:40.555074 16284 consensus_queue.cc:1048] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:40.555846 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:40.556370 15936 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:40.563349 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 47733.2us
I20260709 13:04:40.565912 16284 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.566454 16284 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.568476 16284 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:04:40.577046 15963 tablet.cc:2406] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:04:40.569372 15936 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:04:40.580946 15825 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:40.582880 16440 raft_consensus.cc:2804] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:40.583635 16440 raft_consensus.cc:697] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:04:40.584681 16440 consensus_queue.cc:237] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.570560 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:40.588927 15932 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 0.
I20260709 13:04:40.590400 15825 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:40.575745 16051 ts_tablet_manager.cc:1434] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.041s	user 0.005s	sys 0.003s
I20260709 13:04:40.593959 16284 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:40.594386 16284 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:40.594777 16284 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:40.595604 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:40.578491 16435 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.604133 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:40.604769 15936 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:40.604894 15757 catalog_manager.cc:5697] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:40.596362 16051 tablet_bootstrap.cc:492] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea: Bootstrap starting.
I20260709 13:04:40.596746 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:40.598841 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:40.599453 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:40.608507 16006 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:40.615362 16435 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:40.615330 16006 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:04:40.636665 15936 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:04:40.607040 15757 catalog_manager.cc:5697] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:40.639289 16051 tablet_bootstrap.cc:654] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:40.642072 16055 tablet_bootstrap.cc:492] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:40.642495 16055 ts_tablet_manager.cc:1403] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.096s	user 0.015s	sys 0.039s
I20260709 13:04:40.642948 15825 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:40.644601 16440 raft_consensus.cc:2804] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:40.645020 16440 raft_consensus.cc:697] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:04:40.645838 16440 consensus_queue.cc:237] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.652117 16055 raft_consensus.cc:359] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.652860 16055 raft_consensus.cc:385] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:40.653247 16055 raft_consensus.cc:740] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:40.653999 16055 consensus_queue.cc:260] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.658967 16435 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.659513 16435 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.660844 16055 ts_tablet_manager.cc:1434] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:04:40.662204 16055 tablet_bootstrap.cc:492] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:40.663561 16162 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.664235 16435 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:40.664109 16162 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.666025 16162 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:40.666662 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:40.667240 15937 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 0.
I20260709 13:04:40.667578 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:40.668515 15825 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:40.683049 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:40.685186 16283 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:40.695469 16283 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:40.698514 16283 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:40.705156 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:04:40.713356 15808 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:04:40.717757 15825 leader_election.cc:343] T ceaf02d91e334d5eba077b59ccce3a38 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:40.692483 16284 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.725787 16284 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:40.727625 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:40.728209 15932 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:40.728083 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:40.735033 15932 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:04:40.735986 15861 raft_consensus.cc:2393] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 1: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:04:40.737712 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:04:40.740949 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 44557.5us
I20260709 13:04:40.740447 15900 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:04:40.746332 15827 leader_election.cc:343] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:40.746390 15825 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:40.747784 16122 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:40.748224 16122 raft_consensus.cc:697] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:04:40.752650 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:40.745402 16352 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:40.735509 16283 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.749236 16122 consensus_queue.cc:237] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:04:40.761904 15827 leader_election.cc:343] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:40.764451 15827 leader_election.cc:343] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:40.764304 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:04:40.765300 15827 leader_election.cc:343] T ceaf02d91e334d5eba077b59ccce3a38 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:40.765673 15827 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:40.767304 15974 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:04:40.768532 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:40.768971 16283 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:40.770639 16435 raft_consensus.cc:2749] T ceaf02d91e334d5eba077b59ccce3a38 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:40.770787 16437 raft_consensus.cc:2749] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:40.773000 16055 tablet_bootstrap.cc:654] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:40.785857 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:40.786614 16051 tablet_bootstrap.cc:492] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea: No bootstrap required, opened a new log
I20260709 13:04:40.787057 16051 ts_tablet_manager.cc:1403] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea: Time spent bootstrapping tablet: real 0.191s	user 0.022s	sys 0.014s
W20260709 13:04:40.788959 15827 leader_election.cc:343] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:40.789822 15827 leader_election.cc:343] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:40.794978 15827 leader_election.cc:343] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:40.805171 15900 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:40.805739 16051 raft_consensus.cc:359] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.806677 16051 raft_consensus.cc:385] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:40.807065 16051 raft_consensus.cc:740] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Initialized, Role: FOLLOWER
I20260709 13:04:40.809224 16306 raft_consensus.cc:2749] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:40.812345 15757 catalog_manager.cc:5697] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:40.814756 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:40.815315 15932 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:40.806020 16284 consensus_queue.cc:1048] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 13:04:40.813874 15757 catalog_manager.cc:5697] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:40.807953 16051 consensus_queue.cc:260] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.819754 16284 consensus_queue.cc:1048] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:04:40.823583 16051 ts_tablet_manager.cc:1434] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea: Time spent starting tablet: real 0.036s	user 0.006s	sys 0.002s
I20260709 13:04:40.828640 16055 tablet_bootstrap.cc:492] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:40.829161 16055 ts_tablet_manager.cc:1403] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.167s	user 0.010s	sys 0.013s
I20260709 13:04:40.836838 15932 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:04:40.838526 15825 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:40.840019 16434 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:40.840473 16434 raft_consensus.cc:697] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:04:40.841262 16434 consensus_queue.cc:237] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.842450 16055 raft_consensus.cc:359] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.843109 16055 raft_consensus.cc:385] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:40.843364 16055 raft_consensus.cc:740] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:40.844053 16055 consensus_queue.cc:260] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.856907 16055 ts_tablet_manager.cc:1434] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.001s
I20260709 13:04:40.857702 16055 tablet_bootstrap.cc:492] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:40.862923 16055 tablet_bootstrap.cc:654] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:40.869201 16436 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.869725 16436 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.872179 16436 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:40.873579 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:40.877282 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:40.878015 15932 raft_consensus.cc:2393] T 7b6e87097733428bb2167c6f308fe589 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 1: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:04:40.878957 16005 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:40.886657 16055 tablet_bootstrap.cc:492] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:40.887260 16055 ts_tablet_manager.cc:1403] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.030s	user 0.009s	sys 0.009s
I20260709 13:04:40.888523 15974 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:04:40.889508 16436 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:40.895716 16456 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.896138 16055 raft_consensus.cc:359] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.896169 16456 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.896948 16055 raft_consensus.cc:385] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:40.897251 16055 raft_consensus.cc:740] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:40.897907 16055 consensus_queue.cc:260] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.898484 16456 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:40.900949 16055 ts_tablet_manager.cc:1434] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 13:04:40.901844 16055 tablet_bootstrap.cc:492] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:40.901896 16436 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:40.894901 16122 consensus_queue.cc:1048] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:40.903554 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:04:40.904731 15902 leader_election.cc:343] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:40.904875 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:40.905601 15857 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 0.
I20260709 13:04:40.906800 15900 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:40.907650 16456 raft_consensus.cc:2804] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:40.908003 16456 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:40.908252 16055 tablet_bootstrap.cc:654] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:40.908331 16456 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:40.910205 15756 catalog_manager.cc:5697] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:40.912176 15756 catalog_manager.cc:5697] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:40.918541 16055 tablet_bootstrap.cc:492] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:40.919117 16055 ts_tablet_manager.cc:1403] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.004s
I20260709 13:04:40.921219 16369 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.921638 16369 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.921950 16055 raft_consensus.cc:359] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.922834 16055 raft_consensus.cc:385] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:40.923180 16055 raft_consensus.cc:740] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:40.923813 16369 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:40.923974 16055 consensus_queue.cc:260] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.926653 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:40.922173 16456 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.926885 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:40.927742 15932 raft_consensus.cc:2393] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 1: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:04:40.928508 16055 ts_tablet_manager.cc:1434] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 13:04:40.930238 16055 tablet_bootstrap.cc:492] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:40.935825 16434 consensus_queue.cc:1048] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:40.936513 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:40.936879 16055 tablet_bootstrap.cc:654] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:40.937186 15858 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:40.945381 16440 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.946022 16440 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.957340 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:40.958153 15934 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 0.
I20260709 13:04:40.958477 15858 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:04:40.960685 15825 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:40.961704 16284 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:40.962056 16284 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:40.962390 16284 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:40.967969 15900 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:40.970552 16055 tablet_bootstrap.cc:492] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:40.970992 16055 ts_tablet_manager.cc:1403] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.041s	user 0.014s	sys 0.023s
I20260709 13:04:40.973279 16055 raft_consensus.cc:359] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.973908 16055 raft_consensus.cc:385] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:40.974185 16055 raft_consensus.cc:740] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:40.974815 16055 consensus_queue.cc:260] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.975709 16352 raft_consensus.cc:2804] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:40.976179 16352 raft_consensus.cc:697] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:04:40.976282 16055 ts_tablet_manager.cc:1434] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:04:40.976965 16352 consensus_queue.cc:237] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.977566 16055 tablet_bootstrap.cc:492] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee: Bootstrap starting.
I20260709 13:04:40.973452 16284 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:40.986709 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:40.987401 16005 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 0.
I20260709 13:04:41.002396 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:41.003228 15932 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:41.004199 16440 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:41.008811 16055 tablet_bootstrap.cc:654] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:41.010684 15974 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:04:41.011512 16436 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:41.020594 16436 raft_consensus.cc:2749] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:41.015545 16284 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:41.025384 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:41.025967 16011 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:41.027246 16456 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 1 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:41.030562 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:41.031111 16005 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:41.035653 16440 consensus_queue.cc:1048] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:41.041674 16005 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:04:41.045207 16440 consensus_queue.cc:1048] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:41.053623 16011 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:04:41.055251 15827 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:41.057150 16434 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:41.057741 16434 raft_consensus.cc:697] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:04:41.058543 16434 consensus_queue.cc:237] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:41.074221 16011 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:41.075251 16122 consensus_queue.cc:1048] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:41.080135 15932 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:04:41.079746 15760 catalog_manager.cc:5697] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:41.091778 16440 consensus_queue.cc:1048] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:41.102150 15757 catalog_manager.cc:5697] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:41.111135 16055 tablet_bootstrap.cc:492] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee: No bootstrap required, opened a new log
I20260709 13:04:41.111572 16055 ts_tablet_manager.cc:1403] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent bootstrapping tablet: real 0.134s	user 0.041s	sys 0.030s
I20260709 13:04:41.114113 16055 raft_consensus.cc:359] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:41.114717 16055 raft_consensus.cc:385] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:41.114972 16055 raft_consensus.cc:740] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Initialized, Role: FOLLOWER
I20260709 13:04:41.115746 16055 consensus_queue.cc:260] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:41.118052 16055 ts_tablet_manager.cc:1434] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:04:41.132902 16440 consensus_queue.cc:1048] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:41.155937 16284 consensus_queue.cc:1048] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:41.166828 16440 consensus_queue.cc:1048] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:41.194192 16284 consensus_queue.cc:1048] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:42.401711 16489 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:04:42.420722 16489 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:42.421082 16485 consensus_queue.cc:1048] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:42.422645 16489 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:42.414535 16484 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:04:42.424523 16484 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:42.426518 16484 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:42.431699 16487 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:04:42.432376 16498 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:04:42.432367 16487 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:42.433118 16498 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:42.435026 16498 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:42.435074 16487 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:42.437649 16488 consensus_queue.cc:1048] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:42.444327 16010 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:42.454257 16480 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:04:42.454723 16480 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:42.455533 16491 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:04:42.456022 16491 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:42.458459 16491 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:42.461463 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:42.464069 16482 consensus_queue.cc:1048] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:42.465763 16480 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:42.402041 16494 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:04:42.467332 16494 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:42.477551 16495 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:04:42.478039 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:42.478089 16495 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:42.480377 16495 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:42.487231 16494 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:42.492489 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:42.493435 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:42.504069 16483 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:04:42.504587 16483 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:42.506695 16483 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:42.508041 16480 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:04:42.508605 16480 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:42.510421 16480 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:42.517632 16494 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:04:42.518201 16494 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:42.520268 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:42.520903 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:42.521698 15856 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:04:42.522665 15900 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:42.523440 16497 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:42.523813 16497 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:04:42.524163 16497 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:42.525599 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:42.528020 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:42.532088 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:42.532636 16497 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:42.533874 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:42.535817 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:42.536592 16497 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:42.538626 16504 consensus_queue.cc:1048] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:42.540019 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:42.541024 16489 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.541479 16489 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:42.543746 16489 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:42.545226 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:42.545876 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:42.548239 16483 consensus_queue.cc:1048] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:42.549660 16494 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:42.550629 15902 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:42.551409 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:42.552112 16504 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:42.553818 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:42.554345 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:42.556895 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:42.560279 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:42.560374 15902 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:42.562568 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:42.563808 16497 raft_consensus.cc:2749] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:42.572005 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:42.577551 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:42.593878 15827 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:42.595908 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:42.598924 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:42.601402 16480 raft_consensus.cc:2749] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:42.602865 16491 consensus_queue.cc:1048] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:42.609578 15825 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:42.611843 16488 raft_consensus.cc:2749] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:42.628198 15825 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:04:42.629274 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 47347.8us
I20260709 13:04:42.630689 16488 raft_consensus.cc:2749] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:42.632105 15825 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:42.633539 16488 raft_consensus.cc:2749] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:42.654510 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:42.654687 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:42.668143 15974 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:04:42.668742 16489 raft_consensus.cc:2749] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:42.676240 15974 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:04:42.676896 16489 raft_consensus.cc:2749] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:42.678125 15974 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:04:42.678733 16489 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:42.678998 15974 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:04:42.679637 16489 raft_consensus.cc:2749] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:42.681793 15974 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:04:42.682482 16508 raft_consensus.cc:2749] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:42.686560 15974 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:04:42.687222 16508 raft_consensus.cc:2749] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:42.945250 15933 raft_consensus.cc:1240] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:42.946321 16489 consensus_queue.cc:1059] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:42.947045 16495 raft_consensus.cc:3055] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:42.947340 16495 raft_consensus.cc:740] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:04:42.948096 16495 consensus_queue.cc:260] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:42.949131 16495 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:42.977396 15958 debug-util.cc:398] Leaking SignalData structure 0x7b08000b28c0 after lost signal to thread 14751
W20260709 13:04:42.978229 15958 debug-util.cc:398] Leaking SignalData structure 0x7b080010da60 after lost signal to thread 15794
W20260709 13:04:42.978994 15958 debug-util.cc:398] Leaking SignalData structure 0x7b08002d20e0 after lost signal to thread 15886
W20260709 13:04:42.979635 15958 debug-util.cc:398] Leaking SignalData structure 0x7b08001b2600 after lost signal to thread 15961
W20260709 13:04:42.980329 15958 debug-util.cc:398] Leaking SignalData structure 0x7b08002b9dc0 after lost signal to thread 16035
I20260709 13:04:43.037776 14748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:43.073518 16525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:43.073719 16524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:43.076999 14748 server_base.cc:1061] running on GCE node
W20260709 13:04:43.078516 16527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:43.079795 14748 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:43.080127 14748 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:43.080375 14748 hybrid_clock.cc:648] HybridClock initialized: now 1783602283080353 us; error 0 us; skew 500 ppm
I20260709 13:04:43.081130 14748 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:43.085139 14748 webserver.cc:533] Webserver started at http://127.14.103.4:46291/ using document root <none> and password file <none>
I20260709 13:04:43.085655 14748 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:43.085877 14748 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:43.086987 14748 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:43.088732 14748 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-3-root/instance:
uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
format_stamp: "Formatted at 2026-07-09 13:04:43 on dist-test-slave-fpp7"
I20260709 13:04:43.095083 14748 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.001s	sys 0.004s
I20260709 13:04:43.098834 16532 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:43.099638 14748 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:04:43.100047 14748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-3-root
uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
format_stamp: "Formatted at 2026-07-09 13:04:43 on dist-test-slave-fpp7"
I20260709 13:04:43.100323 14748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:43.131353 14748 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:43.133198 14748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:43.135229 14748 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:43.138813 14748 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:43.139097 14748 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:43.139407 14748 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:43.139652 14748 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:43.241420 14748 rpc_server.cc:307] RPC server started. Bound to: 127.14.103.4:43643
I20260709 13:04:43.241554 16594 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.103.4:43643 every 8 connection(s)
I20260709 13:04:43.272784 14748 auto_leader_rebalancer-test.cc:384] add a tserver: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:04:43.287032 16595 heartbeater.cc:344] Connected to a master server at 127.14.103.62:42985
I20260709 13:04:43.287456 16595 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:43.288232 16595 heartbeater.cc:507] Master 127.14.103.62:42985 requested a full tablet report, sending...
I20260709 13:04:43.290457 15759 ts_manager.cc:194] Registered new tserver with Master: eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:04:43.292268 15759 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58966
I20260709 13:04:43.609170 16009 consensus_queue.cc:237] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:43.615959 15931 raft_consensus.cc:1275] T 888cce851f074f3c969de10b02ff95ad P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:43.616958 15862 raft_consensus.cc:1275] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:43.620167 16489 consensus_queue.cc:1048] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:43.620986 16493 consensus_queue.cc:1048] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:43.639971 16489 raft_consensus.cc:2955] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
W20260709 13:04:43.648645 15976 consensus_peers.cc:597] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 888cce851f074f3c969de10b02ff95ad. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:43.650087 15862 raft_consensus.cc:2955] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:43.657305 15931 raft_consensus.cc:2955] T 888cce851f074f3c969de10b02ff95ad P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:43.669468 15933 consensus_queue.cc:237] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:43.663103 15759 catalog_manager.cc:5697] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:43.700274 15856 raft_consensus.cc:1275] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:43.701648 16494 consensus_queue.cc:1048] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:43.709301 16009 raft_consensus.cc:1275] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:43.711591 16494 consensus_queue.cc:1048] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:04:43.730098 15902 consensus_peers.cc:597] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f4908a0c5a714f5e8c8f2b20cc7f2840. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:43.740160 16599 raft_consensus.cc:2955] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:43.741621 16011 raft_consensus.cc:2955] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:43.756964 15856 raft_consensus.cc:2955] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:43.798527 15862 consensus_queue.cc:237] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:43.798452 15756 catalog_manager.cc:5697] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:43.813141 15930 raft_consensus.cc:1275] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:43.815815 16480 consensus_queue.cc:1048] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:43.822889 16011 raft_consensus.cc:1275] T 78530df4cdb742eda67781671b0431c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:43.825086 16511 consensus_queue.cc:1048] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:43.839428 16511 raft_consensus.cc:2955] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:43.859071 15759 catalog_manager.cc:5697] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:43.847702 15930 raft_consensus.cc:2955] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
W20260709 13:04:43.861845 15827 consensus_peers.cc:597] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 78530df4cdb742eda67781671b0431c6. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:43.862609 16007 raft_consensus.cc:2955] T 78530df4cdb742eda67781671b0431c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:43.873972 15856 consensus_queue.cc:237] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:43.897708 16011 raft_consensus.cc:1275] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:04:43.899610 15827 consensus_peers.cc:597] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c766f90082d54f299b03f3e9d75d2db0. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:43.901006 16511 consensus_queue.cc:1048] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:43.904954 15931 raft_consensus.cc:1275] T c766f90082d54f299b03f3e9d75d2db0 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:43.907317 16511 consensus_queue.cc:1048] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:43.919165 16511 raft_consensus.cc:2955] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:43.919569 16009 raft_consensus.cc:2955] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:43.927798 15930 raft_consensus.cc:2955] T c766f90082d54f299b03f3e9d75d2db0 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:43.949496 15759 catalog_manager.cc:5697] T c766f90082d54f299b03f3e9d75d2db0 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:44.181432 16627 ts_tablet_manager.cc:933] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:44.183328 16627 tablet_copy_client.cc:323] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.3:46251
I20260709 13:04:44.221210 16021 tablet_copy_service.cc:140] P 85e997ba5003439781ec4f1f8db0c3ee: Received BeginTabletCopySession request for tablet 888cce851f074f3c969de10b02ff95ad from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:60156)
I20260709 13:04:44.221779 16021 tablet_copy_service.cc:161] P 85e997ba5003439781ec4f1f8db0c3ee: Beginning new tablet copy session on tablet 888cce851f074f3c969de10b02ff95ad from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:60156: session id = eb1e0493af1e46dbb6dec7e25525bab5-888cce851f074f3c969de10b02ff95ad
I20260709 13:04:44.224738 16631 ts_tablet_manager.cc:933] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
I20260709 13:04:44.233911 16021 tablet_copy_source_session.cc:215] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:44.246191 16627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 888cce851f074f3c969de10b02ff95ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.254578 16633 ts_tablet_manager.cc:933] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:44.255612 16631 tablet_copy_client.cc:323] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.1:38033
I20260709 13:04:44.270735 16633 tablet_copy_client.cc:323] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.2:43009
I20260709 13:04:44.295857 16595 heartbeater.cc:499] Master 127.14.103.62:42985 was elected leader, sending a full tablet report...
I20260709 13:04:44.296294 16627 tablet_copy_client.cc:806] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:44.296948 16627 tablet_copy_client.cc:670] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:44.320796 15872 tablet_copy_service.cc:140] P 31882606e87447fb9a36bbf625129fe8: Received BeginTabletCopySession request for tablet 78530df4cdb742eda67781671b0431c6 from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:58214)
I20260709 13:04:44.321350 15872 tablet_copy_service.cc:161] P 31882606e87447fb9a36bbf625129fe8: Beginning new tablet copy session on tablet 78530df4cdb742eda67781671b0431c6 from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:58214: session id = eb1e0493af1e46dbb6dec7e25525bab5-78530df4cdb742eda67781671b0431c6
I20260709 13:04:44.329799 16626 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:04:44.330298 16626 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:44.331579 16627 tablet_copy_client.cc:538] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:44.334992 16626 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:44.335083 15947 tablet_copy_service.cc:140] P 8430db80a39d4868bcf7641c16de35ea: Received BeginTabletCopySession request for tablet f4908a0c5a714f5e8c8f2b20cc7f2840 from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:53636)
I20260709 13:04:44.337009 15947 tablet_copy_service.cc:161] P 8430db80a39d4868bcf7641c16de35ea: Beginning new tablet copy session on tablet f4908a0c5a714f5e8c8f2b20cc7f2840 from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:53636: session id = eb1e0493af1e46dbb6dec7e25525bab5-f4908a0c5a714f5e8c8f2b20cc7f2840
I20260709 13:04:44.339198 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:44.339810 16009 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:04:44.340966 15827 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:44.341609 16511 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:44.341941 16511 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:04:44.342271 16511 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:44.344061 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:44.344784 15930 raft_consensus.cc:2393] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 2: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:04:44.353444 15872 tablet_copy_source_session.cc:215] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:44.355327 16511 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:44.356626 16631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78530df4cdb742eda67781671b0431c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.357903 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:44.358660 15930 raft_consensus.cc:2393] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 2: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:04:44.361388 15947 tablet_copy_source_session.cc:215] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:44.364203 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:44.364434 16633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4908a0c5a714f5e8c8f2b20cc7f2840. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.370467 16011 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:04:44.371714 15827 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:04:44.372416 16480 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:44.372905 16480 raft_consensus.cc:697] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:04:44.373704 16480 consensus_queue.cc:237] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:44.378319 16504 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.378798 16504 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:44.379998 16633 tablet_copy_client.cc:806] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:44.380600 16633 tablet_copy_client.cc:670] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:44.380971 16504 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:44.381889 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:44.382467 16009 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2.
I20260709 13:04:44.382786 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:44.383610 15902 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:44.384842 16504 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:44.385231 16504 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:44.385648 16504 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:44.388904 16511 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:44.371883 16627 tablet_bootstrap.cc:492] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:04:44.393678 16504 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:44.396000 15757 catalog_manager.cc:5697] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 1 to 2, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:44.398481 16633 tablet_copy_client.cc:538] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:44.402263 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:44.404376 16504 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:44.406782 16633 tablet_bootstrap.cc:492] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:04:44.407272 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:44.407850 16011 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:44.426504 16631 tablet_copy_client.cc:806] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:44.432411 16631 tablet_copy_client.cc:670] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:44.434445 16011 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:04:44.437247 15902 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:04:44.438906 16631 tablet_copy_client.cc:538] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:44.439004 16490 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:44.439553 16490 raft_consensus.cc:697] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:04:44.440445 16490 consensus_queue.cc:237] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:44.464367 16631 tablet_bootstrap.cc:492] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:04:44.465345 15757 catalog_manager.cc:5697] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 2 to 3, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 3 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:44.470693 16627 tablet_bootstrap.cc:492] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:44.471714 16627 tablet_bootstrap.cc:492] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:04:44.472383 16627 ts_tablet_manager.cc:1403] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.101s	user 0.027s	sys 0.012s
I20260709 13:04:44.476398 16633 tablet_bootstrap.cc:492] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:44.476455 16627 raft_consensus.cc:359] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:44.477361 16633 tablet_bootstrap.cc:492] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:04:44.477437 16627 raft_consensus.cc:740] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:04:44.478042 16633 ts_tablet_manager.cc:1403] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.071s	user 0.023s	sys 0.010s
I20260709 13:04:44.478160 16627 consensus_queue.cc:260] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:44.485101 16633 raft_consensus.cc:359] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:44.485894 16633 raft_consensus.cc:740] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:04:44.486492 16633 consensus_queue.cc:260] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:44.496913 16633 ts_tablet_manager.cc:1434] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.019s	user 0.004s	sys 0.004s
I20260709 13:04:44.505832 15947 tablet_copy_service.cc:342] P 8430db80a39d4868bcf7641c16de35ea: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-f4908a0c5a714f5e8c8f2b20cc7f2840 received from {username='slave'} at 127.0.0.1:53636
I20260709 13:04:44.506469 15947 tablet_copy_service.cc:434] P 8430db80a39d4868bcf7641c16de35ea: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-f4908a0c5a714f5e8c8f2b20cc7f2840 on tablet f4908a0c5a714f5e8c8f2b20cc7f2840 with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:04:44.516700 16633 ts_tablet_manager.cc:933] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
I20260709 13:04:44.518419 16627 ts_tablet_manager.cc:1434] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.046s	user 0.014s	sys 0.008s
I20260709 13:04:44.525309 16633 tablet_copy_client.cc:323] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.1:38033
I20260709 13:04:44.526932 15872 tablet_copy_service.cc:140] P 31882606e87447fb9a36bbf625129fe8: Received BeginTabletCopySession request for tablet c766f90082d54f299b03f3e9d75d2db0 from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:58214)
I20260709 13:04:44.527462 15872 tablet_copy_service.cc:161] P 31882606e87447fb9a36bbf625129fe8: Beginning new tablet copy session on tablet c766f90082d54f299b03f3e9d75d2db0 from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:58214: session id = eb1e0493af1e46dbb6dec7e25525bab5-c766f90082d54f299b03f3e9d75d2db0
I20260709 13:04:44.532076 16021 tablet_copy_service.cc:342] P 85e997ba5003439781ec4f1f8db0c3ee: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-888cce851f074f3c969de10b02ff95ad received from {username='slave'} at 127.0.0.1:60156
I20260709 13:04:44.532552 16021 tablet_copy_service.cc:434] P 85e997ba5003439781ec4f1f8db0c3ee: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-888cce851f074f3c969de10b02ff95ad on tablet 888cce851f074f3c969de10b02ff95ad with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:04:44.543004 15872 tablet_copy_source_session.cc:215] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:44.546532 16633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c766f90082d54f299b03f3e9d75d2db0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.548854 16631 tablet_bootstrap.cc:492] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:44.549633 16631 tablet_bootstrap.cc:492] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:04:44.550153 16631 ts_tablet_manager.cc:1403] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.086s	user 0.013s	sys 0.020s
I20260709 13:04:44.552686 16631 raft_consensus.cc:359] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:44.557768 16631 raft_consensus.cc:740] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:04:44.558405 16631 consensus_queue.cc:260] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:44.560768 16631 ts_tablet_manager.cc:1434] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.001s
I20260709 13:04:44.562625 15872 tablet_copy_service.cc:342] P 31882606e87447fb9a36bbf625129fe8: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-78530df4cdb742eda67781671b0431c6 received from {username='slave'} at 127.0.0.1:58214
I20260709 13:04:44.563102 15872 tablet_copy_service.cc:434] P 31882606e87447fb9a36bbf625129fe8: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-78530df4cdb742eda67781671b0431c6 on tablet 78530df4cdb742eda67781671b0431c6 with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:04:44.564777 16633 tablet_copy_client.cc:806] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:44.565361 16633 tablet_copy_client.cc:670] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:44.571609 16633 tablet_copy_client.cc:538] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:44.581202 16633 tablet_bootstrap.cc:492] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:04:44.620694 16633 tablet_bootstrap.cc:492] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:44.621645 16633 tablet_bootstrap.cc:492] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:04:44.622342 16633 ts_tablet_manager.cc:1403] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.041s	user 0.019s	sys 0.002s
I20260709 13:04:44.625368 16633 raft_consensus.cc:359] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:44.626304 16633 raft_consensus.cc:740] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:04:44.627044 16633 consensus_queue.cc:260] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:44.636016 16633 ts_tablet_manager.cc:1434] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.013s	user 0.010s	sys 0.000s
I20260709 13:04:44.644143 15872 tablet_copy_service.cc:342] P 31882606e87447fb9a36bbf625129fe8: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-c766f90082d54f299b03f3e9d75d2db0 received from {username='slave'} at 127.0.0.1:58214
I20260709 13:04:44.644629 15872 tablet_copy_service.cc:434] P 31882606e87447fb9a36bbf625129fe8: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-c766f90082d54f299b03f3e9d75d2db0 on tablet c766f90082d54f299b03f3e9d75d2db0 with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:04:44.692751 16570 raft_consensus.cc:1217] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:44.752633 16570 raft_consensus.cc:1217] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:44.763175 16570 raft_consensus.cc:1217] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:44.859009 15930 raft_consensus.cc:1240] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:44.863324 16483 consensus_queue.cc:1059] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:44.864071 16483 consensus_queue.cc:1243] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer 8430db80a39d4868bcf7641c16de35ea log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:04:44.865298 16483 raft_consensus.cc:3055] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:44.865629 16483 raft_consensus.cc:740] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:04:44.866329 16483 consensus_queue.cc:260] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:44.867452 16483 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:44.946237 16570 raft_consensus.cc:1217] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:45.016201 15856 raft_consensus.cc:1275] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:04:45.016598 15856 pending_rounds.cc:85] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8: Aborting all ops after (but not including) 1
I20260709 13:04:45.016872 15856 pending_rounds.cc:107] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:04:45.017163 15856 raft_consensus.cc:2889] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:45.018959 16494 consensus_queue.cc:1048] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:45.045713 16011 raft_consensus.cc:1275] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:04:45.047290 16494 consensus_queue.cc:1048] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:45.147048 16504 raft_consensus.cc:1064] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:04:45.149988 16504 consensus_queue.cc:237] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:04:45.156003 16489 raft_consensus.cc:1064] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:04:45.168253 15862 raft_consensus.cc:1275] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:45.172535 16504 consensus_queue.cc:1048] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:04:45.185556 16626 raft_consensus.cc:1064] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:04:45.188675 16626 consensus_queue.cc:237] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:04:45.205526 16570 raft_consensus.cc:1275] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:45.209065 16011 raft_consensus.cc:1275] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:45.203150 16489 consensus_queue.cc:237] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:04:45.260170 16504 consensus_queue.cc:1048] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:45.264019 16504 consensus_queue.cc:1048] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:45.271343 16570 raft_consensus.cc:1275] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:45.274600 16480 consensus_queue.cc:1048] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:45.279918 15931 raft_consensus.cc:1275] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:45.281778 16480 consensus_queue.cc:1048] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:45.282908 16007 raft_consensus.cc:1275] T 78530df4cdb742eda67781671b0431c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:45.286397 16480 consensus_queue.cc:1048] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:45.310874 16569 raft_consensus.cc:1275] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:45.311748 15856 raft_consensus.cc:1275] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:45.312543 16493 consensus_queue.cc:1048] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:45.314806 16493 consensus_queue.cc:1048] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:45.324921 16010 raft_consensus.cc:2955] T 78530df4cdb742eda67781671b0431c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:45.344547 15759 catalog_manager.cc:5697] T 78530df4cdb742eda67781671b0431c6 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:45.320264 16483 raft_consensus.cc:2955] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:45.360122 15856 raft_consensus.cc:2955] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:45.359876 16494 raft_consensus.cc:2955] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:45.360144 16646 raft_consensus.cc:2955] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:45.366158 15862 raft_consensus.cc:2955] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:45.370296 15930 raft_consensus.cc:1275] T 888cce851f074f3c969de10b02ff95ad P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:45.377303 16569 raft_consensus.cc:2955] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:45.393532 16666 raft_consensus.cc:1064] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:04:45.396951 16489 consensus_queue.cc:1048] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:45.446065 16569 raft_consensus.cc:2955] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:45.448120 15931 raft_consensus.cc:2955] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:45.449084 15757 catalog_manager.cc:5697] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:45.396227 16666 consensus_queue.cc:237] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:04:45.448313 16570 raft_consensus.cc:2955] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:45.494673 16010 raft_consensus.cc:1275] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:45.495965 16570 raft_consensus.cc:1275] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:45.497262 16668 consensus_queue.cc:1048] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:45.498870 16626 consensus_queue.cc:1048] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:45.500620 15930 raft_consensus.cc:1275] T c766f90082d54f299b03f3e9d75d2db0 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:45.503821 16668 consensus_queue.cc:1048] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:45.514441 16007 raft_consensus.cc:2955] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:45.537576 16626 raft_consensus.cc:2955] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:45.539724 16010 raft_consensus.cc:2955] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:45.550309 16568 raft_consensus.cc:2955] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:45.565388 15930 raft_consensus.cc:2955] T 888cce851f074f3c969de10b02ff95ad P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:45.640576 15933 raft_consensus.cc:2955] T c766f90082d54f299b03f3e9d75d2db0 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:45.651348 15756 catalog_manager.cc:5697] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 reported cstate change: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:45.683323 15759 catalog_manager.cc:5697] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:45.711563 15933 consensus_queue.cc:237] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:04:45.718438 15862 consensus_queue.cc:237] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:04:45.720045 16568 raft_consensus.cc:1275] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:45.723068 16010 raft_consensus.cc:1275] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:45.724258 16670 consensus_queue.cc:1048] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:45.727241 16670 consensus_queue.cc:1048] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:45.727664 16011 consensus_queue.cc:237] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:04:45.737350 16568 raft_consensus.cc:2955] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:45.731068 15856 consensus_queue.cc:237] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:04:45.738477 16010 raft_consensus.cc:2955] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:45.734647 16490 raft_consensus.cc:2955] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:45.750736 15756 catalog_manager.cc:5697] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:45.761443 16570 raft_consensus.cc:1275] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:45.764627 16570 raft_consensus.cc:1275] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:45.765583 15858 raft_consensus.cc:1275] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:45.765483 16626 consensus_queue.cc:1048] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:45.768302 16493 consensus_queue.cc:1048] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:45.768760 16009 raft_consensus.cc:1275] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:45.770027 16569 raft_consensus.cc:1275] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:45.774706 16493 consensus_queue.cc:1048] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:45.776254 15846 tablet_service.cc:1558] Processing DeleteTablet for tablet f4908a0c5a714f5e8c8f2b20cc7f2840 with delete_type TABLET_DATA_TOMBSTONED (TS 31882606e87447fb9a36bbf625129fe8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43504
I20260709 13:04:45.775904 16489 raft_consensus.cc:2955] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:45.780978 16666 consensus_queue.cc:1048] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:45.788030 15936 raft_consensus.cc:1275] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:45.791584 15744 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f4908a0c5a714f5e8c8f2b20cc7f2840 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:45.791790 16668 consensus_queue.cc:1048] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:45.792068 16569 raft_consensus.cc:2955] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:45.799889 15746 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 888cce851f074f3c969de10b02ff95ad with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:45.800364 16678 tablet_replica.cc:333] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:04:45.800845 16668 consensus_queue.cc:1048] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:45.804324 16667 raft_consensus.cc:2955] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:45.817512 16570 raft_consensus.cc:2955] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:45.831910 15933 raft_consensus.cc:2955] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:45.846392 15756 catalog_manager.cc:5697] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:04:45.870493 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 92365.1us
I20260709 13:04:45.905511 15995 tablet_service.cc:1558] Processing DeleteTablet for tablet 78530df4cdb742eda67781671b0431c6 with delete_type TABLET_DATA_TOMBSTONED (TS 85e997ba5003439781ec4f1f8db0c3ee not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41648
I20260709 13:04:45.908330 15757 catalog_manager.cc:5697] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:45.923177 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 67763.5us
I20260709 13:04:45.897091 16664 raft_consensus.cc:2955] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:04:45.964862 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 108554us
I20260709 13:04:45.966670 16678 raft_consensus.cc:2243] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:45.967404 16678 raft_consensus.cc:2272] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:45.978360 16678 ts_tablet_manager.cc:1916] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 31882606e87447fb9a36bbf625129fe8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:45.982878 15744 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 78530df4cdb742eda67781671b0431c6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:45.989562 15862 raft_consensus.cc:2955] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:46.000967 16010 raft_consensus.cc:2955] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:46.008169 15923 tablet_service.cc:1558] Processing DeleteTablet for tablet 888cce851f074f3c969de10b02ff95ad with delete_type TABLET_DATA_TOMBSTONED (TS 8430db80a39d4868bcf7641c16de35ea not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58386
I20260709 13:04:46.014202 15744 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c766f90082d54f299b03f3e9d75d2db0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:46.022133 16685 tablet_replica.cc:333] T 78530df4cdb742eda67781671b0431c6 P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:04:46.024358 16569 raft_consensus.cc:2955] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:46.023432 16685 raft_consensus.cc:2243] T 78530df4cdb742eda67781671b0431c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:46.040077 16685 raft_consensus.cc:2272] T 78530df4cdb742eda67781671b0431c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:46.044163 16686 tablet_replica.cc:333] T 888cce851f074f3c969de10b02ff95ad P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:04:46.045158 15759 catalog_manager.cc:5697] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:46.053253 16685 ts_tablet_manager.cc:1916] T 78530df4cdb742eda67781671b0431c6 P 85e997ba5003439781ec4f1f8db0c3ee: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:46.060380 16686 raft_consensus.cc:2243] T 888cce851f074f3c969de10b02ff95ad P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:46.061094 16686 raft_consensus.cc:2272] T 888cce851f074f3c969de10b02ff95ad P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:46.071542 16678 ts_tablet_manager.cc:1929] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 31882606e87447fb9a36bbf625129fe8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:04:46.072090 16678 log.cc:1199] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 31882606e87447fb9a36bbf625129fe8: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-0-root/wals/f4908a0c5a714f5e8c8f2b20cc7f2840
I20260709 13:04:46.072489 15923 tablet_service.cc:1558] Processing DeleteTablet for tablet c766f90082d54f299b03f3e9d75d2db0 with delete_type TABLET_DATA_TOMBSTONED (TS 8430db80a39d4868bcf7641c16de35ea not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58386
I20260709 13:04:46.073647 15744 catalog_manager.cc:5028] TS 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): tablet f4908a0c5a714f5e8c8f2b20cc7f2840 (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:04:46.076139 16686 ts_tablet_manager.cc:1916] T 888cce851f074f3c969de10b02ff95ad P 8430db80a39d4868bcf7641c16de35ea: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:46.094463 16685 ts_tablet_manager.cc:1929] T 78530df4cdb742eda67781671b0431c6 P 85e997ba5003439781ec4f1f8db0c3ee: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:04:46.094954 16685 log.cc:1199] T 78530df4cdb742eda67781671b0431c6 P 85e997ba5003439781ec4f1f8db0c3ee: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-2-root/wals/78530df4cdb742eda67781671b0431c6
I20260709 13:04:46.098721 16686 ts_tablet_manager.cc:1929] T 888cce851f074f3c969de10b02ff95ad P 8430db80a39d4868bcf7641c16de35ea: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:04:46.100663 16686 log.cc:1199] T 888cce851f074f3c969de10b02ff95ad P 8430db80a39d4868bcf7641c16de35ea: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-1-root/wals/888cce851f074f3c969de10b02ff95ad
I20260709 13:04:46.102309 15746 catalog_manager.cc:5028] TS 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): tablet 78530df4cdb742eda67781671b0431c6 (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:04:46.103787 16686 tablet_replica.cc:333] T c766f90082d54f299b03f3e9d75d2db0 P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:04:46.103760 15744 catalog_manager.cc:5028] TS 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): tablet 888cce851f074f3c969de10b02ff95ad (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:04:46.105094 16686 raft_consensus.cc:2243] T c766f90082d54f299b03f3e9d75d2db0 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:46.105793 16686 raft_consensus.cc:2272] T c766f90082d54f299b03f3e9d75d2db0 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:46.110226 16686 ts_tablet_manager.cc:1916] T c766f90082d54f299b03f3e9d75d2db0 P 8430db80a39d4868bcf7641c16de35ea: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:46.123807 16686 ts_tablet_manager.cc:1929] T c766f90082d54f299b03f3e9d75d2db0 P 8430db80a39d4868bcf7641c16de35ea: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:04:46.124122 16686 log.cc:1199] T c766f90082d54f299b03f3e9d75d2db0 P 8430db80a39d4868bcf7641c16de35ea: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-1-root/wals/c766f90082d54f299b03f3e9d75d2db0
I20260709 13:04:46.125303 15744 catalog_manager.cc:5028] TS 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): tablet c766f90082d54f299b03f3e9d75d2db0 (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:04:47.141705 15933 consensus_queue.cc:237] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:47.164602 16010 raft_consensus.cc:1275] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:04:47.168627 15902 consensus_peers.cc:597] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f8b7d73618ae489990e21e8cc312177d. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:47.169497 15862 raft_consensus.cc:1275] T f8b7d73618ae489990e21e8cc312177d P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:47.170929 16688 consensus_queue.cc:1048] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:47.172931 16490 consensus_queue.cc:1048] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:47.187608 16490 raft_consensus.cc:2955] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:47.190582 16011 raft_consensus.cc:2955] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:47.192951 15862 raft_consensus.cc:2955] T f8b7d73618ae489990e21e8cc312177d P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:47.213511 15859 consensus_queue.cc:237] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:47.234882 15759 catalog_manager.cc:5697] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea reported cstate change: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:47.262979 15827 consensus_peers.cc:597] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4e1a3c90a16e428fb2b328aff66b4ea6. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:47.264933 16011 raft_consensus.cc:1275] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:47.267273 15933 raft_consensus.cc:1275] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:47.272701 16626 consensus_queue.cc:1048] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:47.278414 16699 consensus_queue.cc:1048] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:47.290033 16626 raft_consensus.cc:2955] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:47.293774 16010 raft_consensus.cc:2955] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:47.302822 15930 raft_consensus.cc:2955] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:47.340871 15859 consensus_queue.cc:237] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:47.355329 15757 catalog_manager.cc:5697] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:47.379016 16010 raft_consensus.cc:1275] T 260d22f804a345ed8f055536994fcb14 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:47.383044 16626 consensus_queue.cc:1048] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:04:47.384783 15827 consensus_peers.cc:597] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 260d22f804a345ed8f055536994fcb14. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:47.386071 15930 raft_consensus.cc:1275] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:47.387282 16626 consensus_queue.cc:1048] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:47.391264 16701 raft_consensus.cc:2955] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:47.394307 16010 raft_consensus.cc:2955] T 260d22f804a345ed8f055536994fcb14 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:47.396503 15931 raft_consensus.cc:2955] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
W20260709 13:04:47.459741 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 44462.6us
I20260709 13:04:47.494375 15857 consensus_queue.cc:237] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:47.497084 15759 catalog_manager.cc:5697] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:47.504496 16007 raft_consensus.cc:1275] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:04:47.504688 15827 consensus_peers.cc:597] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7b6e87097733428bb2167c6f308fe589. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:47.505960 15931 raft_consensus.cc:1275] T 7b6e87097733428bb2167c6f308fe589 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:47.507524 16701 consensus_queue.cc:1048] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:47.508332 16700 consensus_queue.cc:1048] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:47.519032 16700 raft_consensus.cc:2955] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:47.523080 15930 raft_consensus.cc:2955] T 7b6e87097733428bb2167c6f308fe589 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:47.524138 16007 raft_consensus.cc:2955] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:47.553642 15759 catalog_manager.cc:5697] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:47.660908 16714 ts_tablet_manager.cc:933] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:47.662714 16714 tablet_copy_client.cc:323] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.2:43009
I20260709 13:04:47.664762 15947 tablet_copy_service.cc:140] P 8430db80a39d4868bcf7641c16de35ea: Received BeginTabletCopySession request for tablet f8b7d73618ae489990e21e8cc312177d from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:53636)
I20260709 13:04:47.665266 15947 tablet_copy_service.cc:161] P 8430db80a39d4868bcf7641c16de35ea: Beginning new tablet copy session on tablet f8b7d73618ae489990e21e8cc312177d from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:53636: session id = eb1e0493af1e46dbb6dec7e25525bab5-f8b7d73618ae489990e21e8cc312177d
I20260709 13:04:47.670441 15947 tablet_copy_source_session.cc:215] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:47.672940 16714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8b7d73618ae489990e21e8cc312177d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:47.683518 16714 tablet_copy_client.cc:806] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:47.684029 16714 tablet_copy_client.cc:670] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:47.687906 16714 tablet_copy_client.cc:538] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:47.697495 16714 tablet_bootstrap.cc:492] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:04:47.714761 16714 tablet_bootstrap.cc:492] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:47.715600 16714 tablet_bootstrap.cc:492] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:04:47.716259 16714 ts_tablet_manager.cc:1403] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.008s
I20260709 13:04:47.719038 16714 raft_consensus.cc:359] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:47.719820 16714 raft_consensus.cc:740] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:04:47.720741 16714 consensus_queue.cc:260] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:47.724048 16714 ts_tablet_manager.cc:1434] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 13:04:47.726898 15947 tablet_copy_service.cc:342] P 8430db80a39d4868bcf7641c16de35ea: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-f8b7d73618ae489990e21e8cc312177d received from {username='slave'} at 127.0.0.1:53636
I20260709 13:04:47.727334 15947 tablet_copy_service.cc:434] P 8430db80a39d4868bcf7641c16de35ea: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-f8b7d73618ae489990e21e8cc312177d on tablet f8b7d73618ae489990e21e8cc312177d with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:04:47.780375 16714 ts_tablet_manager.cc:933] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
I20260709 13:04:47.781879 16714 tablet_copy_client.cc:323] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.1:38033
I20260709 13:04:47.783337 15872 tablet_copy_service.cc:140] P 31882606e87447fb9a36bbf625129fe8: Received BeginTabletCopySession request for tablet 4e1a3c90a16e428fb2b328aff66b4ea6 from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:58214)
I20260709 13:04:47.783842 15872 tablet_copy_service.cc:161] P 31882606e87447fb9a36bbf625129fe8: Beginning new tablet copy session on tablet 4e1a3c90a16e428fb2b328aff66b4ea6 from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:58214: session id = eb1e0493af1e46dbb6dec7e25525bab5-4e1a3c90a16e428fb2b328aff66b4ea6
I20260709 13:04:47.788633 15872 tablet_copy_source_session.cc:215] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:47.791247 16714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e1a3c90a16e428fb2b328aff66b4ea6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:47.806126 16714 tablet_copy_client.cc:806] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:47.806674 16714 tablet_copy_client.cc:670] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:47.811430 16714 tablet_copy_client.cc:538] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:47.820741 16714 tablet_bootstrap.cc:492] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:04:47.837505 16714 tablet_bootstrap.cc:492] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:47.838265 16714 tablet_bootstrap.cc:492] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:04:47.838892 16714 ts_tablet_manager.cc:1403] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.003s
I20260709 13:04:47.841503 16714 raft_consensus.cc:359] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:47.842113 16714 raft_consensus.cc:740] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:04:47.842762 16714 consensus_queue.cc:260] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:47.845024 16714 ts_tablet_manager.cc:1434] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:04:47.846764 15872 tablet_copy_service.cc:342] P 31882606e87447fb9a36bbf625129fe8: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-4e1a3c90a16e428fb2b328aff66b4ea6 received from {username='slave'} at 127.0.0.1:58214
I20260709 13:04:47.847157 15872 tablet_copy_service.cc:434] P 31882606e87447fb9a36bbf625129fe8: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-4e1a3c90a16e428fb2b328aff66b4ea6 on tablet 4e1a3c90a16e428fb2b328aff66b4ea6 with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:04:47.865068 16714 ts_tablet_manager.cc:933] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
I20260709 13:04:47.867126 16714 tablet_copy_client.cc:323] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.1:38033
I20260709 13:04:47.868640 15872 tablet_copy_service.cc:140] P 31882606e87447fb9a36bbf625129fe8: Received BeginTabletCopySession request for tablet 260d22f804a345ed8f055536994fcb14 from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:58214)
I20260709 13:04:47.869158 15872 tablet_copy_service.cc:161] P 31882606e87447fb9a36bbf625129fe8: Beginning new tablet copy session on tablet 260d22f804a345ed8f055536994fcb14 from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:58214: session id = eb1e0493af1e46dbb6dec7e25525bab5-260d22f804a345ed8f055536994fcb14
I20260709 13:04:47.876264 15872 tablet_copy_source_session.cc:215] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:47.878772 16714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 260d22f804a345ed8f055536994fcb14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:47.895013 16714 tablet_copy_client.cc:806] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:47.895506 16714 tablet_copy_client.cc:670] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:47.899263 16714 tablet_copy_client.cc:538] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:47.907498 16714 tablet_bootstrap.cc:492] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:04:47.924134 16714 tablet_bootstrap.cc:492] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:47.924896 16714 tablet_bootstrap.cc:492] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:04:47.925768 16714 ts_tablet_manager.cc:1403] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.018s	user 0.013s	sys 0.008s
I20260709 13:04:47.928859 16714 raft_consensus.cc:359] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:47.929575 16714 raft_consensus.cc:740] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:04:47.930119 16714 consensus_queue.cc:260] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:47.934062 16714 ts_tablet_manager.cc:1434] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:04:47.935559 15872 tablet_copy_service.cc:342] P 31882606e87447fb9a36bbf625129fe8: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-260d22f804a345ed8f055536994fcb14 received from {username='slave'} at 127.0.0.1:58214
I20260709 13:04:47.935976 15872 tablet_copy_service.cc:434] P 31882606e87447fb9a36bbf625129fe8: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-260d22f804a345ed8f055536994fcb14 on tablet 260d22f804a345ed8f055536994fcb14 with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:04:48.162148 16714 ts_tablet_manager.cc:933] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
I20260709 13:04:48.169566 16569 raft_consensus.cc:1217] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:48.226191 16714 tablet_copy_client.cc:323] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.1:38033
I20260709 13:04:48.228861 15872 tablet_copy_service.cc:140] P 31882606e87447fb9a36bbf625129fe8: Received BeginTabletCopySession request for tablet 7b6e87097733428bb2167c6f308fe589 from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:58214)
I20260709 13:04:48.229676 15872 tablet_copy_service.cc:161] P 31882606e87447fb9a36bbf625129fe8: Beginning new tablet copy session on tablet 7b6e87097733428bb2167c6f308fe589 from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:58214: session id = eb1e0493af1e46dbb6dec7e25525bab5-7b6e87097733428bb2167c6f308fe589
I20260709 13:04:48.245404 15872 tablet_copy_source_session.cc:215] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:04:48.246023 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 61450.4us
I20260709 13:04:48.271292 16714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b6e87097733428bb2167c6f308fe589. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:48.275277 16569 raft_consensus.cc:1217] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:48.319444 16714 tablet_copy_client.cc:806] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:48.320458 16714 tablet_copy_client.cc:670] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:48.329970 16714 tablet_copy_client.cc:538] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:48.340942 16714 tablet_bootstrap.cc:492] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:04:48.364432 16714 tablet_bootstrap.cc:492] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:48.365347 16714 tablet_bootstrap.cc:492] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:04:48.365980 16714 ts_tablet_manager.cc:1403] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.025s	user 0.016s	sys 0.003s
I20260709 13:04:48.368966 16714 raft_consensus.cc:359] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:48.370090 16714 raft_consensus.cc:740] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:04:48.370704 16714 consensus_queue.cc:260] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:48.375176 16714 ts_tablet_manager.cc:1434] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:04:48.377671 15872 tablet_copy_service.cc:342] P 31882606e87447fb9a36bbf625129fe8: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-7b6e87097733428bb2167c6f308fe589 received from {username='slave'} at 127.0.0.1:58214
I20260709 13:04:48.378140 15872 tablet_copy_service.cc:434] P 31882606e87447fb9a36bbf625129fe8: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-7b6e87097733428bb2167c6f308fe589 on tablet 7b6e87097733428bb2167c6f308fe589 with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:04:48.386073 16569 raft_consensus.cc:1217] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:48.536687 16723 raft_consensus.cc:1064] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:04:48.543350 16723 consensus_queue.cc:237] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:04:48.555503 16569 raft_consensus.cc:1275] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:48.557763 16010 raft_consensus.cc:1275] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:48.558048 15862 raft_consensus.cc:1275] T f8b7d73618ae489990e21e8cc312177d P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:48.557670 16723 consensus_queue.cc:1048] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:48.559883 16730 consensus_queue.cc:1048] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:48.561226 16691 consensus_queue.cc:1048] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:48.577500 16730 raft_consensus.cc:2955] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:48.590214 16569 raft_consensus.cc:2955] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:48.588572 16010 raft_consensus.cc:2955] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:48.607653 15862 raft_consensus.cc:2955] T f8b7d73618ae489990e21e8cc312177d P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:48.619951 15756 catalog_manager.cc:5697] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:48.652215 15931 consensus_queue.cc:237] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:04:48.656719 16569 raft_consensus.cc:1275] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:48.658668 16691 consensus_queue.cc:1048] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:48.662858 16010 raft_consensus.cc:1275] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:48.664726 16691 consensus_queue.cc:1048] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:48.667572 16730 raft_consensus.cc:2955] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:48.669783 16569 raft_consensus.cc:2955] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:48.672612 16007 raft_consensus.cc:2955] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:48.677803 15744 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f8b7d73618ae489990e21e8cc312177d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:48.684198 16570 raft_consensus.cc:1217] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:48.691610 16700 raft_consensus.cc:1064] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:04:48.692768 15757 catalog_manager.cc:5697] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea reported cstate change: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:48.694226 16700 consensus_queue.cc:237] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:04:48.711320 15846 tablet_service.cc:1558] Processing DeleteTablet for tablet f8b7d73618ae489990e21e8cc312177d with delete_type TABLET_DATA_TOMBSTONED (TS 31882606e87447fb9a36bbf625129fe8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43504
I20260709 13:04:48.725944 16743 tablet_replica.cc:333] T f8b7d73618ae489990e21e8cc312177d P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:04:48.727056 16743 raft_consensus.cc:2243] T f8b7d73618ae489990e21e8cc312177d P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:48.730244 15931 raft_consensus.cc:1275] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:48.731753 16728 consensus_queue.cc:1048] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:48.733624 16009 raft_consensus.cc:1275] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:48.738914 16569 raft_consensus.cc:1275] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:48.739528 16700 consensus_queue.cc:1048] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:48.740200 16743 raft_consensus.cc:2272] T f8b7d73618ae489990e21e8cc312177d P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:48.743867 16728 consensus_queue.cc:1048] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:48.745026 16743 ts_tablet_manager.cc:1916] T f8b7d73618ae489990e21e8cc312177d P 31882606e87447fb9a36bbf625129fe8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:48.766338 16743 ts_tablet_manager.cc:1929] T f8b7d73618ae489990e21e8cc312177d P 31882606e87447fb9a36bbf625129fe8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:04:48.766788 16743 log.cc:1199] T f8b7d73618ae489990e21e8cc312177d P 31882606e87447fb9a36bbf625129fe8: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-0-root/wals/f8b7d73618ae489990e21e8cc312177d
I20260709 13:04:48.770345 15744 catalog_manager.cc:5028] TS 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): tablet f8b7d73618ae489990e21e8cc312177d (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:04:48.775303 16701 raft_consensus.cc:2955] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:48.776965 16007 raft_consensus.cc:2955] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:48.784873 16728 raft_consensus.cc:1064] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:04:48.784298 16569 raft_consensus.cc:2955] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:48.787369 16728 consensus_queue.cc:237] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:04:48.790247 15931 raft_consensus.cc:2955] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:48.801525 15759 catalog_manager.cc:5697] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:48.835657 15856 consensus_queue.cc:237] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:04:48.859890 16009 raft_consensus.cc:1275] T 260d22f804a345ed8f055536994fcb14 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:48.860636 16569 raft_consensus.cc:1275] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:48.859913 16570 raft_consensus.cc:1275] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:48.867601 16700 consensus_queue.cc:1048] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:48.868294 16729 consensus_queue.cc:1048] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:48.867627 15930 raft_consensus.cc:1275] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:48.869751 16700 consensus_queue.cc:1048] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:48.870540 16701 consensus_queue.cc:1048] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:48.867591 15931 raft_consensus.cc:1275] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:48.873939 16701 consensus_queue.cc:1048] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:48.890903 16701 raft_consensus.cc:2955] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:48.894380 16569 raft_consensus.cc:2955] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:48.900498 15933 raft_consensus.cc:2955] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:48.908039 16729 raft_consensus.cc:2955] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:48.912215 15744 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4e1a3c90a16e428fb2b328aff66b4ea6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:48.918205 15759 catalog_manager.cc:5697] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:48.912007 15931 raft_consensus.cc:2955] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:48.933457 16009 raft_consensus.cc:2955] T 260d22f804a345ed8f055536994fcb14 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:48.937500 15995 tablet_service.cc:1558] Processing DeleteTablet for tablet 4e1a3c90a16e428fb2b328aff66b4ea6 with delete_type TABLET_DATA_TOMBSTONED (TS 85e997ba5003439781ec4f1f8db0c3ee not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41648
I20260709 13:04:48.955039 16755 tablet_replica.cc:333] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:04:48.956228 16755 raft_consensus.cc:2243] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:48.957063 16755 raft_consensus.cc:2272] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:48.956516 15759 catalog_manager.cc:5697] T 260d22f804a345ed8f055536994fcb14 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:48.970538 16569 raft_consensus.cc:2955] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:48.973311 16755 ts_tablet_manager.cc:1916] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 85e997ba5003439781ec4f1f8db0c3ee: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:49.001511 15862 consensus_queue.cc:237] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:04:49.007978 16755 ts_tablet_manager.cc:1929] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 85e997ba5003439781ec4f1f8db0c3ee: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:04:49.008477 16755 log.cc:1199] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 85e997ba5003439781ec4f1f8db0c3ee: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-2-root/wals/4e1a3c90a16e428fb2b328aff66b4ea6
I20260709 13:04:49.010519 15746 catalog_manager.cc:5028] TS 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): tablet 4e1a3c90a16e428fb2b328aff66b4ea6 (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:04:49.013800 15931 raft_consensus.cc:1275] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:49.015149 16752 consensus_queue.cc:1048] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:49.019841 16569 raft_consensus.cc:1275] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:49.021660 16754 consensus_queue.cc:1048] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:49.021463 16729 raft_consensus.cc:2955] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:49.023363 15931 raft_consensus.cc:2955] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:49.027050 16569 raft_consensus.cc:2955] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:49.032896 15744 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 260d22f804a345ed8f055536994fcb14 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:49.043061 15759 catalog_manager.cc:5697] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:49.053532 15995 tablet_service.cc:1558] Processing DeleteTablet for tablet 260d22f804a345ed8f055536994fcb14 with delete_type TABLET_DATA_TOMBSTONED (TS 85e997ba5003439781ec4f1f8db0c3ee not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41648
I20260709 13:04:49.054589 16755 tablet_replica.cc:333] T 260d22f804a345ed8f055536994fcb14 P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:04:49.055557 16755 raft_consensus.cc:2243] T 260d22f804a345ed8f055536994fcb14 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:49.056334 16755 raft_consensus.cc:2272] T 260d22f804a345ed8f055536994fcb14 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:49.059453 16755 ts_tablet_manager.cc:1916] T 260d22f804a345ed8f055536994fcb14 P 85e997ba5003439781ec4f1f8db0c3ee: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:49.075210 16755 ts_tablet_manager.cc:1929] T 260d22f804a345ed8f055536994fcb14 P 85e997ba5003439781ec4f1f8db0c3ee: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:04:49.075556 16755 log.cc:1199] T 260d22f804a345ed8f055536994fcb14 P 85e997ba5003439781ec4f1f8db0c3ee: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-2-root/wals/260d22f804a345ed8f055536994fcb14
I20260709 13:04:49.076841 15746 catalog_manager.cc:5028] TS 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): tablet 260d22f804a345ed8f055536994fcb14 (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:04:49.111135 16729 raft_consensus.cc:1064] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:04:49.113584 16729 consensus_queue.cc:237] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:04:49.128211 16569 raft_consensus.cc:1275] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:49.129463 16752 consensus_queue.cc:1048] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:49.131853 15931 raft_consensus.cc:1275] T 7b6e87097733428bb2167c6f308fe589 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:49.133337 16009 raft_consensus.cc:1275] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:49.133471 16752 consensus_queue.cc:1048] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:49.134951 16729 consensus_queue.cc:1048] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:49.147732 16729 raft_consensus.cc:2955] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:49.154498 16009 raft_consensus.cc:2955] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:49.164155 16569 raft_consensus.cc:2955] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:49.177485 15757 catalog_manager.cc:5697] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:49.191308 15930 raft_consensus.cc:2955] T 7b6e87097733428bb2167c6f308fe589 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:49.206812 15862 consensus_queue.cc:237] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:04:49.215051 16569 raft_consensus.cc:1275] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:49.215489 16009 raft_consensus.cc:1275] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:49.216420 16728 consensus_queue.cc:1048] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:49.218221 16752 consensus_queue.cc:1048] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:49.222713 16752 raft_consensus.cc:2955] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:49.227540 16009 raft_consensus.cc:2955] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:49.236138 15744 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7b6e87097733428bb2167c6f308fe589 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:49.234591 16569 raft_consensus.cc:2955] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:49.245864 15759 catalog_manager.cc:5697] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:49.257301 15923 tablet_service.cc:1558] Processing DeleteTablet for tablet 7b6e87097733428bb2167c6f308fe589 with delete_type TABLET_DATA_TOMBSTONED (TS 8430db80a39d4868bcf7641c16de35ea not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58386
I20260709 13:04:49.264786 16760 tablet_replica.cc:333] T 7b6e87097733428bb2167c6f308fe589 P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:04:49.266004 16760 raft_consensus.cc:2243] T 7b6e87097733428bb2167c6f308fe589 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:49.266835 16760 raft_consensus.cc:2272] T 7b6e87097733428bb2167c6f308fe589 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:49.270020 16760 ts_tablet_manager.cc:1916] T 7b6e87097733428bb2167c6f308fe589 P 8430db80a39d4868bcf7641c16de35ea: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:49.280097 16760 ts_tablet_manager.cc:1929] T 7b6e87097733428bb2167c6f308fe589 P 8430db80a39d4868bcf7641c16de35ea: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:04:49.280366 16760 log.cc:1199] T 7b6e87097733428bb2167c6f308fe589 P 8430db80a39d4868bcf7641c16de35ea: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-1-root/wals/7b6e87097733428bb2167c6f308fe589
I20260709 13:04:49.281549 15744 catalog_manager.cc:5028] TS 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): tablet 7b6e87097733428bb2167c6f308fe589 (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:04:50.320886 15933 consensus_queue.cc:237] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:50.326360 16009 raft_consensus.cc:1275] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:50.327109 15862 raft_consensus.cc:1275] T ceaf02d91e334d5eba077b59ccce3a38 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:50.327648 16731 consensus_queue.cc:1048] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:50.328547 16691 consensus_queue.cc:1048] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:04:50.331485 15902 consensus_peers.cc:597] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ceaf02d91e334d5eba077b59ccce3a38. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:50.338605 16691 raft_consensus.cc:2955] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:50.341341 15857 raft_consensus.cc:2955] T ceaf02d91e334d5eba077b59ccce3a38 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:50.344022 16009 raft_consensus.cc:2955] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:50.355551 15933 consensus_queue.cc:237] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:50.359622 15760 catalog_manager.cc:5697] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:50.366772 15902 consensus_peers.cc:597] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c04fe1420dfe4602909deef4baa3f115. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:50.366981 16009 raft_consensus.cc:1275] T c04fe1420dfe4602909deef4baa3f115 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:50.368227 16691 consensus_queue.cc:1048] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:50.368750 15857 raft_consensus.cc:1275] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:50.369861 16763 consensus_queue.cc:1048] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:50.377682 16691 raft_consensus.cc:2955] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:50.381920 16009 raft_consensus.cc:2955] T c04fe1420dfe4602909deef4baa3f115 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:50.383615 15857 raft_consensus.cc:2955] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:50.402532 15857 consensus_queue.cc:237] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:50.410235 16010 raft_consensus.cc:1275] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:50.412276 16775 consensus_queue.cc:1048] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:04:50.414014 15827 consensus_peers.cc:597] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8849099536af497bae91c72da893681c. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:50.419302 15933 raft_consensus.cc:1275] T 8849099536af497bae91c72da893681c P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:50.421830 16752 consensus_queue.cc:1048] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:50.421430 15760 catalog_manager.cc:5697] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:50.436123 16775 raft_consensus.cc:2955] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:50.438166 15933 raft_consensus.cc:2955] T 8849099536af497bae91c72da893681c P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:50.450646 16009 raft_consensus.cc:2955] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:50.462438 15760 catalog_manager.cc:5697] T 8849099536af497bae91c72da893681c P 8430db80a39d4868bcf7641c16de35ea reported cstate change: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:50.472831 15933 consensus_queue.cc:237] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
W20260709 13:04:50.484088 15902 consensus_peers.cc:597] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f9822eb87c7e4819af0dc0b0d2951def. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:50.485419 16009 raft_consensus.cc:1275] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:50.486557 16691 consensus_queue.cc:1048] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:50.487545 15857 raft_consensus.cc:1275] T f9822eb87c7e4819af0dc0b0d2951def P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:50.497255 16763 consensus_queue.cc:1048] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:50.518392 16691 raft_consensus.cc:2955] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:50.520350 15856 raft_consensus.cc:2955] T f9822eb87c7e4819af0dc0b0d2951def P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:50.520605 16009 raft_consensus.cc:2955] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:50.538973 15760 catalog_manager.cc:5697] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:51.215229 16794 ts_tablet_manager.cc:933] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:51.227288 16792 ts_tablet_manager.cc:933] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:51.231922 16795 ts_tablet_manager.cc:933] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:51.236163 16795 tablet_copy_client.cc:323] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.2:43009
I20260709 13:04:51.238269 15947 tablet_copy_service.cc:140] P 8430db80a39d4868bcf7641c16de35ea: Received BeginTabletCopySession request for tablet f9822eb87c7e4819af0dc0b0d2951def from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:53636)
I20260709 13:04:51.238814 15947 tablet_copy_service.cc:161] P 8430db80a39d4868bcf7641c16de35ea: Beginning new tablet copy session on tablet f9822eb87c7e4819af0dc0b0d2951def from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:53636: session id = eb1e0493af1e46dbb6dec7e25525bab5-f9822eb87c7e4819af0dc0b0d2951def
I20260709 13:04:51.240319 16792 tablet_copy_client.cc:323] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.2:43009
I20260709 13:04:51.245260 15946 tablet_copy_service.cc:140] P 8430db80a39d4868bcf7641c16de35ea: Received BeginTabletCopySession request for tablet ceaf02d91e334d5eba077b59ccce3a38 from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:53636)
I20260709 13:04:51.245783 15946 tablet_copy_service.cc:161] P 8430db80a39d4868bcf7641c16de35ea: Beginning new tablet copy session on tablet ceaf02d91e334d5eba077b59ccce3a38 from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:53636: session id = eb1e0493af1e46dbb6dec7e25525bab5-ceaf02d91e334d5eba077b59ccce3a38
I20260709 13:04:51.251276 16794 tablet_copy_client.cc:323] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.2:43009
I20260709 13:04:51.253432 15946 tablet_copy_source_session.cc:215] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:51.260953 16792 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ceaf02d91e334d5eba077b59ccce3a38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:51.261785 15946 tablet_copy_service.cc:140] P 8430db80a39d4868bcf7641c16de35ea: Received BeginTabletCopySession request for tablet c04fe1420dfe4602909deef4baa3f115 from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:53636)
I20260709 13:04:51.262339 15946 tablet_copy_service.cc:161] P 8430db80a39d4868bcf7641c16de35ea: Beginning new tablet copy session on tablet c04fe1420dfe4602909deef4baa3f115 from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:53636: session id = eb1e0493af1e46dbb6dec7e25525bab5-c04fe1420dfe4602909deef4baa3f115
I20260709 13:04:51.266324 15947 tablet_copy_source_session.cc:215] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:51.269425 16795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9822eb87c7e4819af0dc0b0d2951def. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:51.285672 15946 tablet_copy_source_session.cc:215] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:51.315361 16792 tablet_copy_client.cc:806] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:51.316048 16792 tablet_copy_client.cc:670] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:51.317354 16800 ts_tablet_manager.cc:933] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
I20260709 13:04:51.320297 16794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c04fe1420dfe4602909deef4baa3f115. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:51.325502 16792 tablet_copy_client.cc:538] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:51.318454 16795 tablet_copy_client.cc:806] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:51.340885 16795 tablet_copy_client.cc:670] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:51.343046 16792 tablet_bootstrap.cc:492] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:04:51.342643 16800 tablet_copy_client.cc:323] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.1:38033
I20260709 13:04:51.344791 16795 tablet_copy_client.cc:538] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:51.347285 16794 tablet_copy_client.cc:806] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:51.347824 16794 tablet_copy_client.cc:670] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:51.349011 15872 tablet_copy_service.cc:140] P 31882606e87447fb9a36bbf625129fe8: Received BeginTabletCopySession request for tablet 8849099536af497bae91c72da893681c from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:58214)
I20260709 13:04:51.349574 15872 tablet_copy_service.cc:161] P 31882606e87447fb9a36bbf625129fe8: Beginning new tablet copy session on tablet 8849099536af497bae91c72da893681c from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:58214: session id = eb1e0493af1e46dbb6dec7e25525bab5-8849099536af497bae91c72da893681c
I20260709 13:04:51.358385 16794 tablet_copy_client.cc:538] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:51.358947 15872 tablet_copy_source_session.cc:215] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:51.361024 16795 tablet_bootstrap.cc:492] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:04:51.362056 16800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8849099536af497bae91c72da893681c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:51.374305 16794 tablet_bootstrap.cc:492] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:04:51.379514 16800 tablet_copy_client.cc:806] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:51.380821 16800 tablet_copy_client.cc:670] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:51.384871 16800 tablet_copy_client.cc:538] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:51.388794 16792 tablet_bootstrap.cc:492] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:51.389762 16792 tablet_bootstrap.cc:492] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:04:51.390470 16792 ts_tablet_manager.cc:1403] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.048s	user 0.020s	sys 0.016s
I20260709 13:04:51.396404 16792 raft_consensus.cc:359] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:51.397274 16792 raft_consensus.cc:740] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:04:51.397955 16792 consensus_queue.cc:260] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:51.402410 16800 tablet_bootstrap.cc:492] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:04:51.408156 16792 ts_tablet_manager.cc:1434] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.004s
I20260709 13:04:51.410701 15946 tablet_copy_service.cc:342] P 8430db80a39d4868bcf7641c16de35ea: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-ceaf02d91e334d5eba077b59ccce3a38 received from {username='slave'} at 127.0.0.1:53636
I20260709 13:04:51.411101 15946 tablet_copy_service.cc:434] P 8430db80a39d4868bcf7641c16de35ea: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-ceaf02d91e334d5eba077b59ccce3a38 on tablet ceaf02d91e334d5eba077b59ccce3a38 with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:04:51.425141 16794 tablet_bootstrap.cc:492] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:51.425967 16794 tablet_bootstrap.cc:492] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:04:51.426640 16794 ts_tablet_manager.cc:1403] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.053s	user 0.023s	sys 0.006s
I20260709 13:04:51.426790 16795 tablet_bootstrap.cc:492] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:51.427897 16795 tablet_bootstrap.cc:492] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:04:51.428579 16795 ts_tablet_manager.cc:1403] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.070s	user 0.019s	sys 0.010s
I20260709 13:04:51.429608 16794 raft_consensus.cc:359] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:51.430444 16794 raft_consensus.cc:740] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:04:51.431090 16794 consensus_queue.cc:260] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:51.431397 16795 raft_consensus.cc:359] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:51.432324 16795 raft_consensus.cc:740] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:04:51.433000 16795 consensus_queue.cc:260] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:51.433601 16794 ts_tablet_manager.cc:1434] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 13:04:51.435325 15946 tablet_copy_service.cc:342] P 8430db80a39d4868bcf7641c16de35ea: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-c04fe1420dfe4602909deef4baa3f115 received from {username='slave'} at 127.0.0.1:53636
I20260709 13:04:51.435753 15946 tablet_copy_service.cc:434] P 8430db80a39d4868bcf7641c16de35ea: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-c04fe1420dfe4602909deef4baa3f115 on tablet c04fe1420dfe4602909deef4baa3f115 with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:04:51.443599 16795 ts_tablet_manager.cc:1434] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 13:04:51.446291 15946 tablet_copy_service.cc:342] P 8430db80a39d4868bcf7641c16de35ea: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-f9822eb87c7e4819af0dc0b0d2951def received from {username='slave'} at 127.0.0.1:53636
I20260709 13:04:51.446792 15946 tablet_copy_service.cc:434] P 8430db80a39d4868bcf7641c16de35ea: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-f9822eb87c7e4819af0dc0b0d2951def on tablet f9822eb87c7e4819af0dc0b0d2951def with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:04:51.456707 16800 tablet_bootstrap.cc:492] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:51.457723 16800 tablet_bootstrap.cc:492] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:04:51.458292 16800 ts_tablet_manager.cc:1403] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.056s	user 0.016s	sys 0.019s
I20260709 13:04:51.460547 16800 raft_consensus.cc:359] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:51.461151 16800 raft_consensus.cc:740] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:04:51.461658 16800 consensus_queue.cc:260] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:51.463456 16800 ts_tablet_manager.cc:1434] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20260709 13:04:51.465330 15872 tablet_copy_service.cc:342] P 31882606e87447fb9a36bbf625129fe8: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-8849099536af497bae91c72da893681c received from {username='slave'} at 127.0.0.1:58214
I20260709 13:04:51.465798 15872 tablet_copy_service.cc:434] P 31882606e87447fb9a36bbf625129fe8: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-8849099536af497bae91c72da893681c on tablet 8849099536af497bae91c72da893681c with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:04:51.555608 16570 raft_consensus.cc:1217] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:51.556488 16568 raft_consensus.cc:1217] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:51.673668 16568 raft_consensus.cc:1217] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:51.680029 16568 raft_consensus.cc:1217] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 13:04:51.719338 15883 debug-util.cc:398] Leaking SignalData structure 0x7b08001f66c0 after lost signal to thread 14751
W20260709 13:04:51.720419 15883 debug-util.cc:398] Leaking SignalData structure 0x7b08001a31c0 after lost signal to thread 15794
W20260709 13:04:51.721714 15883 debug-util.cc:398] Leaking SignalData structure 0x7b08002973e0 after lost signal to thread 15886
W20260709 13:04:51.722692 15883 debug-util.cc:398] Leaking SignalData structure 0x7b08002f7720 after lost signal to thread 15961
W20260709 13:04:51.723701 15883 debug-util.cc:398] Leaking SignalData structure 0x7b080027c020 after lost signal to thread 16035
W20260709 13:04:51.725971 15883 debug-util.cc:398] Leaking SignalData structure 0x7b08001a2ae0 after lost signal to thread 16594
I20260709 13:04:52.006632 16729 raft_consensus.cc:1064] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:04:52.008425 16729 consensus_queue.cc:237] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:04:52.021806 15930 raft_consensus.cc:1275] T 8849099536af497bae91c72da893681c P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:52.022150 16008 raft_consensus.cc:1275] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:52.023486 16785 consensus_queue.cc:1048] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:52.025663 16568 raft_consensus.cc:1275] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:52.027477 16785 consensus_queue.cc:1048] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:52.031966 16785 consensus_queue.cc:1048] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:04:52.050992 16691 raft_consensus.cc:1064] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:04:52.054603 16691 consensus_queue.cc:237] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:04:52.062301 16731 raft_consensus.cc:1064] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:04:52.069290 16731 consensus_queue.cc:237] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:04:52.101157 16568 raft_consensus.cc:1275] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:52.102814 16799 consensus_queue.cc:1048] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:52.103439 16007 raft_consensus.cc:1275] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:52.112939 16729 raft_consensus.cc:2955] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:52.117300 15859 raft_consensus.cc:1275] T f9822eb87c7e4819af0dc0b0d2951def P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:52.119899 15859 raft_consensus.cc:1275] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:52.124210 16570 raft_consensus.cc:2955] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:52.126784 16568 raft_consensus.cc:1275] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:52.126931 16007 raft_consensus.cc:1275] T c04fe1420dfe4602909deef4baa3f115 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:52.142419 16799 consensus_queue.cc:1048] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:52.147748 16801 consensus_queue.cc:1048] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:52.149502 16801 consensus_queue.cc:1048] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:52.154304 15933 raft_consensus.cc:2955] T 8849099536af497bae91c72da893681c P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:52.155484 15757 catalog_manager.cc:5697] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 reported cstate change: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:52.167999 16008 raft_consensus.cc:2955] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:52.194828 16790 consensus_queue.cc:1048] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:52.195647 16791 consensus_queue.cc:1048] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:52.230052 16820 raft_consensus.cc:1064] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:04:52.233454 16820 consensus_queue.cc:237] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:04:52.252683 15862 raft_consensus.cc:1275] T ceaf02d91e334d5eba077b59ccce3a38 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:52.255033 16570 raft_consensus.cc:1275] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:52.261982 16791 consensus_queue.cc:1048] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:52.262297 16731 raft_consensus.cc:2955] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:52.264986 16008 raft_consensus.cc:1275] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:52.276473 15862 raft_consensus.cc:2955] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:52.288900 15861 consensus_queue.cc:237] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:04:52.273777 16799 raft_consensus.cc:2955] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:52.312808 16007 raft_consensus.cc:1275] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:52.315166 16783 consensus_queue.cc:1048] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:52.305125 15756 catalog_manager.cc:5697] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:52.318722 16570 raft_consensus.cc:1275] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:52.327462 16822 consensus_queue.cc:1048] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:52.328744 16784 raft_consensus.cc:2955] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:52.331753 16007 raft_consensus.cc:2955] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:52.333360 16570 raft_consensus.cc:2955] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:52.354324 15760 catalog_manager.cc:5697] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:52.358584 15930 consensus_queue.cc:237] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:04:52.361127 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 88174.9us
I20260709 13:04:52.372949 15923 tablet_service.cc:1558] Processing DeleteTablet for tablet 8849099536af497bae91c72da893681c with delete_type TABLET_DATA_TOMBSTONED (TS 8430db80a39d4868bcf7641c16de35ea not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58386
I20260709 13:04:52.377661 15855 raft_consensus.cc:1275] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:52.379715 16791 consensus_queue.cc:1048] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:52.379707 15855 raft_consensus.cc:2955] T f9822eb87c7e4819af0dc0b0d2951def P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:52.387877 15744 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8849099536af497bae91c72da893681c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:52.399842 16820 consensus_queue.cc:1048] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:52.401769 16820 raft_consensus.cc:2955] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:52.403589 16829 tablet_replica.cc:333] T 8849099536af497bae91c72da893681c P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:04:52.404624 16829 raft_consensus.cc:2243] T 8849099536af497bae91c72da893681c P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:52.405530 16829 raft_consensus.cc:2272] T 8849099536af497bae91c72da893681c P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:52.409802 15857 raft_consensus.cc:2955] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:52.413194 15760 catalog_manager.cc:5697] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea reported cstate change: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:52.413545 16570 raft_consensus.cc:2955] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:52.420539 16007 raft_consensus.cc:2955] T c04fe1420dfe4602909deef4baa3f115 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:52.422971 16829 ts_tablet_manager.cc:1916] T 8849099536af497bae91c72da893681c P 8430db80a39d4868bcf7641c16de35ea: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:52.424680 15744 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c04fe1420dfe4602909deef4baa3f115 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:52.428452 15930 consensus_queue.cc:237] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:04:52.452133 15757 catalog_manager.cc:5697] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:52.455492 16006 raft_consensus.cc:2955] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:52.457095 16820 consensus_queue.cc:1048] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:52.461390 16829 ts_tablet_manager.cc:1929] T 8849099536af497bae91c72da893681c P 8430db80a39d4868bcf7641c16de35ea: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:04:52.461828 16829 log.cc:1199] T 8849099536af497bae91c72da893681c P 8430db80a39d4868bcf7641c16de35ea: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-1-root/wals/8849099536af497bae91c72da893681c
I20260709 13:04:52.467171 15744 catalog_manager.cc:5028] TS 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): tablet 8849099536af497bae91c72da893681c (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:04:52.458572 16569 raft_consensus.cc:2955] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:52.472415 16568 raft_consensus.cc:1275] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:52.497020 16820 consensus_queue.cc:1048] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:52.500263 15995 tablet_service.cc:1558] Processing DeleteTablet for tablet c04fe1420dfe4602909deef4baa3f115 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 4)) from {username='slave'} at 127.0.0.1:41648
I20260709 13:04:52.508672 16567 raft_consensus.cc:2955] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:52.541076 16820 raft_consensus.cc:2955] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:52.542874 16011 raft_consensus.cc:1275] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:52.545097 16568 raft_consensus.cc:1275] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:52.558135 16832 tablet_replica.cc:333] T c04fe1420dfe4602909deef4baa3f115 P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:04:52.560211 16832 raft_consensus.cc:2243] T c04fe1420dfe4602909deef4baa3f115 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:52.565467 16832 raft_consensus.cc:2272] T c04fe1420dfe4602909deef4baa3f115 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:52.569900 16832 ts_tablet_manager.cc:1916] T c04fe1420dfe4602909deef4baa3f115 P 85e997ba5003439781ec4f1f8db0c3ee: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:52.566105 16569 raft_consensus.cc:2955] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:52.571857 16791 consensus_queue.cc:1048] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:52.577914 16799 consensus_queue.cc:1048] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:52.580504 16011 raft_consensus.cc:2955] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:52.599299 15855 raft_consensus.cc:2955] T ceaf02d91e334d5eba077b59ccce3a38 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:52.615276 15757 catalog_manager.cc:5697] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:52.635644 16832 ts_tablet_manager.cc:1929] T c04fe1420dfe4602909deef4baa3f115 P 85e997ba5003439781ec4f1f8db0c3ee: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:04:52.636106 16832 log.cc:1199] T c04fe1420dfe4602909deef4baa3f115 P 85e997ba5003439781ec4f1f8db0c3ee: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-2-root/wals/c04fe1420dfe4602909deef4baa3f115
I20260709 13:04:52.659477 15746 catalog_manager.cc:5028] TS 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): tablet c04fe1420dfe4602909deef4baa3f115 (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:04:52.687119 16820 raft_consensus.cc:2955] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:52.700037 16569 raft_consensus.cc:2955] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:52.713064 15744 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f9822eb87c7e4819af0dc0b0d2951def with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:52.713481 16008 raft_consensus.cc:2955] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:52.718077 15930 consensus_queue.cc:237] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:04:52.728332 16567 raft_consensus.cc:1275] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:52.732734 15756 catalog_manager.cc:5697] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:52.733946 16799 consensus_queue.cc:1048] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:52.748351 16791 raft_consensus.cc:2955] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:52.757560 15846 tablet_service.cc:1558] Processing DeleteTablet for tablet f9822eb87c7e4819af0dc0b0d2951def with delete_type TABLET_DATA_TOMBSTONED (TS 31882606e87447fb9a36bbf625129fe8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43504
I20260709 13:04:52.758659 15744 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ceaf02d91e334d5eba077b59ccce3a38 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:52.757277 16569 raft_consensus.cc:2955] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:52.760912 16834 tablet_replica.cc:333] T f9822eb87c7e4819af0dc0b0d2951def P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:04:52.766114 16834 raft_consensus.cc:2243] T f9822eb87c7e4819af0dc0b0d2951def P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:52.767812 16834 raft_consensus.cc:2272] T f9822eb87c7e4819af0dc0b0d2951def P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:52.771564 16834 ts_tablet_manager.cc:1916] T f9822eb87c7e4819af0dc0b0d2951def P 31882606e87447fb9a36bbf625129fe8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:52.781713 15756 catalog_manager.cc:5697] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:52.783730 16008 raft_consensus.cc:1275] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:52.785171 16820 consensus_queue.cc:1048] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:52.790498 16008 raft_consensus.cc:2955] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:52.819243 15846 tablet_service.cc:1558] Processing DeleteTablet for tablet ceaf02d91e334d5eba077b59ccce3a38 with delete_type TABLET_DATA_TOMBSTONED (TS 31882606e87447fb9a36bbf625129fe8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43504
I20260709 13:04:52.833647 16834 ts_tablet_manager.cc:1929] T f9822eb87c7e4819af0dc0b0d2951def P 31882606e87447fb9a36bbf625129fe8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:04:52.834146 16834 log.cc:1199] T f9822eb87c7e4819af0dc0b0d2951def P 31882606e87447fb9a36bbf625129fe8: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-0-root/wals/f9822eb87c7e4819af0dc0b0d2951def
I20260709 13:04:52.836032 16834 tablet_replica.cc:333] T ceaf02d91e334d5eba077b59ccce3a38 P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:04:52.835987 15744 catalog_manager.cc:5028] TS 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): tablet f9822eb87c7e4819af0dc0b0d2951def (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:04:52.837255 16834 raft_consensus.cc:2243] T ceaf02d91e334d5eba077b59ccce3a38 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:52.837967 16834 raft_consensus.cc:2272] T ceaf02d91e334d5eba077b59ccce3a38 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:52.842448 16834 ts_tablet_manager.cc:1916] T ceaf02d91e334d5eba077b59ccce3a38 P 31882606e87447fb9a36bbf625129fe8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:52.854257 16834 ts_tablet_manager.cc:1929] T ceaf02d91e334d5eba077b59ccce3a38 P 31882606e87447fb9a36bbf625129fe8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:04:52.854568 16834 log.cc:1199] T ceaf02d91e334d5eba077b59ccce3a38 P 31882606e87447fb9a36bbf625129fe8: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-0-root/wals/ceaf02d91e334d5eba077b59ccce3a38
I20260709 13:04:52.855811 15744 catalog_manager.cc:5028] TS 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): tablet ceaf02d91e334d5eba077b59ccce3a38 (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:04:54.568348 16842 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:04:54.569133 16842 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:54.571779 16842 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:04:54.616039 15975 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.840s	user 0.007s	sys 0.004s
I20260709 13:04:54.621824 16837 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:04:54.622458 16837 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:54.624645 16837 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:04:54.633612 15975 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.858s	user 0.007s	sys 0.004s
I20260709 13:04:54.633737 16839 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:04:54.634280 16839 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:54.636464 16839 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:54.640872 16725 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:04:54.639066 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:54.647912 16725 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:54.654740 15931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 687) took 1010 ms. Trace:
I20260709 13:04:54.664207 15931 rpcz_store.cc:276] 0709 13:04:54.498183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.653795 (+155612us) service_pool.cc:224] Handling call
0709 13:04:54.654135 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:54.654344 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:54.654381 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:54.654431 (+    50us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:54.654472 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.654543 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.654671 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:54.665680 16856 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:04:54.665618 16840 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:04:54.666308 16856 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:54.666792 16840 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:04:54.668783 16840 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:04:54.656276 16845 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:04:54.670183 16845 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:54.672245 16845 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:54.675938 16837 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:04:54.676275 16856 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:54.676682 16837 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:54.681766 16837 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:54.687849 16725 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:54.689783 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:54.698968 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:54.702150 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:54.703215 16859 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:04:54.709548 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:54.711406 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:54.712615 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:54.713006 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:54.703845 16859 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:04:54.721596 15930 consensus_queue.cc:237] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:54.725730 16859 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:54.731787 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:04:54.732028 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:54.733116 16725 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:04:54.733577 16725 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:54.735136 16725 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:54.735519 16725 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:04:54.735911 16725 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:54.740461 16852 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:04:54.741830 16725 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:54.741534 16852 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:54.728029 15974 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:04:54.750634 16725 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:04:54.742108 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:54.753916 16861 raft_consensus.cc:2749] T eabbcb31af804f2dac6af5731e18a4dd P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:54.754525 16861 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:04:54.754940 16861 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:54.756901 16861 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:54.760243 16725 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:04:54.763995 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:54.749778 16859 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:04:54.766769 16725 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:04:54.772681 16852 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:54.773442 16859 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:04:54.774363 16844 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:04:54.774806 16845 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:04:54.776472 15902 consensus_peers.cc:597] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d2be2d6f06bc47a39f26307ae9372194. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:54.776552 16845 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:54.779903 16845 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:54.780472 16858 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:04:54.775403 16844 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:04:54.782588 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:54.781030 16858 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:54.785892 16844 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:04:54.786755 16858 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:54.787380 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:04:54.790894 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:54.792006 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:54.792193 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:54.793401 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:54.793745 15861 raft_consensus.cc:1275] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:54.794584 16844 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:04:54.795127 16844 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:54.797171 16844 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:54.800263 15900 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:04:54.802068 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:54.802985 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:54.803442 16535 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:54.804005 16007 raft_consensus.cc:1275] T d2be2d6f06bc47a39f26307ae9372194 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:54.802501 16844 raft_consensus.cc:2749] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:54.804915 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:54.802794 16852 consensus_queue.cc:1048] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:54.802649 16009 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:04:54.806792 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:54.808007 16862 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:54.808516 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:04:54.813447 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:54.814996 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:54.815413 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:54.815476 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:54.809237 16859 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:54.809898 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:54.810544 15902 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:04:54.820516 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 59231.7us
I20260709 13:04:54.824774 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:54.826253 15902 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:54.826759 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:54.828431 15902 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:54.829100 16533 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:54.830154 16854 raft_consensus.cc:2749] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:54.830509 16845 raft_consensus.cc:2749] T 952bd0ea3bba4d38ad3383fdf572cd01 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:54.831058 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:54.833036 15902 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:04:54.833732 15974 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:04:54.835767 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:54.836314 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:54.836256 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:54.837505 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:54.838492 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:54.840324 16866 consensus_queue.cc:1048] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 13:04:54.840540 15974 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:04:54.841791 15974 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:04:54.842445 16725 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:54.823788 16844 raft_consensus.cc:2749] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:54.846732 16848 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:54.847285 16848 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:04:54.847715 16848 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:54.848325 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:54.845446 16840 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:54.846791 16857 raft_consensus.cc:2749] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:54.847384 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:54.864946 15827 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:54.843884 16859 raft_consensus.cc:2749] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:54.868122 16836 raft_consensus.cc:2749] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:54.868427 15827 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:54.869416 16836 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:54.872340 15827 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:54.873159 16843 raft_consensus.cc:2749] T fd8e1402dafb4300911e5b0189c8fe0a P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:54.874267 15827 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:54.874971 16843 raft_consensus.cc:2749] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:54.874665 16848 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:54.878302 15974 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:04:54.879068 16867 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:54.879344 15974 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:04:54.880221 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:54.881906 16867 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:54.883184 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:54.884193 16848 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:54.884542 15900 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:54.883831 16866 raft_consensus.cc:2955] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:54.880501 15974 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:04:54.887194 15974 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:04:54.887708 16725 raft_consensus.cc:2749] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:54.888214 16857 raft_consensus.cc:2749] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:54.891839 16848 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:54.895740 15861 raft_consensus.cc:2955] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:54.900970 16008 raft_consensus.cc:2955] T d2be2d6f06bc47a39f26307ae9372194 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:54.902478 16006 consensus_queue.cc:237] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:54.909006 15759 catalog_manager.cc:5697] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:54.930841 15933 raft_consensus.cc:1275] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:04:54.932835 15976 consensus_peers.cc:597] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d50c1e8d433a407aa9fae9cb9866f184. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:54.933382 16725 consensus_queue.cc:1048] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:54.942976 15861 raft_consensus.cc:1275] T d50c1e8d433a407aa9fae9cb9866f184 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:54.946542 16725 consensus_queue.cc:1048] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:54.948941 16840 raft_consensus.cc:2955] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:54.950831 15930 raft_consensus.cc:2955] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:54.964675 15861 raft_consensus.cc:2955] T d50c1e8d433a407aa9fae9cb9866f184 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:54.984032 16006 consensus_queue.cc:237] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:54.996852 15757 catalog_manager.cc:5697] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:55.023295 15861 raft_consensus.cc:1275] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:04:55.023840 15976 consensus_peers.cc:597] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3b3e05ffd87b4bbcb04b315d3bfdf8ed. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:55.024868 15935 raft_consensus.cc:1275] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:55.024807 16867 consensus_queue.cc:1048] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:55.028411 16867 consensus_queue.cc:1048] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:55.071822 16867 raft_consensus.cc:2955] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:55.073568 15861 raft_consensus.cc:2955] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:55.088461 15930 consensus_queue.cc:237] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:55.100231 15756 catalog_manager.cc:5697] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:55.119182 15933 raft_consensus.cc:2955] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:55.128216 15861 raft_consensus.cc:1275] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:55.128535 16008 raft_consensus.cc:1275] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:55.129519 16844 consensus_queue.cc:1048] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:55.130995 15931 raft_consensus.cc:1240] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 1. Current term is 2. Ops: []
W20260709 13:04:55.131203 15902 consensus_peers.cc:597] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6e422bf1c2eb46fcb2fe33e4eb00f888. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:55.133649 16839 consensus_queue.cc:1059] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:55.134166 16849 consensus_queue.cc:1048] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:55.134569 16839 raft_consensus.cc:3055] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:55.134956 16839 raft_consensus.cc:740] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:04:55.136977 16839 consensus_queue.cc:260] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:55.139365 16839 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:55.164840 16845 raft_consensus.cc:2955] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:55.168047 15861 raft_consensus.cc:2955] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:55.181301 16008 raft_consensus.cc:2955] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:55.203651 15756 catalog_manager.cc:5697] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:55.268873 16883 ts_tablet_manager.cc:933] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:55.276098 16883 tablet_copy_client.cc:323] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.2:43009
I20260709 13:04:55.282418 15946 tablet_copy_service.cc:140] P 8430db80a39d4868bcf7641c16de35ea: Received BeginTabletCopySession request for tablet d2be2d6f06bc47a39f26307ae9372194 from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:53636)
I20260709 13:04:55.283004 15946 tablet_copy_service.cc:161] P 8430db80a39d4868bcf7641c16de35ea: Beginning new tablet copy session on tablet d2be2d6f06bc47a39f26307ae9372194 from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:53636: session id = eb1e0493af1e46dbb6dec7e25525bab5-d2be2d6f06bc47a39f26307ae9372194
I20260709 13:04:55.297401 15946 tablet_copy_source_session.cc:215] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:55.300750 16883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2be2d6f06bc47a39f26307ae9372194. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:55.314267 16883 tablet_copy_client.cc:806] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:55.314765 16883 tablet_copy_client.cc:670] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:55.317965 16883 tablet_copy_client.cc:538] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:55.323884 16883 tablet_bootstrap.cc:492] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:04:55.344784 16883 tablet_bootstrap.cc:492] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:55.345491 16883 tablet_bootstrap.cc:492] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:04:55.346006 16883 ts_tablet_manager.cc:1403] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.009s
I20260709 13:04:55.349190 16883 raft_consensus.cc:359] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:55.350029 16883 raft_consensus.cc:740] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:04:55.350667 16883 consensus_queue.cc:260] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:55.363039 16883 ts_tablet_manager.cc:1434] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 13:04:55.370882 15946 tablet_copy_service.cc:342] P 8430db80a39d4868bcf7641c16de35ea: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-d2be2d6f06bc47a39f26307ae9372194 received from {username='slave'} at 127.0.0.1:53636
I20260709 13:04:55.371300 15946 tablet_copy_service.cc:434] P 8430db80a39d4868bcf7641c16de35ea: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-d2be2d6f06bc47a39f26307ae9372194 on tablet d2be2d6f06bc47a39f26307ae9372194 with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:04:55.422510 16883 ts_tablet_manager.cc:933] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:55.433346 16883 tablet_copy_client.cc:323] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.3:46251
I20260709 13:04:55.435120 16021 tablet_copy_service.cc:140] P 85e997ba5003439781ec4f1f8db0c3ee: Received BeginTabletCopySession request for tablet d50c1e8d433a407aa9fae9cb9866f184 from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:60156)
I20260709 13:04:55.435565 16021 tablet_copy_service.cc:161] P 85e997ba5003439781ec4f1f8db0c3ee: Beginning new tablet copy session on tablet d50c1e8d433a407aa9fae9cb9866f184 from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:60156: session id = eb1e0493af1e46dbb6dec7e25525bab5-d50c1e8d433a407aa9fae9cb9866f184
W20260709 13:04:55.440431 15791 debug-util.cc:398] Leaking SignalData structure 0x7b080008a700 after lost signal to thread 14751
I20260709 13:04:55.441002 16021 tablet_copy_source_session.cc:215] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:04:55.441365 15791 debug-util.cc:398] Leaking SignalData structure 0x7b080008ade0 after lost signal to thread 15794
W20260709 13:04:55.442502 15791 debug-util.cc:398] Leaking SignalData structure 0x7b0800310d80 after lost signal to thread 15886
W20260709 13:04:55.443507 15791 debug-util.cc:398] Leaking SignalData structure 0x7b08002527a0 after lost signal to thread 15961
I20260709 13:04:55.444631 16883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d50c1e8d433a407aa9fae9cb9866f184. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:04:55.444684 15791 debug-util.cc:398] Leaking SignalData structure 0x7b080026ce00 after lost signal to thread 16035
W20260709 13:04:55.446077 15791 debug-util.cc:398] Leaking SignalData structure 0x7b08002f8a80 after lost signal to thread 16594
W20260709 13:04:55.446470 15791 debug-util.cc:398] Leaking SignalData structure 0x7b0800252120 after lost signal to thread 16820
I20260709 13:04:55.464037 16883 tablet_copy_client.cc:806] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:55.464630 16883 tablet_copy_client.cc:670] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:55.469434 16883 tablet_copy_client.cc:538] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:55.488708 16887 ts_tablet_manager.cc:933] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:55.490556 16883 tablet_bootstrap.cc:492] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:04:55.496837 16887 tablet_copy_client.cc:323] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.3:46251
I20260709 13:04:55.499294 16021 tablet_copy_service.cc:140] P 85e997ba5003439781ec4f1f8db0c3ee: Received BeginTabletCopySession request for tablet 3b3e05ffd87b4bbcb04b315d3bfdf8ed from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:60156)
I20260709 13:04:55.499827 16021 tablet_copy_service.cc:161] P 85e997ba5003439781ec4f1f8db0c3ee: Beginning new tablet copy session on tablet 3b3e05ffd87b4bbcb04b315d3bfdf8ed from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:60156: session id = eb1e0493af1e46dbb6dec7e25525bab5-3b3e05ffd87b4bbcb04b315d3bfdf8ed
I20260709 13:04:55.508818 16021 tablet_copy_source_session.cc:215] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:55.511950 16887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b3e05ffd87b4bbcb04b315d3bfdf8ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:55.557307 16887 tablet_copy_client.cc:806] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:55.559105 16887 tablet_copy_client.cc:670] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:55.564278 16887 tablet_copy_client.cc:538] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:55.579566 16887 tablet_bootstrap.cc:492] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:04:55.580042 16883 tablet_bootstrap.cc:492] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:55.581060 16883 tablet_bootstrap.cc:492] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:04:55.581776 16883 ts_tablet_manager.cc:1403] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.091s	user 0.022s	sys 0.002s
I20260709 13:04:55.585556 16883 raft_consensus.cc:359] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:55.589075 16883 raft_consensus.cc:740] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:04:55.589764 16883 consensus_queue.cc:260] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:55.599939 16883 ts_tablet_manager.cc:1434] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 13:04:55.603168 16021 tablet_copy_service.cc:342] P 85e997ba5003439781ec4f1f8db0c3ee: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-d50c1e8d433a407aa9fae9cb9866f184 received from {username='slave'} at 127.0.0.1:60156
I20260709 13:04:55.603587 16021 tablet_copy_service.cc:434] P 85e997ba5003439781ec4f1f8db0c3ee: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-d50c1e8d433a407aa9fae9cb9866f184 on tablet d50c1e8d433a407aa9fae9cb9866f184 with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:04:55.644791 16883 ts_tablet_manager.cc:933] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:55.659142 16887 tablet_bootstrap.cc:492] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:55.677033 16887 tablet_bootstrap.cc:492] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:04:55.678848 16887 ts_tablet_manager.cc:1403] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.100s	user 0.020s	sys 0.014s
W20260709 13:04:55.702641 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 63209.1us
I20260709 13:04:55.704509 16883 tablet_copy_client.cc:323] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.2:43009
I20260709 13:04:55.706312 15946 tablet_copy_service.cc:140] P 8430db80a39d4868bcf7641c16de35ea: Received BeginTabletCopySession request for tablet 6e422bf1c2eb46fcb2fe33e4eb00f888 from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:53636)
I20260709 13:04:55.706821 15946 tablet_copy_service.cc:161] P 8430db80a39d4868bcf7641c16de35ea: Beginning new tablet copy session on tablet 6e422bf1c2eb46fcb2fe33e4eb00f888 from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:53636: session id = eb1e0493af1e46dbb6dec7e25525bab5-6e422bf1c2eb46fcb2fe33e4eb00f888
I20260709 13:04:55.717061 16887 raft_consensus.cc:359] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:55.719728 16887 raft_consensus.cc:740] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:04:55.720396 15946 tablet_copy_source_session.cc:215] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:55.720412 16887 consensus_queue.cc:260] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:55.723546 16883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e422bf1c2eb46fcb2fe33e4eb00f888. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:55.725701 16887 ts_tablet_manager.cc:1434] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:04:55.727222 16021 tablet_copy_service.cc:342] P 85e997ba5003439781ec4f1f8db0c3ee: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-3b3e05ffd87b4bbcb04b315d3bfdf8ed received from {username='slave'} at 127.0.0.1:60156
I20260709 13:04:55.727589 16021 tablet_copy_service.cc:434] P 85e997ba5003439781ec4f1f8db0c3ee: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-3b3e05ffd87b4bbcb04b315d3bfdf8ed on tablet 3b3e05ffd87b4bbcb04b315d3bfdf8ed with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:04:55.747644 16883 tablet_copy_client.cc:806] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:55.748226 16883 tablet_copy_client.cc:670] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:55.752908 16883 tablet_copy_client.cc:538] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:55.775372 16883 tablet_bootstrap.cc:492] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:04:55.816238 16883 tablet_bootstrap.cc:492] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:55.825615 16567 raft_consensus.cc:1217] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:55.829262 16883 tablet_bootstrap.cc:492] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:04:55.829998 16883 ts_tablet_manager.cc:1403] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.055s	user 0.022s	sys 0.003s
I20260709 13:04:55.832621 16883 raft_consensus.cc:359] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:55.833670 16883 raft_consensus.cc:740] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:04:55.834290 16883 consensus_queue.cc:260] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:55.836427 16883 ts_tablet_manager.cc:1434] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:04:55.844079 15946 tablet_copy_service.cc:342] P 8430db80a39d4868bcf7641c16de35ea: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-6e422bf1c2eb46fcb2fe33e4eb00f888 received from {username='slave'} at 127.0.0.1:53636
I20260709 13:04:55.844561 15946 tablet_copy_service.cc:434] P 8430db80a39d4868bcf7641c16de35ea: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-6e422bf1c2eb46fcb2fe33e4eb00f888 on tablet 6e422bf1c2eb46fcb2fe33e4eb00f888 with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:04:55.865797 16567 raft_consensus.cc:1217] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:55.888417 16567 raft_consensus.cc:1217] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:56.170594 16569 raft_consensus.cc:1217] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 13:04:56.217742 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 97395.3us
I20260709 13:04:56.321336 16856 raft_consensus.cc:1064] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:04:56.324283 16856 consensus_queue.cc:237] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:04:56.333602 16898 raft_consensus.cc:1064] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:04:56.336092 16898 consensus_queue.cc:237] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:04:56.361515 16006 raft_consensus.cc:1275] T d2be2d6f06bc47a39f26307ae9372194 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:56.363037 16895 consensus_queue.cc:1048] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:56.364153 16567 raft_consensus.cc:1275] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:56.369423 16895 consensus_queue.cc:1048] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:04:56.385560 15857 raft_consensus.cc:1275] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:56.387043 16898 consensus_queue.cc:1048] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:56.414148 15933 raft_consensus.cc:1275] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:56.414186 15857 raft_consensus.cc:1275] T d50c1e8d433a407aa9fae9cb9866f184 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:56.416682 16567 raft_consensus.cc:1275] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:56.417485 16840 consensus_queue.cc:1048] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:56.421293 16840 consensus_queue.cc:1048] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:56.432111 16008 raft_consensus.cc:2955] T d2be2d6f06bc47a39f26307ae9372194 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:56.425236 16844 raft_consensus.cc:2955] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:56.446301 16856 consensus_queue.cc:1048] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:56.453874 16900 raft_consensus.cc:1064] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:04:56.456974 16900 consensus_queue.cc:237] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:04:56.459299 16856 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:04:56.459849 16856 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:56.463423 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:56.463778 16856 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:56.463997 15859 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 2.
I20260709 13:04:56.464952 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:04:56.465569 15930 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 2: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:04:56.466637 15974 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:04:56.467388 16856 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:56.467854 16856 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:04:56.468267 16856 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:56.476902 15759 catalog_manager.cc:5697] T d2be2d6f06bc47a39f26307ae9372194 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:56.486972 16856 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:56.490170 16856 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:56.498553 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:04:56.504787 15930 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 2: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:04:56.498557 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:04:56.521423 16901 log.cc:927] Time spent T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:04:56.522552 16569 raft_consensus.cc:1275] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:56.522478 16894 raft_consensus.cc:2955] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:56.524092 16840 consensus_queue.cc:1048] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:56.529145 15861 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 2.
I20260709 13:04:56.532490 15935 raft_consensus.cc:1275] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:56.538352 15974 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:04:56.539371 16840 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:56.540126 16840 raft_consensus.cc:697] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:04:56.540479 15861 raft_consensus.cc:1275] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:56.541574 16840 consensus_queue.cc:237] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:56.543249 16567 raft_consensus.cc:2955] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:56.554144 16900 consensus_queue.cc:1048] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:56.562687 15935 raft_consensus.cc:2955] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:56.579927 16900 consensus_queue.cc:1048] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:56.592773 15857 raft_consensus.cc:2955] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:56.586702 16567 raft_consensus.cc:2955] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:56.607658 15859 raft_consensus.cc:2955] T d50c1e8d433a407aa9fae9cb9866f184 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:56.628002 16891 raft_consensus.cc:2955] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:56.636849 16569 raft_consensus.cc:2955] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:56.637670 15933 raft_consensus.cc:2955] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:56.650964 15859 raft_consensus.cc:2955] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:56.666920 16866 raft_consensus.cc:1064] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:04:56.669943 16844 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.672494 16844 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:56.674371 16844 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:56.669420 16866 consensus_queue.cc:237] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:04:56.679487 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:04:56.680136 15859 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2.
I20260709 13:04:56.681116 15900 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:04:56.681917 16895 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:56.682258 16895 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:56.682579 16895 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:56.683744 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:04:56.692801 16895 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:56.694442 15760 catalog_manager.cc:5697] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:56.708325 15760 catalog_manager.cc:5697] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:56.695641 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:04:56.712028 15859 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:56.696451 16895 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:04:56.697535 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:04:56.722174 15859 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:04:56.723142 16007 raft_consensus.cc:1275] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:56.723546 16569 raft_consensus.cc:1275] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:56.725535 16898 consensus_queue.cc:1048] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:56.725945 15859 raft_consensus.cc:1275] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:56.728003 15900 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:04:56.729043 16844 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:56.729521 16844 raft_consensus.cc:697] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:04:56.730367 16844 consensus_queue.cc:237] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:56.727428 16898 consensus_queue.cc:1048] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:04:56.732735 15756 catalog_manager.cc:5697] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 1 to 2, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 2 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:56.740679 15933 consensus_queue.cc:237] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:04:56.747710 15857 raft_consensus.cc:1275] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:56.750901 16896 consensus_queue.cc:1048] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:56.756645 16567 raft_consensus.cc:1275] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:56.762530 16895 consensus_queue.cc:1048] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 13:04:56.757952 16844 consensus_queue.cc:1048] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:04:56.778735 16008 consensus_queue.cc:237] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:04:56.790467 16006 consensus_queue.cc:237] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:04:56.779213 15760 catalog_manager.cc:5697] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 2 to 3, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 3 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:56.799438 16844 raft_consensus.cc:2955] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:56.808825 16569 raft_consensus.cc:1275] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:56.810344 15744 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d2be2d6f06bc47a39f26307ae9372194 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:56.811746 16891 consensus_queue.cc:1048] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:56.812316 15857 raft_consensus.cc:1275] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:56.813064 16569 raft_consensus.cc:1275] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:56.827173 16856 consensus_queue.cc:1048] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:56.828504 16891 consensus_queue.cc:1048] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:56.832075 16895 raft_consensus.cc:2955] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:56.840116 16568 raft_consensus.cc:2955] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:56.841421 15933 raft_consensus.cc:1275] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:56.837097 16007 raft_consensus.cc:2955] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:56.842075 16900 raft_consensus.cc:2955] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:56.826444 16569 raft_consensus.cc:2955] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:56.844918 16840 consensus_queue.cc:1048] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:56.838315 16894 raft_consensus.cc:2955] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:56.854868 15861 raft_consensus.cc:2955] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:56.867302 15933 raft_consensus.cc:2955] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:56.872177 15760 catalog_manager.cc:5697] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:56.885200 15746 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d50c1e8d433a407aa9fae9cb9866f184 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:56.891553 15857 raft_consensus.cc:2955] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:56.892977 16569 raft_consensus.cc:2955] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:56.905923 15759 catalog_manager.cc:5697] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:56.913959 15846 tablet_service.cc:1558] Processing DeleteTablet for tablet d50c1e8d433a407aa9fae9cb9866f184 with delete_type TABLET_DATA_TOMBSTONED (TS 31882606e87447fb9a36bbf625129fe8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43504
I20260709 13:04:56.849025 15756 catalog_manager.cc:5697] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:56.914441 15746 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3b3e05ffd87b4bbcb04b315d3bfdf8ed with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:56.848277 16567 raft_consensus.cc:2955] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:56.933379 16925 tablet_replica.cc:333] T d50c1e8d433a407aa9fae9cb9866f184 P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:04:56.934655 16925 raft_consensus.cc:2243] T d50c1e8d433a407aa9fae9cb9866f184 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.935463 16925 raft_consensus.cc:2272] T d50c1e8d433a407aa9fae9cb9866f184 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:56.949349 16925 ts_tablet_manager.cc:1916] T d50c1e8d433a407aa9fae9cb9866f184 P 31882606e87447fb9a36bbf625129fe8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:56.967087 15995 tablet_service.cc:1558] Processing DeleteTablet for tablet d2be2d6f06bc47a39f26307ae9372194 with delete_type TABLET_DATA_TOMBSTONED (TS 85e997ba5003439781ec4f1f8db0c3ee not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41648
I20260709 13:04:56.973318 15859 raft_consensus.cc:2955] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:56.974704 15759 catalog_manager.cc:5697] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:56.985137 16925 ts_tablet_manager.cc:1929] T d50c1e8d433a407aa9fae9cb9866f184 P 31882606e87447fb9a36bbf625129fe8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:04:56.985658 16925 log.cc:1199] T d50c1e8d433a407aa9fae9cb9866f184 P 31882606e87447fb9a36bbf625129fe8: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-0-root/wals/d50c1e8d433a407aa9fae9cb9866f184
I20260709 13:04:56.987670 15744 catalog_manager.cc:5028] TS 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): tablet d50c1e8d433a407aa9fae9cb9866f184 (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:04:56.991994 16926 tablet_replica.cc:333] T d2be2d6f06bc47a39f26307ae9372194 P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:04:56.994220 16926 raft_consensus.cc:2243] T d2be2d6f06bc47a39f26307ae9372194 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:56.994948 16926 raft_consensus.cc:2272] T d2be2d6f06bc47a39f26307ae9372194 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.008680 15923 tablet_service.cc:1558] Processing DeleteTablet for tablet 3b3e05ffd87b4bbcb04b315d3bfdf8ed with delete_type TABLET_DATA_TOMBSTONED (TS 8430db80a39d4868bcf7641c16de35ea not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58386
I20260709 13:04:57.013837 15935 consensus_queue.cc:237] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:04:57.018653 16926 ts_tablet_manager.cc:1916] T d2be2d6f06bc47a39f26307ae9372194 P 85e997ba5003439781ec4f1f8db0c3ee: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:57.040771 16567 raft_consensus.cc:1275] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:57.046851 16895 consensus_queue.cc:1048] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:57.050504 15857 raft_consensus.cc:1240] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:57.053512 16900 consensus_queue.cc:1059] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:57.054502 16840 raft_consensus.cc:3055] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:57.054839 16840 raft_consensus.cc:740] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:04:57.055430 16926 ts_tablet_manager.cc:1929] T d2be2d6f06bc47a39f26307ae9372194 P 85e997ba5003439781ec4f1f8db0c3ee: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:04:57.055886 16926 log.cc:1199] T d2be2d6f06bc47a39f26307ae9372194 P 85e997ba5003439781ec4f1f8db0c3ee: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-2-root/wals/d2be2d6f06bc47a39f26307ae9372194
I20260709 13:04:57.054834 16898 raft_consensus.cc:2955] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:57.056113 16840 consensus_queue.cc:260] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:04:57.057298 16840 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:57.059999 15746 catalog_manager.cc:5028] TS 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): tablet d2be2d6f06bc47a39f26307ae9372194 (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:04:57.068132 16567 raft_consensus.cc:2955] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:57.080922 15744 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6e422bf1c2eb46fcb2fe33e4eb00f888 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:57.089793 15760 catalog_manager.cc:5697] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:57.095911 16927 tablet_replica.cc:333] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:04:57.097433 16927 raft_consensus.cc:2243] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:57.097687 16007 raft_consensus.cc:1275] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:57.098167 16927 raft_consensus.cc:2272] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.098922 16898 consensus_queue.cc:1048] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:57.106321 15846 tablet_service.cc:1558] Processing DeleteTablet for tablet 6e422bf1c2eb46fcb2fe33e4eb00f888 with delete_type TABLET_DATA_TOMBSTONED (TS 31882606e87447fb9a36bbf625129fe8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43504
I20260709 13:04:57.107237 16925 tablet_replica.cc:333] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:04:57.113423 16007 raft_consensus.cc:2955] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:04:57.117378 16927 ts_tablet_manager.cc:1916] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 8430db80a39d4868bcf7641c16de35ea: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:57.123765 16925 raft_consensus.cc:2243] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:57.124513 16925 raft_consensus.cc:2272] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.127991 16925 ts_tablet_manager.cc:1916] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 31882606e87447fb9a36bbf625129fe8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:57.139918 16006 raft_consensus.cc:1275] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:04:57.140343 16006 pending_rounds.cc:85] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting all ops after (but not including) 1
I20260709 13:04:57.140597 16006 pending_rounds.cc:107] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:04:57.140884 16006 raft_consensus.cc:2889] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:57.142670 16922 consensus_queue.cc:1048] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:57.144769 16925 ts_tablet_manager.cc:1929] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 31882606e87447fb9a36bbf625129fe8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:04:57.145241 16925 log.cc:1199] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 31882606e87447fb9a36bbf625129fe8: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-0-root/wals/6e422bf1c2eb46fcb2fe33e4eb00f888
I20260709 13:04:57.146970 15744 catalog_manager.cc:5028] TS 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): tablet 6e422bf1c2eb46fcb2fe33e4eb00f888 (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:04:57.162410 15859 raft_consensus.cc:1275] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:04:57.167014 16898 consensus_queue.cc:1048] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:57.186396 16927 ts_tablet_manager.cc:1929] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 8430db80a39d4868bcf7641c16de35ea: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:04:57.186932 16927 log.cc:1199] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 8430db80a39d4868bcf7641c16de35ea: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-1-root/wals/3b3e05ffd87b4bbcb04b315d3bfdf8ed
I20260709 13:04:57.224437 15744 catalog_manager.cc:5028] TS 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): tablet 3b3e05ffd87b4bbcb04b315d3bfdf8ed (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:04:58.291605 15859 consensus_queue.cc:237] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:58.311896 15933 raft_consensus.cc:1275] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:04:58.312672 15827 consensus_peers.cc:597] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: eabbcb31af804f2dac6af5731e18a4dd. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:58.314333 16893 consensus_queue.cc:1048] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:58.320475 16007 raft_consensus.cc:1275] T eabbcb31af804f2dac6af5731e18a4dd P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:58.328430 16893 consensus_queue.cc:1048] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:58.337605 16899 raft_consensus.cc:2955] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:58.344058 15930 raft_consensus.cc:2955] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:58.394315 15760 catalog_manager.cc:5697] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea reported cstate change: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:58.422845 16008 raft_consensus.cc:2955] T eabbcb31af804f2dac6af5731e18a4dd P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:58.444598 15855 consensus_queue.cc:237] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
W20260709 13:04:58.467793 15827 consensus_peers.cc:597] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 471dbd2e16264fe5837e8b7ea3601f94. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:58.470631 16007 raft_consensus.cc:1275] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:58.472141 16836 consensus_queue.cc:1048] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:58.472959 15930 raft_consensus.cc:1275] T 471dbd2e16264fe5837e8b7ea3601f94 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:58.474784 16836 consensus_queue.cc:1048] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:58.490059 16836 raft_consensus.cc:2955] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:58.491505 15933 raft_consensus.cc:2955] T 471dbd2e16264fe5837e8b7ea3601f94 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:58.510510 15759 catalog_manager.cc:5697] T 471dbd2e16264fe5837e8b7ea3601f94 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:58.511253 16007 raft_consensus.cc:2955] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:58.512861 15930 consensus_queue.cc:237] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
W20260709 13:04:58.553628 15902 consensus_peers.cc:597] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0b3676041725488fb6ffad37889b386b. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:58.554148 16008 raft_consensus.cc:1275] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:58.556519 15855 raft_consensus.cc:1275] T 0b3676041725488fb6ffad37889b386b P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:58.557654 16895 consensus_queue.cc:1048] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:58.559762 16933 consensus_queue.cc:1048] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:58.567588 16933 raft_consensus.cc:2955] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:58.571017 16007 raft_consensus.cc:2955] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:58.582384 15855 raft_consensus.cc:2955] T 0b3676041725488fb6ffad37889b386b P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:58.586964 15930 consensus_queue.cc:237] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:58.588454 15760 catalog_manager.cc:5697] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:58.620612 16007 raft_consensus.cc:1275] T 626a74f434ad4e70bed42f8b404a6e81 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:04:58.621811 15902 consensus_peers.cc:597] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 626a74f434ad4e70bed42f8b404a6e81. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:58.623840 15855 raft_consensus.cc:1275] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:58.625672 16895 consensus_queue.cc:1048] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:58.628093 16895 consensus_queue.cc:1048] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:58.659863 16933 raft_consensus.cc:2955] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:58.663021 15857 raft_consensus.cc:2955] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:58.665936 16007 raft_consensus.cc:2955] T 626a74f434ad4e70bed42f8b404a6e81 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:58.685225 15759 catalog_manager.cc:5697] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:04:58.862108 16954 ts_tablet_manager.cc:933] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
I20260709 13:04:58.865567 16954 tablet_copy_client.cc:323] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.1:38033
I20260709 13:04:58.867650 15872 tablet_copy_service.cc:140] P 31882606e87447fb9a36bbf625129fe8: Received BeginTabletCopySession request for tablet eabbcb31af804f2dac6af5731e18a4dd from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:58214)
I20260709 13:04:58.868242 15872 tablet_copy_service.cc:161] P 31882606e87447fb9a36bbf625129fe8: Beginning new tablet copy session on tablet eabbcb31af804f2dac6af5731e18a4dd from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:58214: session id = eb1e0493af1e46dbb6dec7e25525bab5-eabbcb31af804f2dac6af5731e18a4dd
I20260709 13:04:58.876252 15872 tablet_copy_source_session.cc:215] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:58.880163 16954 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eabbcb31af804f2dac6af5731e18a4dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:58.905321 16954 tablet_copy_client.cc:806] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:58.906011 16954 tablet_copy_client.cc:670] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:58.913848 16954 tablet_copy_client.cc:538] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:58.927335 16954 tablet_bootstrap.cc:492] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:04:58.953300 16954 tablet_bootstrap.cc:492] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:58.954026 16954 tablet_bootstrap.cc:492] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:04:58.955194 16954 ts_tablet_manager.cc:1403] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.028s	user 0.010s	sys 0.018s
I20260709 13:04:58.958432 16954 raft_consensus.cc:359] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:58.959301 16954 raft_consensus.cc:740] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:04:58.960013 16954 consensus_queue.cc:260] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:58.966974 16954 ts_tablet_manager.cc:1434] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:04:58.969082 15872 tablet_copy_service.cc:342] P 31882606e87447fb9a36bbf625129fe8: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-eabbcb31af804f2dac6af5731e18a4dd received from {username='slave'} at 127.0.0.1:58214
I20260709 13:04:58.969554 15872 tablet_copy_service.cc:434] P 31882606e87447fb9a36bbf625129fe8: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-eabbcb31af804f2dac6af5731e18a4dd on tablet eabbcb31af804f2dac6af5731e18a4dd with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:04:59.032332 16954 ts_tablet_manager.cc:933] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:59.037056 16954 tablet_copy_client.cc:323] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.2:43009
I20260709 13:04:59.040881 15946 tablet_copy_service.cc:140] P 8430db80a39d4868bcf7641c16de35ea: Received BeginTabletCopySession request for tablet 626a74f434ad4e70bed42f8b404a6e81 from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:53636)
I20260709 13:04:59.041507 15946 tablet_copy_service.cc:161] P 8430db80a39d4868bcf7641c16de35ea: Beginning new tablet copy session on tablet 626a74f434ad4e70bed42f8b404a6e81 from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:53636: session id = eb1e0493af1e46dbb6dec7e25525bab5-626a74f434ad4e70bed42f8b404a6e81
I20260709 13:04:59.050184 15946 tablet_copy_source_session.cc:215] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:59.053568 16954 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 626a74f434ad4e70bed42f8b404a6e81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:59.070070 16959 ts_tablet_manager.cc:933] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:04:59.074812 16954 tablet_copy_client.cc:806] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:59.079717 16954 tablet_copy_client.cc:670] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:59.083141 16959 tablet_copy_client.cc:323] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.2:43009
I20260709 13:04:59.085019 16954 tablet_copy_client.cc:538] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:59.085433 15946 tablet_copy_service.cc:140] P 8430db80a39d4868bcf7641c16de35ea: Received BeginTabletCopySession request for tablet 0b3676041725488fb6ffad37889b386b from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:53636)
I20260709 13:04:59.085960 15946 tablet_copy_service.cc:161] P 8430db80a39d4868bcf7641c16de35ea: Beginning new tablet copy session on tablet 0b3676041725488fb6ffad37889b386b from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:53636: session id = eb1e0493af1e46dbb6dec7e25525bab5-0b3676041725488fb6ffad37889b386b
I20260709 13:04:59.101146 15946 tablet_copy_source_session.cc:215] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:59.101367 16954 tablet_bootstrap.cc:492] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:04:59.104563 16959 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b3676041725488fb6ffad37889b386b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:59.106325 16961 ts_tablet_manager.cc:933] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
I20260709 13:04:59.118616 16961 tablet_copy_client.cc:323] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.1:38033
I20260709 13:04:59.124886 15872 tablet_copy_service.cc:140] P 31882606e87447fb9a36bbf625129fe8: Received BeginTabletCopySession request for tablet 471dbd2e16264fe5837e8b7ea3601f94 from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:58214)
I20260709 13:04:59.125396 15872 tablet_copy_service.cc:161] P 31882606e87447fb9a36bbf625129fe8: Beginning new tablet copy session on tablet 471dbd2e16264fe5837e8b7ea3601f94 from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:58214: session id = eb1e0493af1e46dbb6dec7e25525bab5-471dbd2e16264fe5837e8b7ea3601f94
I20260709 13:04:59.134444 15872 tablet_copy_source_session.cc:215] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:59.138064 16961 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 471dbd2e16264fe5837e8b7ea3601f94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:59.140179 16959 tablet_copy_client.cc:806] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:59.140772 16959 tablet_copy_client.cc:670] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:59.147778 16959 tablet_copy_client.cc:538] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:59.159902 16959 tablet_bootstrap.cc:492] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:04:59.177426 16961 tablet_copy_client.cc:806] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:59.182005 16961 tablet_copy_client.cc:670] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:59.190642 16961 tablet_copy_client.cc:538] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:59.190914 16954 tablet_bootstrap.cc:492] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:59.197299 16954 tablet_bootstrap.cc:492] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:04:59.198318 16954 ts_tablet_manager.cc:1403] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.097s	user 0.015s	sys 0.009s
I20260709 13:04:59.201910 16954 raft_consensus.cc:359] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:59.202952 16954 raft_consensus.cc:740] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:04:59.225056 16961 tablet_bootstrap.cc:492] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:04:59.228729 16954 consensus_queue.cc:260] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:59.232245 16954 ts_tablet_manager.cc:1434] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.002s
I20260709 13:04:59.236694 15946 tablet_copy_service.cc:342] P 8430db80a39d4868bcf7641c16de35ea: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-626a74f434ad4e70bed42f8b404a6e81 received from {username='slave'} at 127.0.0.1:53636
I20260709 13:04:59.237210 15946 tablet_copy_service.cc:434] P 8430db80a39d4868bcf7641c16de35ea: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-626a74f434ad4e70bed42f8b404a6e81 on tablet 626a74f434ad4e70bed42f8b404a6e81 with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:04:59.262331 16959 tablet_bootstrap.cc:492] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:59.263602 16959 tablet_bootstrap.cc:492] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:04:59.264346 16959 ts_tablet_manager.cc:1403] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.105s	user 0.019s	sys 0.037s
I20260709 13:04:59.289441 16959 raft_consensus.cc:359] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:59.290417 16959 raft_consensus.cc:740] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:04:59.291126 16959 consensus_queue.cc:260] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:59.302448 16959 ts_tablet_manager.cc:1434] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.037s	user 0.005s	sys 0.001s
I20260709 13:04:59.305672 15946 tablet_copy_service.cc:342] P 8430db80a39d4868bcf7641c16de35ea: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-0b3676041725488fb6ffad37889b386b received from {username='slave'} at 127.0.0.1:53636
I20260709 13:04:59.306111 15946 tablet_copy_service.cc:434] P 8430db80a39d4868bcf7641c16de35ea: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-0b3676041725488fb6ffad37889b386b on tablet 0b3676041725488fb6ffad37889b386b with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:04:59.341249 16961 tablet_bootstrap.cc:492] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:59.342283 16961 tablet_bootstrap.cc:492] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:04:59.343088 16961 ts_tablet_manager.cc:1403] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.118s	user 0.017s	sys 0.014s
I20260709 13:04:59.346495 16961 raft_consensus.cc:359] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:59.347469 16961 raft_consensus.cc:740] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:04:59.348328 16961 consensus_queue.cc:260] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:04:59.353958 16961 ts_tablet_manager.cc:1434] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.004s
I20260709 13:04:59.355924 15872 tablet_copy_service.cc:342] P 31882606e87447fb9a36bbf625129fe8: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-471dbd2e16264fe5837e8b7ea3601f94 received from {username='slave'} at 127.0.0.1:58214
I20260709 13:04:59.356351 15872 tablet_copy_service.cc:434] P 31882606e87447fb9a36bbf625129fe8: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-471dbd2e16264fe5837e8b7ea3601f94 on tablet 471dbd2e16264fe5837e8b7ea3601f94 with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:04:59.383565 16569 raft_consensus.cc:1217] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:59.420890 16569 raft_consensus.cc:1217] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:59.640936 16569 raft_consensus.cc:1217] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:59.690954 16569 raft_consensus.cc:1217] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:59.851815 16899 raft_consensus.cc:1064] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:04:59.854374 16899 consensus_queue.cc:237] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:04:59.869626 16569 raft_consensus.cc:1275] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:59.872104 16008 raft_consensus.cc:1275] T eabbcb31af804f2dac6af5731e18a4dd P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:59.873188 16966 consensus_queue.cc:1048] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:59.877640 15930 raft_consensus.cc:1275] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:59.879304 16836 consensus_queue.cc:1048] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:59.880271 16966 consensus_queue.cc:1048] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:59.933010 16970 raft_consensus.cc:1064] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:04:59.935706 16970 consensus_queue.cc:237] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:04:59.986295 16976 log.cc:927] Time spent T eabbcb31af804f2dac6af5731e18a4dd P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:04:59.996575 16836 raft_consensus.cc:2955] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:00.000752 15935 raft_consensus.cc:2955] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:00.014576 15760 catalog_manager.cc:5697] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 reported cstate change: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:00.017356 16569 raft_consensus.cc:2955] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:00.035280 16008 raft_consensus.cc:2955] T eabbcb31af804f2dac6af5731e18a4dd P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:00.079650 15857 raft_consensus.cc:1275] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:00.080530 16007 raft_consensus.cc:1275] T 626a74f434ad4e70bed42f8b404a6e81 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:00.083222 16972 consensus_queue.cc:1048] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:00.086809 16569 raft_consensus.cc:1275] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:00.092895 16971 consensus_queue.cc:1048] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:00.101385 15859 consensus_queue.cc:237] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:00.121490 16567 raft_consensus.cc:1275] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:00.122763 16972 consensus_queue.cc:1048] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:00.128942 16899 consensus_queue.cc:1048] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:00.140585 15935 raft_consensus.cc:1275] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:00.143549 16836 consensus_queue.cc:1048] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:00.148062 16895 raft_consensus.cc:1064] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:05:00.152247 16899 raft_consensus.cc:2955] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:00.150676 16895 consensus_queue.cc:237] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:00.182031 16933 raft_consensus.cc:2955] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:00.189656 15935 raft_consensus.cc:2955] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:00.194623 16569 raft_consensus.cc:1275] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:00.196030 16007 raft_consensus.cc:1275] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:00.197290 15744 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet eabbcb31af804f2dac6af5731e18a4dd with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:00.203963 16971 consensus_queue.cc:1048] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:00.204807 16895 consensus_queue.cc:1048] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:00.214501 15759 catalog_manager.cc:5697] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 reported cstate change: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:00.217010 15859 raft_consensus.cc:1275] T 0b3676041725488fb6ffad37889b386b P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:00.225142 16569 raft_consensus.cc:2955] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:00.216140 16567 raft_consensus.cc:2955] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:00.245819 16966 raft_consensus.cc:1064] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:05:00.251740 16972 consensus_queue.cc:1048] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:00.255501 15859 raft_consensus.cc:2955] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:00.257263 16966 consensus_queue.cc:237] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:00.260440 16007 raft_consensus.cc:2955] T 626a74f434ad4e70bed42f8b404a6e81 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:00.264860 15995 tablet_service.cc:1558] Processing DeleteTablet for tablet eabbcb31af804f2dac6af5731e18a4dd with delete_type TABLET_DATA_TOMBSTONED (TS 85e997ba5003439781ec4f1f8db0c3ee not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41648
I20260709 13:05:00.303103 15759 catalog_manager.cc:5697] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:00.370625 15930 consensus_queue.cc:237] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:00.417421 16569 raft_consensus.cc:1275] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:00.418790 15857 raft_consensus.cc:1275] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:00.420090 16971 consensus_queue.cc:1048] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:00.420990 16933 consensus_queue.cc:1048] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:00.430652 16933 raft_consensus.cc:2955] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:00.432262 16569 raft_consensus.cc:2955] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:00.437098 16007 raft_consensus.cc:1275] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:00.438000 15857 raft_consensus.cc:2955] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:00.438928 16567 raft_consensus.cc:1275] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:00.440116 16836 consensus_queue.cc:1048] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:00.440951 16985 consensus_queue.cc:1048] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:00.454710 15759 catalog_manager.cc:5697] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:00.456902 16895 raft_consensus.cc:2955] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:00.465862 15993 tablet_service.cc:1558] Processing DeleteTablet for tablet 626a74f434ad4e70bed42f8b404a6e81 with delete_type TABLET_DATA_TOMBSTONED (TS 85e997ba5003439781ec4f1f8db0c3ee not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41648
I20260709 13:05:00.465421 15861 raft_consensus.cc:2955] T 0b3676041725488fb6ffad37889b386b P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:00.475755 15744 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 626a74f434ad4e70bed42f8b404a6e81 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:00.475317 16569 raft_consensus.cc:2955] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:05:00.494506 16992 log.cc:927] Time spent T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:05:00.494699 16986 raft_consensus.cc:2955] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:00.487891 16994 tablet_replica.cc:333] T eabbcb31af804f2dac6af5731e18a4dd P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:05:00.496713 15930 raft_consensus.cc:1275] T 471dbd2e16264fe5837e8b7ea3601f94 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:00.502512 16899 consensus_queue.cc:1048] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:00.506279 16006 raft_consensus.cc:2955] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:00.513953 15759 catalog_manager.cc:5697] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea reported cstate change: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:00.518685 16994 raft_consensus.cc:2243] T eabbcb31af804f2dac6af5731e18a4dd P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:00.519377 16994 raft_consensus.cc:2272] T eabbcb31af804f2dac6af5731e18a4dd P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:00.524267 16994 ts_tablet_manager.cc:1916] T eabbcb31af804f2dac6af5731e18a4dd P 85e997ba5003439781ec4f1f8db0c3ee: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:00.530467 15760 catalog_manager.cc:5697] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:00.535483 16567 raft_consensus.cc:2955] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:00.535655 16011 raft_consensus.cc:2955] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:00.562983 15935 raft_consensus.cc:2955] T 471dbd2e16264fe5837e8b7ea3601f94 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:00.583559 15857 consensus_queue.cc:237] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:00.594740 16011 raft_consensus.cc:1275] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:00.595892 16569 raft_consensus.cc:1275] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:00.593159 15935 consensus_queue.cc:237] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:00.598752 16986 consensus_queue.cc:1048] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:00.599406 16998 consensus_queue.cc:1048] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:00.611295 16990 raft_consensus.cc:2955] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:00.618084 16569 raft_consensus.cc:2955] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:00.620497 16567 raft_consensus.cc:1275] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:00.622272 16970 consensus_queue.cc:1048] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:00.627489 16011 raft_consensus.cc:2955] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:00.630396 16971 raft_consensus.cc:2955] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:00.635766 16006 raft_consensus.cc:1275] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:00.644253 16933 consensus_queue.cc:1048] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:00.645658 16568 raft_consensus.cc:2955] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:00.654659 15744 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 471dbd2e16264fe5837e8b7ea3601f94 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:00.662567 15744 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0b3676041725488fb6ffad37889b386b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:00.670357 15757 catalog_manager.cc:5697] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:00.681130 16011 raft_consensus.cc:2955] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:00.684111 15755 catalog_manager.cc:5697] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea reported cstate change: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:00.688726 15923 tablet_service.cc:1558] Processing DeleteTablet for tablet 471dbd2e16264fe5837e8b7ea3601f94 with delete_type TABLET_DATA_TOMBSTONED (TS 8430db80a39d4868bcf7641c16de35ea not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58386
I20260709 13:05:00.704373 17000 tablet_replica.cc:333] T 471dbd2e16264fe5837e8b7ea3601f94 P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:05:00.705672 17000 raft_consensus.cc:2243] T 471dbd2e16264fe5837e8b7ea3601f94 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:00.706888 15846 tablet_service.cc:1558] Processing DeleteTablet for tablet 0b3676041725488fb6ffad37889b386b with delete_type TABLET_DATA_TOMBSTONED (TS 31882606e87447fb9a36bbf625129fe8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43504
I20260709 13:05:00.706442 17000 raft_consensus.cc:2272] T 471dbd2e16264fe5837e8b7ea3601f94 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:00.722662 17000 ts_tablet_manager.cc:1916] T 471dbd2e16264fe5837e8b7ea3601f94 P 8430db80a39d4868bcf7641c16de35ea: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:00.733872 17001 tablet_replica.cc:333] T 0b3676041725488fb6ffad37889b386b P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:05:00.734233 16994 ts_tablet_manager.cc:1929] T eabbcb31af804f2dac6af5731e18a4dd P 85e997ba5003439781ec4f1f8db0c3ee: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:00.734834 16994 log.cc:1199] T eabbcb31af804f2dac6af5731e18a4dd P 85e997ba5003439781ec4f1f8db0c3ee: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-2-root/wals/eabbcb31af804f2dac6af5731e18a4dd
I20260709 13:05:00.735128 17001 raft_consensus.cc:2243] T 0b3676041725488fb6ffad37889b386b P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:00.735886 17001 raft_consensus.cc:2272] T 0b3676041725488fb6ffad37889b386b P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:00.736860 15746 catalog_manager.cc:5028] TS 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): tablet eabbcb31af804f2dac6af5731e18a4dd (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:05:00.738241 16994 tablet_replica.cc:333] T 626a74f434ad4e70bed42f8b404a6e81 P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:05:00.741451 16994 raft_consensus.cc:2243] T 626a74f434ad4e70bed42f8b404a6e81 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:00.742364 16994 raft_consensus.cc:2272] T 626a74f434ad4e70bed42f8b404a6e81 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:00.744570 17001 ts_tablet_manager.cc:1916] T 0b3676041725488fb6ffad37889b386b P 31882606e87447fb9a36bbf625129fe8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:00.746057 16994 ts_tablet_manager.cc:1916] T 626a74f434ad4e70bed42f8b404a6e81 P 85e997ba5003439781ec4f1f8db0c3ee: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:00.754752 17000 ts_tablet_manager.cc:1929] T 471dbd2e16264fe5837e8b7ea3601f94 P 8430db80a39d4868bcf7641c16de35ea: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:00.755306 17000 log.cc:1199] T 471dbd2e16264fe5837e8b7ea3601f94 P 8430db80a39d4868bcf7641c16de35ea: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-1-root/wals/471dbd2e16264fe5837e8b7ea3601f94
I20260709 13:05:00.757032 15744 catalog_manager.cc:5028] TS 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): tablet 471dbd2e16264fe5837e8b7ea3601f94 (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:05:00.763145 17001 ts_tablet_manager.cc:1929] T 0b3676041725488fb6ffad37889b386b P 31882606e87447fb9a36bbf625129fe8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:00.763553 17001 log.cc:1199] T 0b3676041725488fb6ffad37889b386b P 31882606e87447fb9a36bbf625129fe8: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-0-root/wals/0b3676041725488fb6ffad37889b386b
I20260709 13:05:00.764948 15744 catalog_manager.cc:5028] TS 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): tablet 0b3676041725488fb6ffad37889b386b (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:05:00.770687 16994 ts_tablet_manager.cc:1929] T 626a74f434ad4e70bed42f8b404a6e81 P 85e997ba5003439781ec4f1f8db0c3ee: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:00.771160 16994 log.cc:1199] T 626a74f434ad4e70bed42f8b404a6e81 P 85e997ba5003439781ec4f1f8db0c3ee: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-2-root/wals/626a74f434ad4e70bed42f8b404a6e81
I20260709 13:05:00.772710 15746 catalog_manager.cc:5028] TS 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): tablet 626a74f434ad4e70bed42f8b404a6e81 (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:05:01.807022 16011 consensus_queue.cc:237] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
W20260709 13:05:01.814826 15976 consensus_peers.cc:597] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e236a7bf118740c9b3d88227bed5670e. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:01.815070 15857 raft_consensus.cc:1275] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:01.815167 15931 raft_consensus.cc:1275] T e236a7bf118740c9b3d88227bed5670e P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:01.816947 16894 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:01.819280 16968 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:01.830106 16968 raft_consensus.cc:2955] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:01.841668 15857 raft_consensus.cc:2955] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:01.849097 16011 consensus_queue.cc:237] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:01.854139 15759 catalog_manager.cc:5697] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:01.858757 15931 raft_consensus.cc:2955] T e236a7bf118740c9b3d88227bed5670e P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
W20260709 13:05:01.898871 15976 consensus_peers.cc:597] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fd8e1402dafb4300911e5b0189c8fe0a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:01.899699 15930 raft_consensus.cc:1275] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:01.901000 16968 consensus_queue.cc:1048] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:01.901424 15857 raft_consensus.cc:1275] T fd8e1402dafb4300911e5b0189c8fe0a P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:01.902760 16894 consensus_queue.cc:1048] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:01.920233 16968 raft_consensus.cc:2955] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:01.928625 15859 raft_consensus.cc:2955] T fd8e1402dafb4300911e5b0189c8fe0a P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:01.932579 15930 raft_consensus.cc:2955] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:01.946030 15760 catalog_manager.cc:5697] T fd8e1402dafb4300911e5b0189c8fe0a P 31882606e87447fb9a36bbf625129fe8 reported cstate change: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:01.950560 15935 consensus_queue.cc:237] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
W20260709 13:05:01.962946 15902 consensus_peers.cc:597] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1d9c19e253ce4d53bb9e2f2241d3cafc. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:01.964021 16011 raft_consensus.cc:1275] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:01.966149 15857 raft_consensus.cc:1275] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:01.966203 16971 consensus_queue.cc:1048] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:01.971132 16971 consensus_queue.cc:1048] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:01.974416 17012 raft_consensus.cc:2955] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:01.983247 16011 raft_consensus.cc:2955] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:01.990999 15859 consensus_queue.cc:237] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:01.998823 15755 catalog_manager.cc:5697] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea reported cstate change: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:02.001121 15861 raft_consensus.cc:2955] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
W20260709 13:05:02.043008 15827 consensus_peers.cc:597] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dd0dfa7599d64160b42b750a38e5c239. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:02.043990 15935 raft_consensus.cc:1275] T dd0dfa7599d64160b42b750a38e5c239 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:02.046346 16988 consensus_queue.cc:1048] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:02.046626 16011 raft_consensus.cc:1275] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:02.048308 16990 consensus_queue.cc:1048] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:02.064821 16988 raft_consensus.cc:2955] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:02.066766 15935 raft_consensus.cc:2955] T dd0dfa7599d64160b42b750a38e5c239 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:02.076895 16006 raft_consensus.cc:2955] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:02.097236 15759 catalog_manager.cc:5697] T dd0dfa7599d64160b42b750a38e5c239 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:02.399897 17025 ts_tablet_manager.cc:933] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:02.404443 17025 tablet_copy_client.cc:323] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.3:46251
I20260709 13:05:02.406262 16021 tablet_copy_service.cc:140] P 85e997ba5003439781ec4f1f8db0c3ee: Received BeginTabletCopySession request for tablet e236a7bf118740c9b3d88227bed5670e from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:60156)
I20260709 13:05:02.406836 16021 tablet_copy_service.cc:161] P 85e997ba5003439781ec4f1f8db0c3ee: Beginning new tablet copy session on tablet e236a7bf118740c9b3d88227bed5670e from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:60156: session id = eb1e0493af1e46dbb6dec7e25525bab5-e236a7bf118740c9b3d88227bed5670e
I20260709 13:05:02.414490 16021 tablet_copy_source_session.cc:215] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:02.417621 17025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e236a7bf118740c9b3d88227bed5670e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:02.433178 17025 tablet_copy_client.cc:806] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:02.434345 17025 tablet_copy_client.cc:670] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:02.434763 17027 ts_tablet_manager.cc:933] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:02.439621 17025 tablet_copy_client.cc:538] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:02.454380 17025 tablet_bootstrap.cc:492] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:05:02.459856 17027 tablet_copy_client.cc:323] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.3:46251
I20260709 13:05:02.463027 16021 tablet_copy_service.cc:140] P 85e997ba5003439781ec4f1f8db0c3ee: Received BeginTabletCopySession request for tablet fd8e1402dafb4300911e5b0189c8fe0a from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:60156)
I20260709 13:05:02.463542 16021 tablet_copy_service.cc:161] P 85e997ba5003439781ec4f1f8db0c3ee: Beginning new tablet copy session on tablet fd8e1402dafb4300911e5b0189c8fe0a from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:60156: session id = eb1e0493af1e46dbb6dec7e25525bab5-fd8e1402dafb4300911e5b0189c8fe0a
I20260709 13:05:02.474110 16021 tablet_copy_source_session.cc:215] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:02.477407 17027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd8e1402dafb4300911e5b0189c8fe0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:02.485911 17025 tablet_bootstrap.cc:492] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:02.487257 17025 tablet_bootstrap.cc:492] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:05:02.487994 17025 ts_tablet_manager.cc:1403] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.034s	user 0.020s	sys 0.004s
I20260709 13:05:02.493088 17025 raft_consensus.cc:359] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:02.493963 17025 raft_consensus.cc:740] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:05:02.494252 17027 tablet_copy_client.cc:806] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:02.494863 17027 tablet_copy_client.cc:670] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:02.495472 17025 consensus_queue.cc:260] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:02.503887 17027 tablet_copy_client.cc:538] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:02.505509 17025 ts_tablet_manager.cc:1434] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.004s
I20260709 13:05:02.509902 16021 tablet_copy_service.cc:342] P 85e997ba5003439781ec4f1f8db0c3ee: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-e236a7bf118740c9b3d88227bed5670e received from {username='slave'} at 127.0.0.1:60156
I20260709 13:05:02.510411 16021 tablet_copy_service.cc:434] P 85e997ba5003439781ec4f1f8db0c3ee: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-e236a7bf118740c9b3d88227bed5670e on tablet e236a7bf118740c9b3d88227bed5670e with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:02.516006 17027 tablet_bootstrap.cc:492] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:05:02.537626 17027 tablet_bootstrap.cc:492] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:02.538525 17027 tablet_bootstrap.cc:492] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:05:02.539188 17027 ts_tablet_manager.cc:1403] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.023s	user 0.017s	sys 0.003s
I20260709 13:05:02.541936 17027 raft_consensus.cc:359] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:02.542734 17027 raft_consensus.cc:740] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:05:02.543360 17027 consensus_queue.cc:260] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:02.547967 17027 ts_tablet_manager.cc:1434] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:05:02.550961 16021 tablet_copy_service.cc:342] P 85e997ba5003439781ec4f1f8db0c3ee: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-fd8e1402dafb4300911e5b0189c8fe0a received from {username='slave'} at 127.0.0.1:60156
I20260709 13:05:02.551362 16021 tablet_copy_service.cc:434] P 85e997ba5003439781ec4f1f8db0c3ee: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-fd8e1402dafb4300911e5b0189c8fe0a on tablet fd8e1402dafb4300911e5b0189c8fe0a with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:02.581396 17027 ts_tablet_manager.cc:933] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
I20260709 13:05:02.582051 17025 ts_tablet_manager.cc:933] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:02.596434 17025 tablet_copy_client.cc:323] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.2:43009
I20260709 13:05:02.598805 15946 tablet_copy_service.cc:140] P 8430db80a39d4868bcf7641c16de35ea: Received BeginTabletCopySession request for tablet 1d9c19e253ce4d53bb9e2f2241d3cafc from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:53636)
I20260709 13:05:02.599606 15946 tablet_copy_service.cc:161] P 8430db80a39d4868bcf7641c16de35ea: Beginning new tablet copy session on tablet 1d9c19e253ce4d53bb9e2f2241d3cafc from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:53636: session id = eb1e0493af1e46dbb6dec7e25525bab5-1d9c19e253ce4d53bb9e2f2241d3cafc
I20260709 13:05:02.600775 17027 tablet_copy_client.cc:323] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.1:38033
I20260709 13:05:02.602787 15872 tablet_copy_service.cc:140] P 31882606e87447fb9a36bbf625129fe8: Received BeginTabletCopySession request for tablet dd0dfa7599d64160b42b750a38e5c239 from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:58214)
I20260709 13:05:02.603375 15872 tablet_copy_service.cc:161] P 31882606e87447fb9a36bbf625129fe8: Beginning new tablet copy session on tablet dd0dfa7599d64160b42b750a38e5c239 from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:58214: session id = eb1e0493af1e46dbb6dec7e25525bab5-dd0dfa7599d64160b42b750a38e5c239
I20260709 13:05:02.608343 15946 tablet_copy_source_session.cc:215] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:02.611265 15872 tablet_copy_source_session.cc:215] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:02.612816 17025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d9c19e253ce4d53bb9e2f2241d3cafc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:02.615322 17027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd0dfa7599d64160b42b750a38e5c239. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:02.632213 17027 tablet_copy_client.cc:806] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:02.632606 17025 tablet_copy_client.cc:806] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:02.632753 17027 tablet_copy_client.cc:670] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:02.633348 17025 tablet_copy_client.cc:670] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:02.638622 17027 tablet_copy_client.cc:538] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:02.638672 17025 tablet_copy_client.cc:538] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:02.652186 17025 tablet_bootstrap.cc:492] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:05:02.658133 17027 tablet_bootstrap.cc:492] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:05:02.688297 17025 tablet_bootstrap.cc:492] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:02.689376 17025 tablet_bootstrap.cc:492] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:05:02.690171 17025 ts_tablet_manager.cc:1403] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.038s	user 0.024s	sys 0.005s
I20260709 13:05:02.693687 17025 raft_consensus.cc:359] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:02.695915 17025 raft_consensus.cc:740] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:05:02.696620 17025 consensus_queue.cc:260] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:02.699395 17025 ts_tablet_manager.cc:1434] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:05:02.701145 15946 tablet_copy_service.cc:342] P 8430db80a39d4868bcf7641c16de35ea: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-1d9c19e253ce4d53bb9e2f2241d3cafc received from {username='slave'} at 127.0.0.1:53636
I20260709 13:05:02.701557 15946 tablet_copy_service.cc:434] P 8430db80a39d4868bcf7641c16de35ea: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-1d9c19e253ce4d53bb9e2f2241d3cafc on tablet 1d9c19e253ce4d53bb9e2f2241d3cafc with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:02.739847 17027 tablet_bootstrap.cc:492] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:02.740840 17027 tablet_bootstrap.cc:492] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:05:02.741500 17027 ts_tablet_manager.cc:1403] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.084s	user 0.038s	sys 0.036s
I20260709 13:05:02.744390 17027 raft_consensus.cc:359] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:02.745183 17027 raft_consensus.cc:740] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:05:02.745805 17027 consensus_queue.cc:260] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:02.748349 17027 ts_tablet_manager.cc:1434] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 13:05:02.750581 15872 tablet_copy_service.cc:342] P 31882606e87447fb9a36bbf625129fe8: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-dd0dfa7599d64160b42b750a38e5c239 received from {username='slave'} at 127.0.0.1:58214
I20260709 13:05:02.751052 15872 tablet_copy_service.cc:434] P 31882606e87447fb9a36bbf625129fe8: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-dd0dfa7599d64160b42b750a38e5c239 on tablet dd0dfa7599d64160b42b750a38e5c239 with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:02.940162 16567 raft_consensus.cc:1217] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:03.019904 16567 raft_consensus.cc:1217] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:03.066797 16568 raft_consensus.cc:1217] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:03.113778 16567 raft_consensus.cc:1217] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:03.458911 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:03.468160 17032 raft_consensus.cc:1064] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:05:03.470589 17032 consensus_queue.cc:237] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:03.490748 16567 raft_consensus.cc:1275] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:03.492096 15935 raft_consensus.cc:1275] T e236a7bf118740c9b3d88227bed5670e P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:03.492312 15859 raft_consensus.cc:1275] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:03.493520 16894 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:03.494946 16968 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:03.496596 16968 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:03.517061 17032 raft_consensus.cc:2955] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:03.519425 15935 raft_consensus.cc:2955] T e236a7bf118740c9b3d88227bed5670e P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:03.531497 15859 raft_consensus.cc:2955] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:03.546538 16567 raft_consensus.cc:2955] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:03.571655 15759 catalog_manager.cc:5697] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:03.574692 17044 raft_consensus.cc:1064] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:05:03.584709 17044 consensus_queue.cc:237] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:03.587747 17032 raft_consensus.cc:1064] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:05:03.590225 17032 consensus_queue.cc:237] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:03.600176 15930 raft_consensus.cc:1275] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:03.601559 16894 consensus_queue.cc:1048] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:03.606803 15861 raft_consensus.cc:1275] T fd8e1402dafb4300911e5b0189c8fe0a P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:03.608562 16567 raft_consensus.cc:1275] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:03.610126 16894 consensus_queue.cc:1048] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:03.615401 15931 raft_consensus.cc:1275] T dd0dfa7599d64160b42b750a38e5c239 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:03.617214 16988 consensus_queue.cc:1048] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:03.613029 17032 consensus_queue.cc:1048] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 13:05:03.636061 16970 raft_consensus.cc:1064] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:05:03.638921 16970 consensus_queue.cc:237] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:03.671917 16567 raft_consensus.cc:1275] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:03.674552 16006 raft_consensus.cc:1275] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:05:03.694653 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 44826.3us
I20260709 13:05:03.732131 16569 raft_consensus.cc:1275] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:03.746449 17053 consensus_queue.cc:1048] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:03.747298 15861 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:05:03.748544 15861 raft_consensus.cc:606] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Received request to transfer leadership to TS eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:03.749910 17059 consensus_queue.cc:1048] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:05:03.753786 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 46027.8us
I20260709 13:05:03.767457 16007 raft_consensus.cc:1275] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:03.775632 17055 consensus_queue.cc:1048] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:03.783259 16568 raft_consensus.cc:2955] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:03.786942 17041 raft_consensus.cc:2955] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:05:03.797621 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 55214.5us
I20260709 13:05:03.811530 15930 raft_consensus.cc:2955] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:05:03.817332 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 62037.5us
W20260709 13:05:03.824386 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 77455.7us
I20260709 13:05:03.826416 17058 consensus_queue.cc:1048] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:03.832401 15861 raft_consensus.cc:2955] T fd8e1402dafb4300911e5b0189c8fe0a P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:03.855036 15861 raft_consensus.cc:1275] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:03.852005 17057 raft_consensus.cc:2955] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:03.875416 17059 raft_consensus.cc:2955] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:03.879913 17010 consensus_queue.cc:1048] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:03.878052 16007 raft_consensus.cc:2955] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:03.890692 16568 raft_consensus.cc:2955] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:03.905296 16011 raft_consensus.cc:2955] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:03.905436 15931 raft_consensus.cc:2955] T dd0dfa7599d64160b42b750a38e5c239 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:03.917893 15859 raft_consensus.cc:2955] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:03.942193 16011 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef"
dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
mode: GRACEFUL
new_leader_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:37262
I20260709 13:05:03.941653 16567 raft_consensus.cc:2955] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:03.942826 16011 raft_consensus.cc:606] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Received request to transfer leadership to TS 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:03.954545 15755 catalog_manager.cc:5697] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:03.957036 15755 catalog_manager.cc:5697] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:03.973018 15760 catalog_manager.cc:5697] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:03.977303 17072 raft_consensus.cc:993] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee: : Instructing follower 31882606e87447fb9a36bbf625129fe8 to start an election
I20260709 13:05:03.977865 17072 raft_consensus.cc:1081] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Signalling peer 31882606e87447fb9a36bbf625129fe8 to start an election
I20260709 13:05:03.979697 15861 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:43524
I20260709 13:05:03.980283 15861 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:03.980643 15861 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:04.000427 15859 consensus_queue.cc:237] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:04.006690 15861 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:04.008133 15857 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "78530df4cdb742eda67781671b0431c6"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:05:04.008693 15857 raft_consensus.cc:606] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Received request to transfer leadership to TS eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:04.016245 16007 consensus_queue.cc:237] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:04.020325 16988 raft_consensus.cc:993] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8: : Instructing follower eb1e0493af1e46dbb6dec7e25525bab5 to start an election
I20260709 13:05:04.020812 16988 raft_consensus.cc:1081] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Signalling peer eb1e0493af1e46dbb6dec7e25525bab5 to start an election
I20260709 13:05:04.022243 16569 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6"
dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:45246
I20260709 13:05:04.022727 16569 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:04.023030 16569 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:04.025988 16011 consensus_queue.cc:237] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:04.030380 16569 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:04.037418 16011 raft_consensus.cc:1275] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:04.032872 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:04.033916 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:04.038115 15859 raft_consensus.cc:3055] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:04.038178 15930 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:04.038509 15859 raft_consensus.cc:740] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:04.039255 15859 consensus_queue.cc:260] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:04.040289 15859 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:04.041186 15935 raft_consensus.cc:1275] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:04.043887 17032 consensus_queue.cc:1048] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:04.044996 15859 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:04.045940 15930 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:04.049896 16533 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:04.058013 17030 raft_consensus.cc:2804] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:04.058202 16968 raft_consensus.cc:2955] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:04.033985 16570 raft_consensus.cc:1275] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:04.034003 15857 raft_consensus.cc:1275] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:04.054598 17059 consensus_queue.cc:1048] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:04.061969 17072 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:04.063741 17072 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:04.055864 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:04.069880 15931 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:04.034965 16567 raft_consensus.cc:1275] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:04.071382 15746 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet fd8e1402dafb4300911e5b0189c8fe0a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:04.073477 16894 consensus_queue.cc:1048] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:04.034112 16569 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:04.075824 15857 raft_consensus.cc:2955] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:04.057014 15859 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:05:04.036231 16566 raft_consensus.cc:1275] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:04.087785 17030 raft_consensus.cc:697] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:04.088843 15859 raft_consensus.cc:606] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Received request to transfer leadership to TS eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:04.088763 17030 consensus_queue.cc:237] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:04.072798 17041 raft_consensus.cc:2955] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:04.091468 15861 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:04.098945 15931 raft_consensus.cc:2468] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:04.100152 15825 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:04.101066 16988 raft_consensus.cc:2804] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:04.101753 16988 raft_consensus.cc:697] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:04.102550 16988 consensus_queue.cc:237] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:04.102329 15755 catalog_manager.cc:5697] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:04.105376 16566 raft_consensus.cc:2955] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:04.117187 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:04.117800 16011 raft_consensus.cc:3055] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:04.118137 16011 raft_consensus.cc:740] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:04.119342 17024 consensus_queue.cc:1048] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:04.119225 15931 raft_consensus.cc:2955] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:04.120409 16011 consensus_queue.cc:260] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:04.123456 16011 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:04.125854 15846 tablet_service.cc:1558] Processing DeleteTablet for tablet fd8e1402dafb4300911e5b0189c8fe0a with delete_type TABLET_DATA_TOMBSTONED (TS 31882606e87447fb9a36bbf625129fe8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43504
I20260709 13:05:04.131758 15746 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e236a7bf118740c9b3d88227bed5670e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:04.136644 15859 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:05:04.137488 15859 raft_consensus.cc:606] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Received request to transfer leadership to TS eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:04.131289 16569 raft_consensus.cc:2955] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:04.143018 17058 raft_consensus.cc:2955] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:04.145231 15758 catalog_manager.cc:5697] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 1 to 2, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:04.158452 16007 raft_consensus.cc:2955] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:04.172582 16011 raft_consensus.cc:2468] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:04.173302 15744 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet dd0dfa7599d64160b42b750a38e5c239 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:04.192425 16566 raft_consensus.cc:2955] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:04.194818 17044 raft_consensus.cc:993] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8: : Instructing follower eb1e0493af1e46dbb6dec7e25525bab5 to start an election
I20260709 13:05:04.195307 17044 raft_consensus.cc:1081] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Signalling peer eb1e0493af1e46dbb6dec7e25525bab5 to start an election
I20260709 13:05:04.195699 15759 catalog_manager.cc:5697] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 1 to 2, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 2 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:04.199519 15758 catalog_manager.cc:5697] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 reported cstate change: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:05:04.203899 17077 log.cc:927] Time spent T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:05:04.203563 15758 catalog_manager.cc:5697] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:04.206122 16569 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0"
dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:45246
I20260709 13:05:04.206664 16569 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:04.207406 16569 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:04.209540 17078 tablet_replica.cc:333] T fd8e1402dafb4300911e5b0189c8fe0a P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:05:04.212819 17078 raft_consensus.cc:2243] T fd8e1402dafb4300911e5b0189c8fe0a P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:04.213544 17078 raft_consensus.cc:2272] T fd8e1402dafb4300911e5b0189c8fe0a P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:04.215169 16569 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:04.219177 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:04.220077 15857 raft_consensus.cc:3055] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:04.220358 15857 raft_consensus.cc:740] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:04.221062 15857 consensus_queue.cc:260] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:04.222011 15857 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:04.224244 16569 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:04.226763 15931 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:46118
I20260709 13:05:04.227345 15931 raft_consensus.cc:606] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Received request to transfer leadership to TS eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:04.227411 17059 raft_consensus.cc:993] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8: : Instructing follower eb1e0493af1e46dbb6dec7e25525bab5 to start an election
I20260709 13:05:04.228010 17059 raft_consensus.cc:1081] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Signalling peer eb1e0493af1e46dbb6dec7e25525bab5 to start an election
I20260709 13:05:04.228863 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:05:04.229466 16568 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589"
dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:45246
I20260709 13:05:04.229996 16568 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:04.229651 17078 ts_tablet_manager.cc:1916] T fd8e1402dafb4300911e5b0189c8fe0a P 31882606e87447fb9a36bbf625129fe8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:04.230327 16568 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:04.232791 15857 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:04.237331 16568 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:04.241783 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:04.245151 16568 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:04.240404 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:04.246312 16011 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:04.246447 15923 tablet_service.cc:1558] Processing DeleteTablet for tablet e236a7bf118740c9b3d88227bed5670e with delete_type TABLET_DATA_TOMBSTONED (TS 8430db80a39d4868bcf7641c16de35ea not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58386
I20260709 13:05:04.247448 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:04.247980 15859 raft_consensus.cc:3055] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:04.248255 15859 raft_consensus.cc:740] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:04.248857 15859 consensus_queue.cc:260] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:04.249874 15859 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:04.250113 15919 tablet_service.cc:1558] Processing DeleteTablet for tablet dd0dfa7599d64160b42b750a38e5c239 with delete_type TABLET_DATA_TOMBSTONED (TS 8430db80a39d4868bcf7641c16de35ea not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58386
I20260709 13:05:04.251196 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:04.251837 16007 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:04.254354 16533 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:04.257633 16011 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:04.258555 17030 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:04.259052 17030 raft_consensus.cc:697] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:04.259919 17030 consensus_queue.cc:237] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:04.270867 16007 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:04.272871 16535 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:04.251813 17080 tablet_replica.cc:333] T e236a7bf118740c9b3d88227bed5670e P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:05:04.274715 17080 raft_consensus.cc:2243] T e236a7bf118740c9b3d88227bed5670e P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:04.274952 17074 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:04.275496 17080 raft_consensus.cc:2272] T e236a7bf118740c9b3d88227bed5670e P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:04.279170 17078 ts_tablet_manager.cc:1929] T fd8e1402dafb4300911e5b0189c8fe0a P 31882606e87447fb9a36bbf625129fe8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:04.279737 17078 log.cc:1199] T fd8e1402dafb4300911e5b0189c8fe0a P 31882606e87447fb9a36bbf625129fe8: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-0-root/wals/fd8e1402dafb4300911e5b0189c8fe0a
I20260709 13:05:04.286276 15744 catalog_manager.cc:5028] TS 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): tablet fd8e1402dafb4300911e5b0189c8fe0a (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:05:04.288856 15859 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:04.292457 17074 raft_consensus.cc:697] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:04.293463 17074 consensus_queue.cc:237] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:04.295845 17080 ts_tablet_manager.cc:1916] T e236a7bf118740c9b3d88227bed5670e P 8430db80a39d4868bcf7641c16de35ea: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:04.297945 15759 catalog_manager.cc:5697] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 1 to 2, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 2 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:04.315840 15759 catalog_manager.cc:5697] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 1 to 2, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 2 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:04.319852 17080 ts_tablet_manager.cc:1929] T e236a7bf118740c9b3d88227bed5670e P 8430db80a39d4868bcf7641c16de35ea: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:04.320429 17080 log.cc:1199] T e236a7bf118740c9b3d88227bed5670e P 8430db80a39d4868bcf7641c16de35ea: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-1-root/wals/e236a7bf118740c9b3d88227bed5670e
I20260709 13:05:04.322244 17080 tablet_replica.cc:333] T dd0dfa7599d64160b42b750a38e5c239 P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:05:04.322268 15744 catalog_manager.cc:5028] TS 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): tablet e236a7bf118740c9b3d88227bed5670e (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:05:04.324262 17080 raft_consensus.cc:2243] T dd0dfa7599d64160b42b750a38e5c239 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:04.325002 17080 raft_consensus.cc:2272] T dd0dfa7599d64160b42b750a38e5c239 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:04.329799 17080 ts_tablet_manager.cc:1916] T dd0dfa7599d64160b42b750a38e5c239 P 8430db80a39d4868bcf7641c16de35ea: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:04.345235 17080 ts_tablet_manager.cc:1929] T dd0dfa7599d64160b42b750a38e5c239 P 8430db80a39d4868bcf7641c16de35ea: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:04.345613 17080 log.cc:1199] T dd0dfa7599d64160b42b750a38e5c239 P 8430db80a39d4868bcf7641c16de35ea: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-1-root/wals/dd0dfa7599d64160b42b750a38e5c239
I20260709 13:05:04.347029 15744 catalog_manager.cc:5028] TS 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): tablet dd0dfa7599d64160b42b750a38e5c239 (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:05:04.427119 17059 raft_consensus.cc:993] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8: : Instructing follower eb1e0493af1e46dbb6dec7e25525bab5 to start an election
I20260709 13:05:04.427721 17044 raft_consensus.cc:1081] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Signalling peer eb1e0493af1e46dbb6dec7e25525bab5 to start an election
I20260709 13:05:04.428725 17056 raft_consensus.cc:993] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea: : Instructing follower eb1e0493af1e46dbb6dec7e25525bab5 to start an election
I20260709 13:05:04.429203 17060 raft_consensus.cc:1081] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Signalling peer eb1e0493af1e46dbb6dec7e25525bab5 to start an election
I20260709 13:05:04.429289 16566 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "78530df4cdb742eda67781671b0431c6"
dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:45246
I20260709 13:05:04.429862 16566 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:04.430245 16566 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:04.430815 16569 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def"
dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:45244
I20260709 13:05:04.431350 16569 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:04.431697 16569 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:04.437129 16566 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:04.437932 16569 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:04.439626 16569 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:04.439311 16566 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:04.441146 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:04.441705 15857 raft_consensus.cc:3055] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:04.442018 15857 raft_consensus.cc:740] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:04.442678 15857 consensus_queue.cc:260] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:04.443734 15857 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:04.444746 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:04.445307 15931 raft_consensus.cc:3055] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:04.445580 15931 raft_consensus.cc:740] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:04.445621 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:04.446144 15930 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:04.446374 15931 consensus_queue.cc:260] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:04.447372 15931 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:04.454762 15857 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:04.455892 16533 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:04.456036 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:04.456599 16007 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:04.456764 17074 raft_consensus.cc:2804] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:04.457288 17074 raft_consensus.cc:697] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:04.458099 17074 consensus_queue.cc:237] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:04.459595 15930 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:04.462924 15931 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:04.466945 16007 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:04.468490 16533 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:04.469314 17030 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:04.469774 17030 raft_consensus.cc:697] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:04.470592 17030 consensus_queue.cc:237] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:04.494792 15759 catalog_manager.cc:5697] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 1 to 2, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 2 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:04.509209 15758 catalog_manager.cc:5697] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 1 to 2, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 2 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:04.525090 15931 consensus_queue.cc:237] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:04.529889 16566 raft_consensus.cc:1275] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:04.529978 15858 raft_consensus.cc:1275] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:04.531239 17057 consensus_queue.cc:1048] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:04.532004 17056 consensus_queue.cc:1048] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:04.540030 17057 raft_consensus.cc:2955] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:04.541272 15858 raft_consensus.cc:2955] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:04.552352 15744 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1d9c19e253ce4d53bb9e2f2241d3cafc with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:04.558107 16569 raft_consensus.cc:2955] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:04.559334 15759 catalog_manager.cc:5697] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 reported cstate change: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:04.570123 15995 tablet_service.cc:1558] Processing DeleteTablet for tablet 1d9c19e253ce4d53bb9e2f2241d3cafc with delete_type TABLET_DATA_TOMBSTONED (TS 85e997ba5003439781ec4f1f8db0c3ee not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41648
I20260709 13:05:04.577886 17087 tablet_replica.cc:333] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:05:04.579784 17087 raft_consensus.cc:2243] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:04.580672 17087 raft_consensus.cc:2272] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:04.585216 17087 ts_tablet_manager.cc:1916] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 85e997ba5003439781ec4f1f8db0c3ee: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:04.601788 17087 ts_tablet_manager.cc:1929] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 85e997ba5003439781ec4f1f8db0c3ee: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:04.602180 17087 log.cc:1199] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 85e997ba5003439781ec4f1f8db0c3ee: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-2-root/wals/1d9c19e253ce4d53bb9e2f2241d3cafc
I20260709 13:05:04.604532 15746 catalog_manager.cc:5028] TS 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): tablet 1d9c19e253ce4d53bb9e2f2241d3cafc (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:05:04.632280 16007 raft_consensus.cc:1275] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:04.633472 16988 consensus_queue.cc:1048] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:04.642656 15930 raft_consensus.cc:1275] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:04.644801 17059 consensus_queue.cc:1048] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:04.680179 15857 raft_consensus.cc:1275] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:04.681236 17074 consensus_queue.cc:1048] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:04.683597 15930 raft_consensus.cc:1275] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:04.685062 17030 consensus_queue.cc:1048] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:04.702745 16011 raft_consensus.cc:1275] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:04.707021 17082 consensus_queue.cc:1048] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:04.708873 16011 raft_consensus.cc:1275] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:04.712657 17030 consensus_queue.cc:1048] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:04.716445 15857 raft_consensus.cc:1275] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:04.720227 17082 consensus_queue.cc:1048] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:04.728050 15858 raft_consensus.cc:1275] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:04.733340 17099 consensus_queue.cc:1048] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:04.839430 15858 raft_consensus.cc:1275] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:04.840626 17101 consensus_queue.cc:1048] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:04.852685 15931 raft_consensus.cc:1275] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:04.853965 17030 consensus_queue.cc:1048] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:04.981374 15931 raft_consensus.cc:1275] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:04.982530 17030 consensus_queue.cc:1048] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:04.990626 16007 raft_consensus.cc:1275] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:04.994451 17030 consensus_queue.cc:1048] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:05.242801 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:05.283034 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:05.317910 15931 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:46118
I20260709 13:05:05.318415 15931 raft_consensus.cc:606] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Received request to transfer leadership to TS 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:05.320866 15858 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:05:05.321288 15858 raft_consensus.cc:606] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Received request to transfer leadership to TS eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:05.323293 15931 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:46118
I20260709 13:05:05.323747 15931 raft_consensus.cc:606] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Received request to transfer leadership to TS eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:05.325768 16011 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f"
dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
mode: GRACEFUL
new_leader_uuid: "8430db80a39d4868bcf7641c16de35ea"
 from {username='slave'} at 127.0.0.1:37262
I20260709 13:05:05.326364 16011 raft_consensus.cc:606] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Received request to transfer leadership to TS 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:05.329339 15931 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:46118
I20260709 13:05:05.329874 15931 raft_consensus.cc:606] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Received request to transfer leadership to TS 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:05.331879 15931 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:46118
I20260709 13:05:05.332361 15931 raft_consensus.cc:606] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Received request to transfer leadership to TS eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:05.333711 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:05:05.335212 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:05.379475 17060 raft_consensus.cc:993] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea: : Instructing follower 31882606e87447fb9a36bbf625129fe8 to start an election
I20260709 13:05:05.380075 17060 raft_consensus.cc:1081] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Signalling peer 31882606e87447fb9a36bbf625129fe8 to start an election
I20260709 13:05:05.382001 15857 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:43520
I20260709 13:05:05.382431 15857 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:05.382714 15857 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:05.386926 15857 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:05.388469 15857 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:05.389602 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:05.390220 15931 raft_consensus.cc:3055] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:05.390504 15931 raft_consensus.cc:740] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:05.390897 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:05.391429 16011 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:05.391557 15931 consensus_queue.cc:260] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:05.392802 15931 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:05.397930 16011 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:05.398548 15931 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:05.399242 15827 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:05.400391 17107 raft_consensus.cc:2804] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:05.400903 17107 raft_consensus.cc:697] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:05.401871 17107 consensus_queue.cc:237] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:05.412734 15755 catalog_manager.cc:5697] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 1 to 2, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:05.448719 17108 raft_consensus.cc:993] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea: : Instructing follower eb1e0493af1e46dbb6dec7e25525bab5 to start an election
I20260709 13:05:05.449177 17057 raft_consensus.cc:1081] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Signalling peer eb1e0493af1e46dbb6dec7e25525bab5 to start an election
I20260709 13:05:05.450522 16566 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81"
dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:45244
I20260709 13:05:05.451048 16566 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:05.451428 16566 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:05.457747 16566 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:05.460158 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:05.460601 15858 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:05.463536 16566 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:05.464468 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:05.465006 15930 raft_consensus.cc:3055] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:05.465215 15930 raft_consensus.cc:740] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:05.465407 15858 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:05.465950 15930 consensus_queue.cc:260] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:05.466634 16533 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:05.467146 15930 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:05.467351 17030 raft_consensus.cc:2804] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:05.467862 17030 raft_consensus.cc:697] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:05.468689 17030 consensus_queue.cc:237] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:05.475461 15930 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:05.478432 15755 catalog_manager.cc:5697] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 1 to 2, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 2 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:05.631865 17107 raft_consensus.cc:993] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8: : Instructing follower eb1e0493af1e46dbb6dec7e25525bab5 to start an election
I20260709 13:05:05.632323 16988 raft_consensus.cc:1081] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Signalling peer eb1e0493af1e46dbb6dec7e25525bab5 to start an election
I20260709 13:05:05.633642 16566 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94"
dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:45246
I20260709 13:05:05.634182 16566 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:05.634526 16566 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:05.641584 16566 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:05.642091 15808 auto_rebalancer.cc:629] Failed to schedule move for tablet 850b6b8a2f54468fb87ea6a22c5c870f: Remote error: Service unavailable: leader transfer in progress
I20260709 13:05:05.644222 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:05.644495 16566 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:05.644759 15857 raft_consensus.cc:3055] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:05.645079 15857 raft_consensus.cc:740] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:05.645759 15857 consensus_queue.cc:260] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:05.646720 15857 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:05.648262 17108 raft_consensus.cc:993] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea: : Instructing follower 31882606e87447fb9a36bbf625129fe8 to start an election
I20260709 13:05:05.648779 17060 raft_consensus.cc:1081] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Signalling peer 31882606e87447fb9a36bbf625129fe8 to start an election
I20260709 13:05:05.649422 15861 consensus_queue.cc:237] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:05.650350 15858 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:43520
I20260709 13:05:05.650873 15858 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:05.651198 15858 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:05.654449 15857 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:05.655643 16533 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:05.656479 17030 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:05.656916 17030 raft_consensus.cc:697] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:05.657709 17030 consensus_queue.cc:237] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:05.664321 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:05.665072 16011 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:05.669584 15858 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:05.672346 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:05.672854 15931 raft_consensus.cc:3055] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:05.673113 15931 raft_consensus.cc:740] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:05.673393 15858 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:05.673564 16011 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:05.673866 15931 consensus_queue.cc:260] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:05.674901 15931 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:05.678603 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:05.679327 16007 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:05.685017 15931 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:05.686524 15825 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:05.687882 17107 raft_consensus.cc:2804] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:05.688328 17107 raft_consensus.cc:697] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:05.689230 17107 consensus_queue.cc:237] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:05:05.690229 15827 consensus_peers.cc:597] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffbfc0e667484157953a66ae2b697013. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:05.692945 15931 raft_consensus.cc:1275] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:05.695439 16988 consensus_queue.cc:1048] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:05.692987 16007 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:05.694844 15755 catalog_manager.cc:5697] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 1 to 2, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 2 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:05.696600 16011 raft_consensus.cc:1275] T ffbfc0e667484157953a66ae2b697013 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:05.701465 16988 consensus_queue.cc:1048] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:05.707298 16988 raft_consensus.cc:2955] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:05.710426 15930 raft_consensus.cc:2955] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:05.713382 15759 catalog_manager.cc:5697] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 1 to 2, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:05.718415 16011 raft_consensus.cc:2955] T ffbfc0e667484157953a66ae2b697013 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:05.728791 15759 catalog_manager.cc:5697] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:05.735114 17108 raft_consensus.cc:993] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea: : Instructing follower eb1e0493af1e46dbb6dec7e25525bab5 to start an election
I20260709 13:05:05.735654 17108 raft_consensus.cc:1081] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Signalling peer eb1e0493af1e46dbb6dec7e25525bab5 to start an election
I20260709 13:05:05.737319 16566 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194"
dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:45244
I20260709 13:05:05.737803 16566 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
W20260709 13:05:05.740506 15808 auto_rebalancer.cc:629] Failed to schedule move for tablet 9b72a30273dd49e883c237c7e5ad237b: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:05:05.746006 16566 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:05.751065 16894 raft_consensus.cc:993] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee: : Instructing follower 8430db80a39d4868bcf7641c16de35ea to start an election
I20260709 13:05:05.751564 16894 raft_consensus.cc:1081] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Signalling peer 8430db80a39d4868bcf7641c16de35ea to start an election
I20260709 13:05:05.752909 15930 consensus_queue.cc:237] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:05.756026 15931 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
 from {username='slave'} at 127.0.0.1:58402
I20260709 13:05:05.756606 15931 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:05.756963 15931 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:05.761107 16566 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:05.762377 15902 consensus_peers.cc:597] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ecbdcd9cd6c944e7a9277f7c8f191e96. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:05.763166 15931 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:05.764376 16011 raft_consensus.cc:1275] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:05.764708 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:05.764873 15931 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:05.765280 15862 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:05.765506 15857 raft_consensus.cc:1275] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:05.763799 16566 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:05.767889 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:05.767975 17113 consensus_queue.cc:1048] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:05.768416 16007 raft_consensus.cc:3055] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:05.768754 16007 raft_consensus.cc:740] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:05.769438 16007 consensus_queue.cc:260] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:05.769708 17108 consensus_queue.cc:1048] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:05.770545 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:05.771142 15857 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:05.770617 16007 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:05.771725 15862 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:05.767988 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:05.772986 15930 raft_consensus.cc:3055] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:05.773017 16533 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:05.773430 15930 raft_consensus.cc:740] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:05.773964 17030 raft_consensus.cc:2804] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:05.774453 17030 raft_consensus.cc:697] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:05.775321 17030 consensus_queue.cc:237] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:05.777326 15857 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2.
I20260709 13:05:05.778445 15900 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:05.779214 17108 raft_consensus.cc:2804] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:05.779662 17108 raft_consensus.cc:697] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:05.774432 15930 consensus_queue.cc:260] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:05.781901 15930 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:05.781630 17108 consensus_queue.cc:237] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:05.785192 17113 raft_consensus.cc:2955] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:05.786199 16011 raft_consensus.cc:2955] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:05.782017 16007 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2.
I20260709 13:05:05.799407 15930 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:05.805857 15758 catalog_manager.cc:5697] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 1 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:05.806041 15862 raft_consensus.cc:2955] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:05.816278 15755 catalog_manager.cc:5697] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 1 to 2, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 2 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:05.842231 15755 catalog_manager.cc:5697] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 1 to 2, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 2 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:05.875541 15862 raft_consensus.cc:1275] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:05.876757 17030 consensus_queue.cc:1048] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:05.892210 15930 raft_consensus.cc:1275] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:05.893714 17030 consensus_queue.cc:1048] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:05.965319 16011 raft_consensus.cc:1275] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:05.966439 17107 consensus_queue.cc:1048] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:05.975705 15931 raft_consensus.cc:1275] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:05.978636 17107 consensus_queue.cc:1048] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:06.163229 15862 raft_consensus.cc:1275] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:06.164757 17126 consensus_queue.cc:1048] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:06.185024 16011 raft_consensus.cc:1275] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:06.186467 17108 consensus_queue.cc:1048] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:06.195017 16008 raft_consensus.cc:1275] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:06.196362 17074 consensus_queue.cc:1048] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:06.249585 15857 raft_consensus.cc:1275] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:06.251319 17101 consensus_queue.cc:1048] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:06.276651 17139 ts_tablet_manager.cc:933] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:06.279860 17139 tablet_copy_client.cc:323] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.2:43009
I20260709 13:05:06.281378 15946 tablet_copy_service.cc:140] P 8430db80a39d4868bcf7641c16de35ea: Received BeginTabletCopySession request for tablet ecbdcd9cd6c944e7a9277f7c8f191e96 from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:53636)
I20260709 13:05:06.281857 15946 tablet_copy_service.cc:161] P 8430db80a39d4868bcf7641c16de35ea: Beginning new tablet copy session on tablet ecbdcd9cd6c944e7a9277f7c8f191e96 from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:53636: session id = eb1e0493af1e46dbb6dec7e25525bab5-ecbdcd9cd6c944e7a9277f7c8f191e96
I20260709 13:05:06.287066 15931 raft_consensus.cc:1275] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:06.287606 15946 tablet_copy_source_session.cc:215] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:06.288977 17107 consensus_queue.cc:1048] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:06.292469 17139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecbdcd9cd6c944e7a9277f7c8f191e96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:06.309474 16008 raft_consensus.cc:1275] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:06.311087 17142 ts_tablet_manager.cc:933] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
I20260709 13:05:06.310932 16988 consensus_queue.cc:1048] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:06.327883 17139 tablet_copy_client.cc:806] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:06.328682 17139 tablet_copy_client.cc:670] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:06.336220 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:06.342218 17142 tablet_copy_client.cc:323] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.1:38033
I20260709 13:05:06.344372 17139 tablet_copy_client.cc:538] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:06.347513 15872 tablet_copy_service.cc:140] P 31882606e87447fb9a36bbf625129fe8: Received BeginTabletCopySession request for tablet ffbfc0e667484157953a66ae2b697013 from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:58214)
I20260709 13:05:06.348104 15872 tablet_copy_service.cc:161] P 31882606e87447fb9a36bbf625129fe8: Beginning new tablet copy session on tablet ffbfc0e667484157953a66ae2b697013 from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:58214: session id = eb1e0493af1e46dbb6dec7e25525bab5-ffbfc0e667484157953a66ae2b697013
I20260709 13:05:06.355477 15857 raft_consensus.cc:1275] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:06.357597 17139 tablet_bootstrap.cc:492] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:05:06.361883 17101 consensus_queue.cc:1048] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:06.373203 15872 tablet_copy_source_session.cc:215] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:06.376958 17142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffbfc0e667484157953a66ae2b697013. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:06.402011 15930 raft_consensus.cc:1275] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:06.408421 17030 consensus_queue.cc:1048] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:06.438292 17142 tablet_copy_client.cc:806] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:06.439150 17142 tablet_copy_client.cc:670] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:06.441792 17139 tablet_bootstrap.cc:492] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:06.443020 17139 tablet_bootstrap.cc:492] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:05:06.443962 17139 ts_tablet_manager.cc:1403] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.087s	user 0.015s	sys 0.013s
I20260709 13:05:06.452163 17139 raft_consensus.cc:359] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:06.453121 17139 raft_consensus.cc:740] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:05:06.455142 17139 consensus_queue.cc:260] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:06.457443 17139 ts_tablet_manager.cc:1434] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:05:06.462405 15946 tablet_copy_service.cc:342] P 8430db80a39d4868bcf7641c16de35ea: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-ecbdcd9cd6c944e7a9277f7c8f191e96 received from {username='slave'} at 127.0.0.1:53636
I20260709 13:05:06.462836 15946 tablet_copy_service.cc:434] P 8430db80a39d4868bcf7641c16de35ea: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-ecbdcd9cd6c944e7a9277f7c8f191e96 on tablet ecbdcd9cd6c944e7a9277f7c8f191e96 with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:06.466313 17142 tablet_copy_client.cc:538] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:06.483193 17142 tablet_bootstrap.cc:492] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:05:06.530824 17142 tablet_bootstrap.cc:492] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:06.531745 17142 tablet_bootstrap.cc:492] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:05:06.532397 17142 ts_tablet_manager.cc:1403] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.049s	user 0.023s	sys 0.027s
I20260709 13:05:06.535274 17142 raft_consensus.cc:359] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:06.536116 17142 raft_consensus.cc:740] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:05:06.536725 17142 consensus_queue.cc:260] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:06.538821 17142 ts_tablet_manager.cc:1434] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:05:06.540802 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:06.541545 15872 tablet_copy_service.cc:342] P 31882606e87447fb9a36bbf625129fe8: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-ffbfc0e667484157953a66ae2b697013 received from {username='slave'} at 127.0.0.1:58214
I20260709 13:05:06.542017 15872 tablet_copy_service.cc:434] P 31882606e87447fb9a36bbf625129fe8: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-ffbfc0e667484157953a66ae2b697013 on tablet ffbfc0e667484157953a66ae2b697013 with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:06.595124 15935 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0b3676041725488fb6ffad37889b386b"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:46118
I20260709 13:05:06.595798 15935 raft_consensus.cc:606] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Received request to transfer leadership to TS eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:06.600180 15857 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "8430db80a39d4868bcf7641c16de35ea"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:05:06.600832 15857 raft_consensus.cc:606] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Received request to transfer leadership to TS 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:06.603899 16008 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9"
dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
mode: GRACEFUL
new_leader_uuid: "8430db80a39d4868bcf7641c16de35ea"
 from {username='slave'} at 127.0.0.1:37262
I20260709 13:05:06.604365 16008 raft_consensus.cc:606] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Received request to transfer leadership to TS 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:06.606747 15935 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:46118
I20260709 13:05:06.607198 15935 raft_consensus.cc:606] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Received request to transfer leadership to TS 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:06.608311 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:05:06.609850 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:06.661664 16566 raft_consensus.cc:1217] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:06.743018 16569 raft_consensus.cc:1217] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:06.788076 17041 raft_consensus.cc:993] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee: : Instructing follower 8430db80a39d4868bcf7641c16de35ea to start an election
I20260709 13:05:06.788532 17041 raft_consensus.cc:1081] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Signalling peer 8430db80a39d4868bcf7641c16de35ea to start an election
I20260709 13:05:06.789880 15930 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
 from {username='slave'} at 127.0.0.1:58402
I20260709 13:05:06.790153 17150 raft_consensus.cc:993] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8: : Instructing follower 8430db80a39d4868bcf7641c16de35ea to start an election
I20260709 13:05:06.790393 15930 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:06.790884 15930 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:06.790891 16988 raft_consensus.cc:1081] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Signalling peer 8430db80a39d4868bcf7641c16de35ea to start an election
I20260709 13:05:06.792569 15935 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
 from {username='slave'} at 127.0.0.1:58390
I20260709 13:05:06.793047 15935 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:06.793390 15935 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:06.800801 15930 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:06.805316 15935 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:06.808601 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:06.809166 16008 raft_consensus.cc:3055] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:06.809475 16008 raft_consensus.cc:740] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:06.810159 16008 consensus_queue.cc:260] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:06.811147 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:06.811411 16008 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:06.811156 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:06.811837 16011 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:06.812004 15862 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:06.816399 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:06.816918 15861 raft_consensus.cc:3055] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:06.817231 15861 raft_consensus.cc:740] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:06.818127 15861 consensus_queue.cc:260] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:06.819275 15861 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:06.821967 15930 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:06.823408 15862 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2.
I20260709 13:05:06.825114 15900 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:06.825155 16011 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:05:06.827924 15861 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:05:06.828006 17108 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:06.835016 17108 raft_consensus.cc:697] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:06.828783 15902 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:06.835916 17108 consensus_queue.cc:237] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:06.810114 15935 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:06.839524 17060 raft_consensus.cc:2804] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:06.840267 17060 raft_consensus.cc:697] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:06.841226 17060 consensus_queue.cc:237] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:06.842633 16008 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2.
I20260709 13:05:06.868256 17126 raft_consensus.cc:993] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea: : Instructing follower 31882606e87447fb9a36bbf625129fe8 to start an election
I20260709 13:05:06.868726 17060 raft_consensus.cc:1081] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Signalling peer 31882606e87447fb9a36bbf625129fe8 to start an election
I20260709 13:05:06.869338 15758 catalog_manager.cc:5697] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 1 to 2, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 2 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:06.870699 15861 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:43520
I20260709 13:05:06.871248 15861 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:06.871585 15861 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:06.877779 15861 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:06.880280 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:06.880818 15935 raft_consensus.cc:3055] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:06.881117 15935 raft_consensus.cc:740] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:06.881256 15861 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:06.881880 15935 consensus_queue.cc:260] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:06.881134 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:06.882845 16008 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:06.883194 15935 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:06.895551 15935 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:06.896850 15825 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:06.896337 15758 catalog_manager.cc:5697] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 2 to 3, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 3 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:06.897635 17059 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:06.898265 17059 raft_consensus.cc:697] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:06.899135 17059 consensus_queue.cc:237] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:06.900058 16008 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:06.913221 15758 catalog_manager.cc:5697] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 1 to 2, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:06.922060 15861 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:43504
I20260709 13:05:06.922562 15861 raft_consensus.cc:606] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Received request to transfer leadership
I20260709 13:05:06.925160 15861 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:43504
I20260709 13:05:06.925596 15861 raft_consensus.cc:606] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Received request to transfer leadership
W20260709 13:05:06.926380 15744 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for ecbdcd9cd6c944e7a9277f7c8f191e96 already in progress [suppressed 4 similar messages]
I20260709 13:05:06.929054 15861 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:43504
I20260709 13:05:06.929494 15861 raft_consensus.cc:606] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Received request to transfer leadership
W20260709 13:05:06.933426 15808 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 13:05:06.933790 15808 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for ecbdcd9cd6c944e7a9277f7c8f191e96 already in progress
W20260709 13:05:06.934043 15808 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for ecbdcd9cd6c944e7a9277f7c8f191e96 already in progress
I20260709 13:05:06.939666 17060 raft_consensus.cc:993] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea: : Instructing follower eb1e0493af1e46dbb6dec7e25525bab5 to start an election
I20260709 13:05:06.940161 17060 raft_consensus.cc:1081] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Signalling peer eb1e0493af1e46dbb6dec7e25525bab5 to start an election
I20260709 13:05:06.942229 16569 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0b3676041725488fb6ffad37889b386b"
dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:45244
I20260709 13:05:06.942704 16569 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:06.943005 16569 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:06.948653 16569 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:06.950141 16569 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:06.950871 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:06.951179 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:06.951408 15935 raft_consensus.cc:3055] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:06.951781 16008 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:06.951840 15935 raft_consensus.cc:740] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:06.952582 15935 consensus_queue.cc:260] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:06.953459 15935 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:06.957717 16008 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:06.957729 15935 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:06.958923 16535 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:06.959795 17074 raft_consensus.cc:2804] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:06.960268 17074 raft_consensus.cc:697] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:06.961102 17074 consensus_queue.cc:237] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:06.969889 15758 catalog_manager.cc:5697] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 1 to 2, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 2 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:07.180650 17150 raft_consensus.cc:1064] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:05:07.182756 17150 consensus_queue.cc:237] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:07.190639 16566 raft_consensus.cc:1275] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:07.192529 16988 consensus_queue.cc:1048] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:07.193141 15930 raft_consensus.cc:1275] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:07.194912 16008 raft_consensus.cc:1275] T ffbfc0e667484157953a66ae2b697013 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:07.196184 16988 consensus_queue.cc:1048] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:07.198562 16988 consensus_queue.cc:1048] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:07.208640 17150 raft_consensus.cc:2955] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:07.210325 16566 raft_consensus.cc:2955] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:07.211828 15930 raft_consensus.cc:2955] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:07.225033 15758 catalog_manager.cc:5697] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:07.225648 16008 raft_consensus.cc:2955] T ffbfc0e667484157953a66ae2b697013 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:07.253659 15861 consensus_queue.cc:237] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:07.265803 16566 raft_consensus.cc:1275] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:07.266369 15935 raft_consensus.cc:1275] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:07.267091 17150 consensus_queue.cc:1048] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:07.268014 16988 consensus_queue.cc:1048] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:07.276130 16988 raft_consensus.cc:2955] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:07.277303 15931 raft_consensus.cc:2955] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:07.284096 16569 raft_consensus.cc:2955] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:07.291008 15744 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ffbfc0e667484157953a66ae2b697013 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:07.295069 16008 raft_consensus.cc:1275] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:07.296591 17108 consensus_queue.cc:1048] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:07.303790 15755 catalog_manager.cc:5697] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:07.328241 15995 tablet_service.cc:1558] Processing DeleteTablet for tablet ffbfc0e667484157953a66ae2b697013 with delete_type TABLET_DATA_TOMBSTONED (TS 85e997ba5003439781ec4f1f8db0c3ee not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41648
I20260709 13:05:07.340312 15857 raft_consensus.cc:1275] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:07.346580 17060 consensus_queue.cc:1048] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:07.361734 17164 tablet_replica.cc:333] T ffbfc0e667484157953a66ae2b697013 P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:05:07.366780 17164 raft_consensus.cc:2243] T ffbfc0e667484157953a66ae2b697013 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:07.367545 17164 raft_consensus.cc:2272] T ffbfc0e667484157953a66ae2b697013 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:07.370842 17164 ts_tablet_manager.cc:1916] T ffbfc0e667484157953a66ae2b697013 P 85e997ba5003439781ec4f1f8db0c3ee: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:07.384581 17164 ts_tablet_manager.cc:1929] T ffbfc0e667484157953a66ae2b697013 P 85e997ba5003439781ec4f1f8db0c3ee: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:07.384902 17164 log.cc:1199] T ffbfc0e667484157953a66ae2b697013 P 85e997ba5003439781ec4f1f8db0c3ee: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-2-root/wals/ffbfc0e667484157953a66ae2b697013
I20260709 13:05:07.386232 15746 catalog_manager.cc:5028] TS 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): tablet ffbfc0e667484157953a66ae2b697013 (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:05:07.394942 16566 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Advancing to term 2
I20260709 13:05:07.399463 16566 raft_consensus.cc:1275] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEARNER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:05:07.400923 17059 consensus_queue.cc:1048] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:07.406944 17059 raft_consensus.cc:1064] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
W20260709 13:05:07.407312 17059 raft_consensus.cc:1068] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8: Unable to promote non-voter eb1e0493af1e46dbb6dec7e25525bab5: Service unavailable: leader transfer in progress
I20260709 13:05:07.423491 16011 raft_consensus.cc:1275] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:07.424731 17030 consensus_queue.cc:1048] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:07.433439 15931 raft_consensus.cc:1275] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:07.435134 17030 consensus_queue.cc:1048] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:07.435709 16008 raft_consensus.cc:1275] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:07.438119 17060 consensus_queue.cc:1048] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:07.446717 15861 raft_consensus.cc:1275] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:07.451017 17108 consensus_queue.cc:1048] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:07.457960 16008 raft_consensus.cc:1275] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:05:07.459945 17059 consensus_queue.cc:1048] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:07.483004 15931 raft_consensus.cc:1275] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:05:07.488209 17150 consensus_queue.cc:1048] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:07.490648 16988 raft_consensus.cc:993] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8: : Instructing follower 85e997ba5003439781ec4f1f8db0c3ee to start an election
I20260709 13:05:07.491036 16988 raft_consensus.cc:1081] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Signalling peer 85e997ba5003439781ec4f1f8db0c3ee to start an election
I20260709 13:05:07.492528 16008 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96"
dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:41674
I20260709 13:05:07.493153 16008 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:07.494247 16008 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:07.507086 16008 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:07.508986 16008 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:07.509322 16011 raft_consensus.cc:1240] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:07.509510 17059 raft_consensus.cc:1064] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
W20260709 13:05:07.510035 17059 raft_consensus.cc:1068] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8: Unable to promote non-voter eb1e0493af1e46dbb6dec7e25525bab5: Service unavailable: leader transfer in progress
I20260709 13:05:07.512374 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:07.512455 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:07.512998 15862 raft_consensus.cc:3055] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:07.513329 15862 raft_consensus.cc:740] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:07.514025 15862 consensus_queue.cc:260] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:07.515127 15862 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:07.528815 15862 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:07.531806 15974 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:07.532629 16894 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:07.533071 16894 raft_consensus.cc:697] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:07.533855 16894 consensus_queue.cc:237] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:07.544876 15759 catalog_manager.cc:5697] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 2 to 3, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 3 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:07.611550 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:07.656436 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:07.695152 16007 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0"
dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
mode: GRACEFUL
new_leader_uuid: "8430db80a39d4868bcf7641c16de35ea"
 from {username='slave'} at 127.0.0.1:37262
I20260709 13:05:07.695737 16007 raft_consensus.cc:606] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Received request to transfer leadership to TS 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:07.698151 15931 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e9577f76f40348e9b009255d441d012a"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:46118
I20260709 13:05:07.698629 15931 raft_consensus.cc:606] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Received request to transfer leadership to TS 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:07.699819 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:05:07.700879 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:07.802513 17120 raft_consensus.cc:993] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee: : Instructing follower 8430db80a39d4868bcf7641c16de35ea to start an election
I20260709 13:05:07.802963 16894 raft_consensus.cc:1081] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Signalling peer 8430db80a39d4868bcf7641c16de35ea to start an election
I20260709 13:05:07.804535 15935 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
 from {username='slave'} at 127.0.0.1:58402
I20260709 13:05:07.805157 15935 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:07.805425 15935 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:07.809995 15935 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:07.811421 15935 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:07.812184 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:07.812373 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:07.812745 15862 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:07.812849 16007 raft_consensus.cc:3055] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:07.813194 16007 raft_consensus.cc:740] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:07.813931 16007 consensus_queue.cc:260] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:07.814980 16007 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:07.818960 15862 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2.
I20260709 13:05:07.820147 15900 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:07.821053 17126 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:07.821225 16007 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2.
I20260709 13:05:07.821765 17126 raft_consensus.cc:697] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:07.822573 17126 consensus_queue.cc:237] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:07.832412 15759 catalog_manager.cc:5697] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 1 to 2, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 2 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:07.847405 17108 raft_consensus.cc:993] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea: : Instructing follower 31882606e87447fb9a36bbf625129fe8 to start an election
I20260709 13:05:07.847879 17126 raft_consensus.cc:1081] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Signalling peer 31882606e87447fb9a36bbf625129fe8 to start an election
I20260709 13:05:07.849257 15861 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e9577f76f40348e9b009255d441d012a"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:43520
I20260709 13:05:07.849786 15861 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:07.850062 15861 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:07.856266 15861 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:05:07.858366 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:07.858517 15861 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:07.858692 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:07.858944 16007 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:07.859229 15935 raft_consensus.cc:3055] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:07.859557 15935 raft_consensus.cc:740] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:07.860365 15935 consensus_queue.cc:260] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:05:07.861322 15935 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:07.865461 16007 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:07.866617 15827 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:07.867403 16988 raft_consensus.cc:2804] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:07.867851 16988 raft_consensus.cc:697] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:07.868604 16988 consensus_queue.cc:237] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:05:07.868906 15935 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:07.904186 15759 catalog_manager.cc:5697] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 1 to 2, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:07.942463 15931 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:07.950238 15931 raft_consensus.cc:1275] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:05:07.952672 17181 consensus_queue.cc:1048] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:07.965721 16566 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEARNER]: Advancing to term 3
I20260709 13:05:07.969971 15857 raft_consensus.cc:1275] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:05:07.972154 16894 consensus_queue.cc:1048] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:07.973150 16566 raft_consensus.cc:1275] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 LEARNER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:05:07.974987 16894 consensus_queue.cc:1048] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:07.988252 17120 raft_consensus.cc:1064] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:05:07.990726 17120 consensus_queue.cc:237] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:07.998322 16569 raft_consensus.cc:1275] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 LEARNER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:05:08.003589 17180 consensus_queue.cc:1048] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:08.004758 15857 raft_consensus.cc:1275] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:05:08.003585 15931 raft_consensus.cc:1275] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:05:08.006047 17180 consensus_queue.cc:1048] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:08.008960 17180 consensus_queue.cc:1048] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:08.016533 16894 raft_consensus.cc:2955] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 2 to 5, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:08.023689 15931 raft_consensus.cc:2955] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:08.027117 15861 raft_consensus.cc:2955] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:08.032677 16569 raft_consensus.cc:2955] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:08.035108 15758 catalog_manager.cc:5697] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: config changed from index 2 to 5, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:08.049031 16007 consensus_queue.cc:237] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:08.056944 16568 raft_consensus.cc:1275] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:05:08.058315 16894 consensus_queue.cc:1048] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:08.066468 17120 raft_consensus.cc:2955] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:08.067461 16568 raft_consensus.cc:2955] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:08.074591 15935 raft_consensus.cc:1275] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:05:08.076808 17181 consensus_queue.cc:1048] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:08.081856 15746 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ecbdcd9cd6c944e7a9277f7c8f191e96 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:08.081347 15935 raft_consensus.cc:2955] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:08.087193 15760 catalog_manager.cc:5697] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: config changed from index 5 to 6, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New cstate: current_term: 3 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:08.097550 15846 tablet_service.cc:1558] Processing DeleteTablet for tablet ecbdcd9cd6c944e7a9277f7c8f191e96 with delete_type TABLET_DATA_TOMBSTONED (TS 31882606e87447fb9a36bbf625129fe8 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:43504
I20260709 13:05:08.101159 17184 tablet_replica.cc:333] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:05:08.102062 17184 raft_consensus.cc:2243] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:08.102895 17184 raft_consensus.cc:2272] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:08.106251 17184 ts_tablet_manager.cc:1916] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:08.118944 17184 ts_tablet_manager.cc:1929] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 13:05:08.119338 17184 log.cc:1199] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 31882606e87447fb9a36bbf625129fe8: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-0-root/wals/ecbdcd9cd6c944e7a9277f7c8f191e96
I20260709 13:05:08.121642 15744 catalog_manager.cc:5028] TS 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): tablet ecbdcd9cd6c944e7a9277f7c8f191e96 (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:05:08.307948 15857 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:08.310036 17060 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:08.330194 16007 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:08.337008 17126 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:08.438591 16011 consensus_queue.cc:237] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:08.446367 15861 raft_consensus.cc:1275] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:08.446696 15935 raft_consensus.cc:1275] T 73f4b066dd3d464699e6b00ebf1cdc44 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:05:08.447460 15976 consensus_peers.cc:597] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 73f4b066dd3d464699e6b00ebf1cdc44. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:08.447870 16894 consensus_queue.cc:1048] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:08.450587 17181 consensus_queue.cc:1048] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:08.460598 15935 raft_consensus.cc:1275] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:08.460673 16894 raft_consensus.cc:2955] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:08.462327 17150 consensus_queue.cc:1048] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:08.462762 15935 raft_consensus.cc:2955] T 73f4b066dd3d464699e6b00ebf1cdc44 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:08.467334 15861 raft_consensus.cc:2955] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:08.475309 16011 raft_consensus.cc:1275] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:08.477945 17150 consensus_queue.cc:1048] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:08.484862 15857 consensus_queue.cc:237] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:08.489892 15760 catalog_manager.cc:5697] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:08.497349 16007 raft_consensus.cc:1275] T 91d0cc5018ee4a9fb3d196c783814c44 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:08.499593 15930 raft_consensus.cc:1275] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:05:08.500873 15827 consensus_peers.cc:597] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 91d0cc5018ee4a9fb3d196c783814c44. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:08.501475 17150 consensus_queue.cc:1048] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:08.503547 17150 consensus_queue.cc:1048] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:08.525348 17186 raft_consensus.cc:2955] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:08.532181 16011 raft_consensus.cc:2955] T 91d0cc5018ee4a9fb3d196c783814c44 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:08.566659 16008 consensus_queue.cc:237] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:08.567488 15931 raft_consensus.cc:2955] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
W20260709 13:05:08.585981 15976 consensus_peers.cc:597] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c138dd864152446f83386a4cd43c6efc. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:08.586378 15931 raft_consensus.cc:1275] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:08.586972 15857 raft_consensus.cc:1275] T c138dd864152446f83386a4cd43c6efc P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:08.588582 17181 consensus_queue.cc:1048] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:08.589433 16894 consensus_queue.cc:1048] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:08.595229 15760 catalog_manager.cc:5697] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:08.605798 17181 raft_consensus.cc:2955] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:08.612129 15857 raft_consensus.cc:2955] T c138dd864152446f83386a4cd43c6efc P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:08.618402 15931 raft_consensus.cc:2955] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:08.627303 16008 consensus_queue.cc:237] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:08.634812 15758 catalog_manager.cc:5697] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:08.660600 15976 consensus_peers.cc:597] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a10bba95cd1421aa3fc332639e97319. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:08.660753 15935 raft_consensus.cc:1275] T 0a10bba95cd1421aa3fc332639e97319 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:08.661787 15857 raft_consensus.cc:1275] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:08.662865 17201 consensus_queue.cc:1048] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:08.668637 17120 consensus_queue.cc:1048] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:08.676393 17201 raft_consensus.cc:2955] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:08.677907 15935 raft_consensus.cc:2955] T 0a10bba95cd1421aa3fc332639e97319 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:08.687179 15857 raft_consensus.cc:2955] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:08.699115 15760 catalog_manager.cc:5697] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:08.701843 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:08.767467 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:08.820114 15861 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:05:08.820912 15861 raft_consensus.cc:606] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Received request to transfer leadership to TS 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:08.823874 16008 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc"
dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
mode: GRACEFUL
new_leader_uuid: "8430db80a39d4868bcf7641c16de35ea"
 from {username='slave'} at 127.0.0.1:37262
I20260709 13:05:08.824416 16008 raft_consensus.cc:606] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Received request to transfer leadership to TS 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:08.828091 15935 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:46118
I20260709 13:05:08.828663 15935 raft_consensus.cc:606] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Received request to transfer leadership to TS 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:08.830619 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:05:08.832365 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:08.920647 17205 ts_tablet_manager.cc:933] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:08.922991 17205 tablet_copy_client.cc:323] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.3:46251
I20260709 13:05:08.924760 16021 tablet_copy_service.cc:140] P 85e997ba5003439781ec4f1f8db0c3ee: Received BeginTabletCopySession request for tablet 73f4b066dd3d464699e6b00ebf1cdc44 from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:60156)
I20260709 13:05:08.925220 16021 tablet_copy_service.cc:161] P 85e997ba5003439781ec4f1f8db0c3ee: Beginning new tablet copy session on tablet 73f4b066dd3d464699e6b00ebf1cdc44 from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:60156: session id = eb1e0493af1e46dbb6dec7e25525bab5-73f4b066dd3d464699e6b00ebf1cdc44
I20260709 13:05:08.931216 16021 tablet_copy_source_session.cc:215] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:08.934988 17205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73f4b066dd3d464699e6b00ebf1cdc44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:08.949312 17205 tablet_copy_client.cc:806] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:08.949916 17205 tablet_copy_client.cc:670] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:08.954288 17205 tablet_copy_client.cc:538] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:08.963503 17205 tablet_bootstrap.cc:492] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:05:08.990279 17205 tablet_bootstrap.cc:492] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:08.991644 17205 tablet_bootstrap.cc:492] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:05:08.992372 17205 ts_tablet_manager.cc:1403] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.029s	user 0.018s	sys 0.010s
I20260709 13:05:08.995203 17205 raft_consensus.cc:359] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:08.995886 17205 raft_consensus.cc:740] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:05:08.996419 17205 consensus_queue.cc:260] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:08.998839 17205 ts_tablet_manager.cc:1434] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.001s
I20260709 13:05:09.000629 16021 tablet_copy_service.cc:342] P 85e997ba5003439781ec4f1f8db0c3ee: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-73f4b066dd3d464699e6b00ebf1cdc44 received from {username='slave'} at 127.0.0.1:60156
I20260709 13:05:09.001214 16021 tablet_copy_service.cc:434] P 85e997ba5003439781ec4f1f8db0c3ee: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-73f4b066dd3d464699e6b00ebf1cdc44 on tablet 73f4b066dd3d464699e6b00ebf1cdc44 with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:09.047647 17126 raft_consensus.cc:993] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea: : Instructing follower 31882606e87447fb9a36bbf625129fe8 to start an election
I20260709 13:05:09.048221 17108 raft_consensus.cc:1081] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Signalling peer 31882606e87447fb9a36bbf625129fe8 to start an election
I20260709 13:05:09.049604 15861 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:43520
I20260709 13:05:09.050160 15861 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:09.050483 15861 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:09.057639 15861 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:09.061465 15861 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:09.062340 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:09.062933 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:09.063035 16008 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:09.063493 15935 raft_consensus.cc:3055] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:09.063833 15935 raft_consensus.cc:740] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:09.064649 15935 consensus_queue.cc:260] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:09.066586 15935 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:09.080188 16008 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:09.081650 15827 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:09.083990 17197 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:09.084550 17197 raft_consensus.cc:697] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:09.085367 17197 consensus_queue.cc:237] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:09.094847 15935 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:09.095194 17205 ts_tablet_manager.cc:933] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
I20260709 13:05:09.098371 17205 tablet_copy_client.cc:323] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.1:38033
I20260709 13:05:09.102807 15872 tablet_copy_service.cc:140] P 31882606e87447fb9a36bbf625129fe8: Received BeginTabletCopySession request for tablet 91d0cc5018ee4a9fb3d196c783814c44 from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:58214)
I20260709 13:05:09.103368 15872 tablet_copy_service.cc:161] P 31882606e87447fb9a36bbf625129fe8: Beginning new tablet copy session on tablet 91d0cc5018ee4a9fb3d196c783814c44 from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:58214: session id = eb1e0493af1e46dbb6dec7e25525bab5-91d0cc5018ee4a9fb3d196c783814c44
I20260709 13:05:09.105276 15759 catalog_manager.cc:5697] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 1 to 2, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:09.111248 15872 tablet_copy_source_session.cc:215] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:09.114140 17205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91d0cc5018ee4a9fb3d196c783814c44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:09.127264 16988 raft_consensus.cc:993] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8: : Instructing follower 85e997ba5003439781ec4f1f8db0c3ee to start an election
I20260709 13:05:09.127774 16988 raft_consensus.cc:1081] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Signalling peer 85e997ba5003439781ec4f1f8db0c3ee to start an election
I20260709 13:05:09.128942 17205 tablet_copy_client.cc:806] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:09.129552 17205 tablet_copy_client.cc:670] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:09.129654 16008 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01"
dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:41674
I20260709 13:05:09.130324 16008 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:09.130717 16008 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:09.139513 16008 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:09.143930 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:09.144470 15861 raft_consensus.cc:3055] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:09.144759 15861 raft_consensus.cc:740] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:09.145485 15861 consensus_queue.cc:260] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:09.146534 15861 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:09.146728 17205 tablet_copy_client.cc:538] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:09.153100 15861 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 2.
I20260709 13:05:09.154510 15974 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:09.155475 17120 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:09.155936 17120 raft_consensus.cc:697] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:09.156738 17120 consensus_queue.cc:237] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:09.157975 17205 tablet_bootstrap.cc:492] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:05:09.164067 16008 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:09.165359 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:09.166035 15935 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:09.178346 15759 catalog_manager.cc:5697] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 1 to 2, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 2 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:09.190925 15935 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 2.
I20260709 13:05:09.196611 17211 ts_tablet_manager.cc:933] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:09.202255 17211 tablet_copy_client.cc:323] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.3:46251
I20260709 13:05:09.203902 16021 tablet_copy_service.cc:140] P 85e997ba5003439781ec4f1f8db0c3ee: Received BeginTabletCopySession request for tablet c138dd864152446f83386a4cd43c6efc from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:60156)
I20260709 13:05:09.204403 16021 tablet_copy_service.cc:161] P 85e997ba5003439781ec4f1f8db0c3ee: Beginning new tablet copy session on tablet c138dd864152446f83386a4cd43c6efc from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:60156: session id = eb1e0493af1e46dbb6dec7e25525bab5-c138dd864152446f83386a4cd43c6efc
I20260709 13:05:09.209731 17205 tablet_bootstrap.cc:492] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:09.210573 17205 tablet_bootstrap.cc:492] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:05:09.211206 17205 ts_tablet_manager.cc:1403] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.053s	user 0.020s	sys 0.007s
I20260709 13:05:09.213476 16021 tablet_copy_source_session.cc:215] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:09.214133 17205 raft_consensus.cc:359] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:09.215023 17205 raft_consensus.cc:740] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:05:09.216017 17205 consensus_queue.cc:260] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:09.217298 17211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c138dd864152446f83386a4cd43c6efc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:09.221184 17205 ts_tablet_manager.cc:1434] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.005s
I20260709 13:05:09.223560 15872 tablet_copy_service.cc:342] P 31882606e87447fb9a36bbf625129fe8: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-91d0cc5018ee4a9fb3d196c783814c44 received from {username='slave'} at 127.0.0.1:58214
I20260709 13:05:09.224041 15872 tablet_copy_service.cc:434] P 31882606e87447fb9a36bbf625129fe8: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-91d0cc5018ee4a9fb3d196c783814c44 on tablet 91d0cc5018ee4a9fb3d196c783814c44 with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:09.232931 17120 raft_consensus.cc:993] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee: : Instructing follower 8430db80a39d4868bcf7641c16de35ea to start an election
I20260709 13:05:09.233351 17120 raft_consensus.cc:1081] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Signalling peer 8430db80a39d4868bcf7641c16de35ea to start an election
I20260709 13:05:09.238041 17211 tablet_copy_client.cc:806] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:09.238690 17211 tablet_copy_client.cc:670] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:09.242851 15931 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
 from {username='slave'} at 127.0.0.1:58402
I20260709 13:05:09.243366 15931 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:09.243718 15931 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:09.253727 15931 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:09.256156 17211 tablet_copy_client.cc:538] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:09.261370 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:09.261919 16008 raft_consensus.cc:3055] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:09.262190 16008 raft_consensus.cc:740] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:09.262883 16008 consensus_queue.cc:260] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:09.263811 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:09.264173 16008 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:09.264322 15861 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:09.266834 17211 tablet_bootstrap.cc:492] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:05:09.268224 17205 ts_tablet_manager.cc:933] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:09.273257 17205 tablet_copy_client.cc:323] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.3:46251
I20260709 13:05:09.278932 16008 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2.
I20260709 13:05:09.279651 15931 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:09.288435 15902 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:09.288861 16021 tablet_copy_service.cc:140] P 85e997ba5003439781ec4f1f8db0c3ee: Received BeginTabletCopySession request for tablet 0a10bba95cd1421aa3fc332639e97319 from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:60156)
I20260709 13:05:09.289165 17060 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:09.289475 16021 tablet_copy_service.cc:161] P 85e997ba5003439781ec4f1f8db0c3ee: Beginning new tablet copy session on tablet 0a10bba95cd1421aa3fc332639e97319 from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:60156: session id = eb1e0493af1e46dbb6dec7e25525bab5-0a10bba95cd1421aa3fc332639e97319
I20260709 13:05:09.289666 17060 raft_consensus.cc:697] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:09.290561 17060 consensus_queue.cc:237] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:09.291824 15861 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2.
I20260709 13:05:09.297746 16021 tablet_copy_source_session.cc:215] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:09.302024 17205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a10bba95cd1421aa3fc332639e97319. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:09.304512 15759 catalog_manager.cc:5697] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 1 to 2, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 2 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:09.317819 17211 tablet_bootstrap.cc:492] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:09.318796 17211 tablet_bootstrap.cc:492] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:05:09.319756 17211 ts_tablet_manager.cc:1403] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.053s	user 0.022s	sys 0.016s
I20260709 13:05:09.322427 17211 raft_consensus.cc:359] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:09.323020 17211 raft_consensus.cc:740] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:05:09.323535 17211 consensus_queue.cc:260] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:09.325985 17211 ts_tablet_manager.cc:1434] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:05:09.326133 17205 tablet_copy_client.cc:806] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:09.326709 17205 tablet_copy_client.cc:670] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:09.331121 16021 tablet_copy_service.cc:342] P 85e997ba5003439781ec4f1f8db0c3ee: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-c138dd864152446f83386a4cd43c6efc received from {username='slave'} at 127.0.0.1:60156
I20260709 13:05:09.331533 16021 tablet_copy_service.cc:434] P 85e997ba5003439781ec4f1f8db0c3ee: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-c138dd864152446f83386a4cd43c6efc on tablet c138dd864152446f83386a4cd43c6efc with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:09.335170 17205 tablet_copy_client.cc:538] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:09.343014 17205 tablet_bootstrap.cc:492] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:05:09.368053 17205 tablet_bootstrap.cc:492] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:09.368827 17205 tablet_bootstrap.cc:492] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:05:09.369313 17205 ts_tablet_manager.cc:1403] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.026s	user 0.020s	sys 0.006s
I20260709 13:05:09.371069 17205 raft_consensus.cc:359] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:09.371609 17205 raft_consensus.cc:740] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:05:09.372102 17205 consensus_queue.cc:260] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:09.374202 17205 ts_tablet_manager.cc:1434] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:05:09.382356 16021 tablet_copy_service.cc:342] P 85e997ba5003439781ec4f1f8db0c3ee: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-0a10bba95cd1421aa3fc332639e97319 received from {username='slave'} at 127.0.0.1:60156
I20260709 13:05:09.382783 16021 tablet_copy_service.cc:434] P 85e997ba5003439781ec4f1f8db0c3ee: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-0a10bba95cd1421aa3fc332639e97319 on tablet 0a10bba95cd1421aa3fc332639e97319 with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:09.494175 16569 raft_consensus.cc:1217] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:09.565553 16008 raft_consensus.cc:1275] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:09.567076 15931 raft_consensus.cc:1275] T 952bd0ea3bba4d38ad3383fdf572cd01 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:09.567342 17197 consensus_queue.cc:1048] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:09.568403 17201 consensus_queue.cc:1048] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:09.605249 15858 raft_consensus.cc:1275] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:09.609194 17217 consensus_queue.cc:1048] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:09.653975 15931 raft_consensus.cc:1275] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:09.656430 17225 consensus_queue.cc:1048] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:09.717883 16011 raft_consensus.cc:1275] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:05:09.748986 16569 raft_consensus.cc:1217] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:09.752888 17108 consensus_queue.cc:1048] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:09.805840 16569 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Advancing to term 2
I20260709 13:05:09.816345 15861 raft_consensus.cc:1275] T c138dd864152446f83386a4cd43c6efc P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:05:09.818517 17108 consensus_queue.cc:1048] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:09.825060 16569 raft_consensus.cc:1275] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEARNER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:05:09.828717 17126 consensus_queue.cc:1048] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:09.833349 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:09.850957 17108 raft_consensus.cc:1064] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:05:09.854738 17108 consensus_queue.cc:237] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:09.861258 16568 raft_consensus.cc:1275] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEARNER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:05:09.861519 16011 raft_consensus.cc:1275] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:05:09.862712 17108 consensus_queue.cc:1048] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:09.863508 15858 raft_consensus.cc:1275] T c138dd864152446f83386a4cd43c6efc P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:05:09.863529 17126 consensus_queue.cc:1048] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:09.865402 17126 consensus_queue.cc:1048] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:09.868264 16566 raft_consensus.cc:1217] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:09.885296 17108 raft_consensus.cc:2955] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 2 to 4, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:09.890118 15861 raft_consensus.cc:2955] T c138dd864152446f83386a4cd43c6efc P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:09.897488 16566 raft_consensus.cc:2955] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:09.891944 16007 raft_consensus.cc:2955] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:09.906934 15760 catalog_manager.cc:5697] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea reported cstate change: config changed from index 2 to 4, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:09.941684 15930 consensus_queue.cc:237] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:09.948210 16566 raft_consensus.cc:1275] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:09.951136 17060 consensus_queue.cc:1048] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:09.951654 16007 raft_consensus.cc:1275] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:09.954473 17060 consensus_queue.cc:1048] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:09.960590 17126 raft_consensus.cc:2955] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:09.966921 16007 raft_consensus.cc:2955] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:09.969681 16566 raft_consensus.cc:2955] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:09.981634 15744 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c138dd864152446f83386a4cd43c6efc with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:10.005090 15755 catalog_manager.cc:5697] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea reported cstate change: config changed from index 4 to 5, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New cstate: current_term: 2 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:10.020978 15846 tablet_service.cc:1558] Processing DeleteTablet for tablet c138dd864152446f83386a4cd43c6efc with delete_type TABLET_DATA_TOMBSTONED (TS 31882606e87447fb9a36bbf625129fe8 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:43504
I20260709 13:05:10.023197 17238 tablet_replica.cc:333] T c138dd864152446f83386a4cd43c6efc P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:05:10.024201 17238 raft_consensus.cc:2243] T c138dd864152446f83386a4cd43c6efc P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:10.025015 17238 raft_consensus.cc:2272] T c138dd864152446f83386a4cd43c6efc P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:10.029609 17238 ts_tablet_manager.cc:1916] T c138dd864152446f83386a4cd43c6efc P 31882606e87447fb9a36bbf625129fe8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:10.050469 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:10.052217 17238 ts_tablet_manager.cc:1929] T c138dd864152446f83386a4cd43c6efc P 31882606e87447fb9a36bbf625129fe8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:05:10.052712 17238 log.cc:1199] T c138dd864152446f83386a4cd43c6efc P 31882606e87447fb9a36bbf625129fe8: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-0-root/wals/c138dd864152446f83386a4cd43c6efc
I20260709 13:05:10.054255 15744 catalog_manager.cc:5028] TS 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): tablet c138dd864152446f83386a4cd43c6efc (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:05:10.099200 17120 raft_consensus.cc:1064] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:05:10.102926 17120 consensus_queue.cc:237] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:10.109510 15930 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "32207d388e0e4185898655b74ae976b1"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:46118
I20260709 13:05:10.109987 15930 raft_consensus.cc:606] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Received request to transfer leadership to TS 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:10.112587 15861 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:05:10.113104 15861 raft_consensus.cc:606] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Received request to transfer leadership to TS 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:10.113754 15935 raft_consensus.cc:1275] T 73f4b066dd3d464699e6b00ebf1cdc44 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:10.115975 16011 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184"
dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
mode: GRACEFUL
new_leader_uuid: "8430db80a39d4868bcf7641c16de35ea"
 from {username='slave'} at 127.0.0.1:37262
I20260709 13:05:10.116465 16011 raft_consensus.cc:606] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Received request to transfer leadership to TS 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:10.116374 17217 consensus_queue.cc:1048] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:10.117771 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:05:10.119333 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:10.113754 16566 raft_consensus.cc:1275] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:10.123471 17201 consensus_queue.cc:1048] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:10.134256 15858 raft_consensus.cc:1275] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:10.141480 17201 consensus_queue.cc:1048] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:10.151484 17243 raft_consensus.cc:2955] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:10.155807 15930 raft_consensus.cc:2955] T 73f4b066dd3d464699e6b00ebf1cdc44 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:10.164211 16569 raft_consensus.cc:2955] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:10.179082 15755 catalog_manager.cc:5697] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:10.198061 15861 raft_consensus.cc:2955] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:10.213815 17226 raft_consensus.cc:993] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8: : Instructing follower 85e997ba5003439781ec4f1f8db0c3ee to start an election
I20260709 13:05:10.214355 17150 raft_consensus.cc:1081] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Signalling peer 85e997ba5003439781ec4f1f8db0c3ee to start an election
I20260709 13:05:10.216351 16011 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef"
dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:41674
I20260709 13:05:10.216856 16011 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:10.217104 16011 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:10.217850 16007 consensus_queue.cc:237] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:10.225014 17225 raft_consensus.cc:1064] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:05:10.229351 16011 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:10.231773 16011 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:10.233425 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:10.233940 15935 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:10.234504 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:10.235025 15858 raft_consensus.cc:3055] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:10.235278 15858 raft_consensus.cc:740] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:10.236004 15858 consensus_queue.cc:260] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:10.237136 15858 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:10.227372 17225 consensus_queue.cc:237] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:10.244838 16566 raft_consensus.cc:1275] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:10.245597 15858 raft_consensus.cc:2468] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:10.235957 15861 raft_consensus.cc:1275] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:10.246410 17243 consensus_queue.cc:1048] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:10.248001 15974 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:10.249784 17243 raft_consensus.cc:2804] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:10.253090 17217 raft_consensus.cc:993] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee: : Instructing follower 8430db80a39d4868bcf7641c16de35ea to start an election
I20260709 13:05:10.253549 17120 raft_consensus.cc:1081] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Signalling peer 8430db80a39d4868bcf7641c16de35ea to start an election
I20260709 13:05:10.253710 17243 raft_consensus.cc:697] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:10.255420 17243 consensus_queue.cc:237] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:10.254269 17244 consensus_queue.cc:1048] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:05:10.258289 17242 raft_consensus.cc:2955] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:10.271227 15746 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 73f4b066dd3d464699e6b00ebf1cdc44 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:10.253460 15935 raft_consensus.cc:2468] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:10.278434 16569 raft_consensus.cc:2955] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:10.280776 15931 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
 from {username='slave'} at 127.0.0.1:58402
I20260709 13:05:10.281289 15931 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:10.281612 15931 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:10.286077 15758 catalog_manager.cc:5697] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 2 to 3, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 3 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:10.293560 16011 raft_consensus.cc:1275] T 91d0cc5018ee4a9fb3d196c783814c44 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:10.295053 16566 raft_consensus.cc:1275] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:10.296685 15930 raft_consensus.cc:1275] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:10.297478 16988 consensus_queue.cc:1048] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:10.298314 17217 raft_consensus.cc:1064] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:05:10.299149 16988 consensus_queue.cc:1048] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:10.300577 15931 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:10.301564 17225 consensus_queue.cc:1048] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:10.306237 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:10.306746 16568 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:10.306550 17217 consensus_queue.cc:237] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:10.315773 16568 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2.
I20260709 13:05:10.316610 15931 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:10.316947 15902 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:10.317936 17060 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:10.318351 17060 raft_consensus.cc:697] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:10.319110 17060 consensus_queue.cc:237] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:10.320806 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:10.321302 16007 raft_consensus.cc:3055] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:10.321550 16007 raft_consensus.cc:740] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:10.322252 16007 consensus_queue.cc:260] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:10.323396 16007 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:10.331039 17225 raft_consensus.cc:2955] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:10.330437 15858 raft_consensus.cc:2955] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:10.332366 16566 raft_consensus.cc:2955] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:10.336351 16568 raft_consensus.cc:1275] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:10.338240 17244 consensus_queue.cc:1048] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:10.346886 17108 raft_consensus.cc:993] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea: : Instructing follower 31882606e87447fb9a36bbf625129fe8 to start an election
I20260709 13:05:10.347498 17234 raft_consensus.cc:1081] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Signalling peer 31882606e87447fb9a36bbf625129fe8 to start an election
I20260709 13:05:10.351608 15930 raft_consensus.cc:1275] T 0a10bba95cd1421aa3fc332639e97319 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:10.352237 15861 raft_consensus.cc:1275] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:10.352062 16011 raft_consensus.cc:2955] T 91d0cc5018ee4a9fb3d196c783814c44 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:10.354110 15755 catalog_manager.cc:5697] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:10.354867 17244 consensus_queue.cc:1048] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:10.353504 15862 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "32207d388e0e4185898655b74ae976b1"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:43520
I20260709 13:05:10.357275 17242 consensus_queue.cc:1048] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:10.356387 15862 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:10.359442 15862 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:10.401911 15759 catalog_manager.cc:5697] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 1 to 2, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 2 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:10.408182 15862 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:10.408492 15760 catalog_manager.cc:5697] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:10.419706 15862 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:10.420014 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:10.422749 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:10.434903 15931 raft_consensus.cc:3055] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:10.435045 16006 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:10.435334 15931 raft_consensus.cc:740] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:10.436179 15931 consensus_queue.cc:260] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:10.437407 15931 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:10.441180 15923 tablet_service.cc:1558] Processing DeleteTablet for tablet 73f4b066dd3d464699e6b00ebf1cdc44 with delete_type TABLET_DATA_TOMBSTONED (TS 8430db80a39d4868bcf7641c16de35ea not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58386
I20260709 13:05:10.409986 16007 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2.
I20260709 13:05:10.451918 16006 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:10.453528 15827 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:10.454387 17225 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:10.454874 17225 raft_consensus.cc:697] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:10.455713 17225 consensus_queue.cc:237] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:10.482275 17256 tablet_replica.cc:333] T 73f4b066dd3d464699e6b00ebf1cdc44 P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:05:10.484273 17256 raft_consensus.cc:2243] T 73f4b066dd3d464699e6b00ebf1cdc44 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:10.481177 15930 raft_consensus.cc:2955] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:10.485158 17256 raft_consensus.cc:2272] T 73f4b066dd3d464699e6b00ebf1cdc44 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:10.489738 17256 ts_tablet_manager.cc:1916] T 73f4b066dd3d464699e6b00ebf1cdc44 P 8430db80a39d4868bcf7641c16de35ea: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:10.492344 15935 raft_consensus.cc:2955] T 0a10bba95cd1421aa3fc332639e97319 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:10.492081 17217 raft_consensus.cc:2955] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:10.504168 15858 raft_consensus.cc:2955] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:10.514240 17256 ts_tablet_manager.cc:1929] T 73f4b066dd3d464699e6b00ebf1cdc44 P 8430db80a39d4868bcf7641c16de35ea: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:10.514928 17256 log.cc:1199] T 73f4b066dd3d464699e6b00ebf1cdc44 P 8430db80a39d4868bcf7641c16de35ea: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-1-root/wals/73f4b066dd3d464699e6b00ebf1cdc44
I20260709 13:05:10.512205 15760 catalog_manager.cc:5697] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 1 to 2, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:10.516971 15744 catalog_manager.cc:5028] TS 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): tablet 73f4b066dd3d464699e6b00ebf1cdc44 (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:05:10.522787 16568 raft_consensus.cc:2955] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:10.524355 15931 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:10.533836 15760 catalog_manager.cc:5697] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:10.559880 16007 consensus_queue.cc:237] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:10.568569 16568 raft_consensus.cc:1275] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:10.569638 15861 raft_consensus.cc:1275] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:10.570297 17244 consensus_queue.cc:1048] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:10.563560 15858 consensus_queue.cc:237] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:10.571883 17120 consensus_queue.cc:1048] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:10.582167 15930 raft_consensus.cc:1275] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:10.583595 16988 consensus_queue.cc:1048] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:10.585225 16569 raft_consensus.cc:1275] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:10.606539 16988 consensus_queue.cc:1048] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:10.607081 15861 raft_consensus.cc:2955] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:10.614580 17243 raft_consensus.cc:2955] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:10.615432 17224 raft_consensus.cc:2955] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:10.621711 15931 raft_consensus.cc:2955] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:10.628149 15746 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0a10bba95cd1421aa3fc332639e97319 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:10.629786 16568 raft_consensus.cc:2955] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:10.638127 15744 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 91d0cc5018ee4a9fb3d196c783814c44 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:10.630955 16566 raft_consensus.cc:2955] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:10.644894 15758 catalog_manager.cc:5697] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:10.657881 15755 catalog_manager.cc:5697] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: config changed from index 3 to 4, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:10.684947 15923 tablet_service.cc:1558] Processing DeleteTablet for tablet 0a10bba95cd1421aa3fc332639e97319 with delete_type TABLET_DATA_TOMBSTONED (TS 8430db80a39d4868bcf7641c16de35ea not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58386
I20260709 13:05:10.686354 17256 tablet_replica.cc:333] T 0a10bba95cd1421aa3fc332639e97319 P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:05:10.687567 17256 raft_consensus.cc:2243] T 0a10bba95cd1421aa3fc332639e97319 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:10.691123 17256 raft_consensus.cc:2272] T 0a10bba95cd1421aa3fc332639e97319 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:10.695346 17256 ts_tablet_manager.cc:1916] T 0a10bba95cd1421aa3fc332639e97319 P 8430db80a39d4868bcf7641c16de35ea: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:10.705303 15995 tablet_service.cc:1558] Processing DeleteTablet for tablet 91d0cc5018ee4a9fb3d196c783814c44 with delete_type TABLET_DATA_TOMBSTONED (TS 85e997ba5003439781ec4f1f8db0c3ee not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41648
I20260709 13:05:10.715050 15858 raft_consensus.cc:1275] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:10.716964 17120 consensus_queue.cc:1048] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:10.717958 17259 tablet_replica.cc:333] T 91d0cc5018ee4a9fb3d196c783814c44 P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:05:10.718976 17259 raft_consensus.cc:2243] T 91d0cc5018ee4a9fb3d196c783814c44 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:10.719926 17259 raft_consensus.cc:2272] T 91d0cc5018ee4a9fb3d196c783814c44 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:10.725776 17259 ts_tablet_manager.cc:1916] T 91d0cc5018ee4a9fb3d196c783814c44 P 85e997ba5003439781ec4f1f8db0c3ee: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:10.728565 15931 raft_consensus.cc:1275] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:10.730396 17120 consensus_queue.cc:1048] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:10.742300 17259 ts_tablet_manager.cc:1929] T 91d0cc5018ee4a9fb3d196c783814c44 P 85e997ba5003439781ec4f1f8db0c3ee: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:10.742897 17259 log.cc:1199] T 91d0cc5018ee4a9fb3d196c783814c44 P 85e997ba5003439781ec4f1f8db0c3ee: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-2-root/wals/91d0cc5018ee4a9fb3d196c783814c44
I20260709 13:05:10.748153 15746 catalog_manager.cc:5028] TS 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): tablet 91d0cc5018ee4a9fb3d196c783814c44 (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:05:10.750452 17256 ts_tablet_manager.cc:1929] T 0a10bba95cd1421aa3fc332639e97319 P 8430db80a39d4868bcf7641c16de35ea: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:10.750873 17256 log.cc:1199] T 0a10bba95cd1421aa3fc332639e97319 P 8430db80a39d4868bcf7641c16de35ea: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-1-root/wals/0a10bba95cd1421aa3fc332639e97319
I20260709 13:05:10.752465 15744 catalog_manager.cc:5028] TS 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): tablet 0a10bba95cd1421aa3fc332639e97319 (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:05:10.815572 16011 raft_consensus.cc:1275] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:10.817153 17108 consensus_queue.cc:1048] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:10.827829 16566 raft_consensus.cc:1275] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:10.829905 17234 consensus_queue.cc:1048] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:10.862417 16011 raft_consensus.cc:1275] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:10.863495 17197 consensus_queue.cc:1048] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:10.874958 15935 raft_consensus.cc:1275] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:10.877272 17224 consensus_queue.cc:1048] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:11.120128 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:11.188977 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:11.221912 15859 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "260d22f804a345ed8f055536994fcb14"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:05:11.222354 15859 raft_consensus.cc:606] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Received request to transfer leadership to TS eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:11.225163 15859 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "32207d388e0e4185898655b74ae976b1"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:05:11.225754 15859 raft_consensus.cc:606] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Received request to transfer leadership to TS 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:11.227828 16006 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef"
dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
mode: GRACEFUL
new_leader_uuid: "8430db80a39d4868bcf7641c16de35ea"
 from {username='slave'} at 127.0.0.1:37262
I20260709 13:05:11.228262 16006 raft_consensus.cc:606] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Received request to transfer leadership to TS 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:11.230393 15931 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:46118
I20260709 13:05:11.230907 15931 raft_consensus.cc:606] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Received request to transfer leadership to TS eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:11.234534 15931 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ac847593d6724522bda12a2207f91c31"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:46118
I20260709 13:05:11.234977 15931 raft_consensus.cc:606] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Received request to transfer leadership to TS 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:11.236063 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:05:11.237705 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:11.240545 17108 raft_consensus.cc:993] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea: : Instructing follower 31882606e87447fb9a36bbf625129fe8 to start an election
I20260709 13:05:11.240998 17108 raft_consensus.cc:1081] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Signalling peer 31882606e87447fb9a36bbf625129fe8 to start an election
I20260709 13:05:11.242298 15859 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ac847593d6724522bda12a2207f91c31"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:43520
I20260709 13:05:11.242712 15859 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:11.242992 15859 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:11.249269 15859 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:11.250633 15859 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:11.251264 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:11.251466 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:11.251833 15931 raft_consensus.cc:3055] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:11.252048 16006 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:11.252173 15931 raft_consensus.cc:740] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:11.253008 15931 consensus_queue.cc:260] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:11.253839 15931 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:11.256765 16006 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:11.257742 15827 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:11.258391 17225 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:11.258432 15931 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:11.258987 17225 raft_consensus.cc:697] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:11.260798 17225 consensus_queue.cc:237] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:11.270562 15759 catalog_manager.cc:5697] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 1 to 2, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:11.315277 17242 raft_consensus.cc:993] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee: : Instructing follower 8430db80a39d4868bcf7641c16de35ea to start an election
I20260709 13:05:11.315829 17274 raft_consensus.cc:1081] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Signalling peer 8430db80a39d4868bcf7641c16de35ea to start an election
I20260709 13:05:11.317123 15935 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
 from {username='slave'} at 127.0.0.1:58402
I20260709 13:05:11.317550 15935 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:11.317829 15935 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:11.323113 15935 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:11.324596 15935 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:11.325222 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:11.325832 15859 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:11.326138 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:11.326752 16006 raft_consensus.cc:3055] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:11.327047 16006 raft_consensus.cc:740] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:11.327823 16006 consensus_queue.cc:260] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:11.328969 16006 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:11.331337 15859 raft_consensus.cc:2468] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:11.332569 15900 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:11.333328 17234 raft_consensus.cc:2804] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:11.338034 17234 raft_consensus.cc:697] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:11.338650 16006 raft_consensus.cc:2468] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:11.338851 17234 consensus_queue.cc:237] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:11.345046 17108 raft_consensus.cc:993] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea: : Instructing follower eb1e0493af1e46dbb6dec7e25525bab5 to start an election
I20260709 13:05:11.345480 17108 raft_consensus.cc:1081] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Signalling peer eb1e0493af1e46dbb6dec7e25525bab5 to start an election
I20260709 13:05:11.346851 16568 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888"
dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:45244
I20260709 13:05:11.347334 16568 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:11.347651 16568 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:11.352738 15759 catalog_manager.cc:5697] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 3 to 4, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 4 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:11.353505 16568 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:11.356683 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:11.356952 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:11.355962 16568 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:11.357240 15935 raft_consensus.cc:3055] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:11.357502 16006 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:11.359242 15935 raft_consensus.cc:740] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:11.360132 15935 consensus_queue.cc:260] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:11.361344 15935 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:11.365929 16006 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:11.367033 16535 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:11.367192 15935 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:11.367764 17179 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:11.368657 17179 raft_consensus.cc:697] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:11.369493 17179 consensus_queue.cc:237] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:11.377741 15759 catalog_manager.cc:5697] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 1 to 2, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 2 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:11.496595 17225 raft_consensus.cc:993] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8: : Instructing follower 85e997ba5003439781ec4f1f8db0c3ee to start an election
I20260709 13:05:11.497072 16988 raft_consensus.cc:1081] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Signalling peer 85e997ba5003439781ec4f1f8db0c3ee to start an election
I20260709 13:05:11.498358 16011 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "32207d388e0e4185898655b74ae976b1"
dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:41674
I20260709 13:05:11.498804 16011 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:11.499094 16011 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:11.504309 16011 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:11.505810 16011 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:11.506683 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:11.507232 15931 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:11.507388 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:11.508019 15859 raft_consensus.cc:3055] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:11.508344 15859 raft_consensus.cc:740] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:11.509112 15859 consensus_queue.cc:260] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:11.510156 15859 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:11.513069 15931 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:11.514058 15974 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:11.514667 17242 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:11.515156 17242 raft_consensus.cc:697] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:11.516032 17242 consensus_queue.cc:237] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:11.516491 15859 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:11.525429 15759 catalog_manager.cc:5697] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 2 to 3, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 3 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:11.741941 16011 raft_consensus.cc:1275] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:11.743382 16988 consensus_queue.cc:1048] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:11.757858 16011 raft_consensus.cc:1275] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:11.760133 17126 consensus_queue.cc:1048] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:11.774446 15936 raft_consensus.cc:1275] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:11.776317 16988 consensus_queue.cc:1048] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:11.807567 15859 raft_consensus.cc:1275] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:11.811605 16988 raft_consensus.cc:993] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8: : Instructing follower eb1e0493af1e46dbb6dec7e25525bab5 to start an election
I20260709 13:05:11.811405 17234 consensus_queue.cc:1048] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:11.812109 16988 raft_consensus.cc:1081] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Signalling peer eb1e0493af1e46dbb6dec7e25525bab5 to start an election
I20260709 13:05:11.814505 16566 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "260d22f804a345ed8f055536994fcb14"
dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:45246
I20260709 13:05:11.815073 16566 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:11.814548 16011 consensus_queue.cc:237] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:11.815419 16566 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:11.828142 15931 raft_consensus.cc:1275] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:11.829437 17179 consensus_queue.cc:1048] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:05:11.849354 16566 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:11.856341 16566 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:11.859090 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:11.859740 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:11.860052 15859 raft_consensus.cc:3055] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:11.860532 15859 raft_consensus.cc:740] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:11.860595 15936 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:11.861368 15859 consensus_queue.cc:260] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:11.871258 15936 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:11.876410 16533 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:11.876739 15859 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:11.877200 17275 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:11.877676 17275 raft_consensus.cc:697] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:11.878470 17275 consensus_queue.cc:237] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:11.880324 16008 raft_consensus.cc:1275] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:11.881920 17179 consensus_queue.cc:1048] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:11.891894 15859 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
W20260709 13:05:11.896617 15976 consensus_peers.cc:597] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 40e7e6076e8d475da059aad457e0e8b1. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:11.899820 15931 raft_consensus.cc:1275] T 40e7e6076e8d475da059aad457e0e8b1 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:11.902884 15861 raft_consensus.cc:1275] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:11.904218 17242 consensus_queue.cc:1048] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:11.906301 17242 consensus_queue.cc:1048] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:11.914357 15760 catalog_manager.cc:5697] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 1 to 2, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 2 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:11.920804 17242 raft_consensus.cc:2955] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:11.921754 15861 raft_consensus.cc:2955] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:11.940590 15931 raft_consensus.cc:2955] T 40e7e6076e8d475da059aad457e0e8b1 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:11.940881 15760 catalog_manager.cc:5697] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:11.945866 16011 consensus_queue.cc:237] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
W20260709 13:05:11.968027 15976 consensus_peers.cc:597] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ae068fc0a8a84b148d068df2b4463b8c. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:11.968993 15935 raft_consensus.cc:1275] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:11.970118 15858 raft_consensus.cc:1275] T ae068fc0a8a84b148d068df2b4463b8c P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:11.970410 17242 consensus_queue.cc:1048] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:11.972227 17242 consensus_queue.cc:1048] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:11.991118 17274 raft_consensus.cc:2955] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:11.993360 15858 raft_consensus.cc:2955] T ae068fc0a8a84b148d068df2b4463b8c P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:11.997095 15935 raft_consensus.cc:2955] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:12.012118 15758 catalog_manager.cc:5697] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea reported cstate change: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:12.012337 15935 consensus_queue.cc:237] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:12.021417 16011 raft_consensus.cc:1275] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260709 13:05:12.021555 15902 consensus_peers.cc:597] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2f401f200ca34a14a539f8a74675eab4. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:12.022922 15858 raft_consensus.cc:1275] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:05:12.025345 17234 consensus_queue.cc:1048] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:12.027192 17234 consensus_queue.cc:1048] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:12.042318 15931 raft_consensus.cc:1275] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:12.043897 17274 consensus_queue.cc:1048] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:12.050146 17126 raft_consensus.cc:2955] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:12.054854 15858 raft_consensus.cc:1275] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:12.055166 16006 raft_consensus.cc:2955] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:12.059839 17274 consensus_queue.cc:1048] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:12.063275 15861 raft_consensus.cc:2955] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:12.065562 16008 consensus_queue.cc:237] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:12.090864 15758 catalog_manager.cc:5697] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: config changed from index -1 to 4, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 3 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:12.097522 15931 raft_consensus.cc:1275] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:12.101164 17274 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:12.102438 15861 raft_consensus.cc:1275] T b5d68c78f0d94e0eb90543b642d1f1af P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:05:12.103883 15976 consensus_peers.cc:597] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b5d68c78f0d94e0eb90543b642d1f1af. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:12.104751 17120 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:12.122174 17274 raft_consensus.cc:2955] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:12.125098 15931 raft_consensus.cc:2955] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:12.131623 15861 raft_consensus.cc:2955] T b5d68c78f0d94e0eb90543b642d1f1af P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:12.144380 15758 catalog_manager.cc:5697] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea reported cstate change: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:12.238565 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:12.330428 15931 raft_consensus.cc:1275] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:12.331950 17179 consensus_queue.cc:1048] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:12.361609 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:12.364938 15861 raft_consensus.cc:1275] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:12.367942 17179 consensus_queue.cc:1048] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:12.436926 17306 ts_tablet_manager.cc:933] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:12.441169 17306 tablet_copy_client.cc:323] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.3:46251
I20260709 13:05:12.443012 16021 tablet_copy_service.cc:140] P 85e997ba5003439781ec4f1f8db0c3ee: Received BeginTabletCopySession request for tablet ae068fc0a8a84b148d068df2b4463b8c from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:60156)
I20260709 13:05:12.443528 17307 ts_tablet_manager.cc:933] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:12.443589 16021 tablet_copy_service.cc:161] P 85e997ba5003439781ec4f1f8db0c3ee: Beginning new tablet copy session on tablet ae068fc0a8a84b148d068df2b4463b8c from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:60156: session id = eb1e0493af1e46dbb6dec7e25525bab5-ae068fc0a8a84b148d068df2b4463b8c
I20260709 13:05:12.454476 16008 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44"
dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
mode: GRACEFUL
new_leader_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:37262
I20260709 13:05:12.455106 16008 raft_consensus.cc:606] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Received request to transfer leadership to TS 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:12.461875 15936 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:46118
I20260709 13:05:12.462415 15936 raft_consensus.cc:606] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Received request to transfer leadership to TS 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:12.468848 15861 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "8430db80a39d4868bcf7641c16de35ea"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:05:12.469669 15861 raft_consensus.cc:606] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Received request to transfer leadership to TS 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:12.471362 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:05:12.473109 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:12.476073 16021 tablet_copy_source_session.cc:215] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:12.478910 17307 tablet_copy_client.cc:323] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.3:46251
I20260709 13:05:12.483625 17306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae068fc0a8a84b148d068df2b4463b8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.484450 16021 tablet_copy_service.cc:140] P 85e997ba5003439781ec4f1f8db0c3ee: Received BeginTabletCopySession request for tablet 40e7e6076e8d475da059aad457e0e8b1 from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:60156)
I20260709 13:05:12.485042 16021 tablet_copy_service.cc:161] P 85e997ba5003439781ec4f1f8db0c3ee: Beginning new tablet copy session on tablet 40e7e6076e8d475da059aad457e0e8b1 from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:60156: session id = eb1e0493af1e46dbb6dec7e25525bab5-40e7e6076e8d475da059aad457e0e8b1
I20260709 13:05:12.493144 16021 tablet_copy_source_session.cc:215] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:12.497365 17307 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40e7e6076e8d475da059aad457e0e8b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.511861 17306 tablet_copy_client.cc:806] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:12.512598 17306 tablet_copy_client.cc:670] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:12.521199 17306 tablet_copy_client.cc:538] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:12.521672 17307 tablet_copy_client.cc:806] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:12.522234 17307 tablet_copy_client.cc:670] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:12.525422 17307 tablet_copy_client.cc:538] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:12.530246 17306 tablet_bootstrap.cc:492] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:05:12.532174 17307 tablet_bootstrap.cc:492] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:05:12.560832 17307 tablet_bootstrap.cc:492] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:12.561795 17307 tablet_bootstrap.cc:492] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:05:12.562325 17307 ts_tablet_manager.cc:1403] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.030s	user 0.016s	sys 0.010s
I20260709 13:05:12.562419 17306 tablet_bootstrap.cc:492] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:12.563369 17306 tablet_bootstrap.cc:492] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:05:12.564043 17306 ts_tablet_manager.cc:1403] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.034s	user 0.018s	sys 0.009s
I20260709 13:05:12.565665 17307 raft_consensus.cc:359] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:12.566553 17307 raft_consensus.cc:740] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:05:12.567233 17307 consensus_queue.cc:260] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:12.568665 17306 raft_consensus.cc:359] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:12.569551 17306 raft_consensus.cc:740] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:05:12.569689 17307 ts_tablet_manager.cc:1434] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:05:12.570246 17306 consensus_queue.cc:260] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:12.571707 16021 tablet_copy_service.cc:342] P 85e997ba5003439781ec4f1f8db0c3ee: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-40e7e6076e8d475da059aad457e0e8b1 received from {username='slave'} at 127.0.0.1:60156
I20260709 13:05:12.572111 16021 tablet_copy_service.cc:434] P 85e997ba5003439781ec4f1f8db0c3ee: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-40e7e6076e8d475da059aad457e0e8b1 on tablet 40e7e6076e8d475da059aad457e0e8b1 with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:12.577262 17306 ts_tablet_manager.cc:1434] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.001s
I20260709 13:05:12.582142 16021 tablet_copy_service.cc:342] P 85e997ba5003439781ec4f1f8db0c3ee: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-ae068fc0a8a84b148d068df2b4463b8c received from {username='slave'} at 127.0.0.1:60156
I20260709 13:05:12.582587 16021 tablet_copy_service.cc:434] P 85e997ba5003439781ec4f1f8db0c3ee: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-ae068fc0a8a84b148d068df2b4463b8c on tablet ae068fc0a8a84b148d068df2b4463b8c with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:12.584306 17307 ts_tablet_manager.cc:933] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:12.586516 17306 ts_tablet_manager.cc:933] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:12.595789 17306 tablet_copy_client.cc:323] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.3:46251
I20260709 13:05:12.597800 16021 tablet_copy_service.cc:140] P 85e997ba5003439781ec4f1f8db0c3ee: Received BeginTabletCopySession request for tablet b5d68c78f0d94e0eb90543b642d1f1af from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:60156)
I20260709 13:05:12.598328 16021 tablet_copy_service.cc:161] P 85e997ba5003439781ec4f1f8db0c3ee: Beginning new tablet copy session on tablet b5d68c78f0d94e0eb90543b642d1f1af from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:60156: session id = eb1e0493af1e46dbb6dec7e25525bab5-b5d68c78f0d94e0eb90543b642d1f1af
I20260709 13:05:12.613641 16021 tablet_copy_source_session.cc:215] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:12.630481 17306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5d68c78f0d94e0eb90543b642d1f1af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.652084 17307 tablet_copy_client.cc:323] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.2:43009
I20260709 13:05:12.653965 15946 tablet_copy_service.cc:140] P 8430db80a39d4868bcf7641c16de35ea: Received BeginTabletCopySession request for tablet 2f401f200ca34a14a539f8a74675eab4 from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:53636)
I20260709 13:05:12.654498 15946 tablet_copy_service.cc:161] P 8430db80a39d4868bcf7641c16de35ea: Beginning new tablet copy session on tablet 2f401f200ca34a14a539f8a74675eab4 from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:53636: session id = eb1e0493af1e46dbb6dec7e25525bab5-2f401f200ca34a14a539f8a74675eab4
I20260709 13:05:12.660728 17306 tablet_copy_client.cc:806] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:12.661326 17306 tablet_copy_client.cc:670] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:12.662081 15946 tablet_copy_source_session.cc:215] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:12.664528 17306 tablet_copy_client.cc:538] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:12.666112 17307 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f401f200ca34a14a539f8a74675eab4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:12.673978 17306 tablet_bootstrap.cc:492] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:05:12.679010 17224 raft_consensus.cc:993] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8: : Instructing follower 8430db80a39d4868bcf7641c16de35ea to start an election
I20260709 13:05:12.679488 17224 raft_consensus.cc:1081] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Signalling peer 8430db80a39d4868bcf7641c16de35ea to start an election
I20260709 13:05:12.681278 15936 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
 from {username='slave'} at 127.0.0.1:58390
I20260709 13:05:12.681794 15936 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:12.682101 15936 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:12.683292 17307 tablet_copy_client.cc:806] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:12.683832 17307 tablet_copy_client.cc:670] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:12.688185 17307 tablet_copy_client.cc:538] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:12.698362 17307 tablet_bootstrap.cc:492] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:05:12.717949 17274 raft_consensus.cc:993] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee: : Instructing follower 31882606e87447fb9a36bbf625129fe8 to start an election
I20260709 13:05:12.718451 17274 raft_consensus.cc:1081] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Signalling peer 31882606e87447fb9a36bbf625129fe8 to start an election
I20260709 13:05:12.720045 15861 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:43524
I20260709 13:05:12.720554 15861 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:12.720881 15861 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:12.707648 15936 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:12.729643 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:12.730189 15858 raft_consensus.cc:3055] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:12.730302 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:12.731144 16566 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:12.730465 15858 raft_consensus.cc:740] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:12.733094 15858 consensus_queue.cc:260] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:12.734099 15858 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:12.742833 15858 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2.
I20260709 13:05:12.751729 15861 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:12.753232 15936 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:12.754446 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:12.765333 16008 raft_consensus.cc:3055] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:12.765620 16008 raft_consensus.cc:740] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:12.766207 16566 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2.
I20260709 13:05:12.766285 16008 consensus_queue.cc:260] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:12.767510 16008 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:12.755492 15861 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:12.755869 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:12.769125 16568 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:12.756700 15900 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:12.772346 17126 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:12.772915 17126 raft_consensus.cc:697] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:12.773886 17126 consensus_queue.cc:237] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:12.764854 17306 tablet_bootstrap.cc:492] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:12.779992 17306 tablet_bootstrap.cc:492] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:05:12.781337 17306 ts_tablet_manager.cc:1403] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.108s	user 0.023s	sys 0.001s
I20260709 13:05:12.782373 16568 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:12.786796 15827 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:12.787647 16988 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:12.784368 17306 raft_consensus.cc:359] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:12.788126 16988 raft_consensus.cc:697] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:12.788360 17306 raft_consensus.cc:740] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:05:12.788950 16988 consensus_queue.cc:237] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:12.789067 17306 consensus_queue.cc:260] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:12.791764 17306 ts_tablet_manager.cc:1434] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:05:12.797885 16021 tablet_copy_service.cc:342] P 85e997ba5003439781ec4f1f8db0c3ee: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-b5d68c78f0d94e0eb90543b642d1f1af received from {username='slave'} at 127.0.0.1:60156
I20260709 13:05:12.798300 16021 tablet_copy_service.cc:434] P 85e997ba5003439781ec4f1f8db0c3ee: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-b5d68c78f0d94e0eb90543b642d1f1af on tablet b5d68c78f0d94e0eb90543b642d1f1af with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:12.794277 16008 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:12.819072 17108 raft_consensus.cc:993] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea: : Instructing follower 31882606e87447fb9a36bbf625129fe8 to start an election
I20260709 13:05:12.819597 17108 raft_consensus.cc:1081] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Signalling peer 31882606e87447fb9a36bbf625129fe8 to start an election
I20260709 13:05:12.829514 15858 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:43520
I20260709 13:05:12.830261 15858 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:12.830706 15858 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:12.843401 15755 catalog_manager.cc:5697] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 1 to 2, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:12.857239 17307 tablet_bootstrap.cc:492] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:12.858392 17307 tablet_bootstrap.cc:492] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:05:12.859057 17307 ts_tablet_manager.cc:1403] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.161s	user 0.065s	sys 0.069s
I20260709 13:05:12.861871 17307 raft_consensus.cc:359] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:12.862666 17307 raft_consensus.cc:740] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:05:12.863355 17307 consensus_queue.cc:260] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:12.865744 17307 ts_tablet_manager.cc:1434] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.002s
I20260709 13:05:12.865825 15858 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:12.867769 15946 tablet_copy_service.cc:342] P 8430db80a39d4868bcf7641c16de35ea: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-2f401f200ca34a14a539f8a74675eab4 received from {username='slave'} at 127.0.0.1:53636
I20260709 13:05:12.868213 15946 tablet_copy_service.cc:434] P 8430db80a39d4868bcf7641c16de35ea: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-2f401f200ca34a14a539f8a74675eab4 on tablet 2f401f200ca34a14a539f8a74675eab4 with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:12.869247 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:12.869884 15931 raft_consensus.cc:3055] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:12.870190 15931 raft_consensus.cc:740] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:12.870944 15931 consensus_queue.cc:260] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:12.871897 15931 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:12.874024 15858 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:12.874457 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:12.874987 16006 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:12.881338 15758 catalog_manager.cc:5697] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 1 to 2, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 2 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:12.882201 15931 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
I20260709 13:05:12.884063 15825 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:12.884136 16006 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
I20260709 13:05:12.884825 16988 raft_consensus.cc:2804] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:12.885469 16988 raft_consensus.cc:697] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:12.886226 16988 consensus_queue.cc:237] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:12.919081 15758 catalog_manager.cc:5697] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 2 to 3, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 3 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:12.968809 16566 raft_consensus.cc:1217] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:12.994640 16566 raft_consensus.cc:1217] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4]   Dedup: 3.4->[]
I20260709 13:05:13.005651 16566 raft_consensus.cc:1217] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:13.075240 16568 raft_consensus.cc:1217] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:13.268100 16568 raft_consensus.cc:1275] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:13.269259 17060 consensus_queue.cc:1048] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:13.284310 15861 raft_consensus.cc:1275] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:13.286353 17060 consensus_queue.cc:1048] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:13.318949 16006 raft_consensus.cc:1275] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:13.324420 16988 consensus_queue.cc:1048] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:13.359213 15935 raft_consensus.cc:1275] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:13.361064 16988 consensus_queue.cc:1048] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:13.389722 16008 raft_consensus.cc:1275] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:13.390887 16988 consensus_queue.cc:1048] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:13.408480 16568 raft_consensus.cc:1275] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:13.410656 17224 consensus_queue.cc:1048] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:13.452020 17242 raft_consensus.cc:1064] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:05:13.455053 17242 consensus_queue.cc:237] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:13.467032 16568 raft_consensus.cc:1275] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:13.468432 15858 raft_consensus.cc:1275] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:13.469236 17274 consensus_queue.cc:1048] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:13.470126 17120 consensus_queue.cc:1048] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:13.470115 15936 raft_consensus.cc:1275] T 40e7e6076e8d475da059aad457e0e8b1 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:13.474095 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:13.484643 17120 consensus_queue.cc:1048] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:13.500396 17060 raft_consensus.cc:1064] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:05:13.502753 17060 consensus_queue.cc:237] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:13.510900 17242 raft_consensus.cc:2955] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:13.521548 15936 raft_consensus.cc:2955] T 40e7e6076e8d475da059aad457e0e8b1 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:13.528151 16566 raft_consensus.cc:2955] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:13.544806 15758 catalog_manager.cc:5697] T 40e7e6076e8d475da059aad457e0e8b1 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:13.546386 17120 raft_consensus.cc:1064] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:05:13.549376 16569 raft_consensus.cc:1275] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 LEARNER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:05:13.549227 17120 consensus_queue.cc:237] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:13.550825 15861 raft_consensus.cc:1275] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:05:13.551268 16008 raft_consensus.cc:1275] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:05:13.552223 17126 consensus_queue.cc:1048] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:13.553777 17319 consensus_queue.cc:1048] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:13.554895 17126 consensus_queue.cc:1048] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:13.571609 15858 raft_consensus.cc:2955] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:05:13.599038 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 45821.5us
I20260709 13:05:13.623167 17126 raft_consensus.cc:2955] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:13.626094 17274 raft_consensus.cc:1064] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:05:13.628756 17274 consensus_queue.cc:237] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:13.630640 16008 raft_consensus.cc:2955] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:13.641062 15858 raft_consensus.cc:1275] T ae068fc0a8a84b148d068df2b4463b8c P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:13.648455 17242 consensus_queue.cc:1048] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:13.664572 15936 raft_consensus.cc:1275] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:13.666492 16566 raft_consensus.cc:2955] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:13.670022 15858 raft_consensus.cc:2955] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:13.672660 17242 consensus_queue.cc:1048] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:13.697293 16008 consensus_queue.cc:237] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:13.698997 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 67185us
I20260709 13:05:13.700737 16568 raft_consensus.cc:1275] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:13.702939 16568 raft_consensus.cc:1275] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:13.706904 17342 consensus_queue.cc:1048] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:13.707170 15859 raft_consensus.cc:1275] T b5d68c78f0d94e0eb90543b642d1f1af P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:13.710481 17350 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:13.714143 15931 raft_consensus.cc:1275] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:13.713500 15755 catalog_manager.cc:5697] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: config changed from index 4 to 5, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:13.714349 17342 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:13.727836 16570 raft_consensus.cc:1275] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:13.731462 17342 consensus_queue.cc:1048] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:13.739180 17350 raft_consensus.cc:2955] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:13.747535 16570 raft_consensus.cc:2955] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:13.765406 15858 raft_consensus.cc:1275] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:13.766227 15936 consensus_queue.cc:237] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:13.773980 17342 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:13.777130 16570 raft_consensus.cc:1275] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:05:13.778561 15746 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 40e7e6076e8d475da059aad457e0e8b1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:13.780956 15759 catalog_manager.cc:5697] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:13.782789 17126 consensus_queue.cc:1048] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:13.785919 17350 raft_consensus.cc:2955] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:13.794452 15861 raft_consensus.cc:1275] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:05:13.801141 17345 consensus_queue.cc:1048] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:13.802261 17060 raft_consensus.cc:2955] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:13.809643 16570 raft_consensus.cc:2955] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:13.822661 15744 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2f401f200ca34a14a539f8a74675eab4 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:13.829871 17274 consensus_queue.cc:1048] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:13.830042 15861 raft_consensus.cc:2955] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:13.832996 17350 raft_consensus.cc:2955] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:13.834967 15923 tablet_service.cc:1558] Processing DeleteTablet for tablet 40e7e6076e8d475da059aad457e0e8b1 with delete_type TABLET_DATA_TOMBSTONED (TS 8430db80a39d4868bcf7641c16de35ea not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58386
I20260709 13:05:13.836292 15862 raft_consensus.cc:2955] T ae068fc0a8a84b148d068df2b4463b8c P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:13.837668 15857 raft_consensus.cc:2955] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:13.854477 15858 raft_consensus.cc:2955] T b5d68c78f0d94e0eb90543b642d1f1af P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:13.874718 15755 catalog_manager.cc:5697] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: config changed from index 5 to 6, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New cstate: current_term: 3 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:13.856770 16568 raft_consensus.cc:2955] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:13.861162 15760 catalog_manager.cc:5697] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:13.892396 15935 raft_consensus.cc:2955] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:13.918058 17361 tablet_replica.cc:333] T 40e7e6076e8d475da059aad457e0e8b1 P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:05:13.926822 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:13.929023 15995 tablet_service.cc:1558] Processing DeleteTablet for tablet 2f401f200ca34a14a539f8a74675eab4 with delete_type TABLET_DATA_TOMBSTONED (TS 85e997ba5003439781ec4f1f8db0c3ee not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:41648
I20260709 13:05:13.926753 15760 catalog_manager.cc:5697] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:13.964751 17363 tablet_replica.cc:333] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:05:13.965091 17361 raft_consensus.cc:2243] T 40e7e6076e8d475da059aad457e0e8b1 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:13.973520 17361 raft_consensus.cc:2272] T 40e7e6076e8d475da059aad457e0e8b1 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:13.984973 17361 ts_tablet_manager.cc:1916] T 40e7e6076e8d475da059aad457e0e8b1 P 8430db80a39d4868bcf7641c16de35ea: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:13.973635 16570 raft_consensus.cc:2955] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:13.992260 15931 raft_consensus.cc:2955] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:13.999166 16011 consensus_queue.cc:237] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:14.014463 16569 raft_consensus.cc:1275] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:14.014618 17363 raft_consensus.cc:2243] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:14.015455 17363 raft_consensus.cc:2272] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:14.021003 17363 ts_tablet_manager.cc:1916] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:14.023006 17350 consensus_queue.cc:1048] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:14.048137 15935 raft_consensus.cc:1275] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:14.048647 16011 consensus_queue.cc:237] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:14.053910 17361 ts_tablet_manager.cc:1929] T 40e7e6076e8d475da059aad457e0e8b1 P 8430db80a39d4868bcf7641c16de35ea: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:14.054438 17361 log.cc:1199] T 40e7e6076e8d475da059aad457e0e8b1 P 8430db80a39d4868bcf7641c16de35ea: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-1-root/wals/40e7e6076e8d475da059aad457e0e8b1
I20260709 13:05:14.056154 15744 catalog_manager.cc:5028] TS 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): tablet 40e7e6076e8d475da059aad457e0e8b1 (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:05:14.058665 17120 consensus_queue.cc:1048] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:14.058092 17342 raft_consensus.cc:2955] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:14.059181 15935 raft_consensus.cc:1275] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:14.060564 17274 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:14.063369 16566 raft_consensus.cc:2955] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:14.068310 15935 raft_consensus.cc:2955] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:14.082070 17120 raft_consensus.cc:2955] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:14.084442 15936 raft_consensus.cc:2955] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:14.086522 16569 raft_consensus.cc:1275] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:14.088114 17274 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:14.094388 16568 raft_consensus.cc:2955] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:14.100819 15760 catalog_manager.cc:5697] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea reported cstate change: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:14.105625 15755 catalog_manager.cc:5697] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:14.109964 15746 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ae068fc0a8a84b148d068df2b4463b8c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:14.114553 15935 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:46118
I20260709 13:05:14.115172 15935 raft_consensus.cc:606] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Received request to transfer leadership to TS 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:14.124496 17363 ts_tablet_manager.cc:1929] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 13:05:14.124990 17363 log.cc:1199] T 2f401f200ca34a14a539f8a74675eab4 P 85e997ba5003439781ec4f1f8db0c3ee: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-2-root/wals/2f401f200ca34a14a539f8a74675eab4
I20260709 13:05:14.127884 15746 catalog_manager.cc:5028] TS 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): tablet 2f401f200ca34a14a539f8a74675eab4 (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:05:14.128600 15746 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b5d68c78f0d94e0eb90543b642d1f1af with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:14.131736 15858 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:05:14.134925 15858 raft_consensus.cc:606] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Received request to transfer leadership to TS 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:14.135211 15846 tablet_service.cc:1558] Processing DeleteTablet for tablet b5d68c78f0d94e0eb90543b642d1f1af with delete_type TABLET_DATA_TOMBSTONED (TS 31882606e87447fb9a36bbf625129fe8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43504
I20260709 13:05:14.137125 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:05:14.138657 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:14.145220 17366 tablet_replica.cc:333] T b5d68c78f0d94e0eb90543b642d1f1af P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:05:14.146209 17366 raft_consensus.cc:2243] T b5d68c78f0d94e0eb90543b642d1f1af P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:14.146901 17366 raft_consensus.cc:2272] T b5d68c78f0d94e0eb90543b642d1f1af P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:14.147723 15846 tablet_service.cc:1558] Processing DeleteTablet for tablet ae068fc0a8a84b148d068df2b4463b8c with delete_type TABLET_DATA_TOMBSTONED (TS 31882606e87447fb9a36bbf625129fe8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43504
I20260709 13:05:14.152328 17366 ts_tablet_manager.cc:1916] T b5d68c78f0d94e0eb90543b642d1f1af P 31882606e87447fb9a36bbf625129fe8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:14.167126 17366 ts_tablet_manager.cc:1929] T b5d68c78f0d94e0eb90543b642d1f1af P 31882606e87447fb9a36bbf625129fe8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:14.167557 17366 log.cc:1199] T b5d68c78f0d94e0eb90543b642d1f1af P 31882606e87447fb9a36bbf625129fe8: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-0-root/wals/b5d68c78f0d94e0eb90543b642d1f1af
I20260709 13:05:14.169552 17366 tablet_replica.cc:333] T ae068fc0a8a84b148d068df2b4463b8c P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:05:14.169587 15744 catalog_manager.cc:5028] TS 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): tablet b5d68c78f0d94e0eb90543b642d1f1af (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:05:14.170715 17366 raft_consensus.cc:2243] T ae068fc0a8a84b148d068df2b4463b8c P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:14.171432 17366 raft_consensus.cc:2272] T ae068fc0a8a84b148d068df2b4463b8c P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:14.174288 17366 ts_tablet_manager.cc:1916] T ae068fc0a8a84b148d068df2b4463b8c P 31882606e87447fb9a36bbf625129fe8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:14.189400 17366 ts_tablet_manager.cc:1929] T ae068fc0a8a84b148d068df2b4463b8c P 31882606e87447fb9a36bbf625129fe8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:14.189751 17366 log.cc:1199] T ae068fc0a8a84b148d068df2b4463b8c P 31882606e87447fb9a36bbf625129fe8: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-0-root/wals/ae068fc0a8a84b148d068df2b4463b8c
I20260709 13:05:14.190968 15744 catalog_manager.cc:5028] TS 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): tablet ae068fc0a8a84b148d068df2b4463b8c (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:05:14.254427 17319 raft_consensus.cc:993] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea: : Instructing follower 31882606e87447fb9a36bbf625129fe8 to start an election
I20260709 13:05:14.254932 17365 raft_consensus.cc:1081] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Signalling peer 31882606e87447fb9a36bbf625129fe8 to start an election
I20260709 13:05:14.256254 15862 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:43520
I20260709 13:05:14.256691 15862 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:14.256987 15862 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:14.261305 15862 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:14.262934 15862 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:14.263515 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:14.263826 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:14.264178 15935 raft_consensus.cc:3055] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:14.264328 16011 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:14.264513 15935 raft_consensus.cc:740] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:14.265275 15935 consensus_queue.cc:260] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:14.266330 15935 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:14.270653 16011 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:05:14.271723 15827 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:14.272430 17224 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:14.272920 17224 raft_consensus.cc:697] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 4 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:14.273427 15935 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:05:14.273679 17224 consensus_queue.cc:237] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:14.277303 17354 raft_consensus.cc:993] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8: : Instructing follower 85e997ba5003439781ec4f1f8db0c3ee to start an election
I20260709 13:05:14.277891 17334 raft_consensus.cc:1081] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Signalling peer 85e997ba5003439781ec4f1f8db0c3ee to start an election
I20260709 13:05:14.279397 16011 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44"
dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:41674
I20260709 13:05:14.279889 16011 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:14.280215 16011 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:14.286494 15760 catalog_manager.cc:5697] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 3 to 4, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 4 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:14.286526 16011 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:14.291306 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:14.291836 16569 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:14.291970 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:14.292744 15862 raft_consensus.cc:3055] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:14.292948 15862 raft_consensus.cc:740] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:14.293404 15862 consensus_queue.cc:260] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:14.294054 15862 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:14.294677 16011 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:14.298743 16569 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:14.300001 15976 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:14.300292 15862 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:14.300632 17242 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:14.301199 17242 raft_consensus.cc:697] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:14.301923 17242 consensus_queue.cc:237] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:14.309877 15760 catalog_manager.cc:5697] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 2 to 3, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 3 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:15.143076 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:15.444309 15862 raft_consensus.cc:1275] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:05:15.448374 17342 consensus_queue.cc:1048] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:15.464927 16011 consensus_queue.cc:237] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
W20260709 13:05:15.491384 15825 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.002s	sys 0.009s
W20260709 13:05:15.491770 15825 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.002s	sys 0.009s
I20260709 13:05:15.492318 15862 raft_consensus.cc:1275] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:05:15.494927 17377 consensus_queue.cc:1048] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:15.495580 15936 raft_consensus.cc:1275] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:05:15.514662 17334 consensus_queue.cc:1048] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:15.546044 16568 raft_consensus.cc:1275] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:05:15.549890 16007 raft_consensus.cc:1275] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:05:15.556537 17368 consensus_queue.cc:1048] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:15.568831 15935 raft_consensus.cc:1275] T 952bd0ea3bba4d38ad3383fdf572cd01 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 13:05:15.569332 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 54128.3us
I20260709 13:05:15.581498 17380 consensus_queue.cc:1048] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:05:15.595270 17390 log.cc:927] Time spent T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:05:15.630466 17380 consensus_queue.cc:1048] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:05:15.631521 15976 consensus_peers.cc:597] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 952bd0ea3bba4d38ad3383fdf572cd01. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:15.659596 15861 raft_consensus.cc:2955] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:15.656782 17380 raft_consensus.cc:2955] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:15.680186 15935 consensus_queue.cc:237] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:15.693135 15760 catalog_manager.cc:5697] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: config changed from index -1 to 3, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 2 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:15.706441 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:15.710312 15861 raft_consensus.cc:1275] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:05:15.705921 16010 raft_consensus.cc:1275] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:05:15.715585 17345 consensus_queue.cc:1048] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:05:15.705572 15902 consensus_peers.cc:597] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ca01a56f016b4291b8abfddb75b74dd9. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:15.722203 17345 consensus_queue.cc:1048] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:15.721607 15935 raft_consensus.cc:2955] T 952bd0ea3bba4d38ad3383fdf572cd01 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:15.743723 17373 raft_consensus.cc:2955] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:15.747421 16003 raft_consensus.cc:2955] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:15.760499 15932 consensus_queue.cc:237] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:15.770589 15862 raft_consensus.cc:2955] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:15.780627 15758 catalog_manager.cc:5697] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: config changed from index -1 to 3, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 2 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:15.794899 15902 consensus_peers.cc:597] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4fcbf678894c4a1eb80c68b5b53acaef. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:15.797676 16010 raft_consensus.cc:1275] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:05:15.799063 17371 consensus_queue.cc:1048] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:15.803021 15861 raft_consensus.cc:1275] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:05:15.806870 17371 consensus_queue.cc:1048] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:15.826328 17373 raft_consensus.cc:2955] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index -1 to 5, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:15.833217 15855 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "8430db80a39d4868bcf7641c16de35ea"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:05:15.833923 15855 raft_consensus.cc:606] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 4 LEADER]: Received request to transfer leadership to TS 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:15.833488 15862 raft_consensus.cc:2955] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index -1 to 5, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:15.836915 16010 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44"
dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
mode: GRACEFUL
new_leader_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:37262
I20260709 13:05:15.839438 16010 raft_consensus.cc:606] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Received request to transfer leadership to TS 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:15.838070 16003 raft_consensus.cc:2955] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index -1 to 5, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:15.842631 15861 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "8430db80a39d4868bcf7641c16de35ea"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:05:15.843317 15861 raft_consensus.cc:606] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Received request to transfer leadership to TS 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:15.844569 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:05:15.846271 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:15.857082 15760 catalog_manager.cc:5697] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 reported cstate change: config changed from index -1 to 5, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 4 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:15.866425 16003 consensus_queue.cc:237] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:15.883713 17350 raft_consensus.cc:993] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee: : Instructing follower 31882606e87447fb9a36bbf625129fe8 to start an election
I20260709 13:05:15.884240 17242 raft_consensus.cc:1081] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Signalling peer 31882606e87447fb9a36bbf625129fe8 to start an election
I20260709 13:05:15.888724 15861 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:43524
I20260709 13:05:15.889266 15861 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:15.889599 15861 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:15.900698 15862 raft_consensus.cc:1275] T 3a538795cdae403fae5941538575596b P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:15.903061 15932 raft_consensus.cc:1275] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:15.902930 17376 consensus_queue.cc:1048] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:15.906278 15861 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:15.919364 15861 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:15.920912 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:15.922091 16565 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:15.927348 17378 consensus_queue.cc:1048] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:05:15.922099 15976 consensus_peers.cc:597] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a538795cdae403fae5941538575596b. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:15.931222 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:15.931912 16003 raft_consensus.cc:3055] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:15.932286 16003 raft_consensus.cc:740] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:15.933173 16003 consensus_queue.cc:260] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:15.937510 16003 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:15.944903 16003 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:05:15.947969 15827 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:15.949537 17370 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:15.950337 17370 raft_consensus.cc:697] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 4 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:15.949630 16565 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:05:15.951261 17370 consensus_queue.cc:237] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:15.966133 15756 catalog_manager.cc:5697] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 3 to 4, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 4 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:15.983839 17242 raft_consensus.cc:2955] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:15.995951 15932 raft_consensus.cc:2955] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:16.000283 15855 raft_consensus.cc:2955] T 3a538795cdae403fae5941538575596b P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:16.108513 17410 ts_tablet_manager.cc:933] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:16.012038 15756 catalog_manager.cc:5697] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:16.133359 17364 raft_consensus.cc:993] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8: : Instructing follower 8430db80a39d4868bcf7641c16de35ea to start an election
I20260709 13:05:16.133996 17364 raft_consensus.cc:1081] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Signalling peer 8430db80a39d4868bcf7641c16de35ea to start an election
I20260709 13:05:16.136457 15932 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
 from {username='slave'} at 127.0.0.1:58390
I20260709 13:05:16.136962 15932 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:16.137265 15932 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:16.137279 17354 raft_consensus.cc:993] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8: : Instructing follower 8430db80a39d4868bcf7641c16de35ea to start an election
I20260709 13:05:16.147883 17354 raft_consensus.cc:1081] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 4 LEADER]: Signalling peer 8430db80a39d4868bcf7641c16de35ea to start an election
I20260709 13:05:16.149392 15935 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
 from {username='slave'} at 127.0.0.1:58390
I20260709 13:05:16.149900 15935 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:16.150218 15935 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:16.150662 15932 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:16.158303 15935 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:16.161372 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:16.161667 15935 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:16.162133 16010 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:16.164685 15932 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:16.165598 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:16.166347 16011 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:16.172099 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:16.172871 15858 raft_consensus.cc:3055] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:16.173233 15858 raft_consensus.cc:740] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:16.174120 15858 consensus_queue.cc:260] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:16.175293 16010 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
I20260709 13:05:16.175484 15858 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:16.176074 16003 raft_consensus.cc:1240] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:16.183590 16011 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:16.184197 17410 tablet_copy_client.cc:323] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.3:46251
I20260709 13:05:16.184643 17370 consensus_queue.cc:1059] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
I20260709 13:05:16.200304 17370 raft_consensus.cc:3055] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:16.200802 17370 raft_consensus.cc:740] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:16.201841 17370 consensus_queue.cc:260] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:16.185837 15902 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:16.204195 17373 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:16.204668 17373 raft_consensus.cc:697] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 5 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:16.204517 15902 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:16.205559 17373 consensus_queue.cc:237] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:16.206007 17371 raft_consensus.cc:2804] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:16.206662 17371 raft_consensus.cc:697] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:16.207720 17371 consensus_queue.cc:237] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:16.203583 17370 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:16.184620 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:16.221608 16021 tablet_copy_service.cc:140] P 85e997ba5003439781ec4f1f8db0c3ee: Received BeginTabletCopySession request for tablet 952bd0ea3bba4d38ad3383fdf572cd01 from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:60156)
I20260709 13:05:16.222122 16021 tablet_copy_service.cc:161] P 85e997ba5003439781ec4f1f8db0c3ee: Beginning new tablet copy session on tablet 952bd0ea3bba4d38ad3383fdf572cd01 from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:60156: session id = eb1e0493af1e46dbb6dec7e25525bab5-952bd0ea3bba4d38ad3383fdf572cd01
I20260709 13:05:16.227238 17412 ts_tablet_manager.cc:933] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:16.228993 17412 tablet_copy_client.cc:323] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.2:43009
I20260709 13:05:16.230535 16021 tablet_copy_source_session.cc:215] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:16.230589 15946 tablet_copy_service.cc:140] P 8430db80a39d4868bcf7641c16de35ea: Received BeginTabletCopySession request for tablet ca01a56f016b4291b8abfddb75b74dd9 from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:53636)
I20260709 13:05:16.231231 15946 tablet_copy_service.cc:161] P 8430db80a39d4868bcf7641c16de35ea: Beginning new tablet copy session on tablet ca01a56f016b4291b8abfddb75b74dd9 from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:53636: session id = eb1e0493af1e46dbb6dec7e25525bab5-ca01a56f016b4291b8abfddb75b74dd9
I20260709 13:05:16.190019 15858 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:16.242949 15946 tablet_copy_source_session.cc:215] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:16.242659 15756 catalog_manager.cc:5697] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 4 to 5, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 5 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:16.244319 15756 catalog_manager.cc:5697] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 3 to 4, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 4 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:16.252043 17412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca01a56f016b4291b8abfddb75b74dd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.256225 17410 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 952bd0ea3bba4d38ad3383fdf572cd01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.258853 15855 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
I20260709 13:05:16.268199 17412 tablet_copy_client.cc:806] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:16.269047 17412 tablet_copy_client.cc:670] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:16.272990 17412 tablet_copy_client.cc:538] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:16.278028 17410 tablet_copy_client.cc:806] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:16.278581 17410 tablet_copy_client.cc:670] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:16.282192 17410 tablet_copy_client.cc:538] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:16.282861 17412 tablet_bootstrap.cc:492] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:05:16.290982 17410 tablet_bootstrap.cc:492] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
W20260709 13:05:16.303141 15791 debug-util.cc:398] Leaking SignalData structure 0x7b080015f0e0 after lost signal to thread 14751
W20260709 13:05:16.304142 15791 debug-util.cc:398] Leaking SignalData structure 0x7b080026cda0 after lost signal to thread 15794
W20260709 13:05:16.305212 15791 debug-util.cc:398] Leaking SignalData structure 0x7b0800092040 after lost signal to thread 15886
W20260709 13:05:16.306174 15791 debug-util.cc:398] Leaking SignalData structure 0x7b080008a7c0 after lost signal to thread 15961
W20260709 13:05:16.307137 15791 debug-util.cc:398] Leaking SignalData structure 0x7b0800350560 after lost signal to thread 16035
W20260709 13:05:16.311045 15791 debug-util.cc:398] Leaking SignalData structure 0x7b080033e840 after lost signal to thread 16594
I20260709 13:05:16.337797 17412 tablet_bootstrap.cc:492] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:16.339982 17412 tablet_bootstrap.cc:492] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:05:16.340952 17412 ts_tablet_manager.cc:1403] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.058s	user 0.020s	sys 0.012s
I20260709 13:05:16.345284 17412 raft_consensus.cc:359] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:16.346149 17412 raft_consensus.cc:740] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:05:16.346853 17412 consensus_queue.cc:260] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:16.349609 17412 ts_tablet_manager.cc:1434] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:05:16.352255 17410 tablet_bootstrap.cc:492] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:16.353674 17410 tablet_bootstrap.cc:492] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:05:16.354662 15946 tablet_copy_service.cc:342] P 8430db80a39d4868bcf7641c16de35ea: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-ca01a56f016b4291b8abfddb75b74dd9 received from {username='slave'} at 127.0.0.1:53636
I20260709 13:05:16.355132 15946 tablet_copy_service.cc:434] P 8430db80a39d4868bcf7641c16de35ea: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-ca01a56f016b4291b8abfddb75b74dd9 on tablet ca01a56f016b4291b8abfddb75b74dd9 with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:16.354351 17410 ts_tablet_manager.cc:1403] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.064s	user 0.037s	sys 0.016s
I20260709 13:05:16.359938 17410 raft_consensus.cc:359] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:16.360857 17410 raft_consensus.cc:740] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:05:16.361536 17410 consensus_queue.cc:260] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:16.364282 17410 ts_tablet_manager.cc:1434] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:05:16.365857 16021 tablet_copy_service.cc:342] P 85e997ba5003439781ec4f1f8db0c3ee: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-952bd0ea3bba4d38ad3383fdf572cd01 received from {username='slave'} at 127.0.0.1:60156
I20260709 13:05:16.366290 16021 tablet_copy_service.cc:434] P 85e997ba5003439781ec4f1f8db0c3ee: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-952bd0ea3bba4d38ad3383fdf572cd01 on tablet 952bd0ea3bba4d38ad3383fdf572cd01 with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:16.386065 17410 ts_tablet_manager.cc:933] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:16.403136 17410 tablet_copy_client.cc:323] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.3:46251
I20260709 13:05:16.406032 16021 tablet_copy_service.cc:140] P 85e997ba5003439781ec4f1f8db0c3ee: Received BeginTabletCopySession request for tablet 3a538795cdae403fae5941538575596b from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:60156)
I20260709 13:05:16.406610 16021 tablet_copy_service.cc:161] P 85e997ba5003439781ec4f1f8db0c3ee: Beginning new tablet copy session on tablet 3a538795cdae403fae5941538575596b from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:60156: session id = eb1e0493af1e46dbb6dec7e25525bab5-3a538795cdae403fae5941538575596b
I20260709 13:05:16.413360 17412 ts_tablet_manager.cc:933] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:16.416469 16021 tablet_copy_source_session.cc:215] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:16.419797 17412 tablet_copy_client.cc:323] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.2:43009
I20260709 13:05:16.420116 17410 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a538795cdae403fae5941538575596b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:16.425235 15946 tablet_copy_service.cc:140] P 8430db80a39d4868bcf7641c16de35ea: Received BeginTabletCopySession request for tablet 4fcbf678894c4a1eb80c68b5b53acaef from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:53636)
I20260709 13:05:16.425833 15946 tablet_copy_service.cc:161] P 8430db80a39d4868bcf7641c16de35ea: Beginning new tablet copy session on tablet 4fcbf678894c4a1eb80c68b5b53acaef from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:53636: session id = eb1e0493af1e46dbb6dec7e25525bab5-4fcbf678894c4a1eb80c68b5b53acaef
I20260709 13:05:16.444388 17410 tablet_copy_client.cc:806] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:16.445201 17410 tablet_copy_client.cc:670] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:16.460397 17410 tablet_copy_client.cc:538] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:16.472456 16567 raft_consensus.cc:1275] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:05:16.483546 17410 tablet_bootstrap.cc:492] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:05:16.488101 17334 consensus_queue.cc:1048] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:16.533519 15946 tablet_copy_source_session.cc:215] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:16.541926 17410 tablet_bootstrap.cc:492] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:16.542802 17410 tablet_bootstrap.cc:492] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:05:16.543452 17410 ts_tablet_manager.cc:1403] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.060s	user 0.030s	sys 0.010s
I20260709 13:05:16.553298 17410 raft_consensus.cc:359] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:16.554119 17410 raft_consensus.cc:740] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:05:16.554771 17410 consensus_queue.cc:260] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:16.561447 16565 raft_consensus.cc:1217] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:05:16.563024 17410 ts_tablet_manager.cc:1434] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.003s
I20260709 13:05:16.564891 16021 tablet_copy_service.cc:342] P 85e997ba5003439781ec4f1f8db0c3ee: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-3a538795cdae403fae5941538575596b received from {username='slave'} at 127.0.0.1:60156
I20260709 13:05:16.565301 16021 tablet_copy_service.cc:434] P 85e997ba5003439781ec4f1f8db0c3ee: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-3a538795cdae403fae5941538575596b on tablet 3a538795cdae403fae5941538575596b with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:16.593446 17412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fcbf678894c4a1eb80c68b5b53acaef. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:05:16.623565 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 81105.8us
I20260709 13:05:16.626492 16010 raft_consensus.cc:1275] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:05:16.634167 15855 raft_consensus.cc:1275] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:16.640393 17368 consensus_queue.cc:1048] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:16.642166 17412 tablet_copy_client.cc:806] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:16.642783 17412 tablet_copy_client.cc:670] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:16.646665 17372 consensus_queue.cc:1048] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:16.648552 17412 tablet_copy_client.cc:538] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:16.657092 17412 tablet_bootstrap.cc:492] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:05:16.689397 16011 raft_consensus.cc:1275] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:05:16.694046 17372 consensus_queue.cc:1048] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:16.703480 17412 tablet_bootstrap.cc:492] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:16.704722 17412 tablet_bootstrap.cc:492] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:05:16.705405 17412 ts_tablet_manager.cc:1403] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.049s	user 0.027s	sys 0.006s
I20260709 13:05:16.708369 17412 raft_consensus.cc:359] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:16.709321 17412 raft_consensus.cc:740] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:05:16.709430 16003 raft_consensus.cc:1275] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:16.710055 17412 consensus_queue.cc:260] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:16.711256 17345 consensus_queue.cc:1048] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:16.725548 16565 raft_consensus.cc:1217] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:05:16.720347 17412 ts_tablet_manager.cc:1434] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.015s	user 0.002s	sys 0.004s
I20260709 13:05:16.731958 15946 tablet_copy_service.cc:342] P 8430db80a39d4868bcf7641c16de35ea: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-4fcbf678894c4a1eb80c68b5b53acaef received from {username='slave'} at 127.0.0.1:53636
I20260709 13:05:16.732380 15946 tablet_copy_service.cc:434] P 8430db80a39d4868bcf7641c16de35ea: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-4fcbf678894c4a1eb80c68b5b53acaef on tablet 4fcbf678894c4a1eb80c68b5b53acaef with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:16.754232 15855 raft_consensus.cc:1275] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:05:16.759080 17371 consensus_queue.cc:1048] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:16.847167 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:16.853459 16565 raft_consensus.cc:1217] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEARNER]: Deduplicated request from leader. Original: 4.4->[4.5-4.5]   Dedup: 4.5->[]
I20260709 13:05:16.865253 16565 raft_consensus.cc:1217] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:17.147535 17381 raft_consensus.cc:1064] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:05:17.147755 17350 raft_consensus.cc:1064] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:05:17.150173 17350 consensus_queue.cc:237] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:17.150298 17381 consensus_queue.cc:237] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:17.157929 16568 raft_consensus.cc:1275] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEARNER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:05:17.159634 16003 raft_consensus.cc:1275] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:05:17.160833 15855 raft_consensus.cc:1275] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:05:17.162226 17372 consensus_queue.cc:1048] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:17.163368 17345 consensus_queue.cc:1048] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:17.165230 17372 consensus_queue.cc:1048] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:17.177523 15931 raft_consensus.cc:1275] T 952bd0ea3bba4d38ad3383fdf572cd01 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:05:17.179369 17242 consensus_queue.cc:1048] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:17.178988 16565 raft_consensus.cc:1275] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEARNER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:05:17.185930 17242 consensus_queue.cc:1048] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:17.193358 17372 raft_consensus.cc:2955] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:17.194998 16568 raft_consensus.cc:2955] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:17.209473 15858 raft_consensus.cc:1275] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:05:17.215085 17350 consensus_queue.cc:1048] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:17.218118 15858 raft_consensus.cc:2955] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:17.241034 17242 raft_consensus.cc:2955] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:17.242111 16011 raft_consensus.cc:2955] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:17.253288 15931 raft_consensus.cc:2955] T 952bd0ea3bba4d38ad3383fdf572cd01 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:17.261780 15759 catalog_manager.cc:5697] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: config changed from index 3 to 4, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:17.267283 16568 raft_consensus.cc:2955] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:17.278764 15757 catalog_manager.cc:5697] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: config changed from index 3 to 4, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:17.299180 15855 raft_consensus.cc:2955] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:05:17.376266 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 48354.8us
W20260709 13:05:17.380893 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 53711.3us
I20260709 13:05:17.390142 15935 consensus_queue.cc:237] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:17.394181 17345 raft_consensus.cc:1064] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:05:17.399308 17345 consensus_queue.cc:237] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:17.461771 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:17.480751 15858 raft_consensus.cc:1275] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:17.486099 17451 raft_consensus.cc:1064] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:05:17.492808 17451 consensus_queue.cc:237] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:17.501156 17372 consensus_queue.cc:1048] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:17.509809 15858 raft_consensus.cc:1275] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:17.512130 17372 consensus_queue.cc:1048] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:17.515190 17452 raft_consensus.cc:2955] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:17.517297 15858 raft_consensus.cc:2955] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:17.539145 16010 consensus_queue.cc:237] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:17.557518 15855 raft_consensus.cc:1275] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:17.559106 17350 consensus_queue.cc:1048] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:17.561833 16568 raft_consensus.cc:1275] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:17.563663 17451 consensus_queue.cc:1048] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:17.567072 15855 raft_consensus.cc:1275] T 3a538795cdae403fae5941538575596b P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:17.569517 17242 consensus_queue.cc:1048] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:17.568899 17425 raft_consensus.cc:2955] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:17.575726 15855 raft_consensus.cc:2955] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:05:17.582060 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 93347.1us
I20260709 13:05:17.582895 16565 raft_consensus.cc:1275] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:17.591744 17454 consensus_queue.cc:1048] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:17.600013 15746 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 952bd0ea3bba4d38ad3383fdf572cd01 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:17.601012 16010 raft_consensus.cc:1275] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:17.605162 16564 raft_consensus.cc:1275] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEARNER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:17.606026 16566 raft_consensus.cc:1275] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:17.604167 16565 raft_consensus.cc:2955] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:17.611369 15744 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ca01a56f016b4291b8abfddb75b74dd9 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:17.620193 15932 raft_consensus.cc:1275] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:17.628684 17381 consensus_queue.cc:1048] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:17.635986 17381 consensus_queue.cc:1048] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:17.638352 17381 consensus_queue.cc:1048] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:17.650738 16567 raft_consensus.cc:2955] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:17.656667 17455 consensus_queue.cc:1048] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:05:17.674576 17460 log.cc:927] Time spent T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:05:17.678776 17373 raft_consensus.cc:2955] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Committing config change with OpId 4.6: config changed from index 5 to 6, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:17.688086 16010 raft_consensus.cc:2955] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:17.692183 15857 raft_consensus.cc:2955] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:17.706638 17451 raft_consensus.cc:2955] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:17.718415 16565 raft_consensus.cc:2955] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:17.714488 15932 raft_consensus.cc:2955] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:17.720552 16567 raft_consensus.cc:2955] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:17.722138 15862 raft_consensus.cc:2955] T 3a538795cdae403fae5941538575596b P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:17.731287 15756 catalog_manager.cc:5697] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: config changed from index 4 to 5, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New cstate: current_term: 2 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:17.738175 15757 catalog_manager.cc:5697] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: config changed from index 4 to 5, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New cstate: current_term: 2 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:17.745292 15995 tablet_service.cc:1558] Processing DeleteTablet for tablet ca01a56f016b4291b8abfddb75b74dd9 with delete_type TABLET_DATA_TOMBSTONED (TS 85e997ba5003439781ec4f1f8db0c3ee not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:41648
I20260709 13:05:17.755771 15923 tablet_service.cc:1558] Processing DeleteTablet for tablet 952bd0ea3bba4d38ad3383fdf572cd01 with delete_type TABLET_DATA_TOMBSTONED (TS 8430db80a39d4868bcf7641c16de35ea not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:58386
I20260709 13:05:17.755982 15756 catalog_manager.cc:5697] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 reported cstate change: config changed from index 5 to 6, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:17.758010 15756 catalog_manager.cc:5697] T 3a538795cdae403fae5941538575596b P 31882606e87447fb9a36bbf625129fe8 reported cstate change: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:17.771070 17469 tablet_replica.cc:333] T 952bd0ea3bba4d38ad3383fdf572cd01 P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:05:17.777375 17469 raft_consensus.cc:2243] T 952bd0ea3bba4d38ad3383fdf572cd01 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:17.783046 17469 raft_consensus.cc:2272] T 952bd0ea3bba4d38ad3383fdf572cd01 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:17.781319 17468 tablet_replica.cc:333] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:05:17.786422 17468 raft_consensus.cc:2243] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:17.789541 17468 raft_consensus.cc:2272] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:17.791460 17469 ts_tablet_manager.cc:1916] T 952bd0ea3bba4d38ad3383fdf572cd01 P 8430db80a39d4868bcf7641c16de35ea: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:17.798213 16010 consensus_queue.cc:237] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:17.791759 15932 consensus_queue.cc:237] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:17.805768 15935 raft_consensus.cc:1275] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:17.807297 15862 raft_consensus.cc:1275] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:05:17.807883 16567 raft_consensus.cc:1275] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:05:17.807821 17451 consensus_queue.cc:1048] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:17.805784 16565 raft_consensus.cc:1275] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:05:17.809470 17468 ts_tablet_manager.cc:1916] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:17.809633 17345 consensus_queue.cc:1048] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:17.816430 17451 consensus_queue.cc:1048] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:17.819211 17373 consensus_queue.cc:1048] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:17.820107 17469 ts_tablet_manager.cc:1929] T 952bd0ea3bba4d38ad3383fdf572cd01 P 8430db80a39d4868bcf7641c16de35ea: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:05:17.820514 17469 log.cc:1199] T 952bd0ea3bba4d38ad3383fdf572cd01 P 8430db80a39d4868bcf7641c16de35ea: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-1-root/wals/952bd0ea3bba4d38ad3383fdf572cd01
I20260709 13:05:17.822072 15744 catalog_manager.cc:5028] TS 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): tablet 952bd0ea3bba4d38ad3383fdf572cd01 (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:05:17.821906 17350 raft_consensus.cc:2955] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:17.827881 15935 raft_consensus.cc:2955] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:17.831910 17372 raft_consensus.cc:2955] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Committing config change with OpId 4.7: config changed from index 6 to 7, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:17.836886 16565 raft_consensus.cc:2955] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:17.837728 16567 raft_consensus.cc:2955] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Committing config change with OpId 4.7: config changed from index 6 to 7, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:17.843458 17468 ts_tablet_manager.cc:1929] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:05:17.843937 17468 log.cc:1199] T ca01a56f016b4291b8abfddb75b74dd9 P 85e997ba5003439781ec4f1f8db0c3ee: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-2-root/wals/ca01a56f016b4291b8abfddb75b74dd9
I20260709 13:05:17.845539 15746 catalog_manager.cc:5028] TS 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): tablet ca01a56f016b4291b8abfddb75b74dd9 (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:05:17.856861 15746 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3a538795cdae403fae5941538575596b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:05:17.861586 15757 catalog_manager.cc:5697] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: config changed from index 6 to 7, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New cstate: current_term: 4 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:17.862025 15862 raft_consensus.cc:2955] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Committing config change with OpId 4.7: config changed from index 6 to 7, VOTER 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:17.863951 15760 catalog_manager.cc:5697] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea reported cstate change: config changed from index 3 to 4, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:17.924878 15846 tablet_service.cc:1558] Processing DeleteTablet for tablet 3a538795cdae403fae5941538575596b with delete_type TABLET_DATA_TOMBSTONED (TS 31882606e87447fb9a36bbf625129fe8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43504
I20260709 13:05:17.927789 15744 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4fcbf678894c4a1eb80c68b5b53acaef with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:05:17.941506 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 53910.3us
I20260709 13:05:17.949909 15995 tablet_service.cc:1558] Processing DeleteTablet for tablet 4fcbf678894c4a1eb80c68b5b53acaef with delete_type TABLET_DATA_TOMBSTONED (TS 85e997ba5003439781ec4f1f8db0c3ee not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:41648
I20260709 13:05:17.950830 17468 tablet_replica.cc:333] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:05:17.952008 17470 tablet_replica.cc:333] T 3a538795cdae403fae5941538575596b P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:05:17.953284 17470 raft_consensus.cc:2243] T 3a538795cdae403fae5941538575596b P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:17.958946 17470 raft_consensus.cc:2272] T 3a538795cdae403fae5941538575596b P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:17.965742 15935 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:46118
I20260709 13:05:17.966102 17470 ts_tablet_manager.cc:1916] T 3a538795cdae403fae5941538575596b P 31882606e87447fb9a36bbf625129fe8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:17.966427 15935 raft_consensus.cc:606] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Received request to transfer leadership to TS 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:17.971692 15935 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:46118
I20260709 13:05:17.972239 15935 raft_consensus.cc:606] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Received request to transfer leadership to TS eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:17.974555 17468 raft_consensus.cc:2243] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:17.975375 15862 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e9577f76f40348e9b009255d441d012a"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:05:17.975997 15862 raft_consensus.cc:606] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Received request to transfer leadership to TS 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:17.975373 17468 raft_consensus.cc:2272] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:17.978824 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:05:17.980430 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:17.990015 17464 raft_consensus.cc:993] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8: : Instructing follower 85e997ba5003439781ec4f1f8db0c3ee to start an election
I20260709 13:05:17.990691 17422 raft_consensus.cc:1081] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Signalling peer 85e997ba5003439781ec4f1f8db0c3ee to start an election
I20260709 13:05:17.991788 17468 ts_tablet_manager.cc:1916] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:05:17.993101 16003 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e9577f76f40348e9b009255d441d012a"
dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:41674
I20260709 13:05:17.993820 16003 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:17.994222 16003 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:17.994820 17470 ts_tablet_manager.cc:1929] T 3a538795cdae403fae5941538575596b P 31882606e87447fb9a36bbf625129fe8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:05:17.995136 17470 log.cc:1199] T 3a538795cdae403fae5941538575596b P 31882606e87447fb9a36bbf625129fe8: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-0-root/wals/3a538795cdae403fae5941538575596b
I20260709 13:05:17.996807 15744 catalog_manager.cc:5028] TS 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): tablet 3a538795cdae403fae5941538575596b (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:05:18.001747 16003 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:05:18.004510 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:18.004771 16003 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:18.005087 15862 raft_consensus.cc:3055] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:18.005447 15862 raft_consensus.cc:740] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:18.005689 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:18.006235 15932 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:18.006125 15862 consensus_queue.cc:260] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:05:18.007171 15862 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:18.014528 17468 ts_tablet_manager.cc:1929] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.6
I20260709 13:05:18.015014 17468 log.cc:1199] T 4fcbf678894c4a1eb80c68b5b53acaef P 85e997ba5003439781ec4f1f8db0c3ee: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-2-root/wals/4fcbf678894c4a1eb80c68b5b53acaef
I20260709 13:05:18.015298 15932 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:18.015444 15862 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:18.016659 15974 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:18.017103 15746 catalog_manager.cc:5028] TS 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): tablet 4fcbf678894c4a1eb80c68b5b53acaef (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:05:18.017298 17451 raft_consensus.cc:2804] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:18.017871 17451 raft_consensus.cc:697] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:18.018622 17451 consensus_queue.cc:237] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:05:18.028509 15759 catalog_manager.cc:5697] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 2 to 3, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 3 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:18.102016 17345 raft_consensus.cc:993] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea: : Instructing follower eb1e0493af1e46dbb6dec7e25525bab5 to start an election
I20260709 13:05:18.102535 17373 raft_consensus.cc:1081] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Signalling peer eb1e0493af1e46dbb6dec7e25525bab5 to start an election
I20260709 13:05:18.103964 16568 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38"
dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:45244
I20260709 13:05:18.104380 16568 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:18.104598 16568 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:18.109190 16568 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:18.110934 16568 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:18.111948 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:18.112195 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:18.112627 16003 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:18.112702 15932 raft_consensus.cc:3055] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:18.113116 15932 raft_consensus.cc:740] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:18.113867 15932 consensus_queue.cc:260] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:18.115036 15932 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:18.121349 16003 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:18.121376 15932 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:18.122994 16535 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:18.124169 17447 raft_consensus.cc:2804] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:18.124634 17447 raft_consensus.cc:697] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:18.125463 17447 consensus_queue.cc:237] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:18.139109 15759 catalog_manager.cc:5697] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 1 to 2, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 2 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:18.413383 15932 raft_consensus.cc:1275] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:18.421599 17451 consensus_queue.cc:1048] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:18.437321 15857 raft_consensus.cc:1275] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:18.440092 17454 consensus_queue.cc:1048] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:18.484225 17458 raft_consensus.cc:993] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea: : Instructing follower 31882606e87447fb9a36bbf625129fe8 to start an election
I20260709 13:05:18.487924 17381 raft_consensus.cc:1081] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Signalling peer 31882606e87447fb9a36bbf625129fe8 to start an election
I20260709 13:05:18.501571 15857 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:43520
I20260709 13:05:18.504348 15857 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:18.504957 15857 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:18.517469 15857 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:18.520134 15857 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:18.520488 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: true dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:18.520890 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: true dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:18.521502 16567 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:18.521310 15935 raft_consensus.cc:3055] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:18.523913 15935 raft_consensus.cc:740] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:18.524930 15935 consensus_queue.cc:260] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:18.526459 15935 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:18.533411 16567 raft_consensus.cc:2468] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5.
I20260709 13:05:18.535050 15827 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:18.536119 17422 raft_consensus.cc:2804] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:18.536624 17422 raft_consensus.cc:697] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:18.537173 15935 raft_consensus.cc:2468] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5.
I20260709 13:05:18.537484 17422 consensus_queue.cc:237] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:18.547935 15759 catalog_manager.cc:5697] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 4 to 5, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 5 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:18.611174 16003 raft_consensus.cc:1275] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:18.612534 17459 consensus_queue.cc:1048] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:18.623198 15932 raft_consensus.cc:1275] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:18.624866 17459 consensus_queue.cc:1048] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:18.948877 15932 raft_consensus.cc:1275] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:05:18.950237 17368 consensus_queue.cc:1048] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:05:18.968241 16567 raft_consensus.cc:1275] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:05:18.971851 17368 consensus_queue.cc:1048] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:18.985642 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:19.076294 15857 consensus_queue.cc:237] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:19.084244 15935 raft_consensus.cc:1275] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:05:19.084546 15827 consensus_peers.cc:597] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 679229ffe7114facacb8e3a2c4c6d226. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:19.085741 17479 consensus_queue.cc:1048] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:19.086794 16011 raft_consensus.cc:1275] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:19.087924 17422 consensus_queue.cc:1048] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:19.095950 17479 raft_consensus.cc:2955] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:19.097437 16011 raft_consensus.cc:2955] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:19.112239 16003 consensus_queue.cc:237] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:19.118794 15935 raft_consensus.cc:2955] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:19.119381 15757 catalog_manager.cc:5697] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 1 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:19.123258 15976 consensus_peers.cc:597] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 15301761b2ec47fe82ae2f431c8b34ee. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:19.124774 15931 raft_consensus.cc:1275] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:19.126075 17451 consensus_queue.cc:1048] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:19.125348 15857 raft_consensus.cc:1275] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:05:19.135666 17451 consensus_queue.cc:1048] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:19.138530 17425 raft_consensus.cc:2955] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:19.140049 15931 raft_consensus.cc:2955] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:19.161394 15931 consensus_queue.cc:237] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
W20260709 13:05:19.176376 15902 consensus_peers.cc:597] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dfeaf1f6660d48848ae69710739427a0. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:19.176431 15855 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:05:19.176817 16003 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:05:19.171454 15857 raft_consensus.cc:2955] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:19.178845 17372 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:19.180876 17372 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:19.181155 15757 catalog_manager.cc:5697] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: config changed from index -1 to 2, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 1 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:19.209277 17381 raft_consensus.cc:2955] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:19.219708 16011 raft_consensus.cc:2955] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:19.219708 15855 raft_consensus.cc:2955] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:19.233485 15757 catalog_manager.cc:5697] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: config changed from index -1 to 3, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 2 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:19.237470 15931 consensus_queue.cc:237] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
W20260709 13:05:19.252149 15902 consensus_peers.cc:597] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 83a1b79cdcf046c78453695c798a14a5. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:19.255904 16007 raft_consensus.cc:1275] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:19.257556 17497 consensus_queue.cc:1048] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:19.258659 15855 raft_consensus.cc:1275] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:19.260172 17497 consensus_queue.cc:1048] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:19.266002 17372 raft_consensus.cc:2955] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:19.270496 16007 raft_consensus.cc:2955] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:19.272658 15855 raft_consensus.cc:2955] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:05:19.297108 15756 catalog_manager.cc:5697] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: config changed from index -1 to 3, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 3 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:19.573032 17501 ts_tablet_manager.cc:933] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
I20260709 13:05:19.575940 17501 tablet_copy_client.cc:323] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.1:38033
I20260709 13:05:19.577952 15872 tablet_copy_service.cc:140] P 31882606e87447fb9a36bbf625129fe8: Received BeginTabletCopySession request for tablet 679229ffe7114facacb8e3a2c4c6d226 from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:58214)
I20260709 13:05:19.578517 15872 tablet_copy_service.cc:161] P 31882606e87447fb9a36bbf625129fe8: Beginning new tablet copy session on tablet 679229ffe7114facacb8e3a2c4c6d226 from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:58214: session id = eb1e0493af1e46dbb6dec7e25525bab5-679229ffe7114facacb8e3a2c4c6d226
I20260709 13:05:19.589437 15872 tablet_copy_source_session.cc:215] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:19.592094 17503 ts_tablet_manager.cc:933] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:19.600999 17503 tablet_copy_client.cc:323] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.3:46251
I20260709 13:05:19.605405 16021 tablet_copy_service.cc:140] P 85e997ba5003439781ec4f1f8db0c3ee: Received BeginTabletCopySession request for tablet 15301761b2ec47fe82ae2f431c8b34ee from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:60156)
I20260709 13:05:19.605935 16021 tablet_copy_service.cc:161] P 85e997ba5003439781ec4f1f8db0c3ee: Beginning new tablet copy session on tablet 15301761b2ec47fe82ae2f431c8b34ee from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:60156: session id = eb1e0493af1e46dbb6dec7e25525bab5-15301761b2ec47fe82ae2f431c8b34ee
I20260709 13:05:19.606958 17501 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 679229ffe7114facacb8e3a2c4c6d226. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:19.617318 16021 tablet_copy_source_session.cc:215] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:19.625280 17503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15301761b2ec47fe82ae2f431c8b34ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:19.636850 17506 ts_tablet_manager.cc:933] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:19.641983 17501 tablet_copy_client.cc:806] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:19.642992 17501 tablet_copy_client.cc:670] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:19.644106 17506 tablet_copy_client.cc:323] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.2:43009
I20260709 13:05:19.645824 17503 tablet_copy_client.cc:806] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:19.646409 17503 tablet_copy_client.cc:670] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:19.647639 15946 tablet_copy_service.cc:140] P 8430db80a39d4868bcf7641c16de35ea: Received BeginTabletCopySession request for tablet dfeaf1f6660d48848ae69710739427a0 from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:53636)
I20260709 13:05:19.648255 15946 tablet_copy_service.cc:161] P 8430db80a39d4868bcf7641c16de35ea: Beginning new tablet copy session on tablet dfeaf1f6660d48848ae69710739427a0 from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:53636: session id = eb1e0493af1e46dbb6dec7e25525bab5-dfeaf1f6660d48848ae69710739427a0
I20260709 13:05:19.650491 17503 tablet_copy_client.cc:538] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:19.657426 15946 tablet_copy_source_session.cc:215] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:19.659746 17501 tablet_copy_client.cc:538] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:19.660256 17503 tablet_bootstrap.cc:492] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:05:19.666107 17506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfeaf1f6660d48848ae69710739427a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:19.696101 17503 tablet_bootstrap.cc:492] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:19.696689 17501 tablet_bootstrap.cc:492] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:05:19.696825 17509 ts_tablet_manager.cc:933] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:19.697155 17503 tablet_bootstrap.cc:492] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:05:19.697940 17503 ts_tablet_manager.cc:1403] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.038s	user 0.024s	sys 0.005s
I20260709 13:05:19.701836 17503 raft_consensus.cc:359] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:19.702876 17503 raft_consensus.cc:740] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:05:19.781805 17503 consensus_queue.cc:260] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:19.788801 17503 ts_tablet_manager.cc:1434] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.090s	user 0.028s	sys 0.041s
I20260709 13:05:19.795485 16021 tablet_copy_service.cc:342] P 85e997ba5003439781ec4f1f8db0c3ee: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-15301761b2ec47fe82ae2f431c8b34ee received from {username='slave'} at 127.0.0.1:60156
I20260709 13:05:19.796066 16021 tablet_copy_service.cc:434] P 85e997ba5003439781ec4f1f8db0c3ee: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-15301761b2ec47fe82ae2f431c8b34ee on tablet 15301761b2ec47fe82ae2f431c8b34ee with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:19.814096 17506 tablet_copy_client.cc:806] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:19.816080 17509 tablet_copy_client.cc:323] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.2:43009
I20260709 13:05:19.819665 15946 tablet_copy_service.cc:140] P 8430db80a39d4868bcf7641c16de35ea: Received BeginTabletCopySession request for tablet 83a1b79cdcf046c78453695c798a14a5 from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:53636)
I20260709 13:05:19.820241 15946 tablet_copy_service.cc:161] P 8430db80a39d4868bcf7641c16de35ea: Beginning new tablet copy session on tablet 83a1b79cdcf046c78453695c798a14a5 from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:53636: session id = eb1e0493af1e46dbb6dec7e25525bab5-83a1b79cdcf046c78453695c798a14a5
I20260709 13:05:19.816166 17506 tablet_copy_client.cc:670] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:19.828739 15946 tablet_copy_source_session.cc:215] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:19.838425 17506 tablet_copy_client.cc:538] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:19.838442 17509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83a1b79cdcf046c78453695c798a14a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:19.851092 17506 tablet_bootstrap.cc:492] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:05:19.905457 17509 tablet_copy_client.cc:806] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:19.907058 17509 tablet_copy_client.cc:670] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
W20260709 13:05:20.059895 17501 log.cc:927] Time spent T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.137s	user 0.004s	sys 0.000s
I20260709 13:05:20.291332 17506 tablet_bootstrap.cc:492] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:20.292730 17506 tablet_bootstrap.cc:492] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:05:20.293612 17506 ts_tablet_manager.cc:1403] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.443s	user 0.020s	sys 0.013s
I20260709 13:05:20.297554 17506 raft_consensus.cc:359] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:20.298650 17501 tablet_bootstrap.cc:492] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:20.578699 17501 tablet_bootstrap.cc:492] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:05:20.298713 17506 raft_consensus.cc:740] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:05:20.584924 17501 ts_tablet_manager.cc:1403] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.888s	user 0.019s	sys 0.012s
I20260709 13:05:20.584714 17506 consensus_queue.cc:260] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:20.587551 17506 ts_tablet_manager.cc:1434] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.294s	user 0.008s	sys 0.000s
I20260709 13:05:20.589414 17501 raft_consensus.cc:359] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:20.590299 17501 raft_consensus.cc:740] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:05:20.590951 17501 consensus_queue.cc:260] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:20.594949 17501 ts_tablet_manager.cc:1434] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 13:05:20.989498 17521 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:20.990311 17521 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:20.991865 17522 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:20.992551 17522 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:20.995115 17522 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:20.996825 17521 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:21.128438 17517 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:21.129204 17517 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.136451 17517 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:21.229583 17517 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:21.230195 17517 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:21.257861 17521 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:21.258625 17521 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:21.261339 17521 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:21.283061 17517 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:21.280846 17525 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:21.283759 17525 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.286919 17523 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:05:21.287803 15976 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.059s	user 0.013s	sys 0.010s
I20260709 13:05:21.287287 17525 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:21.287864 17523 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:21.289001 15976 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.066s	user 0.013s	sys 0.010s
I20260709 13:05:21.290462 17523 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:21.291028 17451 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:21.292382 17526 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:21.292938 17526 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.294930 17526 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:21.291512 17451 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:21.297821 17521 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:21.298648 17451 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:21.298409 17521 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:21.300382 17517 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:21.300449 17521 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:21.301036 17517 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:21.302323 17528 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:21.302563 17527 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:21.302651 17451 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:21.302866 17528 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.303131 17527 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.303320 17451 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:21.305687 17527 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:21.296736 17525 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:21.306679 17522 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:21.307358 17526 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:21.307255 17522 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.306353 17525 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.309540 17522 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:21.305871 17451 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:21.311357 17525 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:21.308483 17526 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.313910 17522 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:21.303355 17517 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:21.311895 17528 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:21.311872 17523 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:21.313467 17521 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:21.314224 17526 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:21.317912 17526 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:21.315860 17520 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:21.318605 17526 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.319206 17520 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.321755 17526 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:21.314917 17523 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.322409 17526 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:21.323637 17526 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.314438 17522 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.324700 17531 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:21.325299 17531 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:21.322431 17520 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:21.317186 17525 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:21.327466 17531 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:21.327375 17525 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.314978 17521 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.333669 17532 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:21.333935 17521 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:21.334242 17532 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:05:21.329459 17525 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:21.336498 17525 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:21.337049 17525 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:21.341277 15977 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.078s	user 0.015s	sys 0.005s
I20260709 13:05:21.316593 17528 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:05:21.341754 15977 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.079s	user 0.015s	sys 0.005s
I20260709 13:05:21.341997 17528 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.344237 17528 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:21.344925 17528 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:21.346493 17528 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.349677 17528 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:21.337359 17527 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:21.350970 17528 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:21.351011 17527 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.324215 17523 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:21.345963 17525 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:21.326566 17522 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:21.351632 17528 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.353296 17527 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:21.337819 17531 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:21.326781 17526 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:21.353940 17531 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.354799 17528 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:21.350327 17520 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:21.355695 17520 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.356655 17521 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:21.357188 17521 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.359051 17521 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:21.317063 17517 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:21.356208 17531 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:21.359637 17517 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:21.361137 17532 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:21.364331 17520 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:21.434165 17528 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:21.434930 17528 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.438138 17528 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:21.438690 17524 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:21.439491 17524 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:05:21.445595 17524 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
I20260709 13:05:21.447806 17524 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:21.362316 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:21.449198 17524 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:21.448390 17528 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:21.450037 17526 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:21.462422 17526 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.464525 17526 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:21.465034 17526 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:21.465494 17526 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.468065 17528 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.470161 17538 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:21.470569 17528 leader_election.cc:290] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:21.467520 17526 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:21.471814 17544 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:21.472321 17544 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.474509 17524 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:21.455689 17523 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:21.481043 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:21.459354 17543 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:21.483084 16569 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 1.
I20260709 13:05:21.461505 17529 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:21.475981 16567 raft_consensus.cc:1217] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:21.477119 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:21.478880 17526 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:21.479308 17528 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:21.479606 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:21.481122 17523 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.456269 17525 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:21.483335 17520 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:21.476332 17544 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:21.361681 17517 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:21.454340 17541 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:21.470872 17538 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:21.499854 17518 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:21.500772 17530 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:21.466349 17539 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:21.483147 17543 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:21.498090 17526 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.501577 17531 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:21.498098 17528 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.552467 17544 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:21.548571 17541 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:21.592499 17542 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:21.546449 17520 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.603530 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:21.615332 15855 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:21.606415 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:21.504655 17525 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.617750 16569 raft_consensus.cc:1217] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:05:21.617769 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:21.589924 17517 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:21.676365 16570 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:05:21.607662 15872 tablet_copy_service.cc:342] P 31882606e87447fb9a36bbf625129fe8: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-679229ffe7114facacb8e3a2c4c6d226 received from {username='slave'} at 127.0.0.1:58214
I20260709 13:05:21.709096 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:21.709695 15855 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
I20260709 13:05:21.711169 17550 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:21.715513 17543 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:21.583555 17518 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.606588 17531 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:21.729020 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:21.730652 17538 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:21.501703 17529 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.728134 15872 tablet_copy_service.cc:434] P 31882606e87447fb9a36bbf625129fe8: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-679229ffe7114facacb8e3a2c4c6d226 on tablet 679229ffe7114facacb8e3a2c4c6d226 with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:21.730939 17540 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:21.615020 17544 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.735354 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:21.718027 17547 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:21.720465 17535 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:05:21.659900 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 107827us
I20260709 13:05:21.582482 17539 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:21.722471 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:21.568845 17530 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:21.724463 17543 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:21.765149 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:05:21.743224 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 46494.6us
I20260709 13:05:21.742312 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:21.708479 17517 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:21.711957 17550 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:21.721261 17519 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:21.749750 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:21.789084 15976 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:21.750989 17546 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:21.755924 17545 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:21.799623 17530 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:21.800175 17530 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:21.800591 17530 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.802505 17530 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:21.802992 17530 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:21.803535 17530 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.804157 17532 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:21.804723 17532 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.805590 17530 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:21.806083 17530 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:21.806689 17532 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:21.806485 17530 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.808586 17530 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:21.731732 17540 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.796453 17546 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:21.797633 17545 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.768343 17523 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:21.816761 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:21.819093 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:21.756683 16565 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:21.822090 17554 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:21.822073 17537 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:21.756762 17547 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.809345 17532 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:21.831045 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:21.705650 17542 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:21.750749 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:21.869899 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:21.757000 17535 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.816443 17530 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:21.756170 17536 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:21.874347 17548 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:21.876922 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:21.877631 17532 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.815636 17543 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.851372 17554 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:21.903602 16570 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:21.973173 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:21.974608 17557 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:05:21.976991 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 233064us
I20260709 13:05:21.993551 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:21.893877 17530 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:21.963989 17554 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:21.954186 17516 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:21.976747 17520 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:21.978823 17528 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:21.872241 17519 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:21.975419 17557 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:21.904605 17537 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:21.974238 16561 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
W20260709 13:05:22.137092 15977 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.017s	sys 0.064s
I20260709 13:05:22.046391 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:22.281919 15977 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.749s	user 0.018s	sys 0.073s
I20260709 13:05:22.047765 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:22.277715 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:22.046185 17552 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:22.279695 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:22.135814 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:22.135855 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:22.283732 16003 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
W20260709 13:05:22.296160 15903 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.010s	sys 0.029s
I20260709 13:05:21.980304 17548 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:22.029001 17531 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:05:22.428412 15903 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.012s	sys 0.035s
I20260709 13:05:22.322832 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:22.425175 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:22.429637 15935 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 1.
I20260709 13:05:22.378623 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:22.393556 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:22.381256 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:22.430739 15931 raft_consensus.cc:2468] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:05:22.434427 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:22.437151 15902 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.729s	user 0.001s	sys 0.018s
W20260709 13:05:22.437604 15902 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.001s	sys 0.018s
I20260709 13:05:22.411867 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:22.430557 15857 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 1.
I20260709 13:05:22.439803 16004 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319"
dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
mode: GRACEFUL
new_leader_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:37262
I20260709 13:05:22.440043 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:22.440552 16004 raft_consensus.cc:606] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Received request to transfer leadership to TS 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:22.440677 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:22.439931 17559 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:22.441789 17559 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:22.442137 17525 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:22.379554 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:22.322283 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:22.324734 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:22.435743 16003 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:22.411613 15862 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 1.
I20260709 13:05:22.382262 17539 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:22.092201 17541 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:22.300951 15946 tablet_copy_service.cc:342] P 8430db80a39d4868bcf7641c16de35ea: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-dfeaf1f6660d48848ae69710739427a0 received from {username='slave'} at 127.0.0.1:53636
I20260709 13:05:22.136323 17544 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:22.414616 17517 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:22.443544 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:22.455953 15932 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 1.
I20260709 13:05:22.322845 17553 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:22.383105 17451 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:22.017645 17536 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:22.462384 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:22.322386 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:22.448052 15856 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:22.457258 17532 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:22.457803 17550 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:22.463129 17451 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:22.449505 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:22.443660 17547 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:22.414849 17518 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:22.383994 17540 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:22.447345 15858 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:05:22.472039 15855 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 1.
I20260709 13:05:22.046763 17555 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:22.465574 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:22.460915 17519 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:22.465570 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:22.374142 16591 debug-util.cc:398] Leaking SignalData structure 0x7b08002e3060 after lost signal to thread 14751
I20260709 13:05:22.466890 15860 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 1.
I20260709 13:05:22.467923 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:22.464015 15930 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 1.
I20260709 13:05:22.461347 15946 tablet_copy_service.cc:434] P 8430db80a39d4868bcf7641c16de35ea: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-dfeaf1f6660d48848ae69710739427a0 on tablet dfeaf1f6660d48848ae69710739427a0 with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:22.463995 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:22.457270 17553 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:22.477434 16007 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 1.
I20260709 13:05:22.470424 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:22.324156 17543 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:22.483296 15975 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.003s	sys 0.014s
W20260709 13:05:22.483628 15975 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.003s	sys 0.015s
I20260709 13:05:22.378258 17554 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:22.462687 17539 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:22.468904 17530 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:22.463807 17542 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:22.481213 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:05:22.496022 15974 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.005s	sys 0.014s
I20260709 13:05:22.465710 17544 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:22.463598 17541 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:22.324785 17564 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:22.497371 15932 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 1.
W20260709 13:05:22.498198 15974 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.759s	user 0.006s	sys 0.015s
I20260709 13:05:22.447309 17552 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:22.502694 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:22.501511 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:22.488006 17555 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:22.501721 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:22.508112 15859 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 1.
I20260709 13:05:22.511401 16008 raft_consensus.cc:2393] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 2: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:22.503494 17544 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:22.514750 17536 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:22.517513 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:22.161724 17516 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:22.537088 16003 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 1.
I20260709 13:05:22.500077 17559 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:22.537907 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:22.499796 17539 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:22.535230 17570 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:22.543293 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:22.546197 16567 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:22.547626 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:22.548205 15930 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 1.
I20260709 13:05:22.549382 17548 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:22.550043 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:22.550544 17539 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:22.550634 16561 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:05:22.551373 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:22.556248 15859 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:05:22.556340 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:22.556881 15859 raft_consensus.cc:606] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Received request to transfer leadership to TS eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:22.556996 16569 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
W20260709 13:05:22.546612 16591 debug-util.cc:398] Leaking SignalData structure 0x7b08002e3740 after lost signal to thread 15794
I20260709 13:05:22.556425 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:22.558979 17555 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:22.559128 16007 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:22.544317 17559 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:22.559777 17559 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
W20260709 13:05:22.560904 16591 debug-util.cc:398] Leaking SignalData structure 0x7b0800378d00 after lost signal to thread 15886
I20260709 13:05:22.561198 17451 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:22.561748 17559 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:05:22.561995 16591 debug-util.cc:398] Leaking SignalData structure 0x7b0800356260 after lost signal to thread 15961
I20260709 13:05:22.507002 17541 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:22.562621 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:22.555399 15827 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:22.518642 17564 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:22.576253 17529 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:22.565793 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:22.577280 17545 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:22.564030 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:22.578823 16566 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:05:22.564735 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:22.552441 17554 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:22.583190 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:22.555217 17531 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:22.583982 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:22.569106 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:22.581941 17519 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:22.585064 17553 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:22.585680 15827 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:22.586303 17526 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:05:22.568665 17551 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.019s	sys 0.012s
I20260709 13:05:22.590190 17519 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:22.586091 17570 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:22.593039 17554 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:22.541165 17561 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:22.594621 17509 tablet_copy_client.cc:538] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:22.594780 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:22.595476 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:22.596793 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:22.602626 16567 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:05:22.595610 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:05:22.597592 15901 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.762s	user 0.017s	sys 0.030s
I20260709 13:05:22.564486 17557 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:22.603718 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:22.599611 17561 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:22.569478 17544 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:22.601382 17519 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:22.606716 15827 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:22.596719 17531 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:05:22.606237 17551 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.792s	user 0.020s	sys 0.013s
I20260709 13:05:22.583942 17537 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:22.611469 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:05:22.600761 16591 debug-util.cc:398] Leaking SignalData structure 0x7b0800365ea0 after lost signal to thread 16035
I20260709 13:05:22.618383 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:22.598351 17532 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:22.620395 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:22.612344 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:22.612797 15901 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.876s	user 0.017s	sys 0.031s
I20260709 13:05:22.589699 17451 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:22.614471 17531 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:22.614869 16533 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:22.615185 17564 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:22.612484 15902 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:22.618373 17518 raft_consensus.cc:993] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8: : Instructing follower eb1e0493af1e46dbb6dec7e25525bab5 to start an election
I20260709 13:05:22.610847 17546 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:22.617707 17535 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:22.623175 15827 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:22.602053 17554 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:22.626052 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:22.611402 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:22.625411 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:22.628960 16570 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:05:22.627218 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:22.604805 16565 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:05:22.631188 17546 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:22.631032 17451 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:22.631951 17524 raft_consensus.cc:2804] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:22.626410 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:22.632350 17524 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:22.632582 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:22.632786 17524 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:22.633127 17451 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:22.631713 17546 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:05:22.621214 16591 debug-util.cc:398] Leaking SignalData structure 0x7b08002dae20 after lost signal to thread 16036
W20260709 13:05:22.634287 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 44347.6us
I20260709 13:05:22.635131 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:22.630023 15827 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:22.636199 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:22.636858 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:22.632608 16009 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:05:22.637390 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:22.635596 17554 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:22.620817 17532 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:22.655010 15827 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:22.655404 16011 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:22.656332 16533 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:22.657223 16533 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:22.626574 17517 raft_consensus.cc:2804] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:22.660100 17533 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:22.659993 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:22.660473 17523 raft_consensus.cc:2804] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:22.660578 17529 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:22.661654 17557 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:22.680794 16533 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:22.664919 17561 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:22.692428 16533 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:22.691915 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:22.692368 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:22.717145 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:22.717408 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:22.718245 16009 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:05:22.635704 17560 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:22.660375 17520 raft_consensus.cc:2804] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:22.677892 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:22.635787 17535 raft_consensus.cc:2804] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:22.634161 17536 raft_consensus.cc:1081] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Signalling peer eb1e0493af1e46dbb6dec7e25525bab5 to start an election
I20260709 13:05:22.680689 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:22.663630 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:22.635198 17518 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:22.696877 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:22.692021 17557 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:22.734766 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:22.691109 17571 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:22.696277 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:22.690615 17528 raft_consensus.cc:2804] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:22.682235 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:22.676704 17533 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:22.738375 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:22.727159 16010 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2.
I20260709 13:05:22.741604 17533 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:22.658361 17532 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:22.718595 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:22.680365 17523 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:22.653266 17570 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:22.665496 17517 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:22.664228 15827 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:22.738348 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:22.680279 17529 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:22.739962 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:22.750826 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 86652.9us
I20260709 13:05:22.740775 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:22.741132 15900 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:05:22.752612 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 151102us
I20260709 13:05:22.741477 17541 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:22.726825 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:22.739120 16533 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:22.680090 16004 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:05:22.697973 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:22.693341 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:22.748667 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:22.749644 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:22.689062 17516 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:22.755759 17536 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:22.755697 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:22.756162 16006 raft_consensus.cc:1240] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:22.751453 15827 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:22.756793 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:22.726796 17535 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:22.726495 17520 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:22.735718 17526 raft_consensus.cc:2804] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:22.759588 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:22.747475 17523 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:22.737638 17528 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:22.770179 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:22.770576 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:05:22.745961 16591 debug-util.cc:398] Leaking SignalData structure 0x7b0800374600 after lost signal to thread 16594
I20260709 13:05:22.775863 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:22.773404 17549 raft_consensus.cc:2804] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:22.813812 15930 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
I20260709 13:05:22.736208 17518 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:22.759663 17529 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:22.814900 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:05:22.766058 17536 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:22.771059 16535 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:22.774644 17520 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:22.775527 17535 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:22.804301 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:22.804981 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:22.806911 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:22.828538 15931 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:05:22.757560 17517 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:22.815872 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:22.817581 16566 raft_consensus.cc:1217] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:05:22.818567 16561 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013"
dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:45246
I20260709 13:05:22.833155 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:22.821237 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:22.768831 17539 raft_consensus.cc:2804] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:22.787209 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:22.787817 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:22.778367 17526 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:22.839927 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:22.840116 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:22.817144 17528 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:22.848138 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:22.848608 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:22.848191 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:22.848131 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:22.834668 16591 debug-util.cc:398] Leaking SignalData structure 0x7b0800374740 after lost signal to thread 17501
I20260709 13:05:22.824133 17518 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:22.846200 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:22.835268 17536 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:22.826550 17537 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:22.778005 17552 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:22.760121 17560 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:22.853452 17525 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:22.862586 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:22.865825 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:22.840572 17539 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:22.832471 17549 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:22.853627 17526 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:22.864415 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:22.807390 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:22.878106 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:22.879770 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:22.878917 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:22.886997 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:22.890697 16008 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2.
W20260709 13:05:22.889285 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 130946us
I20260709 13:05:22.863752 17537 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:22.765837 17571 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:22.897979 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:22.876686 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:22.901196 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:22.876976 17557 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:22.901437 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:22.882972 17539 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:22.902369 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:22.888742 17549 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:22.911949 16533 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:22.913371 17577 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:22.913897 17577 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:22.914387 17577 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:22.915529 16533 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:22.916395 17564 raft_consensus.cc:993] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee: : Instructing follower 31882606e87447fb9a36bbf625129fe8 to start an election
I20260709 13:05:22.916934 17563 raft_consensus.cc:2804] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:22.917636 17563 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:22.875334 17525 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:22.918154 17563 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:22.918532 17525 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:22.904814 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:22.917016 17564 raft_consensus.cc:1081] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Signalling peer 31882606e87447fb9a36bbf625129fe8 to start an election
I20260709 13:05:22.904031 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:22.931212 16533 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:22.898402 17537 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:22.933002 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:22.938745 15935 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:05:22.939826 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:22.942636 16533 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:22.944161 16533 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:22.946951 17534 raft_consensus.cc:2749] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:22.948026 16533 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:22.948530 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:22.951280 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:22.951439 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:22.952319 15862 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 2.
I20260709 13:05:22.949280 16533 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:22.975523 17558 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:22.976251 17558 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:22.976775 17558 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:22.987351 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:22.986301 15974 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:22.998016 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:22.953828 17534 raft_consensus.cc:2804] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:23.000121 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:23.006738 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:23.014094 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:23.033074 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:22.981796 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:23.033336 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:22.976189 17560 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:23.040552 15860 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:43524
I20260709 13:05:22.997771 17554 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:23.048672 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:23.006860 17534 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:23.034399 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:23.041235 15860 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:23.053696 15860 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:23.046348 15900 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:23.022248 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:23.020082 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:23.037197 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:23.060443 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:23.038750 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:23.055783 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:23.036360 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:05:23.092077 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 45841.4us
I20260709 13:05:23.093518 15900 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:23.069527 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:23.089929 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:05:23.095961 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 97347.5us
I20260709 13:05:23.057127 17534 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:23.103534 15974 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:23.108975 15900 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:23.119059 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:05:23.119558 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 170302us
W20260709 13:05:23.120394 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 349383us
I20260709 13:05:23.120659 16533 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:23.121306 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:23.091447 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:23.110260 17541 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:23.115274 17566 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:23.149732 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:23.122181 17561 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:23.159368 15900 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:23.133724 17540 raft_consensus.cc:2749] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:23.244411 17509 tablet_bootstrap.cc:492] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:05:23.246541 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:23.130774 17533 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:23.265986 17533 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:23.266639 17533 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:23.268043 17579 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:23.268639 17579 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:23.269121 17579 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:23.268662 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 284697us
I20260709 13:05:23.192970 17578 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:23.190834 17561 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:23.237337 17541 raft_consensus.cc:2749] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:23.281872 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:23.236300 17580 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:23.282226 17561 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:23.328313 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:23.304018 17580 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:05:23.341621 17519 consensus_meta.cc:287] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.731s	user 0.014s	sys 0.008s
I20260709 13:05:23.338366 17517 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:23.342338 17519 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:23.343014 15858 raft_consensus.cc:1240] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:23.345408 17517 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:23.267360 17533 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:23.352825 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 193488us
I20260709 13:05:23.313310 17571 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:23.362093 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:23.281925 17578 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:23.392807 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:23.353515 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:23.353965 17580 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:23.341789 17549 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:23.404105 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 283530us
I20260709 13:05:23.408429 16533 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:23.415776 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:23.420413 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:05:23.424952 15827 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 1.565s (SENT)
I20260709 13:05:23.425279 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:23.426457 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:23.427524 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:23.428561 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:05:23.405540 17520 consensus_meta.cc:287] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.559s	user 0.017s	sys 0.012s
I20260709 13:05:23.397292 17574 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:23.410358 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:23.434852 15825 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:23.452582 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:23.453172 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:23.431238 16561 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:23.480247 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:05:23.482649 17518 consensus_meta.cc:287] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.607s	user 0.005s	sys 0.014s
I20260709 13:05:23.487545 15974 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:23.491339 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:23.484282 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:23.498759 15974 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:23.500509 15974 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:23.454885 17590 raft_consensus.cc:2804] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:23.505133 17560 raft_consensus.cc:2749] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:23.505877 17552 raft_consensus.cc:2749] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:23.494490 15902 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:23.506371 17591 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:23.505157 17590 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:23.506868 17591 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:23.507051 15902 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:23.507323 17591 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:23.507970 17557 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:23.509660 16535 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:23.508363 17551 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:23.467269 16561 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:23.443583 17520 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:23.517927 17557 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:23.522559 15974 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:23.515081 17551 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:05:23.519218 17524 consensus_meta.cc:287] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.886s	user 0.010s	sys 0.005s
I20260709 13:05:23.519517 17549 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:23.497184 17517 raft_consensus.cc:2749] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:23.453294 17574 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:23.530611 15853 raft_consensus.cc:1240] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:23.537875 17524 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:23.554033 17573 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:23.554280 17524 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:23.557394 17520 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:23.559194 15933 raft_consensus.cc:1240] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:23.507098 17590 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:23.524912 17557 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:23.527891 17551 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:23.524036 17560 raft_consensus.cc:2749] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:23.496042 17595 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:23.539182 17519 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:23.493238 17518 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:23.573508 17549 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:23.616596 17533 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:23.619210 17549 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:05:23.641350 17535 consensus_meta.cc:287] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.791s	user 0.009s	sys 0.030s
W20260709 13:05:23.647444 17558 consensus_meta.cc:287] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.670s	user 0.011s	sys 0.013s
W20260709 13:05:23.666579 17525 consensus_meta.cc:287] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.747s	user 0.009s	sys 0.001s
W20260709 13:05:23.667733 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 145231us
I20260709 13:05:23.678741 15974 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:23.638257 17578 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:23.649134 17526 consensus_meta.cc:287] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.751s	user 0.013s	sys 0.000s
I20260709 13:05:23.647135 17549 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:23.672302 17574 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:23.613493 17595 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:23.672572 17546 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:23.681483 15974 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:05:23.693802 17523 consensus_meta.cc:287] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.863s	user 0.012s	sys 0.006s
I20260709 13:05:23.698822 15974 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:05:23.692512 17563 consensus_meta.cc:287] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.774s	user 0.009s	sys 0.017s
I20260709 13:05:23.723726 15974 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:23.670640 17535 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:23.726508 15974 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:23.721681 17574 raft_consensus.cc:2749] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:05:23.698052 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 289715us
W20260709 13:05:23.700440 17529 consensus_meta.cc:287] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.851s	user 0.017s	sys 0.008s
I20260709 13:05:23.738444 17529 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:23.739521 16563 raft_consensus.cc:1240] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:23.740841 17597 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:23.741271 17529 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:23.741391 17597 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:23.697033 17605 raft_consensus.cc:2749] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:23.743495 16533 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:23.725824 17516 raft_consensus.cc:2749] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:23.695750 17526 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:23.725786 17560 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:23.736541 17574 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:23.748021 16533 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:23.736994 17523 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:23.682724 17525 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:23.747968 17528 consensus_meta.cc:287] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.887s	user 0.012s	sys 0.023s
I20260709 13:05:23.752902 16568 raft_consensus.cc:1240] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:23.677229 17558 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:23.748225 17539 consensus_meta.cc:287] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.813s	user 0.011s	sys 0.007s
W20260709 13:05:23.759816 17537 consensus_meta.cc:287] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.781s	user 0.006s	sys 0.010s
I20260709 13:05:23.760067 17539 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:05:23.760219 17537 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:23.761963 17537 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:23.762053 17539 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
I20260709 13:05:23.755684 16564 raft_consensus.cc:1240] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:23.755214 17523 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:23.758121 17528 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:23.763852 16562 raft_consensus.cc:1240] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:23.764389 17528 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:23.767199 17558 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:23.726383 17563 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:23.761782 17551 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:23.743487 17597 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:23.775030 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 265398us
I20260709 13:05:23.754163 17526 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:23.764775 17535 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:23.757023 17518 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:23.785965 16535 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
W20260709 13:05:23.777956 17536 consensus_meta.cc:287] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.905s	user 0.020s	sys 0.005s
I20260709 13:05:23.787242 16565 raft_consensus.cc:1240] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:23.832803 17584 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:23.833386 17584 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:23.833797 17584 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:23.876044 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 90182.1us
I20260709 13:05:23.878573 16535 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
W20260709 13:05:23.936295 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 198832us
I20260709 13:05:23.937618 15974 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:23.940480 15974 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:05:23.941956 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 63470.8us
I20260709 13:05:23.942257 15974 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:23.943516 15974 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:23.943408 16535 leader_election.cc:304] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:23.783416 17539 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:23.934268 17528 raft_consensus.cc:2804] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:23.977084 16535 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:23.942281 17554 raft_consensus.cc:2749] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:23.970927 17572 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:23.790320 17555 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:23.971657 17529 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:23.979480 16535 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:23.979086 17528 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:23.981213 17582 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:23.981528 17603 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:23.990658 17601 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:23.991588 17549 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:23.972086 17566 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:23.983137 17535 raft_consensus.cc:2804] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:23.995731 17554 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:24.047726 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:24.048393 17549 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:23.997260 17539 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:23.935779 17536 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:23.974493 17605 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:24.024397 17535 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:23.980538 17591 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:24.076012 17535 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:24.078173 17534 consensus_meta.cc:287] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.956s	user 0.006s	sys 0.009s
I20260709 13:05:24.010532 17529 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:23.974694 17595 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:24.010361 17580 consensus_meta.cc:287] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.596s	user 0.004s	sys 0.008s
I20260709 13:05:24.024000 17566 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:24.048090 17554 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:23.994652 16561 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:24.013950 17528 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:24.073035 17604 raft_consensus.cc:2749] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:05:24.073271 17532 consensus_meta.cc:287] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 1.316s	user 0.005s	sys 0.013s
I20260709 13:05:24.664400 17532 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:24.667084 17532 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:24.074266 17536 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:24.081725 17561 consensus_meta.cc:287] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.754s	user 0.006s	sys 0.005s
I20260709 13:05:24.668190 17561 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:24.670783 17561 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:23.978344 17560 raft_consensus.cc:2749] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:24.017961 17555 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:24.673492 17555 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:24.113791 17539 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:24.115043 17525 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:24.022158 17585 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:24.071486 17603 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:24.016917 17572 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:24.078676 17534 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:24.125358 17563 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:24.146173 17529 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:24.122301 17590 consensus_meta.cc:287] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.540s	user 0.003s	sys 0.005s
W20260709 13:05:24.584664 17557 consensus_meta.cc:287] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 1.003s	user 0.010s	sys 0.000s
W20260709 13:05:24.124478 17577 consensus_meta.cc:287] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 1.210s	user 0.010s	sys 0.013s
W20260709 13:05:24.585039 17584 consensus_meta.cc:287] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.751s	user 0.006s	sys 0.004s
I20260709 13:05:24.718340 17566 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:24.593619 17580 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:24.688860 17582 consensus_meta.cc:287] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.664s	user 0.010s	sys 0.000s
I20260709 13:05:24.762604 17551 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:24.733122 17590 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:24.781003 17590 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:24.734308 17557 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:24.734186 17577 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:24.761292 17584 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:24.648324 17535 consensus_meta.cc:287] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.564s	user 0.004s	sys 0.004s
I20260709 13:05:24.778637 17582 raft_consensus.cc:2749] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:24.795424 17535 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:24.800773 16570 raft_consensus.cc:1240] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:24.824333 17557 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:24.815789 17603 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:24.817659 17554 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:24.836589 15975 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.089s	user 0.077s	sys 0.119s
W20260709 13:05:24.836953 15975 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.099s	user 0.077s	sys 0.119s
I20260709 13:05:24.831176 17534 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:24.847153 15974 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.877s	user 0.027s	sys 0.085s
W20260709 13:05:24.847607 15974 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.878s	user 0.027s	sys 0.085s
W20260709 13:05:24.848740 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 905183us
I20260709 13:05:24.848865 17604 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:24.849076 17560 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:24.849432 15974 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:24.849684 17580 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:24.849627 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:24.850019 17561 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:24.850421 17605 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:24.850514 17561 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:24.849438 17604 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:24.852546 17561 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:24.852958 17563 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:24.853449 17604 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:24.853459 17563 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:24.849557 17560 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:24.855290 17605 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:24.855607 17563 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:24.855789 17605 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:24.856353 17560 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:24.847198 17554 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:24.857255 17554 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:24.857776 17605 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:24.858239 15853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 1606) took 1425 ms. Trace:
I20260709 13:05:24.858891 17560 consensus_queue.cc:1059] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:24.859215 17554 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:24.859845 17554 raft_consensus.cc:3055] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:24.860188 17554 raft_consensus.cc:740] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:24.861418 17584 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:24.861124 17554 consensus_queue.cc:260] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:24.861243 17529 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:24.862509 17554 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:24.862619 17577 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:24.863824 17579 consensus_meta.cc:287] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 1.594s	user 0.009s	sys 0.006s
I20260709 13:05:24.863859 17529 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:24.864393 17579 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:24.865470 16569 raft_consensus.cc:1240] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:24.858589 15853 rpcz_store.cc:276] 0709 13:05:23.433043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:23.443298 (+ 10255us) service_pool.cc:224] Handling call
0709 13:05:23.473379 (+ 30081us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:23.542962 (+ 69583us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:24.858030 (+1315068us) spinlock_profiling.cc:243] Waited 57 ms on lock 0x7b54004268d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:24.858174 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":131101440}
I20260709 13:05:24.874543 17528 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:24.878417 17572 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:24.880146 17535 leader_election.cc:290] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:24.880872 16564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 486) took 1429 ms. Trace:
I20260709 13:05:24.881315 16564 rpcz_store.cc:276] 0709 13:05:23.451883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:23.452032 (+   149us) service_pool.cc:224] Handling call
0709 13:05:23.452406 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:23.762713 (+310307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:24.880578 (+1117865us) spinlock_profiling.cc:243] Waited 301 ms on lock 0x7b54007454d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:24.880789 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":691563904}
I20260709 13:05:24.880897 17579 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:24.886638 17528 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:24.892946 17509 tablet_bootstrap.cc:492] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:24.893312 16566 raft_consensus.cc:1240] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:24.894292 16566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 583) took 1392 ms. Trace:
I20260709 13:05:24.894516 17509 tablet_bootstrap.cc:492] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:05:24.894691 16566 rpcz_store.cc:276] 0709 13:05:23.502122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:24.892775 (+1390653us) service_pool.cc:224] Handling call
0709 13:05:24.893104 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:24.894037 (+   933us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:24.894233 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:24.897301 17509 ts_tablet_manager.cc:1403] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 1.653s	user 0.032s	sys 0.001s
I20260709 13:05:24.902266 17509 raft_consensus.cc:359] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:24.902973 17509 raft_consensus.cc:740] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:05:24.903437 17509 consensus_queue.cc:260] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:24.905253 17509 ts_tablet_manager.cc:1434] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:05:24.915805 16562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 420) took 2095 ms. Trace:
I20260709 13:05:24.916693 16562 rpcz_store.cc:276] 0709 13:05:22.820371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.823234 (+  2863us) service_pool.cc:224] Handling call
0709 13:05:22.871556 (+ 48322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:23.764239 (+892683us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:24.915492 (+1151253us) spinlock_profiling.cc:243] Waited 889 ms on lock 0x7b5400714b54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:24.915734 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2044091136}
I20260709 13:05:24.923373 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:24.923970 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:24.924641 15853 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 2.
I20260709 13:05:24.925915 17609 consensus_queue.cc:1059] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:24.926491 15974 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:24.927357 17609 raft_consensus.cc:3055] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:24.927733 17609 raft_consensus.cc:740] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:24.924043 16563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 416) took 2172 ms. Trace:
I20260709 13:05:24.928297 16563 rpcz_store.cc:276] 0709 13:05:22.752089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.756183 (+  4094us) service_pool.cc:224] Handling call
0709 13:05:22.756476 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:23.739912 (+983436us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:24.923759 (+1183847us) spinlock_profiling.cc:243] Waited 983 ms on lock 0x7b54007157d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:24.923959 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2260297472}
I20260709 13:05:24.928668 17609 consensus_queue.cc:260] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:24.929924 17609 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:24.927739 17554 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:24.932364 17554 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:24.932718 17554 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:24.933904 15827 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.508s	user 0.031s	sys 0.068s
W20260709 13:05:24.934419 15827 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.509s	user 0.031s	sys 0.068s
W20260709 13:05:24.934702 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 1.5096e+06us
I20260709 13:05:24.940119 17536 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:24.939657 17554 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:24.940675 17536 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:24.942013 17554 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:24.942631 17536 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:24.942988 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:24.943645 15858 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:24.948019 16568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 428) took 2043 ms. Trace:
I20260709 13:05:24.948429 16568 rpcz_store.cc:276] 0709 13:05:22.904466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.920567 (+ 16101us) service_pool.cc:224] Handling call
0709 13:05:22.920943 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:23.757785 (+836842us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:24.947787 (+1190002us) spinlock_profiling.cc:243] Waited 832 ms on lock 0x7b54005523d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:24.947964 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1912819584}
I20260709 13:05:24.951064 15858 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:24.952138 15974 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:24.953068 17554 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:24.953771 17554 raft_consensus.cc:697] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:05:24.954115 15828 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.562s	user 0.013s	sys 0.044s
W20260709 13:05:24.954586 15828 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.569s	user 0.013s	sys 0.044s
I20260709 13:05:24.954876 17554 consensus_queue.cc:237] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:24.957727 16565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 585) took 1454 ms. Trace:
I20260709 13:05:24.958305 16565 rpcz_store.cc:276] 0709 13:05:23.502883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:23.507212 (+  4329us) service_pool.cc:224] Handling call
0709 13:05:23.507535 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:23.832975 (+325440us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:24.957411 (+1124436us) spinlock_profiling.cc:243] Waited 275 ms on lock 0x7b54005744d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:24.957650 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":632710400}
I20260709 13:05:24.964694 17536 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:24.965353 17536 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:24.967505 17536 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:24.968448 16569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 586) took 1465 ms. Trace:
I20260709 13:05:24.968878 16569 rpcz_store.cc:276] 0709 13:05:23.503237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:23.506774 (+  3537us) service_pool.cc:224] Handling call
0709 13:05:23.507075 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:24.865835 (+1358760us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:24.968185 (+102350us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54002ff154. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:24.968386 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3123501440}
I20260709 13:05:24.969801 15827 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:24.972035 17532 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:24.974100 15827 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:24.975132 17532 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:24.975504 17532 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:24.975885 17532 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:24.981428 16568 raft_consensus.cc:1240] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:24.984351 16565 raft_consensus.cc:1240] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:24.985496 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:24.986825 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:24.988303 16563 raft_consensus.cc:1240] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 1. Current term is 2. Ops: []
W20260709 13:05:24.989159 15860 consensus_meta.cc:287] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 1.932s	user 0.027s	sys 0.014s
I20260709 13:05:24.989398 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:24.989554 15860 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:24.990935 16562 raft_consensus.cc:1240] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:24.992249 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:24.994037 16564 raft_consensus.cc:1240] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:24.995142 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:05:24.995607 15825 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.560s	user 0.019s	sys 0.103s
W20260709 13:05:25.000072 15825 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.564s	user 0.020s	sys 0.103s
W20260709 13:05:25.000599 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 1.56565e+06us
I20260709 13:05:25.001224 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:25.001438 15932 raft_consensus.cc:2393] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 4: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:25.003286 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:05:25.005637 15932 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 1483) took 2200 ms (client timeout 2174 ms). Trace:
I20260709 13:05:25.004366 15928 raft_consensus.cc:1240] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:25.003472 15825 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:24.998883 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:25.011927 17536 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:25.012357 16568 raft_consensus.cc:2393] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 2: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:25.013049 15928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 1484) took 2154 ms. Trace:
I20260709 13:05:25.005249 17532 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:25.016136 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:25.016667 15935 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.013447 15928 rpcz_store.cc:276] 0709 13:05:22.858158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:23.145134 (+286976us) service_pool.cc:224] Handling call
0709 13:05:25.004180 (+1859046us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:25.011022 (+  6842us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:25.012809 (+  1787us) spinlock_profiling.cc:243] Waited 1.76 s on lock 0x7b54002c1850. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:25.012941 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4050756480}
I20260709 13:05:25.018277 15860 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:25.018803 15860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:43524 (request call id 1602) took 2034 ms. Trace:
I20260709 13:05:25.017702 15825 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:25.021430 15825 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:25.023295 17595 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:25.023762 17595 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:25.024096 15827 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:25.024168 17595 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.026513 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:05:25.014880 15932 rpcz_store.cc:269] 0709 13:05:22.804683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.879425 (+ 74742us) service_pool.cc:224] Handling call
0709 13:05:25.004051 (+2124626us) spinlock_profiling.cc:243] Waited 2.12 s on lock 0x7b54002c1854. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:25.005556 (+  1505us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4866498816}
I20260709 13:05:25.171458 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:25.026661 17536 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:25.171815 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:25.171564 15930 raft_consensus.cc:2393] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 2: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:25.438923 15937 raft_consensus.cc:2393] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 2: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:25.172227 15931 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 1.
I20260709 13:05:25.439663 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:25.439970 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:25.439664 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:25.440867 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:25.441107 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:25.441870 15931 raft_consensus.cc:2393] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 4: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:25.172745 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:25.173357 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:25.442597 15932 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.442696 15934 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:25.173759 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:25.174683 15946 tablet_copy_service.cc:342] P 8430db80a39d4868bcf7641c16de35ea: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-83a1b79cdcf046c78453695c798a14a5 received from {username='slave'} at 127.0.0.1:53636
I20260709 13:05:25.176016 15935 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:25.019097 15860 rpcz_store.cc:276] 0709 13:05:22.984224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:23.040368 (+ 56144us) service_pool.cc:224] Handling call
0709 13:05:25.018705 (+1978337us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:25.448014 15946 tablet_copy_service.cc:434] P 8430db80a39d4868bcf7641c16de35ea: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-83a1b79cdcf046c78453695c798a14a5 on tablet 83a1b79cdcf046c78453695c798a14a5 with peer eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:25.177937 15854 raft_consensus.cc:1240] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:25.178629 17595 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:25.450207 17595 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:25.450349 15932 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
W20260709 13:05:25.180032 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 153996us
W20260709 13:05:25.181118 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 157085us
I20260709 13:05:25.451598 15827 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:25.181505 17613 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:25.181720 15857 raft_consensus.cc:1240] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:25.452267 15825 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:05:25.193276 15902 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.682s	user 0.021s	sys 0.016s
I20260709 13:05:25.193589 15862 raft_consensus.cc:1240] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:25.204807 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:25.207429 16570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 421) took 2369 ms. Trace:
I20260709 13:05:25.456413 16570 rpcz_store.cc:276] 0709 13:05:22.838179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.838305 (+   126us) service_pool.cc:224] Handling call
0709 13:05:23.248137 (+409832us) spinlock_profiling.cc:243] Waited 6.02 ms on lock 0x7b5800044798. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef381fc0 00007f73ef36c929 00007f73ef36c3d2 00007f73e88e3bd4
0709 13:05:23.248518 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:24.801345 (+1552827us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:25.207215 (+405870us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b54005654d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:25.207379 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3582972544}
I20260709 13:05:25.457008 17592 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:25.457937 17592 raft_consensus.cc:2804] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:25.458371 17592 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:25.458801 17592 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.182029 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:25.217309 15759 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58434 (request call id 340) took 2394 ms. Trace:
I20260709 13:05:25.221501 17591 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:25.233409 15933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 1490) took 2261 ms. Trace:
I20260709 13:05:25.233421 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:25.255935 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:25.465868 15859 raft_consensus.cc:2393] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 3: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:25.265849 15855 raft_consensus.cc:1240] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:25.268273 17532 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:25.401191 16533 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:25.437489 16561 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:25.447991 15928 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.464350 15759 rpcz_store.cc:276] 0709 13:05:22.823072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.823203 (+   131us) service_pool.cc:224] Handling call
0709 13:05:25.217230 (+2394027us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36350,"reported_tablets":2,"spinlock_wait_cycles":13184}
W20260709 13:05:25.455394 15902 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.944s	user 0.021s	sys 0.016s
I20260709 13:05:25.466616 15934 raft_consensus.cc:2468] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6.
I20260709 13:05:25.465088 15933 rpcz_store.cc:276] 0709 13:05:22.972187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:23.252529 (+280342us) service_pool.cc:224] Handling call
0709 13:05:23.280527 (+ 27998us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:23.571246 (+290719us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:25.233191 (+1661945us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b5400327654. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:25.233354 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":640291712}
W20260709 13:05:25.476384 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 1.96535e+06us
W20260709 13:05:25.476871 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 75791.3us
I20260709 13:05:25.481068 15902 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:25.476178 17580 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:05:25.483891 15825 connection.cc:256] RPC call timeout handler was delayed by -1.0437845479999623s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2174s for a response
I20260709 13:05:25.490568 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:25.490715 17572 raft_consensus.cc:2749] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:25.492892 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:25.493502 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:25.493911 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:25.494431 15858 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:25.494483 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:25.495397 15856 raft_consensus.cc:2393] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 2: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:25.495998 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:25.496136 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:25.496392 17613 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.496870 17613 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:25.497221 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:25.497278 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:25.498473 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:25.499224 17613 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:25.500082 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:25.500211 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:25.500679 15934 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.500820 15860 raft_consensus.cc:2393] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 3: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:25.501583 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:25.501646 15928 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:25.482633 16561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:45246 (request call id 485) took 2676 ms. Trace:
I20260709 13:05:25.484702 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:25.512313 15857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1466) took 2655 ms. Trace:
I20260709 13:05:25.486130 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:25.487252 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:05:25.512806 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 61269.1us
I20260709 13:05:25.487998 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:25.487946 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:25.488899 15935 raft_consensus.cc:1240] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:25.502234 15860 raft_consensus.cc:2393] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 2: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:25.502969 15861 raft_consensus.cc:1240] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:25.515372 15858 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:25.517632 15934 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:25.512653 15857 rpcz_store.cc:276] 0709 13:05:22.854084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.854357 (+   273us) service_pool.cc:224] Handling call
0709 13:05:22.854588 (+   231us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:25.452620 (+2598032us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:25.508895 (+ 56275us) spinlock_profiling.cc:243] Waited 2.33 s on lock 0x7b540032e454. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:25.509047 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5351306624}
I20260709 13:05:25.512038 16561 rpcz_store.cc:276] 0709 13:05:22.816503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.818391 (+  1888us) service_pool.cc:224] Handling call
0709 13:05:25.026599 (+2208208us) spinlock_profiling.cc:243] Waited 581 ms on lock 0x7b54006a5754. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea39fa3d 00007f73ef36fcd0 00007f73e88f4254 00007f73e88f41e2 00007f73e88f4111 00007f73e88f408c 00007f73e88f3342 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:25.476796 (+450197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1337429888}
I20260709 13:05:25.492937 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:25.519546 15854 raft_consensus.cc:2393] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 2: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:25.523465 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:25.524484 15856 raft_consensus.cc:2393] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 2: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:25.525280 17592 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:25.527361 15855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 1574) took 2900 ms. Trace:
I20260709 13:05:25.527871 15855 rpcz_store.cc:276] 0709 13:05:22.626461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.626678 (+   217us) service_pool.cc:224] Handling call
0709 13:05:22.626928 (+   250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:25.466274 (+2839346us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:25.527058 (+ 60784us) spinlock_profiling.cc:243] Waited 2.64 s on lock 0x7b54002481d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:25.527292 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6068183296}
I20260709 13:05:25.530169 15862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 1604) took 2099 ms. Trace:
I20260709 13:05:25.530642 15862 rpcz_store.cc:276] 0709 13:05:23.431560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:23.431758 (+   198us) service_pool.cc:224] Handling call
0709 13:05:23.449688 (+ 17930us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:25.455427 (+2005739us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:25.529899 (+ 74472us) spinlock_profiling.cc:243] Waited 1.74 s on lock 0x7b540008c354. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:25.530100 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4010053504}
I20260709 13:05:25.533854 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:25.534614 15929 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.536351 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:25.539937 15827 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:25.541103 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:25.541729 17536 raft_consensus.cc:2749] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:25.540421 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:25.542847 15827 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:25.545017 17536 raft_consensus.cc:2749] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:25.546201 15827 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:25.547646 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:25.547892 17536 raft_consensus.cc:2749] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:25.549594 17561 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:25.549772 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:25.550032 15756 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58448 (request call id 306) took 2181 ms. Trace:
I20260709 13:05:25.550125 17561 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:25.550462 15756 rpcz_store.cc:276] 0709 13:05:23.368770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:23.368960 (+   190us) service_pool.cc:224] Handling call
0709 13:05:25.549975 (+2181015us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":3}
I20260709 13:05:25.551360 15827 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:25.551488 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:25.552469 17561 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:25.552983 17561 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:25.553246 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:25.553406 17561 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:25.553876 17536 raft_consensus.cc:2749] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:25.554548 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:25.555333 17561 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:25.527392 17592 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:25.542874 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.572116 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:25.575618 16567 raft_consensus.cc:1240] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:25.568727 15825 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:25.573129 16568 raft_consensus.cc:1240] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:25.575047 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:25.566198 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:25.573544 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:25.578150 16010 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:25.579946 16568 raft_consensus.cc:1240] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 1. Current term is 3. Ops: []
I20260709 13:05:25.582664 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:25.583858 16569 raft_consensus.cc:2393] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 2: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:25.585532 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:25.585711 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:25.586467 16008 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:25.586473 15825 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:25.588829 16570 raft_consensus.cc:1240] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:25.567539 17552 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:25.592177 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:25.592231 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:25.593045 16569 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:25.593250 16567 raft_consensus.cc:2393] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 2: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:25.595160 15976 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:25.597813 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:25.599574 16568 raft_consensus.cc:2393] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 2: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:25.600258 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:25.591079 17532 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:25.601691 17532 raft_consensus.cc:697] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:25.602581 17532 consensus_queue.cc:237] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:25.603922 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:25.604903 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:25.606011 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:25.607241 16002 raft_consensus.cc:1240] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:25.609342 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:25.610029 16566 raft_consensus.cc:2393] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 3: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:25.585371 17536 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:25.612794 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:25.588110 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:05:25.616366 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 132575us
I20260709 13:05:25.617098 15902 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:25.620453 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:25.621223 16003 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.621398 16006 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:25.623452 16569 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
I20260709 13:05:25.616091 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:25.579689 17604 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:25.627199 15902 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:25.629647 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:25.657156 16008 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:05:25.657603 16007 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.577778 15929 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:25.616142 16565 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.591209 17595 consensus_queue.cc:1059] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:05:25.616954 17552 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:25.634543 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:25.628005 17566 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:25.634828 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:25.628224 17557 consensus_queue.cc:1059] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 13:05:25.635660 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:25.641037 16005 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
W20260709 13:05:25.627112 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 146581us
I20260709 13:05:25.648725 16010 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
W20260709 13:05:25.657998 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 124758us
W20260709 13:05:25.675765 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 80680.5us
I20260709 13:05:25.663908 16570 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:25.676832 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.677287 15976 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:25.661482 15902 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:25.679838 15902 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:25.677055 16533 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:25.684554 15825 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:25.675428 16563 raft_consensus.cc:2393] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 4: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:25.688071 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:25.685637 16533 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:25.689376 16533 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:25.690847 16533 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:25.692004 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:25.692291 16533 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:25.694599 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:25.695289 15937 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:25.695379 17621 consensus_queue.cc:1059] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:25.675854 17540 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:25.694765 17620 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:25.680248 17561 raft_consensus.cc:2749] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:25.684679 17551 raft_consensus.cc:2804] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:25.684685 17623 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:25.663554 17604 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:25.686898 17617 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:25.691663 17584 raft_consensus.cc:2764] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260709 13:05:25.699497 17525 raft_consensus.cc:2804] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:25.719976 17623 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:25.684729 17618 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:25.755023 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:25.709370 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 101414us
I20260709 13:05:25.731920 16009 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:25.717355 16007 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:25.735131 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:05:25.762965 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 63159.7us
I20260709 13:05:25.763526 16010 raft_consensus.cc:2393] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 2: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:25.735284 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:25.736312 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:25.704299 17528 raft_consensus.cc:2804] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:25.740765 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:25.741218 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:25.699398 17580 raft_consensus.cc:2804] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:25.699239 17540 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:25.709052 17551 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:05:25.743721 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 62219.8us
I20260709 13:05:25.754102 15858 raft_consensus.cc:1240] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:25.767344 15902 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:25.756973 17613 raft_consensus.cc:2804] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:25.771401 15860 raft_consensus.cc:1240] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:25.765357 15856 raft_consensus.cc:1240] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:25.752696 17627 raft_consensus.cc:3055] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:25.744585 17617 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:25.755225 17595 raft_consensus.cc:3055] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:25.768071 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:05:25.755590 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 71102.6us
I20260709 13:05:25.768462 15902 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:25.768533 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:25.796154 15827 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:25.796340 16003 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:25.796752 16007 raft_consensus.cc:2393] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 3: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:25.795320 17595 raft_consensus.cc:740] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:25.797130 15902 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:25.797477 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:25.797777 17595 consensus_queue.cc:260] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:25.798260 15902 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:25.798897 17595 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:25.799553 17618 raft_consensus.cc:2749] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:25.799822 17630 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:25.769212 15862 raft_consensus.cc:1240] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:25.800474 17618 raft_consensus.cc:2749] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:25.769552 15859 raft_consensus.cc:1240] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:25.770370 15937 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
I20260709 13:05:25.717689 15976 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:25.801246 17618 raft_consensus.cc:2749] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:05:25.801975 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 84380.6us
I20260709 13:05:25.802637 17572 consensus_queue.cc:1059] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:25.802886 15976 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:25.776993 17528 raft_consensus.cc:697] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:25.803745 17621 raft_consensus.cc:2749] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:25.760129 17525 raft_consensus.cc:697] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:25.804215 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:25.804543 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:25.804687 17620 raft_consensus.cc:2749] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:25.773948 17536 raft_consensus.cc:2749] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:25.804898 17525 consensus_queue.cc:237] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:25.805507 16008 raft_consensus.cc:1240] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:25.766294 16533 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:25.806680 17584 raft_consensus.cc:2749] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:25.767187 17540 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.777478 17629 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:25.807830 15855 raft_consensus.cc:1240] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:25.808177 17629 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:25.807936 16533 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:25.808561 17629 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:25.785375 17613 raft_consensus.cc:697] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:25.825024 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:25.755577 17605 consensus_queue.cc:1059] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:05:25.779772 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:25.756440 17561 consensus_queue.cc:1059] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:05:25.759944 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:25.840462 16565 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:25.784704 17566 raft_consensus.cc:3055] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:25.851054 17566 raft_consensus.cc:740] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:25.788940 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:25.771083 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:05:25.852751 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 44892.5us
I20260709 13:05:25.768554 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:25.782105 17580 raft_consensus.cc:697] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:25.803759 17555 raft_consensus.cc:3055] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:25.853472 17555 raft_consensus.cc:740] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:25.851997 17566 consensus_queue.cc:260] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:25.853940 17580 consensus_queue.cc:237] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:25.854390 17566 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.854295 17555 consensus_queue.cc:260] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:25.779525 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:25.855965 16011 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:05:25.810065 15902 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:25.810336 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:25.857172 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 47205.3us
I20260709 13:05:25.818950 17528 consensus_queue.cc:237] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:25.780884 17552 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:25.855584 17555 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.784842 17623 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:25.777223 17627 raft_consensus.cc:740] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:05:25.863030 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.822507 15937 raft_consensus.cc:1240] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 1. Current term is 3. Ops: []
I20260709 13:05:25.828051 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:25.794634 16570 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:25.831597 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:25.864230 15858 raft_consensus.cc:3055] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:25.864535 15858 raft_consensus.cc:740] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:25.832465 15859 raft_consensus.cc:1275] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:05:25.835069 15827 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:25.795424 17617 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.821146 17625 raft_consensus.cc:1064] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:05:25.839001 17613 consensus_queue.cc:237] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:25.771018 17551 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.840118 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:25.844149 17605 raft_consensus.cc:3055] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:25.836998 17577 consensus_queue.cc:1059] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 13:05:25.842026 17591 raft_consensus.cc:2804] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:25.845614 17561 raft_consensus.cc:3055] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:25.846720 17620 consensus_queue.cc:1059] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:25.846577 17540 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:25.887964 17532 consensus_queue.cc:1059] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:25.888408 17620 raft_consensus.cc:3055] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:25.888808 17620 raft_consensus.cc:740] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:25.842841 15757 catalog_manager.cc:5697] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 2 to 3, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 3 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:25.889561 17620 consensus_queue.cc:260] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:05:25.890676 17620 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:25.891712 15937 raft_consensus.cc:1240] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:25.797523 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:25.817142 16009 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
I20260709 13:05:25.858040 15902 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:25.894897 15902 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:25.858249 16533 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:25.859978 17572 raft_consensus.cc:2749] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:25.818030 17604 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:25.877184 17621 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:25.877785 15930 raft_consensus.cc:1240] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:25.871119 17623 consensus_queue.cc:1048] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:05:25.871053 17552 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.879427 17536 consensus_queue.cc:1059] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:25.874368 15858 consensus_queue.cc:260] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:25.873603 17605 raft_consensus.cc:740] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:25.880261 17592 consensus_queue.cc:1059] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:05:25.887307 15854 raft_consensus.cc:1240] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:25.862740 17627 consensus_queue.cc:260] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:25.904883 15858 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:05:25.873754 17615 raft_consensus.cc:2749] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:05:25.906941 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 104113us
I20260709 13:05:25.884162 17561 raft_consensus.cc:740] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:05:25.909725 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 51528us
I20260709 13:05:25.889587 17540 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:25.909786 17636 raft_consensus.cc:2749] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:25.898733 17557 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:25.904243 17566 raft_consensus.cc:2749] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:25.946081 17604 consensus_queue.cc:1059] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:25.946640 17637 raft_consensus.cc:3055] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:25.947047 17637 raft_consensus.cc:740] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:25.947916 17637 consensus_queue.cc:260] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:25.950327 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 69387.6us
I20260709 13:05:25.951108 15827 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:25.956877 15858 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
I20260709 13:05:25.957819 17620 consensus_queue.cc:1059] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:25.958058 17561 consensus_queue.cc:260] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:25.958901 17620 consensus_queue.cc:1059] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:25.959601 17561 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.961333 17640 raft_consensus.cc:3055] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:25.961779 17640 raft_consensus.cc:740] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:25.962734 17640 consensus_queue.cc:260] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:25.945506 17629 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:25.973541 15827 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:25.949781 17557 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:25.907349 17552 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:25.976619 15934 raft_consensus.cc:1240] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:25.964486 17640 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.904397 17621 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:25.977435 17557 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.985455 17621 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:25.985975 17621 raft_consensus.cc:3055] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:25.986292 17621 raft_consensus.cc:740] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:25.980283 17552 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:25.987123 17621 consensus_queue.cc:260] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:25.988150 17621 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:25.989078 17625 consensus_queue.cc:237] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:25.987991 17552 consensus_queue.cc:1059] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:25.990535 17552 consensus_queue.cc:1243] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer 31882606e87447fb9a36bbf625129fe8 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:05:25.966324 17605 consensus_queue.cc:260] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:25.977408 17635 raft_consensus.cc:3055] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:25.966398 17627 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:25.951803 17632 consensus_queue.cc:1059] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.081s
I20260709 13:05:26.045923 17632 consensus_queue.cc:1059] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:26.047868 17635 raft_consensus.cc:740] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:26.049005 17635 consensus_queue.cc:260] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:26.050469 17635 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:26.050959 17561 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:26.050740 17617 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:26.051509 17561 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:26.053515 17561 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:26.053689 17605 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:26.054023 17561 raft_consensus.cc:3055] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:26.054418 17561 raft_consensus.cc:740] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:26.057699 17621 raft_consensus.cc:3055] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:26.057587 16005 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:26.058137 17621 raft_consensus.cc:740] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:26.060961 17532 raft_consensus.cc:2749] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:26.061587 17532 raft_consensus.cc:2749] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:26.063972 17640 raft_consensus.cc:3055] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:26.064469 17640 raft_consensus.cc:740] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:26.065002 17629 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:26.065410 17640 consensus_queue.cc:260] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:26.055179 17561 consensus_queue.cc:260] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:26.043354 17557 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:26.068655 17624 raft_consensus.cc:1064] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:05:26.068807 15929 raft_consensus.cc:1240] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:26.072199 17557 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:26.072940 17637 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:26.082952 17633 raft_consensus.cc:3055] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:26.084093 17551 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:26.083452 17633 raft_consensus.cc:740] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:05:25.905423 17630 consensus_queue.cc:1175] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8430db80a39d4868bcf7641c16de35ea" responder_term: 3 status { last_received { term: 1 index: 4 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 1. Current term is 3. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:05:26.127454 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 163289us
W20260709 13:05:26.127789 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:26.128208 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:26.128602 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:26.128885 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:26.088786 17634 log.cc:927] Time spent T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.142s	user 0.002s	sys 0.000s
W20260709 13:05:26.093281 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 128502us
I20260709 13:05:26.132820 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:26.133932 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:26.059118 17621 consensus_queue.cc:260] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:26.147652 15936 raft_consensus.cc:1240] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 1. Current term is 2. Ops: []
W20260709 13:05:26.149860 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 184555us
W20260709 13:05:26.121008 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 226199us
I20260709 13:05:26.123174 16533 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:26.151968 15902 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:26.135051 17591 raft_consensus.cc:697] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:05:26.126027 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 151056us
I20260709 13:05:26.128880 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:26.154652 15827 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:05:26.158341 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.160888 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:26.161196 16566 raft_consensus.cc:1240] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:26.164430 15827 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:26.158172 17644 raft_consensus.cc:2804] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:26.165529 17653 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:05:26.165756 15958 debug-util.cc:398] Leaking SignalData structure 0x7b0800100a40 after lost signal to thread 14751
I20260709 13:05:26.165441 17630 consensus_queue.cc:1059] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:26.166116 17653 raft_consensus.cc:697] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:26.166683 17630 raft_consensus.cc:3055] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:26.167079 17630 raft_consensus.cc:740] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:26.167979 17630 consensus_queue.cc:260] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:26.169092 17630 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:26.169945 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.169899 17591 consensus_queue.cc:237] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:26.170972 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:26.166708 15958 debug-util.cc:398] Leaking SignalData structure 0x7b08002d1100 after lost signal to thread 15794
W20260709 13:05:26.172993 15958 debug-util.cc:398] Leaking SignalData structure 0x7b08000adb40 after lost signal to thread 15886
I20260709 13:05:26.173516 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:26.174079 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:26.174305 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:26.174459 15827 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:26.172922 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:26.176033 15827 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:05:26.174101 15958 debug-util.cc:398] Leaking SignalData structure 0x7b08002b1f80 after lost signal to thread 15961
W20260709 13:05:26.178045 15958 debug-util.cc:398] Leaking SignalData structure 0x7b08003234c0 after lost signal to thread 16035
I20260709 13:05:26.150307 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:26.148826 17595 consensus_queue.cc:1059] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
W20260709 13:05:26.179505 15958 debug-util.cc:398] Leaking SignalData structure 0x7b08001fc000 after lost signal to thread 16594
I20260709 13:05:26.181625 16535 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:26.149441 17615 consensus_queue.cc:1059] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 13:05:26.183076 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:26.185256 16567 raft_consensus.cc:2393] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 2: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:26.184226 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:26.185559 17577 raft_consensus.cc:2749] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:05:26.189240 15902 connection.cc:256] RPC call timeout handler was delayed by -1.3412057960000539s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1503s for a response
W20260709 13:05:26.189747 15902 connection.cc:256] RPC call timeout handler was delayed by -1.1927057960000411s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1668s for a response
I20260709 13:05:26.176702 17644 raft_consensus.cc:697] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:26.190454 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:26.190796 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:26.191181 17644 consensus_queue.cc:237] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:26.179191 17646 raft_consensus.cc:2749] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:26.195535 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:26.196944 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:26.200380 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:26.197538 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:26.198073 15933 raft_consensus.cc:1240] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:26.201673 16011 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2.
W20260709 13:05:26.127650 17626 consensus_queue.cc:1175] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" responder_term: 3 status { last_received { term: 1 index: 4 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 1. Current term is 3. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:05:26.197369 15932 raft_consensus.cc:1240] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:26.196444 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:26.200584 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:26.175870 17592 raft_consensus.cc:2749] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:26.156766 17651 raft_consensus.cc:2804] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:26.204418 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:26.205868 16009 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:26.167006 17653 consensus_queue.cc:237] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:26.211830 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:26.204901 16003 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2.
I20260709 13:05:26.205325 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:26.219079 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:26.216729 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:26.214567 17557 consensus_queue.cc:1059] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:05:26.221355 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:26.221665 17557 raft_consensus.cc:3055] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:26.222134 17557 raft_consensus.cc:740] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:05:26.223804 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 47828us
I20260709 13:05:26.223804 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:26.204857 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:26.224462 15827 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:26.208540 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:26.226306 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:26.228091 17637 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:26.228595 17637 raft_consensus.cc:697] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:26.228946 15935 raft_consensus.cc:1240] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:26.229475 17637 consensus_queue.cc:237] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:26.235924 15936 raft_consensus.cc:2393] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 2: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:26.236737 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:26.216709 16565 raft_consensus.cc:1240] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:26.238678 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:26.241237 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:26.242030 16570 raft_consensus.cc:2393] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 2: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
W20260709 13:05:26.247280 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.247483 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:26.250698 15827 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:26.251893 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:26.251366 15976 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:26.215830 17651 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:26.258042 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:26.219321 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:26.259825 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:26.260392 15827 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:05:26.259487 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 74529.6us
I20260709 13:05:26.261099 16010 raft_consensus.cc:2376] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3. Re-sending same reply.
I20260709 13:05:26.220391 17629 consensus_queue.cc:1059] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:05:26.262670 15861 raft_consensus.cc:1240] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:26.263451 16535 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:26.262940 16003 raft_consensus.cc:1240] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:26.264673 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:26.265694 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:26.265974 17616 raft_consensus.cc:3055] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:26.266069 16565 raft_consensus.cc:1275] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:26.264402 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:26.223038 17557 consensus_queue.cc:260] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:26.270521 16011 raft_consensus.cc:2393] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 2: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:26.246878 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:26.259443 15928 raft_consensus.cc:1240] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:26.270756 16535 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:26.276933 15862 raft_consensus.cc:1240] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:26.296530 16010 raft_consensus.cc:1240] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:26.271059 17628 raft_consensus.cc:2749] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:26.271420 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:26.298453 16535 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:26.270678 17613 raft_consensus.cc:2749] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:26.280673 17617 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:26.258121 17627 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:26.282229 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:26.264189 17625 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:05:26.283212 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.265405 15827 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:26.199509 17643 consensus_queue.cc:1059] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 13:05:26.257339 17592 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:26.179744 17633 consensus_queue.cc:260] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:26.263923 17615 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:26.332402 17592 raft_consensus.cc:697] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:26.333447 17592 consensus_queue.cc:237] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:26.334257 16562 raft_consensus.cc:1240] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:26.259262 17651 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:26.339267 17555 raft_consensus.cc:3055] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:26.279989 17551 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:26.299994 16535 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:26.300808 17605 consensus_queue.cc:1059] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:26.301920 17540 consensus_queue.cc:1059] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:26.290022 16004 raft_consensus.cc:2376] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5. Re-sending same reply.
I20260709 13:05:26.303357 16008 raft_consensus.cc:1240] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:26.303925 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:26.297549 17618 consensus_queue.cc:1059] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:05:26.305016 17658 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:26.308614 17640 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:26.309139 15854 raft_consensus.cc:2468] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6.
I20260709 13:05:26.340698 17555 raft_consensus.cc:740] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:26.310392 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:26.288544 17616 raft_consensus.cc:740] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:26.308233 16007 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:26.314172 15976 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:05:26.380770 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 68630.5us
I20260709 13:05:26.313843 15974 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:26.315326 17627 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:26.316359 15902 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:26.383957 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:05:26.328303 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 62957.9us
I20260709 13:05:26.321165 17557 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:26.350214 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 50323.8us
I20260709 13:05:26.355285 17580 raft_consensus.cc:2749] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:26.329535 17626 raft_consensus.cc:2749] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:26.356644 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:26.309980 17561 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:26.311478 17591 raft_consensus.cc:2749] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:26.386992 16568 raft_consensus.cc:2393] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 4: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:26.390430 15827 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:26.405611 15827 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:26.349673 15856 raft_consensus.cc:1275] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:05:26.396358 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 82616.6us
I20260709 13:05:26.406806 15974 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:26.393858 16535 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:26.358446 17633 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:26.402266 15932 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 2.
I20260709 13:05:26.415740 16535 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:26.354144 17629 consensus_queue.cc:1059] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:05:26.352820 17641 consensus_queue.cc:1059] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:05:26.356465 17566 consensus_queue.cc:1059] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:05:26.424881 15974 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:26.423925 16535 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:26.432461 17657 raft_consensus.cc:2804] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:26.425719 17613 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:26.433017 17657 raft_consensus.cc:697] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:26.429327 17605 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:26.434159 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:26.433017 17635 raft_consensus.cc:3055] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:26.434000 17657 consensus_queue.cc:237] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:26.434746 17635 raft_consensus.cc:740] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:26.435181 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:26.401612 17648 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:26.434751 16569 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 2.
I20260709 13:05:26.440982 15974 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:26.442502 15974 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:26.439368 15827 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:26.437994 16535 leader_election.cc:304] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:26.422330 17638 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:26.443912 17525 raft_consensus.cc:2804] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:05:26.453722 17655 log.cc:927] Time spent T 00000000000000000000000000000000 P d33cebb189b14a989836214fdbafb898: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 13:05:26.453888 17623 raft_consensus.cc:3055] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:26.454345 17623 raft_consensus.cc:740] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:26.453660 17525 raft_consensus.cc:697] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:26.456863 15932 raft_consensus.cc:1240] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:26.451922 17644 raft_consensus.cc:2804] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:26.457492 15827 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:26.403173 17625 consensus_queue.cc:1048] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 13:05:26.420003 17661 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:26.460004 17595 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:26.436565 17528 raft_consensus.cc:2804] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:26.461345 17639 raft_consensus.cc:3055] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:26.462936 17637 consensus_queue.cc:1059] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:26.463456 17528 raft_consensus.cc:697] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:26.463541 17639 raft_consensus.cc:740] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:26.464494 17528 consensus_queue.cc:237] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:26.464556 17639 consensus_queue.cc:260] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:26.465763 17639 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:26.468698 15932 raft_consensus.cc:1240] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:26.444442 17642 raft_consensus.cc:2749] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:26.447937 17552 raft_consensus.cc:2749] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:26.460355 16535 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:26.449700 17605 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:26.425796 17572 consensus_queue.cc:1059] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:05:26.457314 17629 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:26.458811 17566 raft_consensus.cc:3055] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:26.472355 16535 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:26.466364 17624 consensus_queue.cc:237] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:26.483553 16535 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:26.461632 17626 raft_consensus.cc:2749] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:26.463003 17627 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:26.482133 17566 raft_consensus.cc:740] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:26.450408 17616 consensus_queue.cc:260] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:26.502225 17616 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:26.518695 17621 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:26.510998 17566 consensus_queue.cc:260] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:26.518991 17647 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:05:26.529983 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 172552us
W20260709 13:05:26.530118 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 213817us
I20260709 13:05:26.533041 15902 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:26.532424 15900 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:26.535318 17645 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:05:26.508011 17643 consensus_queue.cc:1175] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "85e997ba5003439781ec4f1f8db0c3ee" responder_term: 5 status { last_received { term: 4 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 4. Current term is 5. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:05:26.525717 17618 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:26.544727 17670 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:26.545073 17645 raft_consensus.cc:697] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:26.545617 17679 raft_consensus.cc:2749] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:26.547288 15900 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:26.547055 17678 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:26.525980 17566 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:26.511554 17525 consensus_queue.cc:237] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:26.562985 17618 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:26.576606 17670 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:05:26.608549 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.573240 17644 raft_consensus.cc:697] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:05:26.620138 17659 log.cc:927] Time spent T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.298s	user 0.002s	sys 0.000s
I20260709 13:05:26.649627 17618 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:26.650128 17644 consensus_queue.cc:237] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:26.656880 16006 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:26.658682 16009 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:26.596426 17645 consensus_queue.cc:237] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:26.666173 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:26.667208 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.668133 17623 consensus_queue.cc:260] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:26.668678 17651 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:26.669379 17623 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:26.669662 15928 raft_consensus.cc:1240] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:26.666535 17604 raft_consensus.cc:3055] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:26.671571 17604 raft_consensus.cc:740] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:26.672468 17604 consensus_queue.cc:260] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:26.673533 17604 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:26.679515 17665 log.cc:927] Time spent T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.265s	user 0.002s	sys 0.000s
W20260709 13:05:26.696837 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 239424us
I20260709 13:05:26.698410 15827 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
W20260709 13:05:26.701735 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 218240us
I20260709 13:05:26.702565 16535 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:26.700145 16563 raft_consensus.cc:1240] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:26.699995 17653 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:26.704563 17653 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:26.704969 17653 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:26.705317 17658 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:26.704097 15929 raft_consensus.cc:1275] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:26.706655 15976 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:26.709028 15827 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:26.710917 17592 raft_consensus.cc:2749] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:26.713248 16570 raft_consensus.cc:1275] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:05:26.714088 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:26.719225 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:26.705886 17658 raft_consensus.cc:697] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:26.712297 17640 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:26.726595 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:26.737783 15858 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:05:26.740365 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:05:26.739497 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 297056us
I20260709 13:05:26.743309 15974 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:26.750749 15974 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:05:26.753270 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.781028 16563 raft_consensus.cc:1240] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:26.781658 15974 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:26.783272 15974 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:26.725916 17640 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.783835 16567 raft_consensus.cc:1275] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:26.792217 15974 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:26.793805 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:26.794322 16569 raft_consensus.cc:1275] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:05:26.783655 17621 consensus_queue.cc:1048] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:05:26.795859 17651 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:26.724023 17639 consensus_queue.cc:1048] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 13:05:26.800773 15974 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:26.802217 15974 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:26.803499 15974 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:26.806862 15861 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2.
I20260709 13:05:26.754596 17561 raft_consensus.cc:2749] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:26.790791 17552 raft_consensus.cc:2749] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:26.790208 17640 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:26.789599 17620 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:26.723579 17658 consensus_queue.cc:237] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:26.831900 17561 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:26.796957 17661 raft_consensus.cc:2749] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:26.829646 15934 raft_consensus.cc:1275] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:26.836336 16008 raft_consensus.cc:1275] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:26.798388 17647 raft_consensus.cc:697] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:26.798388 17632 raft_consensus.cc:2749] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:26.838559 17632 raft_consensus.cc:2749] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:26.838438 17673 consensus_queue.cc:1048] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:05:26.811117 17648 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:26.837651 16010 raft_consensus.cc:1275] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:05:26.845199 17672 log.cc:927] Time spent T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.257s	user 0.000s	sys 0.002s
I20260709 13:05:26.854542 16007 raft_consensus.cc:1275] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:26.854805 17642 consensus_queue.cc:1059] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 13:05:26.865797 17642 raft_consensus.cc:3055] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:26.866231 17642 raft_consensus.cc:740] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:26.870419 17561 raft_consensus.cc:2749] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:26.869340 17647 consensus_queue.cc:237] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:26.874929 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 327701us
I20260709 13:05:26.830200 17635 consensus_queue.cc:260] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:26.887413 17540 raft_consensus.cc:3055] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:26.888029 17540 raft_consensus.cc:740] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:26.889057 17540 consensus_queue.cc:260] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:26.890197 17540 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:26.891438 17551 raft_consensus.cc:3055] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:26.891965 17551 raft_consensus.cc:740] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:26.892992 17551 consensus_queue.cc:260] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:26.895247 17551 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:26.882558 17651 consensus_queue.cc:1059] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 13:05:26.903396 17536 consensus_queue.cc:1048] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:05:26.867159 17642 consensus_queue.cc:260] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:26.909482 17651 consensus_queue.cc:1059] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:05:26.916790 17642 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:26.917235 17572 raft_consensus.cc:3055] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:26.917676 17572 raft_consensus.cc:740] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:26.917722 17555 consensus_queue.cc:260] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:26.919139 17555 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:26.918731 17572 consensus_queue.cc:260] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:26.920040 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.913038 15900 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:26.924269 15900 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:26.927289 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:05:26.928632 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 396477us
I20260709 13:05:26.930080 15902 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:26.932616 15902 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:26.933565 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:26.937891 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:26.938491 15936 raft_consensus.cc:1275] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:26.939394 16533 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:26.928107 17557 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:26.942144 15902 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:26.925140 17651 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:26.947733 17688 raft_consensus.cc:2749] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:26.950353 15902 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:26.910745 17637 consensus_queue.cc:1048] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:05:26.968891 16006 raft_consensus.cc:1275] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:26.928301 17572 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:26.939761 17635 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:26.967962 17667 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:26.970418 15902 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:26.957722 17682 raft_consensus.cc:2749] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:05:26.976646 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 173227us
I20260709 13:05:26.969833 17557 raft_consensus.cc:697] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:26.987625 15974 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:26.945957 17630 raft_consensus.cc:3055] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:26.945669 17618 raft_consensus.cc:2749] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:26.986634 15902 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:26.989969 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:26.997778 15856 raft_consensus.cc:2393] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 3: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
W20260709 13:05:26.990170 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:26.993496 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:26.998311 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.969007 17679 consensus_queue.cc:1048] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:05:27.033773 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.980877 17651 raft_consensus.cc:697] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:26.987473 17629 raft_consensus.cc:2804] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:26.989120 17566 raft_consensus.cc:2804] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:26.989156 17552 raft_consensus.cc:2749] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:27.056883 15935 raft_consensus.cc:1275] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:05:26.981086 17532 consensus_queue.cc:1048] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:05:26.994169 17630 raft_consensus.cc:740] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:26.994306 17667 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:27.091064 15862 raft_consensus.cc:1275] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:27.055085 15936 raft_consensus.cc:1275] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:05:27.069391 17686 log.cc:927] Time spent T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:05:27.035954 17688 raft_consensus.cc:2749] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:27.070324 15933 raft_consensus.cc:1275] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
I20260709 13:05:27.077147 17669 raft_consensus.cc:3055] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:27.082049 16535 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:27.104117 16568 raft_consensus.cc:1275] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:27.086813 15858 raft_consensus.cc:1275] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:05:27.087628 15859 raft_consensus.cc:1275] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
W20260709 13:05:27.094836 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 48010.4us
I20260709 13:05:27.036352 17639 consensus_queue.cc:1059] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 13:05:27.039898 17605 consensus_queue.cc:1048] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
I20260709 13:05:27.115249 15933 raft_consensus.cc:1240] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:27.036224 17613 consensus_queue.cc:1048] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.061s
I20260709 13:05:27.096141 17667 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:27.100433 17566 raft_consensus.cc:697] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:27.131707 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:05:27.168076 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.117022 17669 raft_consensus.cc:740] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:27.110904 17525 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:05:27.219702 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 82532.1us
I20260709 13:05:27.229959 17667 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:27.109021 17640 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:27.124084 17584 consensus_queue.cc:1048] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.095s
W20260709 13:05:27.220374 17689 log.cc:927] Time spent T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 13:05:27.244999 15858 raft_consensus.cc:1240] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:27.247097 17566 consensus_queue.cc:237] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:27.220803 17629 raft_consensus.cc:697] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:27.237377 17540 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:27.265617 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 158900us
W20260709 13:05:27.269387 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.131129 17557 consensus_queue.cc:237] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:27.135525 17630 consensus_queue.cc:260] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:27.233716 17644 consensus_queue.cc:1048] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:05:27.291208 15902 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:27.292392 15902 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:27.294718 15933 raft_consensus.cc:1275] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:27.295615 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:27.218971 17651 consensus_queue.cc:237] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:27.242547 17682 consensus_queue.cc:1048] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:05:27.244701 17633 consensus_queue.cc:1048] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
W20260709 13:05:27.303701 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.313987 17685 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:27.314600 17667 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:27.310426 17639 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:27.283967 17540 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:27.290202 17615 consensus_queue.cc:1059] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:05:27.303069 17528 consensus_queue.cc:1048] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:05:27.302886 17618 consensus_queue.cc:1059] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:05:27.322647 17682 consensus_queue.cc:1048] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 13:05:27.321171 17705 consensus_queue.cc:1048] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 13:05:27.321609 17639 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:27.266474 17669 consensus_queue.cc:260] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:27.306923 17700 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:27.336042 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:05:27.366480 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:27.428550 17687 log.cc:927] Time spent T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.288s	user 0.002s	sys 0.000s
I20260709 13:05:27.306802 17691 consensus_queue.cc:1048] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.103s
I20260709 13:05:27.332053 17629 consensus_queue.cc:237] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:05:27.354183 17627 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:27.436487 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:27.440989 16566 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:27.447839 17555 consensus_meta.cc:287] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.517s	user 0.009s	sys 0.003s
I20260709 13:05:27.421837 17605 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.462001 15930 raft_consensus.cc:1275] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 13:05:27.425042 17592 consensus_queue.cc:1175] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" responder_term: 1 status { last_received { term: 1 index: 2 } last_committed_idx: 2 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)" } } last_received_current_leader { term: 1 index: 2 } }
I20260709 13:05:27.436525 17618 raft_consensus.cc:3055] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:27.457096 17627 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:27.436661 17561 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.436439 17640 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:27.498448 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 243958us
I20260709 13:05:27.436111 17700 raft_consensus.cc:697] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:05:27.480895 17698 log.cc:927] Time spent T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
I20260709 13:05:27.481475 17566 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:27.478420 17695 log.cc:927] Time spent T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.224s	user 0.000s	sys 0.001s
W20260709 13:05:27.467306 17653 consensus_meta.cc:287] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.762s	user 0.006s	sys 0.005s
W20260709 13:05:27.493078 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.421656 17696 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:05:27.518412 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 263449us
I20260709 13:05:27.493723 17618 raft_consensus.cc:740] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:27.470469 17605 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:27.528949 16002 raft_consensus.cc:1275] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:27.507138 17552 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.511057 17685 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.530356 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:05:27.539289 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.526440 17540 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:27.612497 17715 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.612773 15902 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:27.516892 17667 consensus_queue.cc:1059] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 13:05:27.507529 17561 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:27.616734 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.529333 17696 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:27.547644 17525 raft_consensus.cc:697] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:27.520004 17566 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:27.525436 17653 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:27.634742 15902 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:27.637975 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:27.643458 15932 raft_consensus.cc:1240] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:27.539592 17552 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:27.620672 17540 raft_consensus.cc:2749] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:27.670821 17640 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.680802 16002 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94"
dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
mode: GRACEFUL
new_leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:37262
I20260709 13:05:27.682572 15861 raft_consensus.cc:1240] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:27.699326 15757 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58448 (request call id 307) took 1910 ms. Trace:
I20260709 13:05:27.703090 15859 raft_consensus.cc:1275] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:27.541960 17685 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:27.706589 16002 raft_consensus.cc:606] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Received request to transfer leadership to TS eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:27.716846 15862 raft_consensus.cc:1275] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:27.681704 17639 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:27.708146 17635 consensus_meta.cc:287] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.722s	user 0.009s	sys 0.001s
I20260709 13:05:27.697832 17693 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.727409 17592 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:27.702906 17640 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:27.730029 17627 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:27.621469 17700 consensus_queue.cc:237] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:27.743556 17693 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:27.621629 17618 consensus_queue.cc:260] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:27.757221 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 122565us
W20260709 13:05:27.762354 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 72644.5us
W20260709 13:05:27.627925 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 92927.8us
I20260709 13:05:27.770064 16566 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:27.782618 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:27.620940 17715 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:27.799230 15902 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:27.741364 17613 consensus_queue.cc:1059] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:05:27.970649 17696 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:27.748392 17532 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:05:27.803856 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.975152 17532 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:27.976063 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:27.976521 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.808395 16566 raft_consensus.cc:1240] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:27.978164 17532 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:27.809826 15862 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "8430db80a39d4868bcf7641c16de35ea"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:05:27.806262 17712 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:27.979313 17712 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:27.800707 17577 consensus_queue.cc:1048] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 13:05:27.811017 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:27.786212 17702 raft_consensus.cc:2749] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:27.785951 17627 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:27.746418 17592 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:27.786728 17635 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.818219 16569 raft_consensus.cc:1217] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
I20260709 13:05:27.819000 17618 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:27.791251 17540 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:27.748648 17525 consensus_queue.cc:237] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:27.840574 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:27.841055 17561 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:27.842939 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:27.844565 17566 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:27.703363 15757 rpcz_store.cc:276] 0709 13:05:25.771849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:25.771977 (+   128us) service_pool.cc:224] Handling call
0709 13:05:27.682286 (+1910309us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:26.138210 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:26.138989 (+   779us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:26.139032 (+    43us) write_op.cc:435] Acquired schema lock
0709 13:05:26.139058 (+    26us) tablet.cc:662] Decoding operations
0709 13:05:26.139492 (+   434us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:26.139581 (+    89us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:26.139625 (+    44us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:05:26.139867 (+   242us) tablet.cc:701] Row locks acquired
0709 13:05:26.139891 (+    24us) write_op.cc:260] PREPARE: finished
0709 13:05:26.140088 (+   197us) write_op.cc:270] Start()
0709 13:05:26.140221 (+   133us) write_op.cc:276] Timestamp: P: 1783602326140036 usec, L: 0
0709 13:05:26.140252 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 13:05:26.141103 (+   851us) log.cc:844] Serialized 533 byte log entry
0709 13:05:26.571733 (+430630us) op_driver.cc:464] REPLICATION: finished
0709 13:05:27.127196 (+555463us) write_op.cc:301] APPLY: starting
0709 13:05:27.127296 (+   100us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:27.127416 (+   120us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:27.127525 (+   109us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:27.127783 (+   258us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:27.127932 (+   149us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:05:27.127973 (+    41us) write_op.cc:312] APPLY: finished
0709 13:05:27.128161 (+   188us) log.cc:844] Serialized 23 byte log entry
0709 13:05:27.133884 (+  5723us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:27.216335 (+ 82451us) write_op.cc:454] Released schema lock
0709 13:05:27.219349 (+  3014us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:27.249956 (+ 30607us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:27.249988 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":536789,"apply.run_cpu_time_us":3972,"apply.run_wall_time_us":118119,"num_ops":1,"prepare.queue_time_us":246645,"prepare.run_cpu_time_us":9696,"prepare.run_wall_time_us":66454,"raft.queue_time_us":107867,"raft.run_cpu_time_us":36771,"raft.run_wall_time_us":374613,"replication_time_us":426463,"thread_start_us":707726,"threads_started":4,"wal-append.queue_time_us":121259}]]}
I20260709 13:05:27.848366 17640 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:27.824250 17605 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:27.809273 17591 consensus_queue.cc:1048] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:05:27.866760 17685 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:27.836033 17653 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:27.879379 17693 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:27.881049 17645 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.153s	user 0.053s	sys 0.088s
I20260709 13:05:27.881832 17621 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:27.884756 17551 consensus_meta.cc:287] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.986s	user 0.013s	sys 0.000s
W20260709 13:05:27.896740 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 112015us
W20260709 13:05:27.898144 17572 consensus_meta.cc:287] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.919s	user 0.012s	sys 0.001s
I20260709 13:05:28.109810 15930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 331) took 1054 ms. Trace:
I20260709 13:05:27.897027 17552 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:27.905874 17673 raft_consensus.cc:3055] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:05:27.904388 17714 log.cc:927] Time spent T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.282s	user 0.005s	sys 0.000s
I20260709 13:05:28.115149 15929 raft_consensus.cc:1275] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:05:27.934226 15861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1494) took 1001 ms. Trace:
I20260709 13:05:27.934875 17715 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:27.941062 17735 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.116056 15861 rpcz_store.cc:276] 0709 13:05:26.932700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:26.932849 (+   149us) service_pool.cc:224] Handling call
0709 13:05:26.933130 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:27.699874 (+766744us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:27.933932 (+234058us) spinlock_profiling.cc:243] Waited 749 ms on lock 0x7b540008cd54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:27.934154 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1723100416}
W20260709 13:05:27.954326 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 155153us
I20260709 13:05:27.960458 17557 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:27.961091 15854 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2.
I20260709 13:05:27.990948 17657 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.992645 17528 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.032338 17532 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.995757 17584 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:28.123958 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:05:28.064215 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 114508us
I20260709 13:05:28.064435 15936 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:28.070658 17712 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:28.032295 17696 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.081211 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:28.132344 15856 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 2.
I20260709 13:05:28.133260 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:28.083244 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:28.035197 17709 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.100363 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:28.040454 17730 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.110405 15930 rpcz_store.cc:276] 0709 13:05:27.055142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:27.057242 (+  2100us) service_pool.cc:224] Handling call
0709 13:05:27.057523 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:27.462457 (+404934us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:28.109576 (+647119us) spinlock_profiling.cc:243] Waited 401 ms on lock 0x7b5400197b54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:28.109731 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":921634176}
I20260709 13:05:28.032250 17627 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:28.112483 15934 raft_consensus.cc:1275] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:05:28.101090 17645 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.374s	user 0.054s	sys 0.088s
I20260709 13:05:28.069398 17561 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:28.072669 17592 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:28.114267 17673 raft_consensus.cc:740] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:28.077632 17624 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:28.085417 17640 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.102095 17605 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.124629 15902 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:28.069311 17540 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:28.109788 17621 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:05:28.126096 17528 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:28.126874 17532 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:28.129992 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:28.140332 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:28.116448 17735 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:28.064427 17635 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:28.139968 17584 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:28.600564 17712 raft_consensus.cc:2749] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:05:28.601234 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 476649us
W20260709 13:05:28.475828 15977 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.032s	sys 0.026s
I20260709 13:05:28.260380 15974 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:28.143208 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:28.604624 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:28.605865 16568 raft_consensus.cc:2393] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 2: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:28.159718 15856 raft_consensus.cc:1275] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:05:28.358559 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:28.609710 15977 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.038s	sys 0.027s
W20260709 13:05:28.613039 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 352751us
I20260709 13:05:28.617074 15930 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 2: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
W20260709 13:05:28.603317 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:28.619278 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:28.503648 15936 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:28.637858 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:05:28.166132 17731 log.cc:927] Time spent T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.219s	user 0.000s	sys 0.002s
I20260709 13:05:28.692636 16566 raft_consensus.cc:1275] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 13:05:28.292162 17658 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.763s	user 0.020s	sys 0.080s
I20260709 13:05:28.362444 15756 catalog_manager.cc:5697] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 1 to 2, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:28.599748 17737 log.cc:927] Time spent T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.357s	user 0.000s	sys 0.001s
I20260709 13:05:28.697420 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:28.701994 15857 raft_consensus.cc:1275] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:28.248950 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:28.604041 17651 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:28.380661 17733 log.cc:927] Time spent T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.308s	user 0.000s	sys 0.002s
I20260709 13:05:28.202819 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:28.122673 17657 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:28.497431 17679 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.026s	sys 0.033s
I20260709 13:05:28.358471 17639 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.242636 17637 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.714040 17577 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.714617 17645 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.371992 15929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1531) took 1084 ms. Trace:
I20260709 13:05:28.707893 15756 catalog_manager.cc:5697] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 1 to 2, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:28.701802 17658 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.172s	user 0.028s	sys 0.087s
I20260709 13:05:28.751426 15756 catalog_manager.cc:5697] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 1 to 2, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:28.691658 17691 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.642146 17715 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.642527 17552 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:28.292236 17700 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.695176 17551 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:28.696201 17682 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.694796 17557 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:28.703227 16562 raft_consensus.cc:1240] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:28.699474 17626 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.155020 17696 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:28.730931 17657 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:28.731073 17584 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:28.731688 17722 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:28.733677 15854 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 1491) took 1994 ms (client timeout 1896 ms). Trace:
W20260709 13:05:28.737385 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:28.741093 17540 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:28.741384 17613 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.742136 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:28.743355 17627 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:28.742156 17641 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.744127 17615 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.743393 15934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1529) took 1506 ms. Trace:
I20260709 13:05:28.705718 17636 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.746619 17710 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:28.636705 17605 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:28.636965 17561 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:28.642488 17640 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:28.751875 15900 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.772s	user 0.022s	sys 0.026s
W20260709 13:05:28.729485 17679 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.740s	user 0.027s	sys 0.034s
I20260709 13:05:28.636626 17709 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:05:28.796053 17551 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:28.784868 15933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 343) took 1024 ms. Trace:
I20260709 13:05:28.786942 17704 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.722136 17651 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:28.642385 17624 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:28.749248 17639 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:28.803915 15975 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.769s	user 0.022s	sys 0.064s
W20260709 13:05:28.868546 15975 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.834s	user 0.022s	sys 0.064s
I20260709 13:05:28.748378 16565 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2.
I20260709 13:05:28.636830 17730 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:28.761241 17645 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:28.811373 15932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 339) took 1088 ms. Trace:
I20260709 13:05:28.881945 15932 rpcz_store.cc:276] 0709 13:05:27.722819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:27.787944 (+ 65125us) service_pool.cc:224] Handling call
0709 13:05:27.788370 (+   426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:27.802677 (+ 14307us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:27.802710 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:27.815000 (+ 12290us) log.cc:844] Serialized 26 byte log entry
0709 13:05:27.956519 (+141519us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:27.956575 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:27.956668 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:28.811152 (+854484us) raft_consensus.cc:1685] finished
0709 13:05:28.811173 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:28.811317 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13867,"raft.queue_time_us":14363,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":4680,"thread_start_us":141092,"threads_started":1,"wal-append.queue_time_us":131886}
I20260709 13:05:28.814225 15934 rpcz_store.cc:276] 0709 13:05:27.236748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:27.237367 (+   619us) service_pool.cc:224] Handling call
0709 13:05:27.237640 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:28.603906 (+1366266us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:28.743115 (+139209us) spinlock_profiling.cc:243] Waited 874 ms on lock 0x7b540034ba54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:28.743325 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2011108096}
I20260709 13:05:28.750840 17577 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:28.887748 15855 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 2.
I20260709 13:05:28.784977 17691 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:28.751049 17637 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:28.796531 17673 consensus_queue.cc:260] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:28.863003 17738 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.871945 17528 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:28.789746 17715 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:28.794560 17700 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:28.787930 17552 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:28.817715 17627 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.801257 17557 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:28.859427 15900 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.879s	user 0.022s	sys 0.026s
W20260709 13:05:28.898073 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 918197us
I20260709 13:05:28.897863 17627 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:28.899583 17557 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:28.900197 17627 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:28.900563 17715 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:28.900712 17627 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:28.750499 15929 rpcz_store.cc:276] 0709 13:05:27.287346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:27.289266 (+  1920us) service_pool.cc:224] Handling call
0709 13:05:27.332923 (+ 43657us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:28.115694 (+782771us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:28.371629 (+255935us) spinlock_profiling.cc:243] Waited 779 ms on lock 0x7b540034a154. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:28.371912 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1792101248}
I20260709 13:05:28.902053 15936 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:28.803342 17626 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:28.902230 17540 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.902876 17700 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:28.867700 17532 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:28.861761 15933 rpcz_store.cc:276] 0709 13:05:27.760269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:27.874648 (+114379us) service_pool.cc:224] Handling call
0709 13:05:27.874998 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:27.875397 (+   399us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:27.875435 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:27.876085 (+   650us) log.cc:844] Serialized 26 byte log entry
0709 13:05:28.005878 (+129793us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:28.005926 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:28.006017 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:28.784638 (+778621us) raft_consensus.cc:1685] finished
0709 13:05:28.784659 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:28.784805 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":506,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":11147,"thread_start_us":128109,"threads_started":1,"wal-append.queue_time_us":264145}
I20260709 13:05:28.798429 17682 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:28.811120 15854 rpcz_store.cc:269] 0709 13:05:26.739535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:26.753513 (+ 13978us) service_pool.cc:224] Handling call
0709 13:05:28.733435 (+1979922us) spinlock_profiling.cc:243] Waited 947 ms on lock 0x7b540008d254. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:28.733575 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2177923968}
I20260709 13:05:28.879384 17709 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:28.817790 17613 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:28.917749 17735 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:28.922289 17639 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:28.870850 17621 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:28.870357 17635 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:28.819499 17615 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:28.897114 17624 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:28.819432 17636 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:28.819320 17641 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:28.901211 17627 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:28.852054 17710 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:28.912707 17691 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:28.913800 17696 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:28.902916 17540 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:28.930024 17730 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:28.931937 17613 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:28.927634 17557 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:28.929103 17532 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.883209 17739 consensus_queue.cc:1048] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.065s
I20260709 13:05:28.938841 17636 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:28.940858 17682 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:28.941709 17551 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:28.928649 17605 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:05:28.954411 16567 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.967s	user 0.030s	sys 0.046s
I20260709 13:05:28.928825 17621 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.889536 17704 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:28.954820 16567 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.972s	user 0.030s	sys 0.046s
W20260709 13:05:28.950547 17732 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.700s	user 0.016s	sys 0.000s
W20260709 13:05:28.962610 17732 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.802s	user 0.017s	sys 0.000s
I20260709 13:05:28.896749 17738 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:28.954541 17645 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:28.958868 17605 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:05:28.960245 17643 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.012s	sys 0.001s
I20260709 13:05:28.961746 17640 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:28.956777 17615 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:28.959129 17692 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.888s	user 0.009s	sys 0.015s
I20260709 13:05:28.965075 17626 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:28.979219 17637 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:28.981607 17710 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:28.982831 17627 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:28.981653 17618 consensus_meta.cc:287] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.915s	user 0.010s	sys 0.000s
I20260709 13:05:28.963374 17621 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:28.983376 17641 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:28.974426 17653 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.806s	user 0.008s	sys 0.019s
W20260709 13:05:29.000803 17653 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.840s	user 0.008s	sys 0.019s
I20260709 13:05:29.007119 17738 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:29.001890 17673 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:28.997226 17669 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:28.998852 17670 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.012876 17670 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:28.997299 17740 log.cc:927] Time spent T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
W20260709 13:05:29.004988 17692 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.969s	user 0.009s	sys 0.016s
I20260709 13:05:29.018153 17670 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:29.020224 17719 log.cc:927] Time spent T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.098s	user 0.000s	sys 0.003s
I20260709 13:05:28.986657 17704 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:29.029691 17722 raft_consensus.cc:2749] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:05:28.985106 17724 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.858s	user 0.012s	sys 0.025s
W20260709 13:05:29.032015 17724 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.905s	user 0.012s	sys 0.025s
I20260709 13:05:28.768869 15756 catalog_manager.cc:5697] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 1 to 2, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:29.062120 15756 catalog_manager.cc:5697] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 2 to 3, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 3 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:28.836405 15827 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.715s	user 0.002s	sys 0.017s
W20260709 13:05:29.067268 15827 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.946s	user 0.003s	sys 0.018s
W20260709 13:05:29.067840 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 946660us
I20260709 13:05:29.068919 15862 raft_consensus.cc:606] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Received request to transfer leadership to TS 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:29.069913 15862 raft_consensus.cc:614] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:05:29.066248 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:29.085407 15928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 347) took 1123 ms. Trace:
I20260709 13:05:29.086162 15928 rpcz_store.cc:276] 0709 13:05:27.948431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:27.954459 (+  6028us) service_pool.cc:224] Handling call
0709 13:05:27.971593 (+ 17134us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:27.971990 (+   397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:27.972024 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:27.972523 (+   499us) log.cc:844] Serialized 26 byte log entry
0709 13:05:28.295518 (+322995us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:28.336391 (+ 40873us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:28.353344 (+ 16953us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:28.996554 (+643210us) spinlock_profiling.cc:243] Waited 9.32 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:05:29.071412 (+ 74858us) raft_consensus.cc:1685] finished
0709 13:05:29.071468 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:29.072111 (+   643us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":110,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":122,"spinlock_wait_cycles":21428480,"thread_start_us":285170,"threads_started":1,"wal-append.queue_time_us":949936}
I20260709 13:05:29.122620 16561 raft_consensus.cc:1240] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:29.125283 16561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 604) took 1309 ms. Trace:
I20260709 13:05:29.125711 16561 rpcz_store.cc:276] 0709 13:05:27.815705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:27.817634 (+  1929us) service_pool.cc:224] Handling call
0709 13:05:28.960687 (+1143053us) spinlock_profiling.cc:243] Waited 5.67 ms on lock 0x7b5800044798. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef381fc0 00007f73ef36c929 00007f73ef36c3d2 00007f73e88e3bd4
0709 13:05:28.968529 (+  7842us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:29.124807 (+156278us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:29.125186 (+   379us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13043584}
I20260709 13:05:29.122637 16570 raft_consensus.cc:1240] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:29.128463 16567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 453) took 2075 ms. Trace:
I20260709 13:05:29.128973 16567 rpcz_store.cc:276] 0709 13:05:27.048053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:27.048242 (+   189us) service_pool.cc:224] Handling call
0709 13:05:27.048671 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:27.092868 (+ 44197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:27.092912 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:27.093435 (+   523us) log.cc:844] Serialized 26 byte log entry
0709 13:05:27.825000 (+731565us) spinlock_profiling.cc:243] Waited 8.45 ms on lock 0x7b1800350200. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f480d1 00007f73ea363a0f
0709 13:05:27.938523 (+113523us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:28.962132 (+1023609us) log.cc:844] Serialized 14 byte log entry
0709 13:05:29.122494 (+160362us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:29.122707 (+   213us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:29.122786 (+    79us) raft_consensus.cc:1685] finished
0709 13:05:29.123449 (+   663us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:29.123738 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":185481,"raft.queue_time_us":35586,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":18262,"spinlock_wait_cycles":19427200,"thread_start_us":1092644,"threads_started":2,"wal-append.queue_time_us":930806}
W20260709 13:05:29.132059 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:29.069464 15827 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:28.951515 17557 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:28.952402 17532 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:29.135098 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 66006us
I20260709 13:05:29.137331 17532 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:28.968438 17577 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:29.154274 17532 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:29.137352 17557 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:29.159662 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:29.165108 16010 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:28.976548 17540 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:29.163322 17532 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:29.168578 15935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 340) took 1445 ms. Trace:
I20260709 13:05:29.161099 17551 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.169031 15935 rpcz_store.cc:276] 0709 13:05:27.723155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:27.723742 (+   587us) service_pool.cc:224] Handling call
0709 13:05:27.724005 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:29.070881 (+1346876us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:29.070949 (+    68us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:29.071025 (+    76us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:29.071085 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:29.168107 (+ 97022us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b540008a7d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:29.168176 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:29.168388 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":63774,"raft.run_cpu_time_us":310,"raft.run_wall_time_us":311,"spinlock_wait_cycles":3095114624}
I20260709 13:05:29.170329 17532 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:29.170202 17551 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:29.171023 17532 raft_consensus.cc:2749] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:29.172631 15855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 1638) took 1096 ms. Trace:
I20260709 13:05:29.173050 15855 rpcz_store.cc:276] 0709 13:05:28.076475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.100036 (+ 23561us) service_pool.cc:224] Handling call
0709 13:05:29.172252 (+1072216us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b54003b6354. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:29.172423 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":661000064}
I20260709 13:05:29.164259 17557 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.174834 17557 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:29.178426 17557 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:29.173240 17551 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:29.188728 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:29.189663 16009 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5.
I20260709 13:05:29.190040 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:29.191298 16002 raft_consensus.cc:2393] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 2: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:29.193145 17532 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:29.193992 17532 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:29.196570 17532 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:29.198068 17551 consensus_queue.cc:1059] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:28.974642 17605 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:29.234401 16010 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:29.234668 17640 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:29.199928 17551 raft_consensus.cc:3055] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:05:28.985049 17643 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.012s	sys 0.001s
I20260709 13:05:29.266633 17551 raft_consensus.cc:740] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:28.998080 17630 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:29.298770 15862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:57012 (request call id 16) took 1514 ms. Trace:
I20260709 13:05:29.299121 17722 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:29.299381 15862 rpcz_store.cc:276] 0709 13:05:27.783903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:27.784064 (+   161us) service_pool.cc:224] Handling call
0709 13:05:29.298368 (+1514304us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b540032da54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a2853 00007f73ef37060a 00007f73e88f67d4 00007f73e88f6762 00007f73e88f6691 00007f73e88f660c 00007f73e88f58c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:29.298610 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2506434944}
W20260709 13:05:29.299906 15931 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.006s	sys 0.023s
I20260709 13:05:29.300505 17627 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:29.301100 17640 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:05:29.302630 15977 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.010s	sys 0.021s
I20260709 13:05:29.303050 17551 consensus_queue.cc:260] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:29.000438 17621 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:05:29.303859 16536 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.005s	sys 0.020s
W20260709 13:05:29.305251 15901 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.003s	sys 0.009s
W20260709 13:05:29.306757 15974 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.018s	sys 0.070s
I20260709 13:05:29.307797 17653 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:05:29.309725 17736 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.031s	user 0.007s	sys 0.044s
W20260709 13:05:29.401503 15974 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.759s	user 0.018s	sys 0.077s
W20260709 13:05:29.406680 15931 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.006s	sys 0.035s
W20260709 13:05:29.402937 15901 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.711s	user 0.004s	sys 0.021s
W20260709 13:05:29.407424 15977 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.762s	user 0.012s	sys 0.034s
I20260709 13:05:29.310575 17561 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:29.311632 16562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 603) took 1499 ms. Trace:
W20260709 13:05:29.406487 16536 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.007s	sys 0.032s
I20260709 13:05:29.309854 17738 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:29.421218 16570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 507) took 2678 ms. Trace:
W20260709 13:05:29.407184 17736 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.159s	user 0.007s	sys 0.045s
I20260709 13:05:29.314606 17552 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:29.291759 17641 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:29.315392 17744 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:05:29.436476 15825 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.004s	sys 0.011s
W20260709 13:05:29.423517 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 805442us
I20260709 13:05:29.436682 17722 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:29.438614 16534 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.009s	sys 0.015s
W20260709 13:05:29.438889 16535 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.017s	sys 0.009s
W20260709 13:05:29.439175 17525 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.018s	sys 0.009s
W20260709 13:05:29.439765 15976 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.009s	sys 0.023s
I20260709 13:05:29.439986 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:29.440101 17651 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:29.440367 16533 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.026s	sys 0.011s
W20260709 13:05:29.452526 16533 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.760s	user 0.026s	sys 0.011s
W20260709 13:05:29.452692 17525 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.019s	sys 0.009s
W20260709 13:05:29.453049 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 760574us
I20260709 13:05:29.454238 17592 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:29.454775 17715 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:29.455175 17592 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:29.455832 17605 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:29.455925 17715 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:29.419250 17627 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:29.458427 17592 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:29.458945 17715 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:29.459137 17627 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:05:29.452560 15976 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.692s	user 0.010s	sys 0.023s
W20260709 13:05:29.432438 17697 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.773s	user 0.019s	sys 0.040s
W20260709 13:05:29.461304 17697 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.991s	user 0.019s	sys 0.040s
W20260709 13:05:29.463379 16535 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.751s	user 0.017s	sys 0.009s
I20260709 13:05:29.437794 17738 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:29.425683 16570 rpcz_store.cc:276] 0709 13:05:26.740791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:26.741179 (+   388us) service_pool.cc:224] Handling call
0709 13:05:26.799739 (+ 58560us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:29.127978 (+2328239us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:29.401221 (+273243us) spinlock_profiling.cc:243] Waited 2.32 s on lock 0x7b5400744854. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:29.419128 (+ 17907us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5340106112}
I20260709 13:05:29.421723 16562 rpcz_store.cc:276] 0709 13:05:27.811879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:27.815880 (+  4001us) service_pool.cc:224] Handling call
0709 13:05:27.850353 (+ 34473us) spinlock_profiling.cc:243] Waited 9.44 ms on lock 0x7b5800044798. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283eee 0000000000375939 00007f73ef2f71e0 00007f73ef3c00c4 00007f73ef3c0131 00007f73ef381fc0 00007f73ef36c929 00007f73ef36c3d2 00007f73e88e3bd4
0709 13:05:27.850547 (+   194us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:28.803496 (+952949us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:29.310973 (+507477us) spinlock_profiling.cc:243] Waited 795 ms on lock 0x7b54000cdad4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:29.311413 (+   440us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1850715264}
I20260709 13:05:29.450285 17640 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:29.464112 17621 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:29.464797 16534 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.751s	user 0.009s	sys 0.015s
I20260709 13:05:29.464897 16009 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
W20260709 13:05:29.466349 15902 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 1.596s (SENT) [suppressed 13 similar messages]
I20260709 13:05:29.466732 17735 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:29.467144 17755 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:29.467520 17605 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.440837 17653 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:29.467945 17763 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:29.468333 17761 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:05:29.494982 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:29.518639 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:29.502008 15860 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.075s	user 0.007s	sys 0.020s
W20260709 13:05:29.517765 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 63482.1us
W20260709 13:05:29.528690 15860 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.321s	user 0.008s	sys 0.021s
I20260709 13:05:29.527447 17651 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.555368 17746 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:29.556108 17605 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:29.556469 17746 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:29.559330 17605 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:29.549193 17744 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:29.547472 17621 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:29.555099 17735 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:05:29.670750 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 122273us
I20260709 13:05:29.576114 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:05:29.587318 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 121143us
I20260709 13:05:29.585191 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:29.571133 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:29.584465 15858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 349) took 1425 ms. Trace:
I20260709 13:05:29.664582 15931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1539) took 1061 ms. Trace:
I20260709 13:05:29.529748 17641 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:29.665252 17775 log.cc:927] Time spent T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 13:05:29.545359 17712 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:29.693557 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:29.698712 17621 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:29.698963 17628 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:29.699877 17628 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:29.700940 17735 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:29.701220 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:29.702122 16005 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:29.703171 17641 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:29.703594 17633 consensus_queue.cc:1048] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:29.547269 17755 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:29.707957 17525 consensus_queue.cc:1048] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:29.708931 17758 consensus_queue.cc:1048] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:29.709149 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:29.710299 17755 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:29.710901 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:29.711589 17722 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:29.711735 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:29.708940 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:29.712379 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:29.712742 16569 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:29.712921 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:29.713697 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:29.710294 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:29.714217 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:29.714964 17628 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:29.713114 16566 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
I20260709 13:05:29.715804 16565 raft_consensus.cc:1275] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:29.567052 17720 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:05:29.715850 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:29.574066 17724 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:29.584926 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:05:29.578184 17774 log.cc:927] Time spent T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.185s	user 0.002s	sys 0.000s
I20260709 13:05:29.576511 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:29.703120 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:29.561151 15974 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:29.666682 16002 raft_consensus.cc:1275] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:29.714356 16569 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:29.563835 17551 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:29.718931 16010 raft_consensus.cc:1275] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:29.559643 17746 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:29.576126 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:29.572520 17584 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.585427 16004 raft_consensus.cc:1275] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:29.584221 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:29.586562 17653 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:29.577414 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:29.578840 17738 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:29.733412 16007 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:05:29.734249 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:29.726429 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:29.735006 16006 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:29.572144 17700 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.734786 17584 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:29.735348 16005 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:29.736477 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:29.738250 17769 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:29.739107 16005 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:05:29.739532 17755 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:29.583890 17627 raft_consensus.cc:2955] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:29.740669 17746 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:29.741392 17746 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:29.742537 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:29.743212 16005 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
I20260709 13:05:29.734386 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:29.744011 17615 consensus_queue.cc:1048] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:29.744591 16007 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:05:29.728047 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:29.745751 17615 consensus_queue.cc:1048] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:29.746207 16010 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
I20260709 13:05:29.747195 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:29.744616 16006 raft_consensus.cc:1275] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:05:29.748086 16010 raft_consensus.cc:2410] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:05:29.748275 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:29.729396 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:29.737275 17584 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:29.738219 17736 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:29.756104 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:29.739231 17769 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:29.756680 17736 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:29.728837 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:29.548024 17763 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:29.736557 17700 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:29.685516 15931 rpcz_store.cc:276] 0709 13:05:28.602862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.608460 (+  5598us) service_pool.cc:224] Handling call
0709 13:05:28.636110 (+ 27650us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:28.636323 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:05:29.433025 (+796702us) log.cc:844] Serialized 14 byte log entry
0709 13:05:29.547885 (+114860us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:29.587191 (+ 39306us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:29.622719 (+ 35528us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:29.622835 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:29.664513 (+ 41678us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":521907,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":69,"raft.queue_time_us":11856,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":29043,"thread_start_us":797400,"threads_started":2,"wal-append.queue_time_us":139158}
I20260709 13:05:29.586308 17744 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:29.748952 16003 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2.
I20260709 13:05:29.751868 16009 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
I20260709 13:05:29.567432 17651 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:29.740442 15825 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:29.716972 17712 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:05:29.774995 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 213975us
I20260709 13:05:29.560539 17761 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:05:30.134485 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:29.803686 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:05:30.144055 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:30.010067 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:05:29.936542 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:29.685228 15858 rpcz_store.cc:276] 0709 13:05:28.158427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.365732 (+207305us) service_pool.cc:224] Handling call
0709 13:05:28.366143 (+   411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:29.307847 (+941704us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:29.307892 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:29.308573 (+   681us) log.cc:844] Serialized 26 byte log entry
0709 13:05:29.426942 (+118369us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:29.426992 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:29.582957 (+155965us) spinlock_profiling.cc:243] Waited 452 ms on lock 0x7b540008d754. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:29.583009 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:29.583077 (+    68us) raft_consensus.cc:1685] finished
0709 13:05:29.583093 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:29.584256 (+  1163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":110444,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":2862,"spinlock_wait_cycles":1039133056,"thread_start_us":606997,"threads_started":2,"wal-append.queue_time_us":258431}
I20260709 13:05:29.962179 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:29.961098 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:29.759248 17769 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:29.730710 17722 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:29.817452 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:05:29.765643 17772 log.cc:927] Time spent T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.072s	user 0.004s	sys 0.000s
I20260709 13:05:29.959260 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.162083 16567 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
I20260709 13:05:29.764869 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:30.165047 17744 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:29.734280 17643 raft_consensus.cc:476] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Not starting pre-election -- already a leader
I20260709 13:05:29.803543 17641 raft_consensus.cc:2804] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:30.171865 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.176509 15861 raft_consensus.cc:2410] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 2 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:05:29.988960 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.172427 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:30.177886 16565 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2.
I20260709 13:05:29.739250 17653 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.759590 17736 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:29.836244 17715 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:29.961423 17765 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:29.760002 17767 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.171073 15936 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:30.180694 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:29.799141 16562 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 2.
I20260709 13:05:30.150187 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:30.010094 17746 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:30.156230 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:30.156136 17748 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:05:30.190347 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 449957us
I20260709 13:05:30.194451 16564 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:05:30.194761 17700 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:30.195013 17767 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:30.204396 16009 raft_consensus.cc:1275] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:30.217095 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:30.183501 15857 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2.
I20260709 13:05:30.213936 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:29.836743 17724 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:30.209172 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:05:30.215790 15860 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.022s	sys 0.016s
W20260709 13:05:30.223716 15860 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.022s	sys 0.016s
I20260709 13:05:29.924758 17688 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:30.179867 17641 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:29.940575 17755 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:29.830657 17720 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:30.226245 17643 raft_consensus.cc:2804] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:30.235402 15862 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:05:30.243290 16010 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2.
I20260709 13:05:30.235575 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.245021 16568 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
I20260709 13:05:30.245021 16567 raft_consensus.cc:1275] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:30.181298 17769 consensus_queue.cc:1048] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
W20260709 13:05:30.246810 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 63659.4us
I20260709 13:05:30.274116 16009 raft_consensus.cc:1275] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:05:30.276499 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.239241 16002 raft_consensus.cc:1275] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:05:30.271538 16533 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.022s	sys 0.049s
I20260709 13:05:30.218384 17715 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:30.221175 17744 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:30.283851 17697 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:30.284957 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.285733 17761 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:30.285630 17697 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:30.285986 15932 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
I20260709 13:05:30.245280 17641 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:30.277256 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:30.282742 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:30.264901 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:30.247990 17643 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:30.246701 17763 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:30.238440 17700 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:30.235456 17722 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
W20260709 13:05:30.292526 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:30.280798 16533 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.023s	sys 0.049s
I20260709 13:05:30.296109 16009 raft_consensus.cc:1275] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:30.296478 16565 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2.
I20260709 13:05:30.294106 15860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 348) took 2135 ms. Trace:
I20260709 13:05:30.299304 15861 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
W20260709 13:05:30.302042 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 595445us
I20260709 13:05:30.284264 17653 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:30.294885 17643 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:30.304123 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:30.307960 16010 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:05:30.275470 17748 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:30.297570 17712 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:30.331965 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:30.271725 17765 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:30.300408 15860 rpcz_store.cc:276] 0709 13:05:28.158192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.206745 (+ 48553us) service_pool.cc:224] Handling call
0709 13:05:28.207093 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:28.207312 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:05:29.544768 (+1337456us) log.cc:844] Serialized 14 byte log entry
0709 13:05:30.239946 (+695178us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:30.244775 (+  4829us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:30.276998 (+ 32223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:30.283272 (+  6274us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:30.294065 (+ 10793us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1039183,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":96,"raft.queue_time_us":538,"raft.run_cpu_time_us":289,"raft.run_wall_time_us":32789,"thread_start_us":1877875,"threads_started":2}
W20260709 13:05:30.354054 17630 consensus_meta.cc:287] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.893s	user 0.008s	sys 0.007s
I20260709 13:05:30.354821 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:30.355541 15853 raft_consensus.cc:2410] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 2 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:05:30.298511 17746 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:05:30.369211 16005 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.049s	sys 0.060s
I20260709 13:05:30.292047 17688 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:30.374025 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.374706 15861 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:05:30.371503 15856 raft_consensus.cc:2376] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2. Re-sending same reply.
I20260709 13:05:30.367347 17712 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.377897 15858 raft_consensus.cc:1275] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:05:30.371840 16005 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.050s	sys 0.060s
W20260709 13:05:30.369493 17777 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.021s	sys 0.067s
I20260709 13:05:30.370998 17536 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.374897 17651 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:30.372398 17767 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:30.381053 17777 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.022s	sys 0.068s
I20260709 13:05:30.366918 17700 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:30.392498 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.393232 15861 raft_consensus.cc:2410] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:05:30.395869 15974 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:30.403544 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.390592 17651 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:30.386410 17712 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:30.400942 17755 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:30.414448 15860 raft_consensus.cc:1275] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:30.397049 17746 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:30.401372 17536 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:30.420257 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.412986 17763 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:05:30.416993 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 483932us
I20260709 13:05:30.413728 17724 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:30.417308 17744 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:30.423064 17720 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:30.423101 17763 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:30.426714 17627 consensus_meta.cc:287] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.686s	user 0.006s	sys 0.006s
I20260709 13:05:30.427579 17722 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:30.428872 17763 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:30.426498 17651 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:30.434933 17786 log.cc:927] Time spent T 00000000000000000000000000000000 P d33cebb189b14a989836214fdbafb898: Append to log took a long time: real 0.052s	user 0.000s	sys 0.002s
I20260709 13:05:30.440383 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.434718 17653 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:30.442035 15858 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2.
I20260709 13:05:30.440466 15902 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:30.445339 17788 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:30.445768 17788 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:30.446169 17788 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:30.446622 15902 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:30.447575 17720 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:30.451200 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:30.443989 17748 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:30.461365 15902 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:30.450634 17720 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:30.452195 17758 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.453055 16569 raft_consensus.cc:2410] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:05:30.453589 17712 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:30.457058 17715 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:30.457774 17643 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:30.459280 15859 raft_consensus.cc:1275] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:30.460839 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.461839 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.476291 17712 consensus_queue.cc:1048] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:30.476712 15860 raft_consensus.cc:2393] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 2: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:30.476424 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.480050 15861 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2.
I20260709 13:05:30.484331 17755 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:30.484831 17755 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:30.485157 17755 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:30.488339 15855 raft_consensus.cc:2393] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 3: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:30.489234 17643 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:30.489161 17788 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:30.491360 17788 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:30.495425 17758 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:30.496411 17651 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:30.466681 17720 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:30.497691 17758 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:30.465698 17700 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:30.503654 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 88171.7us
I20260709 13:05:30.504140 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:30.507157 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:30.508217 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:05:30.499603 17759 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.793s	user 0.011s	sys 0.023s
I20260709 13:05:30.508857 15858 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 1.
I20260709 13:05:30.509349 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.509922 15861 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 4.
I20260709 13:05:30.508311 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:30.511826 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.512920 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:30.513077 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.513772 15861 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:30.515380 17755 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:30.516297 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:30.517604 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.518257 15932 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 1.
W20260709 13:05:30.519197 15853 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 1637) took 2443 ms (client timeout 1810 ms). Trace:
I20260709 13:05:30.519573 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:05:30.509438 17759 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.803s	user 0.011s	sys 0.023s
W20260709 13:05:30.519634 15853 rpcz_store.cc:269] 0709 13:05:28.076112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.077989 (+  1877us) service_pool.cc:224] Handling call
0709 13:05:30.518911 (+2440922us) spinlock_profiling.cc:243] Waited 2.21 s on lock 0x7b54003886d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:30.519116 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5087307008}
I20260709 13:05:30.525050 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:30.525732 16564 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:30.520488 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.528092 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:30.520802 17788 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.529455 17788 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:30.529798 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.522114 17746 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:30.522248 17700 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:05:30.531719 17782 log.cc:927] Time spent T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.811s	user 0.011s	sys 0.022s
I20260709 13:05:30.532184 17700 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:30.532720 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.533659 15931 raft_consensus.cc:2393] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 4: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:30.534348 17700 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:30.536669 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.520268 15932 raft_consensus.cc:2393] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 3: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:30.542400 16564 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
W20260709 13:05:30.544193 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:30.546243 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.546411 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.546928 15859 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:05:30.547256 15934 raft_consensus.cc:2410] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:05:30.547834 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.548161 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.545365 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.549124 15934 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:30.550783 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.548111 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.552147 15933 raft_consensus.cc:1240] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:30.552414 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.553093 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.536669 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.558701 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.560925 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:05:30.561540 17781 log.cc:927] Time spent T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.865s	user 0.005s	sys 0.018s
I20260709 13:05:30.561767 15854 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:30.520669 17651 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:30.559840 15934 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:05:30.564957 15859 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:30.519225 17763 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:30.567305 15861 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:30.556915 17755 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:30.588941 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:30.660735 15902 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:30.584868 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.593881 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:30.640666 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.653214 15976 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:30.663033 16009 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:05:30.666906 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.667766 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:05:30.594805 17551 consensus_meta.cc:287] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.868s	user 0.009s	sys 0.002s
I20260709 13:05:30.668135 15932 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:30.587956 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.670609 16005 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:30.580353 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.591971 15928 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 2.
I20260709 13:05:30.676277 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:30.660305 17700 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:30.678750 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.679565 15928 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
I20260709 13:05:30.679308 17700 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:30.680486 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:30.680609 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.681128 16564 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:30.681675 17700 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:30.681978 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.650918 17651 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:30.683187 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.683882 15934 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 2.
I20260709 13:05:30.655282 17763 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:30.684092 15930 raft_consensus.cc:2410] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 7 because replica has last-logged OpId of term: 6 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 8.
W20260709 13:05:30.687106 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:30.688829 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:30.690196 16009 raft_consensus.cc:2410] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:05:30.594923 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:30.689414 16010 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:05:30.684137 17651 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:30.693378 17627 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:30.703042 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.703306 17627 raft_consensus.cc:697] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:30.703990 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.704073 15976 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:30.704422 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.704972 15937 raft_consensus.cc:2410] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:05:30.705040 17744 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:30.705567 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.705842 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.706278 15932 raft_consensus.cc:2393] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 4: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:30.706429 15937 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 1.
I20260709 13:05:30.706911 15976 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:30.707379 15932 raft_consensus.cc:1275] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:30.707628 17744 raft_consensus.cc:2749] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:30.708801 15902 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:30.708999 16004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 297) took 1245 ms. Trace:
I20260709 13:05:30.709406 15976 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:30.702920 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.716704 15934 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:30.726459 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.727298 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:30.727463 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.694566 15928 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:30.730362 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:30.733239 15825 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:05:30.733970 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:30.731192 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.734380 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:30.735126 16010 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:05:30.739482 16009 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:05:30.736609 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.730937 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.732965 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.732744 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.724808 17551 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:05:30.732879 15974 connection.cc:256] RPC call timeout handler was delayed by -1.6447273669999731s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1597s for a response
I20260709 13:05:30.726197 17722 raft_consensus.cc:2749] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:30.732734 17744 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.720218 16004 rpcz_store.cc:276] 0709 13:05:29.463556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:29.463869 (+   313us) service_pool.cc:224] Handling call
0709 13:05:29.464450 (+   581us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:29.732971 (+268521us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:30.708724 (+975753us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b54003edd54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:30.708913 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":276916480}
I20260709 13:05:30.730991 17641 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:30.704448 17627 consensus_queue.cc:237] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:30.756348 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:30.758502 15855 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2.
I20260709 13:05:30.764222 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:30.756071 17796 log.cc:927] Time spent T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:05:30.740165 17759 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:30.730486 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.759099 15934 raft_consensus.cc:2393] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 3: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:30.732635 17763 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:30.761866 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.752120 15857 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
I20260709 13:05:30.759949 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:30.732348 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.780095 15928 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 2.
W20260709 13:05:30.781154 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 54516.4us
I20260709 13:05:30.781265 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.763128 17755 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:30.756071 17551 raft_consensus.cc:697] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:30.785274 17551 consensus_queue.cc:237] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:30.790881 16005 raft_consensus.cc:1275] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:30.750869 16564 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:30.748427 17643 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:30.771327 17759 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:30.771787 17763 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:30.772818 17641 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:30.779217 15860 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:30.783906 16003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1629) took 1032 ms. Trace:
I20260709 13:05:30.757802 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.795734 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.795334 16003 rpcz_store.cc:276] 0709 13:05:29.768214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.192967 (+424753us) service_pool.cc:224] Handling call
0709 13:05:30.227503 (+ 34536us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:30.311874 (+ 84371us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:30.349538 (+ 37664us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:30.352359 (+  2821us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:30.352400 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:30.783666 (+431266us) spinlock_profiling.cc:243] Waited 81.5 ms on lock 0x7b54003ee754. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:30.783744 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:30.783859 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":187432192}
I20260709 13:05:30.796753 15858 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
I20260709 13:05:30.799957 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:30.800587 16566 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:30.801206 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.801970 15857 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
I20260709 13:05:30.802747 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.804250 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.804858 15934 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 1.
I20260709 13:05:30.807996 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.805006 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:30.809479 16566 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 2.
I20260709 13:05:30.810905 15853 raft_consensus.cc:2410] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:05:30.811098 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.811662 15934 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 1.
I20260709 13:05:30.792296 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:30.812644 16002 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2.
I20260709 13:05:30.792820 17720 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:30.814175 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.815420 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:30.808215 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.815985 15853 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:30.816206 15928 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
I20260709 13:05:30.816939 17759 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:30.817606 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:30.811614 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.818814 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:30.819453 16570 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 4.
I20260709 13:05:30.819748 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:30.820350 15855 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:30.821064 15931 raft_consensus.cc:2393] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 3: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:30.826293 17722 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:30.815529 17720 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:30.826857 17722 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:30.827847 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.830955 15932 raft_consensus.cc:1240] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:30.828931 17722 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:05:30.833173 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 77531.2us
I20260709 13:05:30.834237 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:30.834444 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.795101 17643 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:30.838275 15825 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:30.840097 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.843767 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.847218 15827 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:30.834039 15974 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:30.842371 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.793649 17744 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:30.855818 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:30.857299 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.842603 17643 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:30.869645 15858 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:05:30.891346 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:05:30.893148 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 59175.3us
I20260709 13:05:30.897544 15855 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:30.898626 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.899647 15974 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:30.899952 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:30.854841 17748 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:30.838413 17756 consensus_queue.cc:1048] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:05:30.899694 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.902787 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.903381 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:30.863940 17627 raft_consensus.cc:2749] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:30.914641 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.915307 15932 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 2.
I20260709 13:05:30.915496 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.916903 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.900943 17653 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:30.917636 15857 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 2.
I20260709 13:05:30.902390 17748 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:30.916828 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.922065 15858 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2.
I20260709 13:05:30.924350 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:30.925052 16570 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 2.
I20260709 13:05:30.926041 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:30.913165 17641 raft_consensus.cc:2804] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:30.926771 15827 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:30.929555 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:30.930150 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:30.931252 16011 raft_consensus.cc:1240] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:30.935814 16011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1624) took 1189 ms. Trace:
I20260709 13:05:30.915408 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:30.924714 15974 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:30.936342 16011 rpcz_store.cc:276] 0709 13:05:29.746255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:29.760165 (+ 13910us) service_pool.cc:224] Handling call
0709 13:05:30.923703 (+1163538us) spinlock_profiling.cc:243] Waited 9.02 ms on lock 0x7b5800114398. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef381fc0 00007f73ef36c929 00007f73ef36c3d2 00007f73e88e3bd4
0709 13:05:30.925465 (+  1762us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:30.935467 (+ 10002us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:30.935738 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20736256}
I20260709 13:05:30.915982 16002 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
I20260709 13:05:30.938957 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:30.928462 17641 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:30.941032 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:30.924335 17748 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:30.950912 15974 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:30.952657 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:30.952950 17765 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:30.947877 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 96500.2us
I20260709 13:05:30.921509 17653 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:30.931529 15853 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:30.956890 17813 raft_consensus.cc:2764] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:05:30.958946 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:05:30.959069 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 45337.6us
I20260709 13:05:30.953896 15935 raft_consensus.cc:2955] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:30.948770 17811 consensus_queue.cc:1048] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:05:30.948263 16005 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:30.953819 17641 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:30.960628 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:31.017339 15932 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 2.
I20260709 13:05:30.984772 15857 raft_consensus.cc:1240] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:30.984076 16008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 299) took 1313 ms. Trace:
W20260709 13:05:31.016533 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 70300.6us
I20260709 13:05:30.953542 17627 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:31.019382 16008 rpcz_store.cc:276] 0709 13:05:29.586833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:29.667782 (+ 80949us) service_pool.cc:224] Handling call
0709 13:05:29.668088 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:29.836352 (+168264us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:29.836403 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:29.836471 (+    68us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:29.836520 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:30.900285 (+1063765us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b54000eddd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:30.900330 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:30.900435 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":375013632}
I20260709 13:05:31.021152 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:31.022841 15860 raft_consensus.cc:1275] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:05:30.985164 17801 log.cc:927] Time spent T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:05:30.956360 17653 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:31.023866 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 119224us
I20260709 13:05:31.028247 15974 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:31.030661 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:31.032696 15932 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 2.
I20260709 13:05:31.038012 15827 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:31.037272 17551 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.985169 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:31.016040 17722 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:05:31.054548 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 97233.5us
I20260709 13:05:30.985891 17744 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:31.049638 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:31.031852 16535 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:31.050451 15825 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:31.052793 15974 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:31.026722 17627 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:31.060866 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:05:31.069393 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 52642.3us
I20260709 13:05:31.048991 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:31.072760 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:31.053944 17813 raft_consensus.cc:2764] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:05:31.072852 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:31.055011 17722 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:31.075240 16565 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:31.071560 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:31.079283 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:31.079720 17818 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:31.080358 17818 raft_consensus.cc:697] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:31.081642 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:31.081395 17818 consensus_queue.cc:237] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:31.082947 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:31.082712 17653 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:31.079813 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:31.084265 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:31.085114 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:31.085425 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:31.086679 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:31.087791 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:31.068722 17627 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:31.066349 17744 raft_consensus.cc:2749] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:31.091924 17688 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:31.092175 17812 log.cc:927] Time spent T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 13:05:31.093624 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:31.076951 16008 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:31.058099 15929 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:31.096678 15825 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:31.117795 17688 raft_consensus.cc:1064] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:05:31.118026 17813 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:31.118429 17813 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:31.118789 17813 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:31.120612 17688 consensus_queue.cc:237] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:31.104075 17806 log.cc:927] Time spent T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
I20260709 13:05:31.106917 16565 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:05:31.107501 17715 consensus_queue.cc:1048] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:31.118425 15825 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:31.123819 15860 raft_consensus.cc:2955] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:31.108224 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:31.125281 17746 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:31.125696 17746 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:31.126034 17746 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:31.126292 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:05:31.127310 15900 leader_election.cc:336] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.896s (SENT)
I20260709 13:05:31.089309 17722 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:31.072700 17628 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:31.109150 17720 consensus_queue.cc:1048] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:31.110543 17802 consensus_queue.cc:1048] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:31.111498 17651 consensus_queue.cc:1048] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:31.089162 16002 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:31.154790 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:31.155476 16002 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:31.156528 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:31.157625 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:31.157846 17627 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:31.124689 15825 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:31.124747 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:31.054432 17551 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:31.127912 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:31.128798 17788 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:31.129235 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:31.131392 15976 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:31.131457 17641 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:05:31.132153 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:31.132719 16004 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:05:31.134409 16007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1626) took 1387 ms. Trace:
W20260709 13:05:31.169996 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 117257us
I20260709 13:05:31.135211 15862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 1504) took 1175 ms. Trace:
I20260709 13:05:31.137913 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:31.068939 17822 raft_consensus.cc:2804] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:05:31.139348 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 67063.2us
I20260709 13:05:31.159756 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:31.139976 15900 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:31.112408 15902 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:05:31.193321 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 80999.7us
I20260709 13:05:31.194016 15902 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:31.151161 17628 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:31.163316 17551 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:31.167899 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:31.196597 16006 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:05:31.197248 16006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41662 (request call id 1623) took 1451 ms. Trace:
I20260709 13:05:31.166615 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:05:31.171820 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 81900.4us
W20260709 13:05:31.197757 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 70469.1us
I20260709 13:05:31.198601 15976 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:31.172966 15862 rpcz_store.cc:276] 0709 13:05:29.960520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:29.960721 (+   201us) service_pool.cc:224] Handling call
0709 13:05:31.135034 (+1174313us) spinlock_profiling.cc:243] Waited 72.1 ms on lock 0x7b540008b1d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:31.135157 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":165871488}
W20260709 13:05:31.173843 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:31.212370 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:31.177515 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:31.179297 15974 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:31.171340 16007 rpcz_store.cc:276] 0709 13:05:29.746879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:29.837398 (+ 90519us) service_pool.cc:224] Handling call
0709 13:05:30.787288 (+949890us) spinlock_profiling.cc:243] Waited 9.12 ms on lock 0x7b5800114398. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef381fc0 00007f73ef36c929 00007f73ef36c3d2 00007f73e88e3bd4
0709 13:05:31.133581 (+346293us) spinlock_profiling.cc:243] Waited 5.62 ms on lock 0x7b5800114398. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283e9c 00007f73e7283a5b 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef381fc0 00007f73ef36c929 00007f73ef36c3d2 00007f73e88e3bd4
0709 13:05:31.133771 (+   190us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:31.134006 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:31.134032 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:31.134153 (+   121us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:31.134187 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:31.134243 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:31.134351 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33896192}
I20260709 13:05:31.108407 15827 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:31.188946 17831 consensus_queue.cc:1059] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:31.117605 17748 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
W20260709 13:05:31.226678 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:31.167433 17630 raft_consensus.cc:2804] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:31.197646 16006 rpcz_store.cc:276] 0709 13:05:29.745941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:29.748588 (+  2647us) service_pool.cc:224] Handling call
0709 13:05:31.167779 (+1419191us) spinlock_profiling.cc:243] Waited 391 ms on lock 0x7b5800114398. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283e9c 00007f73e7283a5b 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:05:31.197153 (+ 29374us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":899299072}
W20260709 13:05:31.239732 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 71130.3us
I20260709 13:05:31.241688 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:31.242370 16007 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2.
I20260709 13:05:31.181447 17744 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:05:31.245581 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 51528.3us
I20260709 13:05:31.246676 15902 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:31.225595 17720 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:31.225955 17628 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:31.182185 17822 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:31.250577 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:31.240934 17630 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:31.258121 15900 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:05:31.259117 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 150784us
I20260709 13:05:31.259986 15827 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:31.261607 15902 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:31.243017 17651 raft_consensus.cc:2749] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:31.266907 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:31.267596 16007 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:31.248783 17744 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:31.255353 17769 raft_consensus.cc:2749] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:31.257687 17715 raft_consensus.cc:2804] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:31.254005 17720 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:31.269147 17630 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:31.258931 17822 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:31.260905 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:31.302732 17833 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:31.302927 17835 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:31.274896 17744 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:31.310585 17697 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:31.257992 17813 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:31.348254 17835 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:31.353842 17813 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:31.353976 17835 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:31.317338 17826 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:31.354921 17826 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:31.272073 17700 consensus_queue.cc:1048] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 13:05:31.271178 17712 consensus_queue.cc:1048] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 13:05:31.360971 17763 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:31.299347 16008 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af"
dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
mode: GRACEFUL
new_leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:37262
I20260709 13:05:31.300539 17651 raft_consensus.cc:2749] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:05:31.318552 17834 log.cc:927] Time spent T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:05:31.321750 17748 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:31.328400 17653 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:31.299520 17715 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:31.270161 17803 raft_consensus.cc:2804] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:31.329438 17767 raft_consensus.cc:2749] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:05:31.336206 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 76339.4us
I20260709 13:05:31.340775 16002 raft_consensus.cc:1275] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:31.345322 17641 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:05:31.350455 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 54703.9us
I20260709 13:05:31.255280 17643 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:31.349697 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 170447us
W20260709 13:05:31.366843 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 122578us
I20260709 13:05:31.305933 17720 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:31.355432 17826 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:31.375298 15827 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
W20260709 13:05:31.383916 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 123771us
I20260709 13:05:31.430972 17653 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:31.431530 17653 raft_consensus.cc:697] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:31.431612 17651 consensus_queue.cc:1048] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:31.432619 17653 consensus_queue.cc:237] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:31.349371 17833 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:31.452421 15855 raft_consensus.cc:2393] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 3: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:31.374151 17715 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:31.457660 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:31.484414 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 105916us
I20260709 13:05:31.373925 17803 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:31.371904 16004 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:31.495036 16533 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:05:31.496237 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 244705us
I20260709 13:05:31.492560 17633 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:05:31.521500 16568 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45246 (request call id 521) took 1809 ms (client timeout 1509 ms). Trace:
I20260709 13:05:31.523910 17633 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:31.524093 16568 rpcz_store.cc:269] 0709 13:05:29.706509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:29.711329 (+  4820us) service_pool.cc:224] Handling call
0709 13:05:31.515987 (+1804658us) spinlock_profiling.cc:243] Waited 533 ms on lock 0x7b5400695ad4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:31.516158 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1224952064}
I20260709 13:05:31.512003 17643 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:31.492931 17803 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:31.526099 17633 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:31.526273 15825 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:31.514998 15976 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:05:31.512158 17825 log.cc:927] Time spent T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.196s	user 0.004s	sys 0.000s
I20260709 13:05:31.527431 17842 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:31.453459 17833 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:31.529783 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:31.530460 15931 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:31.512594 17755 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:31.532109 17755 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:31.532670 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:31.533268 15931 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:31.536132 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:05:31.518630 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 87526.6us
I20260709 13:05:31.545387 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:31.546170 15932 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:31.547477 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:05:31.547808 17838 log.cc:927] Time spent T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.117s	user 0.016s	sys 0.000s
I20260709 13:05:31.551026 15825 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:31.453250 17627 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:31.552067 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:05:31.553536 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 178343us
I20260709 13:05:31.558713 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:31.559350 15930 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:05:31.561991 17536 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:31.567878 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 72991us
I20260709 13:05:31.567960 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:31.568706 15936 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:31.556780 15900 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:31.569590 16533 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:31.577617 15932 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:05:31.577112 15825 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:05:31.580808 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 150788us
I20260709 13:05:31.569970 17628 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:31.587980 17633 consensus_queue.cc:1048] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:31.588363 17722 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:31.589855 17628 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:31.590605 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:31.590683 17722 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:31.589572 17720 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:31.600884 15934 raft_consensus.cc:1240] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:31.601271 17720 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:31.601605 17692 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:31.601960 17720 raft_consensus.cc:2749] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:31.593851 15827 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:31.602944 17748 raft_consensus.cc:2804] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:31.556414 17842 raft_consensus.cc:2804] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:31.603318 17748 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:31.603199 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:31.603814 17842 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:31.603961 17748 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:31.604220 15862 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:31.604521 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:31.592474 16569 raft_consensus.cc:2393] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 3: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:31.605805 17784 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:31.606251 17784 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:31.606321 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:31.493546 17746 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:31.612131 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:31.591235 15935 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:31.606729 17784 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:31.627300 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 70591.6us
I20260709 13:05:31.602006 17692 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:31.618944 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:05:31.628057 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 374478us
I20260709 13:05:31.630599 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:31.633425 15862 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:31.632671 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:31.630189 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:31.635010 15859 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 2.
I20260709 13:05:31.613076 17746 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:31.636969 15857 raft_consensus.cc:2410] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 2 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:05:31.637396 17692 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:31.648150 17843 log.cc:927] Time spent T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:05:31.637331 15902 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:31.637286 17803 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:31.629976 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:31.631016 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:31.628367 17836 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:05:31.650532 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 56786us
I20260709 13:05:31.607369 16533 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:31.651734 15862 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
I20260709 13:05:31.652314 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:05:31.652472 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 137582us
I20260709 13:05:31.650972 15931 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
I20260709 13:05:31.604241 17842 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:31.657290 15857 raft_consensus.cc:2393] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 4: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:31.627576 17536 raft_consensus.cc:2804] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:31.658953 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:31.659942 15859 raft_consensus.cc:2393] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 4: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:05:31.661417 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 54131.4us
I20260709 13:05:31.647132 17822 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:31.661643 15976 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:31.664601 15827 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:31.663555 16533 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:31.667243 15902 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:31.660980 17818 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:31.670151 17724 raft_consensus.cc:2804] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:31.670056 17818 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:31.670601 17724 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:31.671005 17724 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:31.669463 17651 raft_consensus.cc:2749] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:31.672204 17818 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:31.672958 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:31.657157 15928 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:31.655012 17836 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:31.668954 15902 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:31.662585 17627 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:31.676142 15856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 1514) took 1410 ms. Trace:
I20260709 13:05:31.680841 15900 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:31.675843 17769 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:31.699038 15974 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:31.683037 15856 rpcz_store.cc:276] 0709 13:05:30.278309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.282530 (+  4221us) service_pool.cc:224] Handling call
0709 13:05:31.675979 (+1393449us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b540008d254. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:31.676090 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29995392}
I20260709 13:05:31.704494 15902 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:31.702714 17769 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:31.706467 15827 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:31.709172 17627 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:31.700208 17848 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:05:31.734948 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:31.682936 17836 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:31.780098 17837 raft_consensus.cc:2804] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:31.780534 17837 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:31.780952 17837 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:31.702229 17628 raft_consensus.cc:2764] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
W20260709 13:05:31.779582 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 97895.1us
I20260709 13:05:31.782335 15974 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:31.699434 17746 raft_consensus.cc:2764] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:05:31.745796 17769 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:31.748718 17627 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:31.749423 17744 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:31.788676 16006 raft_consensus.cc:2393] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 3: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:31.789060 16008 raft_consensus.cc:606] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Received request to transfer leadership to TS eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:31.789404 16008 raft_consensus.cc:614] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:31.789522 17744 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:31.751034 17692 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:31.751785 17826 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:31.784969 17722 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:31.792816 17826 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:31.792836 17722 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:31.793670 17722 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:31.755084 17755 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:31.730723 17846 raft_consensus.cc:2764] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:05:31.797317 17627 raft_consensus.cc:2804] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:31.797814 17627 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:31.798177 17627 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:31.798803 17692 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:31.759416 17803 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:31.763778 17536 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:31.764478 17643 raft_consensus.cc:2804] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:31.839634 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:31.778245 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 73830.2us
I20260709 13:05:31.768728 17765 raft_consensus.cc:3055] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:31.841780 17765 raft_consensus.cc:740] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:31.842166 15902 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:31.842706 17765 consensus_queue.cc:260] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:05:31.779140 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 115691us
I20260709 13:05:31.845824 16533 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:31.846694 17759 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:31.847150 16533 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:31.847986 17712 raft_consensus.cc:2804] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:31.847916 17846 consensus_queue.cc:1048] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:31.848449 17712 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:31.848868 17712 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:31.782138 17651 raft_consensus.cc:2749] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:05:31.791020 17844 log.cc:927] Time spent T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
W20260709 13:05:31.779829 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 99074.1us
I20260709 13:05:31.807008 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:31.898671 15930 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:31.825604 15937 raft_consensus.cc:1240] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:31.799942 15974 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:31.797044 15928 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
I20260709 13:05:31.790477 17842 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:31.838188 15825 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:05:31.838194 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 764408us
W20260709 13:05:31.906199 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 54013.1us
I20260709 13:05:31.839861 17643 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:31.844169 17765 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:31.844379 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:31.745242 15827 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:31.901919 15855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 1534) took 1048 ms. Trace:
W20260709 13:05:31.906318 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 68211.9us
I20260709 13:05:31.917423 15855 rpcz_store.cc:276] 0709 13:05:30.811883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.899742 (+ 87859us) service_pool.cc:224] Handling call
0709 13:05:31.860146 (+960404us) spinlock_profiling.cc:243] Waited 551 ms on lock 0x7b540032e454. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:31.860345 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1268338560}
I20260709 13:05:31.907122 15902 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:31.909047 17746 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:31.919353 15902 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:31.852406 17536 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:31.916571 16009 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:05:31.925215 16009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41662 (request call id 1645) took 1334 ms. Trace:
I20260709 13:05:31.917930 15825 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:31.931711 15825 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:05:31.960012 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 50015.6us
I20260709 13:05:31.959797 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:31.851799 17822 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:31.898311 17692 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:31.925639 16009 rpcz_store.cc:276] 0709 13:05:30.590776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.784704 (+193928us) service_pool.cc:224] Handling call
0709 13:05:31.844249 (+1059545us) spinlock_profiling.cc:243] Waited 8.2 ms on lock 0x7b5800114398. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:05:31.925142 (+ 80893us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18853632}
I20260709 13:05:31.964669 16006 raft_consensus.cc:1240] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:05:31.959728 17688 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:31.988847 17746 raft_consensus.cc:697] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 4 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
W20260709 13:05:31.990154 17830 log.cc:927] Time spent T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.214s	user 0.001s	sys 0.000s
I20260709 13:05:31.964437 17840 raft_consensus.cc:2804] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:31.915930 17643 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:31.959791 17641 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.964517 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:05:32.012409 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:32.013890 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:32.014988 15934 raft_consensus.cc:2393] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 4: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:05:31.986713 17852 log.cc:927] Time spent T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 13:05:31.988370 17767 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:31.990569 17746 consensus_queue.cc:237] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:31.976039 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:31.922267 17758 consensus_queue.cc:1048] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 13:05:31.993710 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:32.035571 15935 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 2.
I20260709 13:05:31.995239 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:31.995968 17813 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:31.924937 17846 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:31.961234 17847 raft_consensus.cc:2749] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:31.967741 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 314) took 1099 ms. Trace:
I20260709 13:05:31.994632 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:32.037526 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 58538.2us
I20260709 13:05:31.989236 17688 raft_consensus.cc:697] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:32.061786 15929 raft_consensus.cc:2393] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 3: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
W20260709 13:05:32.063057 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 215962us
I20260709 13:05:32.081140 16533 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:05:32.064845 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 166419us
W20260709 13:05:32.068818 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:32.082126 15900 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:32.075356 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:32.066840 16011 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:05:32.033932 17767 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:32.082772 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 337581us
W20260709 13:05:32.084463 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 284576us
I20260709 13:05:32.085326 15974 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:32.088239 15902 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:32.097842 16006 raft_consensus.cc:1240] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:05:32.102784 15974 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:32.081312 17846 raft_consensus.cc:697] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:32.053838 17840 raft_consensus.cc:697] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:32.006875 17641 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:32.108786 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:32.110301 15902 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:32.096391 17831 consensus_queue.cc:1059] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 13:05:32.114450 17846 consensus_queue.cc:237] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:32.115342 17831 raft_consensus.cc:3055] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:32.115751 17831 raft_consensus.cc:740] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:32.116585 17831 consensus_queue.cc:260] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:32.117759 17831 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:32.104940 17720 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:32.108428 17847 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:32.066646 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:05:32.131285 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 50232.4us
I20260709 13:05:32.131883 16009 raft_consensus.cc:1275] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260709 13:05:32.132666 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 48584.9us
I20260709 13:05:32.126111 15935 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:32.128119 16566 raft_consensus.cc:1275] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:05:32.133332 15827 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
W20260709 13:05:32.128377 15936 consensus_meta.cc:287] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.559s	user 0.014s	sys 0.021s
I20260709 13:05:32.133956 15936 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:05:32.134542 15931 raft_consensus.cc:1240] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:32.134941 15825 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:32.136970 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:32.137468 17820 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:32.137897 17853 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:32.081995 17813 raft_consensus.cc:697] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:32.086869 17767 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:32.144169 17853 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:32.152659 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:32.153151 15937 raft_consensus.cc:1275] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:05:32.161235 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:32.162504 17784 consensus_meta.cc:287] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.507s	user 0.009s	sys 0.000s
I20260709 13:05:32.162955 17784 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:32.165468 17784 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:32.067593 17793 consensus_queue.cc:1048] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 13:05:32.145747 15974 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:32.167371 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:32.171440 15974 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:32.145076 15858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 371) took 1414 ms. Trace:
I20260709 13:05:32.158274 16010 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2.
I20260709 13:05:32.130400 17803 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:32.108711 17842 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:32.149776 16535 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:32.200883 17720 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:05:32.201032 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 90809.5us
I20260709 13:05:32.201500 17720 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:32.201956 17720 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:32.202298 15902 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:32.202731 15935 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
I20260709 13:05:32.203583 15902 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:32.100606 17822 consensus_queue.cc:1059] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.058s
W20260709 13:05:32.208175 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 126106us
I20260709 13:05:32.210536 15974 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:32.210680 15900 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:32.137897 17820 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:32.212968 15900 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:32.144415 17755 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:32.211668 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 83447.1us
I20260709 13:05:32.216228 15974 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:32.210716 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:32.156991 16564 raft_consensus.cc:1275] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 LEARNER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260709 13:05:32.199009 16004 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.007s	sys 0.025s
W20260709 13:05:32.236747 16004 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.007s	sys 0.026s
I20260709 13:05:32.166630 17853 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:32.226553 15756 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58426 (request call id 341) took 4886 ms. Trace:
I20260709 13:05:32.237926 15756 rpcz_store.cc:276] 0709 13:05:27.289202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:27.289566 (+   364us) service_pool.cc:224] Handling call
0709 13:05:32.175663 (+4886097us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:29.691463 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:29.716569 (+ 25106us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:29.716630 (+    61us) write_op.cc:435] Acquired schema lock
0709 13:05:29.716674 (+    44us) tablet.cc:662] Decoding operations
0709 13:05:29.728718 (+ 12044us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:29.728821 (+   103us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:29.728852 (+    31us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:05:29.729583 (+   731us) tablet.cc:701] Row locks acquired
0709 13:05:29.729645 (+    62us) write_op.cc:260] PREPARE: finished
0709 13:05:29.729839 (+   194us) write_op.cc:270] Start()
0709 13:05:29.729994 (+   155us) write_op.cc:276] Timestamp: P: 1783602329729774 usec, L: 0
0709 13:05:29.730029 (+    35us) op_driver.cc:348] REPLICATION: starting
0709 13:05:29.755543 (+ 25514us) log.cc:844] Serialized 2064 byte log entry
0709 13:05:30.610468 (+854925us) op_driver.cc:464] REPLICATION: finished
0709 13:05:30.903964 (+293496us) write_op.cc:301] APPLY: starting
0709 13:05:30.904066 (+   102us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:30.904243 (+   177us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:30.928829 (+ 24586us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:31.032317 (+103488us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:31.032701 (+   384us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 13:05:31.032771 (+    70us) write_op.cc:312] APPLY: finished
0709 13:05:31.033084 (+   313us) log.cc:844] Serialized 47 byte log entry
0709 13:05:31.033271 (+   187us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:31.033391 (+   120us) write_op.cc:454] Released schema lock
0709 13:05:31.033471 (+    80us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:31.033950 (+   479us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:31.033989 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":259547,"reported_tablets":7,"child_traces":[["op",{"apply.queue_time_us":258948,"apply.run_cpu_time_us":4093,"apply.run_wall_time_us":129956,"num_ops":5,"prepare.queue_time_us":468803,"prepare.run_cpu_time_us":95201,"prepare.run_wall_time_us":727831,"raft.queue_time_us":138958,"raft.run_cpu_time_us":20371,"raft.run_wall_time_us":263945,"replication_time_us":854545,"thread_start_us":1050881,"threads_started":4,"wal-append.queue_time_us":624370,"wal-append.run_cpu_time_us":16310,"wal-append.run_wall_time_us":1664986}]]}
I20260709 13:05:32.244513 15974 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:32.205137 17818 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:32.211817 17653 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:32.212260 17848 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:32.145062 17819 consensus_queue.cc:1048] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.079s
I20260709 13:05:32.198045 17803 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:05:32.253096 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 49568.9us
I20260709 13:05:32.216696 17820 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:32.256196 15902 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:32.217530 17651 raft_consensus.cc:2804] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:32.227809 17758 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:05:32.246604 17835 consensus_meta.cc:287] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.892s	user 0.019s	sys 0.022s
I20260709 13:05:32.245501 16563 raft_consensus.cc:2393] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 4: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:32.235878 17842 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:32.242419 17867 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:05:32.268846 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 133975us
I20260709 13:05:32.260135 17744 raft_consensus.cc:2804] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:32.247527 17862 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:32.218657 17802 consensus_queue.cc:1059] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 13:05:32.266335 17758 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:32.248039 17763 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:32.258023 17803 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:32.263692 17651 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:32.262156 17755 raft_consensus.cc:2749] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:32.264070 16535 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:32.267958 17653 raft_consensus.cc:697] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 4 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:32.268543 17861 raft_consensus.cc:2804] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:05:32.303452 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 170184us
W20260709 13:05:32.317237 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 72798.9us
I20260709 13:05:32.315591 15827 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
W20260709 13:05:32.304418 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 91928us
I20260709 13:05:32.278355 17842 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:32.288887 17744 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:32.320138 15825 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:32.320700 15974 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:32.300109 17758 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:32.321667 15900 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:32.327467 15974 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:05:32.315838 17769 consensus_meta.cc:287] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.529s	user 0.015s	sys 0.000s
I20260709 13:05:32.258191 17688 consensus_queue.cc:237] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:32.310236 17651 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:32.299291 17763 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:05:32.317752 17836 consensus_meta.cc:287] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.534s	user 0.000s	sys 0.017s
I20260709 13:05:32.269815 17840 consensus_queue.cc:237] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:32.331782 15974 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:32.332828 15900 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:32.337296 17767 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:05:32.319496 17630 consensus_meta.cc:287] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.970s	user 0.008s	sys 0.006s
I20260709 13:05:32.315140 17861 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:32.320426 17833 consensus_meta.cc:287] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.792s	user 0.006s	sys 0.007s
I20260709 13:05:32.343958 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:32.344545 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:32.345098 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:05:32.390647 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 57889.9us
W20260709 13:05:32.346503 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 82554.3us
I20260709 13:05:32.343626 17769 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:32.066512 16002 rpcz_store.cc:276] 0709 13:05:30.862787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:31.492307 (+629520us) service_pool.cc:224] Handling call
0709 13:05:31.494393 (+  2086us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:31.574975 (+ 80582us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:31.577519 (+  2544us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:31.600178 (+ 22659us) log.cc:844] Serialized 26 byte log entry
0709 13:05:31.665473 (+ 65295us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:31.665519 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:31.665660 (+   141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:31.909422 (+243762us) raft_consensus.cc:1685] finished
0709 13:05:31.909445 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:31.962032 (+ 52587us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59901,"raft.queue_time_us":63887,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":45983,"thread_start_us":64849,"threads_started":1,"wal-append.queue_time_us":151498}
I20260709 13:05:32.279050 17835 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:32.326079 17842 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:32.331835 17744 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:32.330454 17584 raft_consensus.cc:2764] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:05:32.323057 16004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 318) took 1423 ms. Trace:
I20260709 13:05:32.357801 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:32.338637 17848 raft_consensus.cc:2749] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:05:32.388676 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 56999.4us
W20260709 13:05:32.397897 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 77819.4us
I20260709 13:05:32.339861 17846 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:32.295254 17813 consensus_queue.cc:237] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:32.353516 17763 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:32.404357 16535 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:32.338176 17867 raft_consensus.cc:2749] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:32.428354 16535 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:32.419973 15933 raft_consensus.cc:1240] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:32.430518 16569 raft_consensus.cc:2393] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 3: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:32.422266 15974 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:32.403169 17630 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:32.432739 17864 raft_consensus.cc:3055] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:05:32.423374 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 107863us
I20260709 13:05:32.396857 17641 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:32.397840 17861 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:32.430089 17854 log.cc:927] Time spent T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.215s	user 0.001s	sys 0.000s
I20260709 13:05:32.423509 17876 raft_consensus.cc:2804] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:32.431650 15974 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:32.433188 17864 raft_consensus.cc:740] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
W20260709 13:05:32.486693 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 339069us
W20260709 13:05:32.438526 17860 log.cc:927] Time spent T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
W20260709 13:05:32.439327 17863 log.cc:927] Time spent T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 13:05:32.445689 17767 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:32.447093 17763 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:32.465673 17819 raft_consensus.cc:2804] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:32.450450 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:32.399251 17818 raft_consensus.cc:2804] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:32.400032 17822 consensus_queue.cc:1048] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 13:05:32.357425 17836 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:32.470474 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:32.517361 16006 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
W20260709 13:05:32.475593 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:32.479279 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 223161us
I20260709 13:05:32.424357 17846 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:05:32.483127 15930 consensus_meta.cc:287] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.571s	user 0.006s	sys 0.001s
I20260709 13:05:32.519968 15930 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
I20260709 13:05:32.520565 16567 raft_consensus.cc:1240] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 3. Current term is 4. Ops: []
W20260709 13:05:32.488492 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 56914.2us
I20260709 13:05:32.464676 17793 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:32.442883 17536 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:32.487489 16533 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:32.491187 15827 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:32.399909 17833 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:32.492836 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:32.493718 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:32.449013 17867 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:32.494272 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:32.494831 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:32.173259 15858 rpcz_store.cc:276] 0709 13:05:30.730698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.985433 (+254735us) service_pool.cc:224] Handling call
0709 13:05:30.985729 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:30.986006 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:31.029192 (+ 43186us) log.cc:844] Serialized 14 byte log entry
0709 13:05:31.834519 (+805327us) spinlock_profiling.cc:243] Waited 1.51 ms on lock 0x7b1800340240. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f4823a 00007f73ea3b2a2a
0709 13:05:32.144734 (+310215us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:32.144789 (+    55us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:32.144847 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.144917 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:32.145025 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":42831,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":2096,"raft.queue_time_us":10043,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":20822,"spinlock_wait_cycles":3484032,"thread_start_us":295617,"threads_started":1}
W20260709 13:05:32.484109 17724 consensus_meta.cc:287] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.813s	user 0.010s	sys 0.003s
W20260709 13:05:32.448031 17855 log.cc:927] Time spent T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
I20260709 13:05:32.500994 17819 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:32.416321 17653 consensus_queue.cc:237] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:32.520841 17755 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:32.523218 15974 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:32.525760 15825 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:32.495019 17847 raft_consensus.cc:2749] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:32.527606 17869 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:32.529767 17793 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:32.534724 15827 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:32.534493 15902 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:32.504410 17769 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:32.508965 17818 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:05:32.545789 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:32.545872 16008 raft_consensus.cc:2368] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 2. Current term is 3.
I20260709 13:05:32.525233 17846 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:32.512673 17876 raft_consensus.cc:697] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:32.528240 17875 consensus_queue.cc:1059] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:32.531178 17867 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:05:32.556262 17874 log.cc:927] Time spent T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:05:32.544235 17724 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:32.552258 17792 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.117s
I20260709 13:05:32.537637 17870 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:32.577366 17818 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:32.590636 16566 raft_consensus.cc:1240] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:32.547088 17819 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:32.600080 17833 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:32.554409 17627 consensus_meta.cc:287] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.756s	user 0.010s	sys 0.025s
I20260709 13:05:32.604930 16011 raft_consensus.cc:1240] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
W20260709 13:05:32.585199 17722 consensus_meta.cc:287] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.791s	user 0.026s	sys 0.009s
I20260709 13:05:32.556617 17835 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:32.578418 17793 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:32.576470 17767 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:32.612579 15937 raft_consensus.cc:1240] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
W20260709 13:05:32.616411 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 93262us
I20260709 13:05:32.589390 17867 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:32.579440 17862 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:32.602178 17848 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:32.588678 17815 raft_consensus.cc:2764] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:05:32.588917 17536 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:32.603808 17870 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:32.605295 17865 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:32.415133 16004 rpcz_store.cc:276] 0709 13:05:30.897724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:31.524214 (+626490us) service_pool.cc:224] Handling call
0709 13:05:31.524543 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:31.524855 (+   312us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:32.057504 (+532649us) spinlock_profiling.cc:243] Waited 27.3 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e729e0ac 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72b8455 00007f73edda03c7
0709 13:05:32.237211 (+179707us) log.cc:844] Serialized 14 byte log entry
0709 13:05:32.318233 (+ 81022us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:32.318295 (+    62us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:32.318361 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.318442 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:32.320413 (+  1971us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3475,"prepare.queue_time_us":711929,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":13205,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":117,"spinlock_wait_cycles":62827392,"thread_start_us":779480,"threads_started":2,"wal-append.queue_time_us":126262}
I20260709 13:05:32.611752 17627 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:32.576161 17864 consensus_queue.cc:260] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:32.541292 17700 consensus_queue.cc:1048] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 13:05:32.590610 17836 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:32.611501 17715 consensus_meta.cc:287] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 1.114s	user 0.007s	sys 0.007s
I20260709 13:05:32.636588 15974 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:32.576534 17826 consensus_queue.cc:1059] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:05:32.560109 17853 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:32.698787 17765 consensus_meta.cc:287] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.766s	user 0.007s	sys 0.000s
I20260709 13:05:32.713681 16003 raft_consensus.cc:2393] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 3: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:32.617468 17881 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:05:32.693481 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 118661us
I20260709 13:05:32.697695 17720 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:32.653952 17862 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:32.661142 17848 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:05:32.703898 17837 consensus_meta.cc:287] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.923s	user 0.010s	sys 0.005s
W20260709 13:05:32.708443 17868 log.cc:927] Time spent T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.280s	user 0.000s	sys 0.001s
W20260709 13:05:32.706686 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 70188.4us
I20260709 13:05:32.727730 15974 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:32.728789 15974 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:32.730388 17826 raft_consensus.cc:2749] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:32.731417 15974 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:32.733163 17826 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:32.733547 17826 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:32.733865 17826 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:32.737787 15974 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:05:32.670250 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 126765us
W20260709 13:05:32.708214 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 130240us
I20260709 13:05:32.665057 17865 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:32.664844 17870 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:32.619879 17722 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:32.652459 17651 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:32.724066 17848 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:32.742165 15974 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:32.742621 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:32.743115 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:32.713589 17835 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:32.757875 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:05:32.722674 17831 consensus_meta.cc:287] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.605s	user 0.012s	sys 0.000s
I20260709 13:05:32.720134 17862 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:32.715517 17881 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:32.740280 17763 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:05:32.716952 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 229526us
I20260709 13:05:32.757077 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:32.710325 17715 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:32.745029 15936 raft_consensus.cc:1240] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:32.758829 15974 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:32.740492 17866 raft_consensus.cc:2804] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:32.745371 17836 raft_consensus.cc:2804] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:32.784107 17836 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:32.760021 17869 raft_consensus.cc:2804] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:32.755224 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:32.755388 17758 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:32.777454 16010 raft_consensus.cc:2393] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 4: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:32.784564 17836 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:32.728524 17837 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:32.749881 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 87043.7us
I20260709 13:05:32.757361 17802 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:32.784808 17720 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:32.794430 17722 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:32.794581 17792 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:32.795882 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:32.746311 17865 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:32.761232 17835 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:32.770437 17881 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:32.836069 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:32.800880 16004 raft_consensus.cc:2393] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 4: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:32.784116 17866 raft_consensus.cc:697] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:32.852829 16565 raft_consensus.cc:2393] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 4: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
W20260709 13:05:32.849617 15900 leader_election.cc:336] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.549s (SENT)
I20260709 13:05:32.713451 17876 consensus_queue.cc:237] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:32.853608 15900 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:05:32.837271 17803 consensus_meta.cc:287] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.521s	user 0.008s	sys 0.002s
I20260709 13:05:32.861927 15932 raft_consensus.cc:1240] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
W20260709 13:05:32.863481 17892 log.cc:927] Time spent T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:05:32.797485 17756 consensus_queue.cc:1059] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 13:05:32.864132 17853 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:32.863427 17802 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:32.865832 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:32.801463 17763 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:32.779557 17724 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:32.870317 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:32.881301 16568 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
W20260709 13:05:32.885722 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 129000us
I20260709 13:05:32.854439 17835 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:32.867435 17720 raft_consensus.cc:2749] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:32.890888 17867 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:32.871819 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:32.872714 17722 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:32.944617 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:32.948432 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:32.949860 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:05:32.952653 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 190185us
I20260709 13:05:32.872083 17803 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:32.957038 16535 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:32.961105 15929 raft_consensus.cc:1275] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:05:32.961467 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:32.962255 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:32.927717 17627 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:32.973469 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:32.950253 17882 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:32.962278 16564 raft_consensus.cc:2393] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 3: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:32.980620 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:32.985261 15902 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:05:32.973011 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 182233us
I20260709 13:05:32.951864 17722 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:32.990612 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:32.992107 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:32.990969 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:32.991622 15858 raft_consensus.cc:2410] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 2 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
W20260709 13:05:32.969439 17748 consensus_meta.cc:287] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 1.365s	user 0.016s	sys 0.026s
I20260709 13:05:33.025092 15827 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:33.044422 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:05:33.063723 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 78536.8us
I20260709 13:05:33.073120 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:33.074153 15860 raft_consensus.cc:1240] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:05:33.074364 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:33.074925 15931 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:05:33.075326 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:32.958952 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:33.019411 15929 raft_consensus.cc:2393] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 2: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:33.079393 16004 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:33.079730 15860 raft_consensus.cc:2393] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 3: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:33.080894 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:33.081421 15860 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:33.061998 17890 consensus_queue.cc:1048] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.091s
I20260709 13:05:33.084443 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:33.085036 15929 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:33.087136 16004 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:05:33.013624 17848 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:33.088150 15860 raft_consensus.cc:2410] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:05:33.034761 15858 raft_consensus.cc:1240] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:05:33.088990 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:05:33.078401 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 118470us
I20260709 13:05:33.078815 15974 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:33.089308 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:33.090060 15858 raft_consensus.cc:2393] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 3: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:33.090204 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:33.090505 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:32.991989 17869 raft_consensus.cc:697] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 5 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:33.091156 15858 raft_consensus.cc:1275] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:05:33.091759 15860 raft_consensus.cc:1240] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:33.091434 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:33.093752 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:33.095125 15855 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2.
W20260709 13:05:33.097638 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 72733.8us
I20260709 13:05:32.985291 17813 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:33.100498 15931 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
W20260709 13:05:33.102617 17842 consensus_meta.cc:287] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.673s	user 0.009s	sys 0.001s
I20260709 13:05:33.103156 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:05:33.019951 17712 consensus_meta.cc:287] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 1.171s	user 0.011s	sys 0.002s
I20260709 13:05:33.107384 15827 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:05:33.032322 17643 consensus_meta.cc:287] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 1.025s	user 0.002s	sys 0.009s
W20260709 13:05:33.081561 17861 consensus_meta.cc:287] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.595s	user 0.006s	sys 0.003s
I20260709 13:05:33.095235 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:33.098879 17902 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:33.116184 17767 raft_consensus.cc:2749] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:33.131034 15860 raft_consensus.cc:1240] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 4 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:33.118326 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:33.005581 17651 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:33.062367 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:33.119904 16566 raft_consensus.cc:2393] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 3: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
W20260709 13:05:33.123308 17895 log.cc:927] Time spent T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:05:33.124414 17847 consensus_queue.cc:1048] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:33.145385 17802 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:33.146003 17903 log.cc:927] Time spent T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:05:33.146833 16563 raft_consensus.cc:1240] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:05:33.099337 17748 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:33.062922 17692 consensus_queue.cc:1059] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 13:05:33.129489 17643 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:33.148430 17692 raft_consensus.cc:3055] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:33.148806 17692 raft_consensus.cc:740] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:33.071516 17853 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:33.149421 17853 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:33.149616 17692 consensus_queue.cc:260] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:33.150760 17692 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:33.062309 17873 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:33.096007 17715 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:33.089339 17848 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:33.131006 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:33.117683 15827 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:32.993379 17627 consensus_queue.cc:1059] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.121s
I20260709 13:05:33.135914 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:33.136053 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:33.099563 17653 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:33.133909 17902 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:33.173033 15853 raft_consensus.cc:1275] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:05:33.173847 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:33.133898 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:32.988737 17688 consensus_queue.cc:1048] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.133s
I20260709 13:05:33.138804 17904 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:33.139007 17894 log.cc:927] Time spent T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 13:05:33.140239 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:05:33.175906 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 58258.6us
I20260709 13:05:33.177193 15827 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:05:33.140982 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 50335.9us
I20260709 13:05:33.151508 17853 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:33.179325 15974 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:33.155766 17873 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:33.179347 15827 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:33.175005 15931 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:33.180297 17767 raft_consensus.cc:2749] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:33.182075 17873 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:33.182381 17627 raft_consensus.cc:3055] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:33.182811 17627 raft_consensus.cc:740] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:33.185343 17643 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:33.185635 17744 consensus_meta.cc:287] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.761s	user 0.009s	sys 0.000s
I20260709 13:05:33.186050 17630 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:33.178746 17840 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:33.175438 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:32.968885 17866 consensus_queue.cc:237] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:33.130023 17907 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:33.135488 17758 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:33.132810 17837 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:33.187547 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:33.198354 15933 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 2.
I20260709 13:05:33.120684 17842 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:33.140602 17836 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:33.119493 17712 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:33.191774 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:05:33.163439 17820 consensus_meta.cc:287] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.892s	user 0.011s	sys 0.000s
I20260709 13:05:33.135639 17813 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:33.174034 17867 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:33.146865 17862 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:33.178485 16004 raft_consensus.cc:2393] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 3: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:33.132885 17861 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:33.094424 17869 consensus_queue.cc:237] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:05:33.136423 17881 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:33.204547 15858 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:33.177126 17902 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:33.179199 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:33.193646 17767 raft_consensus.cc:2749] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:33.193794 17853 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:33.162237 17793 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:33.243511 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:33.243705 17712 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:33.244009 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:33.186048 17744 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:33.254820 15934 raft_consensus.cc:2393] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 3: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
W20260709 13:05:33.253911 17846 consensus_meta.cc:287] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.646s	user 0.007s	sys 0.003s
I20260709 13:05:33.247218 17748 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:33.266325 16562 raft_consensus.cc:1240] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:33.204277 17826 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:33.270505 16011 raft_consensus.cc:1240] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:05:33.273775 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:33.254499 17722 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:33.276832 15931 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:33.277733 15933 raft_consensus.cc:2393] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 3: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:33.262885 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:33.279098 16004 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:33.285066 17744 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:33.286237 17896 log.cc:927] Time spent T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 13:05:33.237578 17627 consensus_queue.cc:260] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:33.292744 17627 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:33.296378 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:33.297765 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:33.283412 15929 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:33.304091 16569 raft_consensus.cc:3055] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:33.304462 16569 raft_consensus.cc:740] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:33.305292 17826 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:33.305214 16569 consensus_queue.cc:260] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:33.305918 17905 log.cc:927] Time spent T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:05:33.306386 16569 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:33.308030 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:33.260993 17853 raft_consensus.cc:697] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:33.236582 17820 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:33.322691 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:05:33.331445 17910 log.cc:927] Time spent T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:05:33.289307 17653 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.318038 17882 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:33.338743 16570 raft_consensus.cc:1240] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:33.319401 17688 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:33.340162 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 349541us
I20260709 13:05:33.269766 17846 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:33.346022 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:33.344182 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:33.346673 16009 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:05:33.349895 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:33.350325 17820 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:33.350643 16009 raft_consensus.cc:2393] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 4: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:33.332517 17871 raft_consensus.cc:3055] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:33.336611 17826 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:33.354671 16533 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:33.358129 16010 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:33.343981 17653 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:33.347271 17836 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:33.360237 16569 raft_consensus.cc:1275] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:05:33.344969 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:33.367697 17835 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:33.370962 17853 consensus_queue.cc:237] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:33.372821 17835 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:33.368013 17769 raft_consensus.cc:3055] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:33.376015 17769 raft_consensus.cc:740] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:33.363054 17871 raft_consensus.cc:740] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:33.359299 17846 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:33.376809 17769 consensus_queue.cc:260] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:33.384238 17769 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:33.385646 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:33.386435 17870 consensus_meta.cc:287] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.630s	user 0.005s	sys 0.005s
I20260709 13:05:33.386886 17870 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:33.390148 15900 leader_election.cc:336] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.726s (SENT)
I20260709 13:05:33.382198 17871 consensus_queue.cc:260] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:05:33.390838 16005 raft_consensus.cc:2393] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 3: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:33.371021 17913 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:33.398712 17861 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:33.377465 17793 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:33.379298 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:33.393129 15900 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:33.393662 17884 raft_consensus.cc:3055] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:33.345152 17688 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:33.423198 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:05:33.424218 17724 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
W20260709 13:05:33.424415 15900 leader_election.cc:336] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.761s (SENT)
I20260709 13:05:33.424817 15900 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:33.425016 17688 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:33.425554 17688 raft_consensus.cc:2749] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:33.425671 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:33.425972 17846 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:33.400658 17803 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:33.426995 15858 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:33.428146 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:33.428328 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:33.428866 15937 raft_consensus.cc:2393] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 3: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
W20260709 13:05:33.401078 17819 consensus_meta.cc:287] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.789s	user 0.006s	sys 0.005s
I20260709 13:05:33.399884 17864 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:33.390725 17902 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:33.406729 16004 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2.
I20260709 13:05:33.392050 17842 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:33.369278 17862 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:33.412616 17904 raft_consensus.cc:2749] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:33.378268 16002 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:33.416429 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:33.364114 17826 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:05:33.421136 17818 consensus_meta.cc:287] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.769s	user 0.005s	sys 0.005s
W20260709 13:05:33.421294 16008 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41662 (request call id 1654) took 1493 ms (client timeout 1952 ms). Trace:
I20260709 13:05:33.421681 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:33.422290 17884 raft_consensus.cc:740] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:33.433863 15860 raft_consensus.cc:1240] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:33.434857 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:33.441790 15936 raft_consensus.cc:1240] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:33.429374 17819 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:33.442344 17881 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:33.444104 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:33.445402 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:33.447882 17803 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:05:33.430246 17865 consensus_meta.cc:287] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.561s	user 0.007s	sys 0.003s
I20260709 13:05:33.663658 16564 raft_consensus.cc:1275] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:33.469286 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:05:34.225276 17916 log.cc:927] Time spent T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.637s	user 0.002s	sys 0.000s
I20260709 13:05:33.513571 17627 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:34.245048 17627 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:34.246968 17627 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:33.490588 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:33.948232 17653 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:34.248286 15854 raft_consensus.cc:2393] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 3: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:34.249320 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:33.531435 17899 consensus_queue.cc:1059] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.733s
I20260709 13:05:34.250458 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:34.250916 17815 raft_consensus.cc:3055] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:33.773125 16004 raft_consensus.cc:1240] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:33.867291 15935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 369) took 1465 ms. Trace:
I20260709 13:05:34.251699 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:33.448889 16563 raft_consensus.cc:1275] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:05:33.896240 16565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45246 (request call id 541) took 1148 ms. Trace:
W20260709 13:05:33.589946 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 47943.8us
W20260709 13:05:33.464015 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:34.252307 16566 raft_consensus.cc:2393] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 2: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:33.659696 16007 raft_consensus.cc:1240] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:34.019650 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:05:33.814277 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:34.013950 15856 raft_consensus.cc:1240] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:34.253396 17803 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:33.650105 17769 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.667409 15900 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:33.469515 16009 raft_consensus.cc:1240] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:33.474005 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:34.029382 15932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 375) took 1623 ms. Trace:
W20260709 13:05:33.652019 16003 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41662 (request call id 1653) took 1724 ms (client timeout 1591 ms). Trace:
W20260709 13:05:33.718345 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:34.021205 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:05:34.254086 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:34.261278 15902 leader_election.cc:336] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 1.591s (SENT)
I20260709 13:05:34.268805 16007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1667) took 1510 ms. Trace:
I20260709 13:05:34.262701 15856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1551) took 1242 ms. Trace:
I20260709 13:05:34.334275 16007 rpcz_store.cc:276] 0709 13:05:32.758214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.631621 (+873407us) service_pool.cc:224] Handling call
0709 13:05:33.657461 (+ 25840us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.268589 (+611128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.268752 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:34.331637 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:34.333820 15902 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:05:34.332661 16003 rpcz_store.cc:269] 0709 13:05:31.927094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:31.968538 (+ 41444us) service_pool.cc:224] Handling call
0709 13:05:33.650737 (+1682199us) spinlock_profiling.cc:243] Waited 700 ms on lock 0x7b540010efd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:33.650894 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1610382208}
W20260709 13:05:34.254531 16008 rpcz_store.cc:269] 0709 13:05:31.927482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:31.959473 (+ 31991us) service_pool.cc:224] Handling call
0709 13:05:33.421087 (+1461614us) spinlock_profiling.cc:243] Waited 582 ms on lock 0x7b54001f7cd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:33.421225 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1337980288}
I20260709 13:05:34.333724 15859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 1543) took 1352 ms. Trace:
I20260709 13:05:34.330941 15932 rpcz_store.cc:276] 0709 13:05:32.406154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.406260 (+   106us) service_pool.cc:224] Handling call
0709 13:05:32.406519 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:32.868042 (+461523us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.029093 (+1161051us) spinlock_profiling.cc:243] Waited 455 ms on lock 0x7b54002c1d54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:34.029310 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1046785792}
I20260709 13:05:34.334313 15856 rpcz_store.cc:276] 0709 13:05:33.020416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.020567 (+   151us) service_pool.cc:224] Handling call
0709 13:05:34.013219 (+992652us) spinlock_profiling.cc:243] Waited 1.55 ms on lock 0x7b5800005798. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef381fc0 00007f73ef36c929 00007f73ef36c3d2 00007f73e88e3bd4
0709 13:05:34.013509 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.262478 (+248969us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.262647 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3574656}
I20260709 13:05:34.257373 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:34.260074 15933 raft_consensus.cc:2393] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 3: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:34.339397 15859 rpcz_store.cc:276] 0709 13:05:32.981198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.001701 (+ 20503us) service_pool.cc:224] Handling call
0709 13:05:34.021045 (+1019344us) spinlock_profiling.cc:243] Waited 1.3 ms on lock 0x7b5800005798. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:05:34.333665 (+312620us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2991360}
I20260709 13:05:34.256196 16565 rpcz_store.cc:276] 0709 13:05:32.747965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.748101 (+   136us) service_pool.cc:224] Handling call
0709 13:05:33.895925 (+1147824us) spinlock_profiling.cc:243] Waited 94.1 ms on lock 0x7b54005654d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:33.896155 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":218397056}
I20260709 13:05:34.344151 16006 raft_consensus.cc:1240] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
W20260709 13:05:34.349334 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 68537.7us
I20260709 13:05:34.255916 15854 raft_consensus.cc:2393] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 4: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:05:34.331744 17756 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.206s	user 0.012s	sys 0.016s
I20260709 13:05:34.357350 16006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 326) took 1880 ms. Trace:
I20260709 13:05:34.352185 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:34.243494 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:33.690641 17813 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:34.240757 17871 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:33.531320 15929 raft_consensus.cc:1240] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
W20260709 13:05:34.359081 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 691692us
I20260709 13:05:34.417153 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:34.358733 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:34.462486 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:33.524645 17724 consensus_queue.cc:1059] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.892s
I20260709 13:05:33.864171 17902 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:34.446055 17756 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.318s	user 0.015s	sys 0.026s
I20260709 13:05:34.269152 17880 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.258044 16563 raft_consensus.cc:1275] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:05:34.443491 15932 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:34.254469 17826 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:34.268477 17627 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:34.268963 17763 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:34.454522 16533 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:05:34.351078 17875 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.018s	sys 0.019s
I20260709 13:05:34.461382 15900 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:33.449452 17876 consensus_queue.cc:1059] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.967s
I20260709 13:05:34.465137 16007 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:34.350869 17904 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:34.443989 16006 rpcz_store.cc:276] 0709 13:05:32.477017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.541470 (+ 64453us) service_pool.cc:224] Handling call
0709 13:05:34.338109 (+1796639us) spinlock_profiling.cc:243] Waited 2.73 ms on lock 0x7b5800114398. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef381fc0 00007f73ef36c929 00007f73ef36c3d2 00007f73e88e3bd4
0709 13:05:34.338300 (+   191us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.357003 (+ 18703us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.357276 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6272384}
I20260709 13:05:34.493342 15929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 380) took 1054 ms. Trace:
I20260709 13:05:34.532922 15929 rpcz_store.cc:276] 0709 13:05:33.438464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.439619 (+  1155us) service_pool.cc:224] Handling call
0709 13:05:33.517404 (+ 77785us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.465092 (+947688us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.487521 (+ 22429us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:34.534111 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:34.534700 15929 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:34.504886 17870 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:34.536082 17870 raft_consensus.cc:2749] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:34.536696 17870 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:33.911791 17858 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:34.356041 17917 log.cc:927] Time spent T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.612s	user 0.000s	sys 0.001s
I20260709 13:05:34.349500 17862 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:34.540630 17862 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:34.541242 17858 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:05:34.542006 16010 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41674 (request call id 1569) took 1793 ms (client timeout 1797 ms). Trace:
I20260709 13:05:34.542196 15936 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
I20260709 13:05:34.542804 17862 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:34.254227 17815 raft_consensus.cc:740] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:34.543280 15936 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 1594) took 1097 ms. Trace:
I20260709 13:05:34.516587 17902 raft_consensus.cc:2749] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:34.544023 17815 consensus_queue.cc:260] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:34.517237 17763 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:34.545258 17815 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:34.543730 15936 rpcz_store.cc:276] 0709 13:05:33.446265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.251470 (+805205us) service_pool.cc:224] Handling call
0709 13:05:34.543120 (+291650us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b54003c50d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:34.543221 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":659141888}
I20260709 13:05:34.546366 17763 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:34.546886 17763 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.337518 17869 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:34.548067 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:05:34.548604 17864 consensus_meta.cc:287] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 1.107s	user 0.011s	sys 0.000s
I20260709 13:05:34.476718 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:34.548934 15933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 1591) took 1103 ms. Trace:
I20260709 13:05:34.549176 16008 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:34.549324 15933 rpcz_store.cc:276] 0709 13:05:33.445466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.547795 (+1102329us) service_pool.cc:224] Handling call
0709 13:05:34.548875 (+  1080us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:34.550618 15933 raft_consensus.cc:1240] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:05:34.551155 15933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1586) took 1107 ms. Trace:
I20260709 13:05:34.259284 17818 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:34.553423 17818 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:34.485267 15976 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:05:34.554404 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 69226.3us
I20260709 13:05:34.478294 16564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 647) took 1061 ms. Trace:
I20260709 13:05:34.555757 17748 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.556242 17748 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:34.460489 15854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58214 (request call id 383) took 1025 ms. Trace:
I20260709 13:05:34.558074 15854 rpcz_store.cc:276] 0709 13:05:33.434751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.250178 (+815427us) service_pool.cc:224] Handling call
0709 13:05:34.456024 (+205846us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:34.559109 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:34.560114 15857 raft_consensus.cc:2410] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:05:34.555795 16564 rpcz_store.cc:276] 0709 13:05:33.416525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.419523 (+  2998us) service_pool.cc:224] Handling call
0709 13:05:33.422802 (+  3279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.241971 (+819169us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.478058 (+236087us) spinlock_profiling.cc:243] Waited 191 ms on lock 0x7b5400550ad4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:34.478231 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":438152448}
W20260709 13:05:34.561864 15857 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 1545) took 1571 ms (client timeout 1799 ms). Trace:
I20260709 13:05:34.561986 16008 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
W20260709 13:05:34.515347 17875 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.856s	user 0.021s	sys 0.033s
I20260709 13:05:34.504257 15931 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
W20260709 13:05:34.512126 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 57731.8us
I20260709 13:05:34.565248 15928 raft_consensus.cc:1240] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:34.458366 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:34.565919 15928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 377) took 1138 ms. Trace:
I20260709 13:05:34.565943 16533 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:34.566378 15928 rpcz_store.cc:276] 0709 13:05:33.427481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.441002 (+ 13521us) service_pool.cc:224] Handling call
0709 13:05:34.564853 (+1123851us) spinlock_profiling.cc:243] Waited 20.2 ms on lock 0x7b5800003698. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef381fc0 00007f73ef36c929 00007f73ef36c3d2 00007f73e88e3bd4
0709 13:05:34.565071 (+   218us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.565664 (+   593us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.565863 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":46474624}
I20260709 13:05:34.254776 16004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1571) took 1490 ms. Trace:
I20260709 13:05:34.464636 17884 consensus_queue.cc:260] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:34.568029 16004 rpcz_store.cc:276] 0709 13:05:32.764073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.671316 (+907243us) service_pool.cc:224] Handling call
0709 13:05:33.750630 (+ 79314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.254504 (+503874us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.254720 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:34.568593 17884 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:34.568699 17784 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:34.569154 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:34.569438 17784 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:34.569752 16004 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:34.570083 17784 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:34.571353 15858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 1675) took 1440 ms. Trace:
I20260709 13:05:34.515102 17627 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:34.566396 16003 raft_consensus.cc:2393] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 2: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:34.572885 16003 raft_consensus.cc:1240] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:34.573400 16003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1574) took 1784 ms. Trace:
I20260709 13:05:34.573582 17627 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:34.573712 16003 rpcz_store.cc:276] 0709 13:05:32.789040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.572420 (+1783380us) service_pool.cc:224] Handling call
0709 13:05:34.572708 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.573186 (+   478us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.573345 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:34.542441 16010 rpcz_store.cc:269] 0709 13:05:32.749004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.870075 (+121071us) service_pool.cc:224] Handling call
0709 13:05:34.541842 (+1671767us) spinlock_profiling.cc:243] Waited 479 ms on lock 0x7b540025a8d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:34.541953 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1102048128}
I20260709 13:05:34.575431 16010 raft_consensus.cc:1240] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:34.523969 17876 raft_consensus.cc:3055] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:34.576056 17876 raft_consensus.cc:740] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:34.576184 16010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 332) took 1629 ms. Trace:
I20260709 13:05:34.547986 17869 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:34.576980 17876 consensus_queue.cc:260] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:05:34.577517 16005 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41662 (request call id 1670) took 1500 ms (client timeout 1788 ms). Trace:
I20260709 13:05:34.578219 17876 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:34.565807 17584 consensus_queue.cc:1059] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:34.516824 17826 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.590967 16007 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:05:34.591615 16009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1567) took 1843 ms. Trace:
I20260709 13:05:34.592020 16009 rpcz_store.cc:276] 0709 13:05:32.747817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.351280 (+603463us) service_pool.cc:224] Handling call
0709 13:05:33.450059 (+ 98779us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.338890 (+888831us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.591437 (+252547us) spinlock_profiling.cc:243] Waited 98.4 ms on lock 0x7b54000ed8d0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:34.591559 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":226356480}
I20260709 13:05:34.328652 17865 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:34.518757 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:34.571753 15858 rpcz_store.cc:276] 0709 13:05:33.130696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.135848 (+  5152us) service_pool.cc:224] Handling call
0709 13:05:34.571200 (+1435352us) spinlock_profiling.cc:243] Waited 26.2 ms on lock 0x7b54003c1254. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:34.571298 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":60346624}
I20260709 13:05:34.350361 17769 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:34.255213 15935 rpcz_store.cc:276] 0709 13:05:32.402274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.402409 (+   135us) service_pool.cc:224] Handling call
0709 13:05:32.490523 (+ 88114us) spinlock_profiling.cc:243] Waited 6.06 ms on lock 0x7b5800003698. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef381fc0 00007f73ef36c929 00007f73ef36c3d2 00007f73e88e3bd4
0709 13:05:32.490716 (+   193us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:32.490932 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:32.704230 (+213298us) log.cc:844] Serialized 14 byte log entry
0709 13:05:33.860871 (+1156641us) spinlock_profiling.cc:243] Waited 28.9 ms on lock 0x7b1800119aa0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e9f504bc 00007f73e729e48b 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f4823a
0709 13:05:33.866660 (+  5789us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:33.866766 (+   106us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:33.866891 (+   125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:33.867019 (+   128us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:33.867214 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":310311,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":49,"raft.queue_time_us":102,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":2754,"spinlock_wait_cycles":80461056,"thread_start_us":275890,"threads_started":2,"wal-append.queue_time_us":214345,"wal-append.run_cpu_time_us":3040,"wal-append.run_wall_time_us":1386179}
I20260709 13:05:34.575456 16003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 331) took 1628 ms. Trace:
I20260709 13:05:34.652968 16003 rpcz_store.cc:276] 0709 13:05:32.946647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.574640 (+1627993us) service_pool.cc:224] Handling call
0709 13:05:34.574930 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.575113 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:34.575136 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:34.575175 (+    39us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:34.575212 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.575260 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:34.575388 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:34.609131 16570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 484) took 2269 ms. Trace:
I20260709 13:05:34.546391 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:34.653986 16570 rpcz_store.cc:276] 0709 13:05:32.340171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.340313 (+   142us) service_pool.cc:224] Handling call
0709 13:05:32.340647 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:33.341715 (+1001068us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.608803 (+1267088us) spinlock_profiling.cc:243] Waited 998 ms on lock 0x7b5400695ad4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:34.609043 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2294920064}
I20260709 13:05:34.654439 15936 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 1593) took 1208 ms. Trace:
I20260709 13:05:34.654752 15936 rpcz_store.cc:276] 0709 13:05:33.445967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.546165 (+1100198us) service_pool.cc:224] Handling call
0709 13:05:34.654381 (+108216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:34.563112 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:34.551419 15933 rpcz_store.cc:276] 0709 13:05:33.444227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.550090 (+1105863us) service_pool.cc:224] Handling call
0709 13:05:34.550412 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:34.550922 (+   510us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.551109 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:34.657517 15933 raft_consensus.cc:1275] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:05:34.657974 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:34.658195 15933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1596) took 1211 ms. Trace:
I20260709 13:05:34.658663 15933 rpcz_store.cc:276] 0709 13:05:33.446820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.656793 (+1209973us) service_pool.cc:224] Handling call
0709 13:05:34.657173 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.657967 (+   794us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.658147 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:34.576686 16010 rpcz_store.cc:276] 0709 13:05:32.947167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.574803 (+1627636us) service_pool.cc:224] Handling call
0709 13:05:34.575152 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.575925 (+   773us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.576124 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:34.585942 15937 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
W20260709 13:05:34.562242 15857 rpcz_store.cc:269] 0709 13:05:32.990544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.990744 (+   200us) service_pool.cc:224] Handling call
0709 13:05:34.561694 (+1570950us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b54003b6854. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:34.561810 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3602778880}
I20260709 13:05:34.660899 15932 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:34.661698 15932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53636 (request call id 385) took 1079 ms. Trace:
I20260709 13:05:34.661868 16010 raft_consensus.cc:1240] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:34.662101 15932 rpcz_store.cc:276] 0709 13:05:33.582114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.351858 (+769744us) service_pool.cc:224] Handling call
0709 13:05:34.661627 (+309769us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:34.662703 16010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 337) took 1705 ms. Trace:
I20260709 13:05:34.664120 15932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 382) took 1178 ms. Trace:
I20260709 13:05:34.595956 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:34.587620 15931 raft_consensus.cc:1240] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 1. Current term is 2. Ops: []
W20260709 13:05:34.665876 15930 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 1563) took 1718 ms (client timeout 1534 ms). Trace:
I20260709 13:05:34.663187 16010 rpcz_store.cc:276] 0709 13:05:32.957646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.661304 (+1703658us) service_pool.cc:224] Handling call
0709 13:05:34.661655 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.662419 (+   764us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.662640 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:34.596678 16011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 330) took 2115 ms. Trace:
I20260709 13:05:34.598181 15862 raft_consensus.cc:1275] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:05:34.547298 17763 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:34.665024 16006 raft_consensus.cc:2393] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 3: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:34.665500 15932 rpcz_store.cc:276] 0709 13:05:33.486081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.663159 (+1177078us) service_pool.cc:224] Handling call
0709 13:05:34.663501 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.663732 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:34.663770 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:34.663827 (+    57us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:34.663874 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.663945 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:34.664059 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:34.670667 15858 raft_consensus.cc:1240] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:34.667271 15937 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53636 (request call id 379) took 1221 ms. Trace:
I20260709 13:05:34.682431 15862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1546) took 1691 ms. Trace:
I20260709 13:05:34.665885 15931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1590) took 1220 ms. Trace:
I20260709 13:05:34.681095 16011 rpcz_store.cc:276] 0709 13:05:32.481110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.610641 (+129531us) service_pool.cc:224] Handling call
0709 13:05:32.653476 (+ 42835us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:33.277070 (+623594us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.596501 (+1319431us) spinlock_profiling.cc:243] Waited 617 ms on lock 0x7b54003edd54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:34.596634 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1418258304}
I20260709 13:05:34.666055 16008 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:34.669114 16009 raft_consensus.cc:1240] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
W20260709 13:05:34.668324 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:34.682822 15858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 387) took 1247 ms. Trace:
W20260709 13:05:34.666436 15930 rpcz_store.cc:269] 0709 13:05:32.947717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.947830 (+   113us) service_pool.cc:224] Handling call
0709 13:05:34.595809 (+1647979us) spinlock_profiling.cc:243] Waited 2.03 ms on lock 0x7b5800003698. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:05:34.665775 (+ 69966us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4671744}
I20260709 13:05:34.682830 15862 rpcz_store.cc:276] 0709 13:05:32.991264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.992045 (+   781us) service_pool.cc:224] Handling call
0709 13:05:34.587018 (+1594973us) spinlock_profiling.cc:243] Waited 33.3 ms on lock 0x7b5800005798. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef381fc0 00007f73ef36c929 00007f73ef36c3d2 00007f73e88e3bd4
0709 13:05:34.587198 (+   180us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.682121 (+ 94923us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.682360 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":76704128}
I20260709 13:05:34.657537 15756 catalog_manager.cc:5697] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 1 to 2, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 2 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:34.689477 15932 raft_consensus.cc:1240] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:34.689708 15857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 389) took 1169 ms. Trace:
I20260709 13:05:34.688011 15858 rpcz_store.cc:276] 0709 13:05:33.435322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.670158 (+1234836us) service_pool.cc:224] Handling call
0709 13:05:34.670468 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.682530 (+ 12062us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.682756 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:34.690070 15857 rpcz_store.cc:276] 0709 13:05:33.520069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.688942 (+1168873us) service_pool.cc:224] Handling call
0709 13:05:34.689231 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.689396 (+   165us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:34.689421 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:34.689461 (+    40us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:34.689493 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.689551 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:34.689656 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:34.683104 15931 rpcz_store.cc:276] 0709 13:05:33.445215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.587208 (+1141993us) service_pool.cc:224] Handling call
0709 13:05:34.587472 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.665680 (+ 78208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.665834 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:34.574384 17627 raft_consensus.cc:2749] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:34.692104 16009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 334) took 1741 ms. Trace:
I20260709 13:05:34.690057 15932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 384) took 1155 ms. Trace:
I20260709 13:05:34.695688 16003 raft_consensus.cc:1240] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:34.700938 16009 rpcz_store.cc:276] 0709 13:05:32.951128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.592787 (+1641659us) service_pool.cc:224] Handling call
0709 13:05:34.668860 (+ 76073us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:34.691640 (+ 22780us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.692049 (+   409us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:34.701499 16003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 335) took 1748 ms. Trace:
I20260709 13:05:34.701941 16003 rpcz_store.cc:276] 0709 13:05:32.953297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.653978 (+1700681us) service_pool.cc:224] Handling call
0709 13:05:34.695518 (+ 41540us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.701186 (+  5668us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.701424 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:34.695816 15935 raft_consensus.cc:1275] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:05:34.696466 15861 raft_consensus.cc:1240] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:05:34.696249 15756 catalog_manager.cc:5697] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 2 to 3, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 3 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:34.599622 17904 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.698840 15936 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:34.705395 15933 raft_consensus.cc:1275] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:34.705994 15933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1597) took 1223 ms. Trace:
I20260709 13:05:34.706593 15935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1592) took 1260 ms. Trace:
I20260709 13:05:34.707057 15935 rpcz_store.cc:276] 0709 13:05:33.445696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.658888 (+1213192us) service_pool.cc:224] Handling call
0709 13:05:34.695427 (+ 36539us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.706308 (+ 10881us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.706533 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:34.706607 15930 raft_consensus.cc:1240] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:34.706365 15933 rpcz_store.cc:276] 0709 13:05:33.482246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.659486 (+1177240us) service_pool.cc:224] Handling call
0709 13:05:34.700278 (+ 40792us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.705762 (+  5484us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.705940 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:34.708225 15932 rpcz_store.cc:276] 0709 13:05:33.535088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.689099 (+1154011us) service_pool.cc:224] Handling call
0709 13:05:34.689322 (+   223us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.689855 (+   533us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.690013 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:34.709422 15930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 383) took 1194 ms. Trace:
I20260709 13:05:34.710034 15932 raft_consensus.cc:1240] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:34.709934 15930 rpcz_store.cc:276] 0709 13:05:33.515067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.688311 (+1173244us) service_pool.cc:224] Handling call
0709 13:05:34.706457 (+ 18146us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.709152 (+  2695us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.709360 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:34.704404 15756 catalog_manager.cc:5697] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 5 to 6, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 6 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:34.722287 15756 catalog_manager.cc:5697] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 1 to 2, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 2 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:34.723345 15932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 386) took 1090 ms. Trace:
I20260709 13:05:34.724169 15932 rpcz_store.cc:276] 0709 13:05:33.632630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.709655 (+1077025us) service_pool.cc:224] Handling call
0709 13:05:34.709893 (+   238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.710565 (+   672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.723266 (+ 12701us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:34.682550 15937 rpcz_store.cc:276] 0709 13:05:33.438075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.438185 (+   110us) service_pool.cc:224] Handling call
0709 13:05:34.659897 (+1221712us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:34.667716 15928 raft_consensus.cc:1240] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:34.724671 15756 catalog_manager.cc:5697] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:34.733649 15756 catalog_manager.cc:5697] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 1 to 2, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 2 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:34.732115 16009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1672) took 1653 ms. Trace:
I20260709 13:05:34.735832 16009 rpcz_store.cc:276] 0709 13:05:33.078334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.701697 (+1623363us) service_pool.cc:224] Handling call
0709 13:05:34.731470 (+ 29773us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.731714 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:34.731749 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:34.731804 (+    55us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:05:34.731848 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.731915 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:34.732051 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:34.514271 17880 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:34.693629 17885 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.691519 17820 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:34.745337 17820 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:34.746919 17869 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:34.747334 17820 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:05:34.577951 16005 rpcz_store.cc:269] 0709 13:05:33.076938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.153393 (+ 76455us) service_pool.cc:224] Handling call
0709 13:05:34.577316 (+1423923us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b5400304654. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:34.577456 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":285400960}
I20260709 13:05:34.738778 15937 raft_consensus.cc:1275] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:05:34.740069 15930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 388) took 1080 ms. Trace:
I20260709 13:05:34.759797 15930 rpcz_store.cc:276] 0709 13:05:33.660162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.711114 (+1050952us) service_pool.cc:224] Handling call
0709 13:05:34.739387 (+ 28273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.739626 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:34.739666 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:34.739746 (+    80us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:34.739796 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.739866 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:34.739993 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:34.722203 17584 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.735347 15756 catalog_manager.cc:5697] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 3 to 4, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 4 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:34.589329 17818 consensus_queue.cc:1059] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.117s
I20260709 13:05:34.744875 16009 raft_consensus.cc:1240] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:34.765532 16009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1575) took 1573 ms. Trace:
I20260709 13:05:34.651993 17826 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:34.722172 17858 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:34.727332 17748 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:34.765862 16009 rpcz_store.cc:276] 0709 13:05:33.192676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.736953 (+1544277us) service_pool.cc:224] Handling call
0709 13:05:34.744710 (+  7757us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.765297 (+ 20587us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.765481 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:34.741425 15862 raft_consensus.cc:1240] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:34.749082 17627 consensus_queue.cc:1048] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:05:34.774252 17880 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:34.778832 15756 catalog_manager.cc:5697] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 3 to 4, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 4 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:34.780689 15934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 376) took 1353 ms. Trace:
I20260709 13:05:34.783763 17584 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:34.784616 15928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1589) took 1320 ms. Trace:
I20260709 13:05:34.729835 17904 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:34.787259 17769 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:34.761868 17803 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:34.786965 15854 raft_consensus.cc:1240] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:34.812556 16009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1577) took 1608 ms. Trace:
W20260709 13:05:34.814002 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:34.914283 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:34.807497 16002 consensus_meta.cc:287] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.562s	user 0.009s	sys 0.001s
I20260709 13:05:34.793907 16008 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
W20260709 13:05:34.918829 15826 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.015s	sys 0.037s
W20260709 13:05:34.919265 15826 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.015s	sys 0.037s
I20260709 13:05:34.805939 17818 raft_consensus.cc:3055] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:34.808794 15934 rpcz_store.cc:276] 0709 13:05:33.427157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.430649 (+  3492us) service_pool.cc:224] Handling call
0709 13:05:33.430933 (+   284us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:33.438369 (+  7436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:33.438398 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:33.438897 (+   499us) log.cc:844] Serialized 26 byte log entry
0709 13:05:33.771940 (+333043us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:33.771999 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:33.772098 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:34.780372 (+1008274us) raft_consensus.cc:1685] finished
0709 13:05:34.780404 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:34.780608 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1483,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":30003,"thread_start_us":284902,"threads_started":1,"wal-append.queue_time_us":1101919}
I20260709 13:05:34.761868 17885 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:34.849494 15756 catalog_manager.cc:5697] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 1 to 3, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 3 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:34.919221 16008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1578) took 1713 ms. Trace:
I20260709 13:05:34.920837 16008 rpcz_store.cc:276] 0709 13:05:33.205392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.918250 (+1712858us) service_pool.cc:224] Handling call
0709 13:05:34.918617 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.918833 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:34.918870 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:34.918923 (+    53us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:34.918972 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.919041 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:34.919172 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:34.913269 16009 rpcz_store.cc:276] 0709 13:05:33.204536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.804869 (+1600333us) service_pool.cc:224] Handling call
0709 13:05:34.811884 (+  7015us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.812144 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:34.812181 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:34.812239 (+    58us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:05:34.812287 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.812366 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:34.812489 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:34.823428 16562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 642) took 2061 ms. Trace:
I20260709 13:05:34.947512 16008 raft_consensus.cc:1240] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:05:34.948021 15861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1548) took 1948 ms. Trace:
I20260709 13:05:34.949148 15756 catalog_manager.cc:5697] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 1 to 2, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 2 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:05:34.954087 15860 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.683s	user 0.007s	sys 0.036s
I20260709 13:05:34.948540 15861 rpcz_store.cc:276] 0709 13:05:32.999619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.000184 (+   565us) service_pool.cc:224] Handling call
0709 13:05:33.401910 (+401726us) spinlock_profiling.cc:243] Waited 6.98 ms on lock 0x7b5800005798. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef381fc0 00007f73ef36c929 00007f73ef36c3d2 00007f73e88e3bd4
0709 13:05:33.402098 (+   188us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:34.703201 (+1301103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.947665 (+244464us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54002490d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:34.947937 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2992260736}
I20260709 13:05:34.963866 15756 catalog_manager.cc:5697] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 2 to 3, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 3 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:05:35.021409 15860 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.750s	user 0.010s	sys 0.038s
W20260709 13:05:35.021965 16533 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 1.594s (SENT) [suppressed 17 similar messages]
I20260709 13:05:35.022365 15756 catalog_manager.cc:5697] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 2 to 3, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 3 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:34.916002 17819 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:34.915512 16005 raft_consensus.cc:1240] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:35.009662 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:35.033998 15857 raft_consensus.cc:2393] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 3: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:35.013593 16008 raft_consensus.cc:1275] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:05:34.951455 17748 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:34.924090 17818 raft_consensus.cc:740] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:34.917555 16002 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
W20260709 13:05:34.955933 15853 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.892s	user 0.002s	sys 0.042s
I20260709 13:05:35.014714 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:34.951701 17822 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:34.811559 15928 rpcz_store.cc:276] 0709 13:05:33.444990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.567455 (+1122465us) service_pool.cc:224] Handling call
0709 13:05:34.587401 (+ 19946us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.742162 (+154761us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.765627 (+ 23465us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:34.967365 17803 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.044679 17803 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:35.046895 17803 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:35.036247 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:05:35.047772 16002 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41674 (request call id 1581) took 1794 ms (client timeout 1868 ms). Trace:
W20260709 13:05:35.048233 16002 rpcz_store.cc:269] 0709 13:05:33.253783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.316265 (+ 62482us) service_pool.cc:224] Handling call
0709 13:05:35.041930 (+1725665us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:35.041857 17865 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:35.049729 17818 consensus_queue.cc:260] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:35.051229 17818 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:35.039932 16007 raft_consensus.cc:1240] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
W20260709 13:05:35.021155 17792 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.009s	sys 0.041s
I20260709 13:05:35.036471 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:35.043426 17822 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:35.055289 15858 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:35.031430 17769 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.057394 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:05:35.042104 15853 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.978s	user 0.004s	sys 0.045s
W20260709 13:05:35.056258 17792 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.009s	sys 0.042s
I20260709 13:05:35.055449 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:34.946250 16562 rpcz_store.cc:276] 0709 13:05:32.751134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.752164 (+  1030us) service_pool.cc:224] Handling call
0709 13:05:32.793732 (+ 41568us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:33.266784 (+473052us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.803580 (+1536796us) spinlock_profiling.cc:243] Waited 470 ms on lock 0x7b54002b0f54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:34.812045 (+  8465us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1081348352}
I20260709 13:05:35.080171 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:35.082422 15861 raft_consensus.cc:2393] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 3: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:05:35.063218 15929 consensus_meta.cc:287] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.528s	user 0.010s	sys 0.000s
I20260709 13:05:35.062207 17822 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:35.085958 15936 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1595) took 1639 ms. Trace:
I20260709 13:05:35.087766 15936 rpcz_store.cc:276] 0709 13:05:33.446543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.664877 (+1218334us) service_pool.cc:224] Handling call
0709 13:05:34.665208 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.705690 (+ 40482us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:35.085676 (+379986us) spinlock_profiling.cc:243] Waited 33.3 ms on lock 0x7b540034a654. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:35.085885 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":76497664}
I20260709 13:05:35.060853 16005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1576) took 1860 ms. Trace:
W20260709 13:05:35.089107 15827 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.726s	user 0.008s	sys 0.012s
W20260709 13:05:35.089517 15827 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.008s	sys 0.012s
I20260709 13:05:35.055516 17748 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:35.062037 17769 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:05:35.086795 15929 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5.
I20260709 13:05:35.098384 15854 raft_consensus.cc:2393] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 6: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:05:35.116742 15929 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 1587) took 1672 ms (client timeout 1890 ms). Trace:
W20260709 13:05:35.117398 15929 rpcz_store.cc:269] 0709 13:05:33.444457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.533833 (+1089376us) service_pool.cc:224] Handling call
0709 13:05:35.116661 (+582828us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:35.093221 17826 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:05:35.120180 15977 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.700s	user 0.002s	sys 0.018s
W20260709 13:05:35.120635 15977 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.002s	sys 0.018s
W20260709 13:05:35.121935 15855 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.008s	sys 0.012s
W20260709 13:05:35.122350 15855 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.706s	user 0.008s	sys 0.012s
W20260709 13:05:35.122763 15976 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.004s	sys 0.017s
W20260709 13:05:35.123119 15976 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.004s	sys 0.017s
W20260709 13:05:35.123431 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 565608us
W20260709 13:05:35.124030 15974 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.012s	sys 0.008s
W20260709 13:05:35.124405 15974 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.012s	sys 0.008s
W20260709 13:05:35.125015 15856 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.004s	sys 0.016s
W20260709 13:05:35.125418 15856 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.004s	sys 0.016s
W20260709 13:05:35.125823 15825 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.008s	sys 0.012s
W20260709 13:05:35.126169 15825 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.008s	sys 0.012s
W20260709 13:05:35.126482 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 673013us
W20260709 13:05:35.128935 15900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.005s	sys 0.014s
W20260709 13:05:35.129369 15900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.005s	sys 0.014s
W20260709 13:05:35.129966 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 668593us
W20260709 13:05:35.130955 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:35.131450 15900 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:05:35.132830 15902 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.757s	user 0.020s	sys 0.000s
W20260709 13:05:35.133323 15902 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.768s	user 0.020s	sys 0.000s
W20260709 13:05:35.133963 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 872611us
I20260709 13:05:35.135301 15902 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:35.137558 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:05:35.093470 15827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:35.138477 15861 raft_consensus.cc:2393] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 3: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:35.139842 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:35.142180 15858 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:35.142486 15827 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:35.146718 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:05:35.132246 17724 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.004s	sys 0.016s
I20260709 13:05:35.115394 15853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 380) took 1681 ms. Trace:
I20260709 13:05:35.150070 17858 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:35.150504 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:35.152769 15853 rpcz_store.cc:276] 0709 13:05:33.433545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.437671 (+  4126us) service_pool.cc:224] Handling call
0709 13:05:33.524133 (+ 86462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:33.581481 (+ 57348us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:05:34.010175 (+428694us) log.cc:844] Serialized 14 byte log entry
0709 13:05:35.088936 (+1078761us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:35.091287 (+  2351us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:35.092458 (+  1171us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:35.098942 (+  6484us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:35.115310 (+ 16368us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":903380,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":48,"raft.queue_time_us":12482,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":5018,"thread_start_us":1357959,"threads_started":2,"wal-append.queue_time_us":1133646}
I20260709 13:05:35.154840 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:35.163374 17584 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:35.164577 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:35.165270 16008 raft_consensus.cc:2393] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 5: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:35.165093 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:05:35.155718 15975 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.744s	user 0.018s	sys 0.024s
W20260709 13:05:35.166131 15975 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.829s	user 0.018s	sys 0.025s
I20260709 13:05:35.157356 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:35.168349 15933 raft_consensus.cc:1240] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:35.168484 16562 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 1.
I20260709 13:05:35.169880 17885 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:35.171164 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:35.157891 17748 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:35.172035 17865 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.158685 17928 raft_consensus.cc:2749] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:35.159596 17826 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:35.174146 17826 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:35.173458 17865 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:35.176287 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:35.177029 17876 consensus_meta.cc:287] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.598s	user 0.006s	sys 0.003s
I20260709 13:05:35.177222 17865 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:35.177548 17826 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:35.178151 17826 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:35.178683 17826 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:35.172065 16004 consensus_meta.cc:287] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.602s	user 0.010s	sys 0.000s
I20260709 13:05:35.180275 16004 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:35.181409 16004 raft_consensus.cc:1240] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:35.182384 17826 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:35.152174 17769 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
I20260709 13:05:35.183300 17902 raft_consensus.cc:3055] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:35.185475 17902 raft_consensus.cc:740] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:35.186589 17902 consensus_queue.cc:260] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:35.182705 16004 raft_consensus.cc:1240] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:35.147406 17803 consensus_queue.cc:1059] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 13:05:35.188005 17871 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:35.188306 17803 raft_consensus.cc:3055] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:35.188733 15929 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
I20260709 13:05:35.188983 17784 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:35.189189 17902 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:35.133445 17869 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.189908 17869 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:35.190663 16004 raft_consensus.cc:1240] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:35.182713 17818 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.192005 17869 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:35.192538 17869 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:35.193009 17869 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:35.194447 16562 raft_consensus.cc:1275] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:05:35.194555 17784 consensus_queue.cc:1059] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:35.194995 17869 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:35.195564 17784 raft_consensus.cc:3055] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:35.195943 17784 raft_consensus.cc:740] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:35.180843 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:35.190301 16007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 333) took 2239 ms. Trace:
I20260709 13:05:35.132962 17904 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:35.188736 17803 raft_consensus.cc:740] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:35.197152 16007 rpcz_store.cc:276] 0709 13:05:32.950611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.591841 (+1641230us) service_pool.cc:224] Handling call
0709 13:05:35.039712 (+447871us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:35.051911 (+ 12199us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:35.190112 (+138201us) spinlock_profiling.cc:243] Waited 447 ms on lock 0x7b54001f86d0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:35.190242 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1028740992}
I20260709 13:05:35.198690 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:35.147217 15902 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:05:35.177029 17884 consensus_meta.cc:287] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.608s	user 0.006s	sys 0.004s
I20260709 13:05:35.159461 15827 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:35.200908 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:05:35.160075 15974 connection.cc:256] RPC call timeout handler was delayed by -1.02014275800002s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1534s for a response
I20260709 13:05:35.090315 16005 rpcz_store.cc:276] 0709 13:05:33.198845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.765513 (+1566668us) service_pool.cc:224] Handling call
0709 13:05:34.812214 (+ 46701us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:35.038963 (+226749us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:35.058829 (+ 19866us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:35.199224 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:35.216877 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 69760.7us
I20260709 13:05:35.220675 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:05:35.217211 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 57809.4us
W20260709 13:05:35.152848 17724 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.004s	sys 0.016s
I20260709 13:05:35.243562 15827 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:35.221590 16568 raft_consensus.cc:2393] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 3: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:35.217005 15974 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:05:35.202903 17866 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.016s	sys 0.027s
I20260709 13:05:35.245419 15974 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:35.236829 17904 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:35.254871 17869 raft_consensus.cc:2749] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:35.243484 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:35.242882 17822 consensus_queue.cc:1048] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:05:35.256204 15974 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:35.237216 17876 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:35.261843 17822 consensus_queue.cc:1059] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:35.262207 17869 raft_consensus.cc:2804] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:35.262800 17869 raft_consensus.cc:697] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:35.252249 17819 consensus_queue.cc:1059] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:05:35.263909 17724 raft_consensus.cc:3055] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:35.196813 17784 consensus_queue.cc:260] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:35.264298 17724 raft_consensus.cc:740] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:35.264182 17869 consensus_queue.cc:237] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:35.265195 17724 consensus_queue.cc:260] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:35.266471 17724 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:35.266958 17826 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.242956 17928 consensus_queue.cc:1048] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 13:05:35.267462 17826 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:35.268064 17925 consensus_queue.cc:1059] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:35.269284 17928 raft_consensus.cc:3055] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:35.269598 17826 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:35.269704 17928 raft_consensus.cc:740] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:35.259020 17904 raft_consensus.cc:697] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:35.270653 17928 consensus_queue.cc:260] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:35.270812 17904 consensus_queue.cc:237] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:35.236469 17865 consensus_queue.cc:1059] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 13:05:35.271982 16011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1671) took 2194 ms. Trace:
I20260709 13:05:35.272197 17865 raft_consensus.cc:3055] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:35.272710 17865 raft_consensus.cc:740] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:35.272360 16011 rpcz_store.cc:276] 0709 13:05:33.077724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.688686 (+1610962us) service_pool.cc:224] Handling call
0709 13:05:34.725811 (+ 37125us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:34.726126 (+   315us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:34.726152 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:34.770585 (+ 44433us) log.cc:844] Serialized 556 byte log entry
0709 13:05:35.121694 (+351109us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:35.121748 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:35.121839 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:35.271781 (+149942us) raft_consensus.cc:1685] finished
0709 13:05:35.271799 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:35.271930 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":350694,"threads_started":1,"wal-append.queue_time_us":497439}
I20260709 13:05:35.243052 17653 consensus_queue.cc:1048] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 13:05:35.243157 17880 consensus_queue.cc:1048] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:05:35.236040 17584 consensus_queue.cc:1059] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:05:35.250751 17884 raft_consensus.cc:2804] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:35.242663 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:35.259958 17803 consensus_queue.cc:260] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:35.251904 17820 consensus_queue.cc:1059] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 13:05:35.242833 17748 consensus_queue.cc:1059] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 13:05:35.273968 15902 leader_election.cc:400] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 2. Current term is 3.
W20260709 13:05:35.274312 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:35.246994 17866 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.017s	sys 0.027s
I20260709 13:05:35.292378 15902 leader_election.cc:403] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:35.274646 15974 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:35.279935 15976 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:35.192550 17818 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:35.238096 17792 consensus_queue.cc:1059] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.022s
W20260709 13:05:35.296277 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:35.300477 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:35.302261 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 60000.7us
I20260709 13:05:35.303552 15902 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:35.303835 17627 consensus_queue.cc:1048] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:35.304744 15902 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:35.306264 15929 raft_consensus.cc:2393] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 2: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:35.307492 17931 raft_consensus.cc:2749] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 2. Current term is 3.
I20260709 13:05:35.307835 17929 raft_consensus.cc:2804] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:35.308701 17929 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:35.309147 17929 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:35.308847 17931 consensus_queue.cc:1059] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:35.310400 17931 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:35.310971 17931 raft_consensus.cc:697] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:35.311954 17871 raft_consensus.cc:2804] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:35.312237 17913 raft_consensus.cc:3055] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:35.312525 17748 raft_consensus.cc:3055] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:35.312670 17913 raft_consensus.cc:740] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:35.312945 17748 raft_consensus.cc:740] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:35.313581 17913 consensus_queue.cc:260] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:35.313735 17748 consensus_queue.cc:260] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:35.314340 17858 raft_consensus.cc:3055] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:35.314801 17913 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:35.284245 17826 consensus_queue.cc:1059] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:05:35.316519 17936 raft_consensus.cc:3055] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:35.316718 17826 raft_consensus.cc:3055] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:35.316963 17936 raft_consensus.cc:740] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:35.317287 17826 raft_consensus.cc:740] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:35.316036 17803 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:35.317903 17936 consensus_queue.cc:260] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:35.319178 17936 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:35.325830 17763 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:35.286054 17884 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:35.328032 17884 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:35.329664 15859 raft_consensus.cc:1240] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:35.306342 15902 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:35.337037 15902 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:35.257524 17930 raft_consensus.cc:2804] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:35.353596 15902 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:35.351423 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:35.355329 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:35.360834 16011 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
I20260709 13:05:35.314847 17858 raft_consensus.cc:740] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:35.363082 16568 raft_consensus.cc:2410] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
W20260709 13:05:35.375664 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 75987.6us
W20260709 13:05:35.376609 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:35.314805 17748 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:35.382006 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 46316us
I20260709 13:05:35.383023 15974 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:35.384547 15974 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:35.330940 17938 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:35.351631 17769 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:35.358177 17937 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:35.391947 17815 consensus_queue.cc:1059] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:35.392628 17902 raft_consensus.cc:3055] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:35.393281 17819 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:35.397325 17820 consensus_queue.cc:1059] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:05:35.397980 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:35.398334 17885 consensus_queue.cc:1059] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:35.399127 17792 consensus_queue.cc:1059] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:35.404547 17929 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:35.410912 17803 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:35.390658 17931 consensus_queue.cc:237] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:05:35.415606 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 393618us
I20260709 13:05:35.388590 17939 raft_consensus.cc:2749] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:35.284423 17865 consensus_queue.cc:260] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:05:35.422076 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 272013us
I20260709 13:05:35.461156 15900 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:35.329054 17826 consensus_queue.cc:260] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:35.462826 17826 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:35.402979 17884 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:35.387022 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:35.388779 17584 raft_consensus.cc:3055] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:35.465564 17584 raft_consensus.cc:740] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:35.358837 17930 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:35.418191 17902 raft_consensus.cc:740] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:35.418301 17938 raft_consensus.cc:697] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:35.432597 17763 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.424975 17937 raft_consensus.cc:697] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:35.517256 15855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 384) took 2082 ms. Trace:
I20260709 13:05:35.493394 15859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 386) took 2058 ms. Trace:
I20260709 13:05:35.521492 15859 rpcz_store.cc:276] 0709 13:05:33.435178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.342784 (+907606us) service_pool.cc:224] Handling call
0709 13:05:34.417217 (+ 74433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:35.330072 (+912855us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:35.493082 (+163010us) spinlock_profiling.cc:243] Waited 912 ms on lock 0x7b5400218c54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:35.493317 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2097955712}
I20260709 13:05:35.495023 17724 raft_consensus.cc:3055] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:35.496966 15856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 385) took 2061 ms. Trace:
I20260709 13:05:35.431802 17769 raft_consensus.cc:2804] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:35.455425 17818 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:35.435667 17913 raft_consensus.cc:3055] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:35.459589 17881 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:35.531338 16535 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:05:35.512568 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:35.484615 17932 raft_consensus.cc:2749] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:05:35.545058 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 299009us
I20260709 13:05:35.546084 16003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 338) took 2415 ms. Trace:
I20260709 13:05:35.541590 17871 raft_consensus.cc:697] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:35.549623 17871 consensus_queue.cc:237] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:35.530871 15856 rpcz_store.cc:276] 0709 13:05:33.435035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.344171 (+909136us) service_pool.cc:224] Handling call
0709 13:05:34.445510 (+101339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.445720 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:35.136124 (+690404us) log.cc:844] Serialized 14 byte log entry
0709 13:05:35.496487 (+360363us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:35.496568 (+    81us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:35.496636 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:35.496723 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:35.496902 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":81662,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":348,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95,"thread_start_us":1018090,"threads_started":2}
I20260709 13:05:35.390118 17858 consensus_queue.cc:260] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:35.486773 17930 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:35.529407 17876 raft_consensus.cc:697] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 6 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:35.434526 17819 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:35.562840 15825 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:35.565483 16006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1669) took 2489 ms. Trace:
I20260709 13:05:35.520061 17865 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:35.566380 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 212847us
I20260709 13:05:35.572515 16009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1679) took 1217 ms. Trace:
I20260709 13:05:35.530736 17724 raft_consensus.cc:740] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:35.546470 16003 rpcz_store.cc:276] 0709 13:05:33.130350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.723775 (+1593425us) service_pool.cc:224] Handling call
0709 13:05:34.742596 (+ 18821us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.742784 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:35.217760 (+474976us) log.cc:844] Serialized 14 byte log entry
0709 13:05:35.545757 (+327997us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:35.545815 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:35.545859 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:35.545930 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:35.546034 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17328,"prepare.queue_time_us":516831,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":37,"thread_start_us":732527,"threads_started":2}
W20260709 13:05:35.576388 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 325437us
I20260709 13:05:35.545461 17913 raft_consensus.cc:740] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:05:35.591022 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:35.591440 15827 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:05:35.601496 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 217016us
I20260709 13:05:35.607987 15857 raft_consensus.cc:1240] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:35.521006 17763 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:35.588392 16009 rpcz_store.cc:276] 0709 13:05:34.354737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.946666 (+591929us) service_pool.cc:224] Handling call
0709 13:05:35.014935 (+ 68269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:35.015155 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:35.359162 (+344007us) log.cc:844] Serialized 14 byte log entry
0709 13:05:35.572166 (+213004us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:35.572219 (+    53us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:35.572267 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:35.572336 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:35.572459 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":244655,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":26,"raft.queue_time_us":83,"thread_start_us":535513,"threads_started":2}
I20260709 13:05:35.623831 15859 raft_consensus.cc:1240] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:35.624656 15857 raft_consensus.cc:1240] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:35.486379 17584 consensus_queue.cc:260] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:35.618120 15974 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:35.561894 17653 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
W20260709 13:05:35.628712 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 167630us
I20260709 13:05:35.633350 15827 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:35.563338 17881 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:35.630679 15856 raft_consensus.cc:1240] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:35.608412 17866 raft_consensus.cc:2764] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:05:35.636483 16010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1668) took 2563 ms. Trace:
I20260709 13:05:35.567373 17880 consensus_queue.cc:1059] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.055s
I20260709 13:05:35.643457 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:35.643518 16010 rpcz_store.cc:276] 0709 13:05:33.073609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.666895 (+1593286us) service_pool.cc:224] Handling call
0709 13:05:34.667154 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.681517 (+ 14363us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:05:35.060064 (+378547us) log.cc:844] Serialized 14 byte log entry
0709 13:05:35.636092 (+576028us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:35.636145 (+    53us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:35.636206 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:35.636280 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:35.636418 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1086,"prepare.queue_time_us":558904,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":78896,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":12289,"thread_start_us":930076,"threads_started":3,"wal-append.queue_time_us":568242}
I20260709 13:05:35.638058 15974 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:35.622386 17803 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:35.670426 15931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 389) took 2003 ms. Trace:
W20260709 13:05:35.673682 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:35.630004 17858 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:35.671025 15931 rpcz_store.cc:276] 0709 13:05:33.667044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.722614 (+1055570us) service_pool.cc:224] Handling call
0709 13:05:34.742497 (+ 19883us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.742722 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:05:35.236898 (+494176us) log.cc:844] Serialized 14 byte log entry
0709 13:05:35.669999 (+433101us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:35.670054 (+    55us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:35.670097 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:35.670167 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:35.670356 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":716990,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":6868,"thread_start_us":905292,"threads_started":3}
I20260709 13:05:35.670812 15974 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:35.675148 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:35.678843 15859 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:35.679920 15974 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:35.682082 15974 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:35.689209 15974 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:35.672981 17784 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:35.589922 17902 consensus_queue.cc:260] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:35.690575 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 159349us
I20260709 13:05:35.650523 17869 raft_consensus.cc:2749] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:35.675457 17818 raft_consensus.cc:2749] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:35.627878 17932 consensus_queue.cc:1048] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
W20260709 13:05:35.699983 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 66727.8us
I20260709 13:05:35.680435 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:35.695905 16533 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:35.685197 17584 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:35.687485 17820 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:35.684479 17939 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:35.676644 17958 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:05:35.695308 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:35.707866 16533 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:35.717764 16007 raft_consensus.cc:1275] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:35.724483 16568 raft_consensus.cc:1240] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 2. Current term is 3. Ops: []
W20260709 13:05:35.726796 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 139022us
I20260709 13:05:35.727655 15902 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:35.737514 16010 raft_consensus.cc:1275] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:05:35.724926 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:35.746621 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:35.747853 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:35.748699 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:35.697613 17884 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:35.632567 17938 consensus_queue.cc:237] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:35.669035 17724 consensus_queue.cc:260] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:35.702214 17769 raft_consensus.cc:697] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:35.699950 17792 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.723230 17869 raft_consensus.cc:2749] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:35.804178 16566 raft_consensus.cc:3055] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:35.794420 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:05:35.806195 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:35.735554 17958 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:35.808439 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:35.651489 17937 consensus_queue.cc:237] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:35.809602 16569 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEARNER]: Advancing to term 4
I20260709 13:05:35.685779 17913 consensus_queue.cc:260] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:35.810093 16566 raft_consensus.cc:740] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:35.740047 17929 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:35.744470 17902 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:35.584019 16006 rpcz_store.cc:276] 0709 13:05:33.076357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.681400 (+1605043us) service_pool.cc:224] Handling call
0709 13:05:34.681776 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.692872 (+ 11096us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:05:35.242961 (+550089us) log.cc:844] Serialized 14 byte log entry
0709 13:05:35.564890 (+321929us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:35.564960 (+    70us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:35.565109 (+   149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:35.565199 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:35.565354 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15736,"prepare.queue_time_us":566484,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":2630,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":7421,"thread_start_us":813230,"threads_started":2}
I20260709 13:05:35.815126 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:35.735793 17818 raft_consensus.cc:2749] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:35.687740 17876 consensus_queue.cc:237] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:35.749876 17653 raft_consensus.cc:2804] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:05:35.834517 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 106946us
I20260709 13:05:35.724185 17866 consensus_queue.cc:1059] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.100s
I20260709 13:05:35.789649 17627 raft_consensus.cc:2764] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260709 13:05:35.799142 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:35.799233 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:35.852351 16568 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
W20260709 13:05:35.610685 17904 consensus_queue.cc:1175] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "31882606e87447fb9a36bbf625129fe8" responder_term: 6 status { last_received { term: 5 index: 4 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 5. Current term is 6. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:05:35.885360 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:35.855400 16569 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEARNER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:35.798991 17939 raft_consensus.cc:697] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:35.851882 16566 consensus_queue.cc:260] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:35.887276 16566 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:35.832899 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:35.876459 17969 consensus_queue.cc:1048] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:05:35.871460 17961 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:35.894467 17961 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:35.881500 17957 consensus_queue.cc:1059] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 13:05:35.896596 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:35.896934 17957 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:35.897475 16009 raft_consensus.cc:2393] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 3: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:35.897557 17957 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:35.899364 17960 log.cc:927] Time spent T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 13:05:35.900180 17955 log.cc:927] Time spent T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
W20260709 13:05:35.900911 17963 log.cc:927] Time spent T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.216s	user 0.001s	sys 0.000s
W20260709 13:05:35.901726 17950 log.cc:927] Time spent T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.226s	user 0.000s	sys 0.001s
I20260709 13:05:35.904356 17981 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.904847 17981 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:35.905891 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:35.906652 17819 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:35.906910 17981 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:35.906507 17769 consensus_queue.cc:237] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:05:35.907691 17975 log.cc:927] Time spent T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:05:35.908634 15860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 382) took 2474 ms. Trace:
I20260709 13:05:35.812970 17792 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:35.871060 16008 raft_consensus.cc:1240] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:35.844813 17968 raft_consensus.cc:2804] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:35.743701 17820 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:35.877722 16565 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
I20260709 13:05:35.839318 17958 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:35.879727 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:35.881798 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:35.882598 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:35.882886 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:35.929744 16007 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:35.930099 16011 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
I20260709 13:05:35.888815 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:05:35.929607 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 183824us
I20260709 13:05:35.932013 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:35.871858 17724 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:35.874112 17871 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.935995 17871 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:35.938578 17871 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:35.891551 17956 log.cc:927] Time spent T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.153s	user 0.000s	sys 0.000s
W20260709 13:05:35.892283 17951 log.cc:927] Time spent T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.197s	user 0.000s	sys 0.000s
I20260709 13:05:35.909068 15860 rpcz_store.cc:276] 0709 13:05:33.434602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.270181 (+835579us) service_pool.cc:224] Handling call
0709 13:05:34.270804 (+   623us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:35.021809 (+751005us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:35.021837 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:35.039323 (+ 17486us) log.cc:844] Serialized 26 byte log entry
0709 13:05:35.458816 (+419493us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:35.458957 (+   141us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:35.461460 (+  2503us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:35.908429 (+446969us) raft_consensus.cc:1685] finished
0709 13:05:35.908448 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:35.908576 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":188824,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":8079,"thread_start_us":1124432,"threads_started":2,"wal-append.queue_time_us":533795}
I20260709 13:05:35.870836 15902 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:35.916980 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:35.907471 17981 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.940490 15935 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
I20260709 13:05:35.879277 16004 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:35.940598 17981 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:05:35.926630 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:35.942724 15934 raft_consensus.cc:2393] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 3: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:35.888619 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:35.943779 16570 raft_consensus.cc:2393] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 2: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:35.945602 17803 raft_consensus.cc:697] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:35.519440 15855 rpcz_store.cc:276] 0709 13:05:33.434898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.342387 (+907489us) service_pool.cc:224] Handling call
0709 13:05:34.342664 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:34.342851 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:05:35.140626 (+797775us) log.cc:844] Serialized 14 byte log entry
0709 13:05:35.516816 (+376190us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:35.516877 (+    61us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:35.516937 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:35.517067 (+   130us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:35.517197 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":184271,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":2425,"thread_start_us":1081620,"threads_started":2}
I20260709 13:05:35.968676 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:05:35.972844 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 53359us
I20260709 13:05:35.959522 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:35.992477 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:35.990293 15934 raft_consensus.cc:2376] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5. Re-sending same reply.
I20260709 13:05:35.911170 17763 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:36.006799 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:05:35.970456 17826 consensus_meta.cc:287] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.507s	user 0.007s	sys 0.003s
I20260709 13:05:36.006274 15825 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:05:36.011890 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 322728us
I20260709 13:05:36.014927 15974 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:05:36.013643 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:36.016395 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:35.960074 17819 raft_consensus.cc:2749] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:35.968312 17653 raft_consensus.cc:697] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:35.926921 17983 raft_consensus.cc:2749] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:36.029058 17971 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.029662 17971 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:35.968308 17957 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:36.023973 17954 log.cc:927] Time spent T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 13:05:36.024474 17979 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:35.996555 17959 raft_consensus.cc:3055] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:35.999626 17982 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:36.021529 17932 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:35.928464 17968 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:36.059420 17978 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:36.002530 17881 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:36.065112 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 194335us
I20260709 13:05:36.027174 17976 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:36.064531 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:36.150040 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:36.157132 16570 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:36.066370 17991 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:36.066956 17987 consensus_queue.cc:1059] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:05:36.164714 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 326513us
I20260709 13:05:36.027856 17992 consensus_queue.cc:1059] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:05:36.073900 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:36.170555 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:36.064967 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:36.081274 16566 raft_consensus.cc:1275] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:05:36.171859 15937 raft_consensus.cc:2410] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 13:05:36.132150 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 187880us
I20260709 13:05:36.171993 15935 raft_consensus.cc:2468] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:05:36.063446 17979 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:36.096266 17968 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:36.065299 17959 raft_consensus.cc:740] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:35.997367 17958 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:35.957366 17803 consensus_queue.cc:237] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:36.078382 15928 raft_consensus.cc:1275] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:05:36.144085 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:36.145910 15825 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:36.199864 15862 raft_consensus.cc:2368] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 3. Current term is 4.
W20260709 13:05:36.199949 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 54104.5us
W20260709 13:05:36.164407 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 149555us
W20260709 13:05:36.165179 15900 connection.cc:256] RPC call timeout handler was delayed by -1.6214880830000311s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1799s for a response
I20260709 13:05:36.066993 17974 consensus_queue.cc:1059] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:05:36.166800 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:36.170341 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:36.202847 16008 raft_consensus.cc:2393] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 4: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:36.203789 16009 raft_consensus.cc:2393] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 3: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
W20260709 13:05:36.202298 15900 connection.cc:256] RPC call timeout handler was delayed by -1.5413880830000153s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1888s for a response
W20260709 13:05:36.171456 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:36.173313 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:36.068802 17971 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:36.176121 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:36.178275 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:36.174160 17930 consensus_meta.cc:287] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.582s	user 0.008s	sys 0.002s
I20260709 13:05:36.184212 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:05:36.185992 15827 connection.cc:256] RPC call timeout handler was delayed by -1.4560579740000321s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1696s for a response
I20260709 13:05:36.192061 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:05:36.193231 17858 consensus_meta.cc:287] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.509s	user 0.008s	sys 0.000s
I20260709 13:05:36.194922 16564 raft_consensus.cc:2955] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:36.196412 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:36.196856 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:36.197781 17992 consensus_queue.cc:1059] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:36.197605 17653 consensus_queue.cc:237] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:36.201730 15974 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:36.214247 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:36.178260 17979 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:36.233387 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:36.234764 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:36.203356 17991 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:36.159549 17931 consensus_queue.cc:1048] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:05:36.222326 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:36.243995 15932 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:05:36.212875 17866 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:36.187321 17820 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:36.225999 16567 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:05:36.199187 17881 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.197532 17784 consensus_queue.cc:1059] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
W20260709 13:05:36.220849 15827 connection.cc:256] RPC call timeout handler was delayed by -1.3651579739999988s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1797s for a response
I20260709 13:05:36.225404 17957 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:36.218812 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:36.235690 16003 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
I20260709 13:05:36.242178 15855 raft_consensus.cc:1275] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:05:36.242769 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:36.243409 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:36.212778 17986 raft_consensus.cc:2804] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:36.203881 17973 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:36.078212 17871 raft_consensus.cc:2955] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:36.155527 17978 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:36.224406 17959 consensus_queue.cc:260] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:05:36.249112 15974 connection.cc:256] RPC call timeout handler was delayed by -1.0942160670000476s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21730000000000002s for a response
I20260709 13:05:36.310406 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:05:36.366217 15974 connection.cc:256] RPC call timeout handler was delayed by -1.0672160670000039s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2203s for a response
I20260709 13:05:36.362254 15857 raft_consensus.cc:2393] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 2: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
W20260709 13:05:36.362835 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:36.230098 17763 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:36.354605 16008 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:36.300870 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:36.298233 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:05:36.362775 15827 connection.cc:256] RPC call timeout handler was delayed by -1.2814579739999772s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.189s for a response
I20260709 13:05:36.297783 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:05:36.370882 15827 connection.cc:256] RPC call timeout handler was delayed by -1.0564666919999581s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1626s for a response
I20260709 13:05:36.298955 16566 raft_consensus.cc:1240] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:36.140204 17939 consensus_queue.cc:237] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:36.247629 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 66072us
I20260709 13:05:36.249054 17987 raft_consensus.cc:3055] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:36.374545 17987 raft_consensus.cc:740] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:36.375344 17987 consensus_queue.cc:260] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:36.376624 17987 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:36.314566 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:36.377784 15854 raft_consensus.cc:2410] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:05:36.378765 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:36.330264 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:36.371928 15858 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 2.
I20260709 13:05:36.381328 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:36.383324 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:36.398022 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:36.238848 16005 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:05:36.319289 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:05:36.428335 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:36.429441 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:36.434085 16003 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:05:36.380935 18000 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.314954 17792 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:36.381989 16002 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
W20260709 13:05:36.472585 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 258347us
I20260709 13:05:36.371454 17973 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:36.367713 17957 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:36.271878 17866 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:36.431810 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:36.485934 15935 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
W20260709 13:05:36.433527 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:36.248314 17958 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:36.431584 17820 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.437111 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:36.441015 17952 log.cc:927] Time spent T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.256s	user 0.001s	sys 0.000s
I20260709 13:05:36.443749 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:36.445539 17978 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:36.407739 17968 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:36.432833 17990 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:36.448796 17880 raft_consensus.cc:3055] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:36.448890 18004 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.448714 17885 raft_consensus.cc:3055] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:36.272135 17991 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:36.457583 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:05:36.458211 17940 log.cc:927] Time spent T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.906s	user 0.008s	sys 0.005s
W20260709 13:05:36.458622 17936 consensus_meta.cc:287] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 1.139s	user 0.013s	sys 0.000s
I20260709 13:05:36.440804 16570 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 2.
I20260709 13:05:36.459199 17982 raft_consensus.cc:2749] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:05:36.459684 17998 log.cc:927] Time spent T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.079s	user 0.002s	sys 0.000s
I20260709 13:05:36.461016 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:36.509783 15976 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:36.462033 17724 raft_consensus.cc:3055] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:36.462298 17979 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:36.226043 17930 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:36.469671 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:05:36.473516 15900 leader_election.cc:336] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.799s (SENT)
I20260709 13:05:36.401084 15855 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
W20260709 13:05:36.519105 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 45614.9us
W20260709 13:05:36.475325 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 83992.3us
I20260709 13:05:36.428442 15854 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
I20260709 13:05:36.368438 17881 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:36.484052 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 117339us
W20260709 13:05:36.548259 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:36.472023 18000 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:36.508131 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:36.496923 15932 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:36.497130 15860 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
I20260709 13:05:36.397891 17763 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:36.504467 15825 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:36.509240 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:36.553337 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:36.554216 15936 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
W20260709 13:05:36.554461 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 50111.1us
I20260709 13:05:36.524046 17627 raft_consensus.cc:2804] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:05:36.516657 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:36.520378 15900 leader_election.cc:336] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.888s (SENT)
I20260709 13:05:36.523057 15827 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:36.527848 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:36.562304 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:36.562413 15900 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:36.537523 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:36.502960 17885 raft_consensus.cc:740] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:36.487952 16533 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:36.517865 16005 raft_consensus.cc:1240] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:36.497622 17820 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:36.551824 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:36.577625 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:36.516434 17724 raft_consensus.cc:740] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:36.507474 17986 raft_consensus.cc:697] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:36.498539 17880 raft_consensus.cc:740] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:36.529799 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:36.563189 15936 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:36.563297 15855 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
W20260709 13:05:36.563793 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 75943.1us
I20260709 13:05:36.566676 18016 raft_consensus.cc:2749] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:36.556865 16561 raft_consensus.cc:2368] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 3. Current term is 4.
I20260709 13:05:36.505723 18004 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:36.568181 15853 raft_consensus.cc:2393] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 5: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:36.567996 17967 consensus_queue.cc:1048] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
W20260709 13:05:36.569176 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 48820.5us
I20260709 13:05:36.569381 17973 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:36.569715 18000 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:36.569741 17982 raft_consensus.cc:2749] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:36.551426 17939 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.561592 17999 raft_consensus.cc:2804] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:36.572500 15856 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:36.629851 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:05:36.580515 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 57565.9us
W20260709 13:05:36.556058 17902 consensus_meta.cc:287] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.699s	user 0.011s	sys 0.002s
I20260709 13:05:36.584156 16008 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
W20260709 13:05:36.587383 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:36.591506 15932 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:36.600354 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:36.574664 17792 raft_consensus.cc:2749] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:05:36.613961 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:36.572153 17627 raft_consensus.cc:697] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:36.649613 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:36.634341 17939 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:36.650588 18009 log.cc:927] Time spent T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:05:36.650353 17627 consensus_queue.cc:237] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:36.652130 18003 log.cc:927] Time spent T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 13:05:36.652382 17939 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:36.652619 17994 consensus_queue.cc:1059] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:36.631958 15827 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:36.640133 15855 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:36.654222 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:36.655153 16568 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:36.642225 15936 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:36.655737 15827 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:36.644044 16533 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:36.648844 15857 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2.
I20260709 13:05:36.651257 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:36.665853 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:36.616104 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:36.672087 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:36.673192 15934 raft_consensus.cc:2393] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 3: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:36.674762 16533 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:36.658851 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:36.677861 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:36.679044 15936 raft_consensus.cc:2410] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:05:36.621482 17999 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:36.601984 17653 consensus_queue.cc:1048] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 13:05:36.687130 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:36.694967 15936 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
I20260709 13:05:36.689488 16533 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:36.697068 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:36.697469 16533 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:36.656672 17881 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:36.699183 16533 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:36.699918 15825 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:36.698004 15934 raft_consensus.cc:2393] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 6: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:05:36.748876 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 49100.7us
I20260709 13:05:36.660164 18016 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:36.633020 17983 consensus_queue.cc:1059] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:05:36.677464 17958 raft_consensus.cc:2749] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:36.751025 15855 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:05:36.702943 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:36.762836 17993 raft_consensus.cc:2749] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:36.763969 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:36.764653 15855 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
W20260709 13:05:36.764891 18010 log.cc:927] Time spent T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.216s	user 0.001s	sys 0.000s
I20260709 13:05:36.765628 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:36.683758 17792 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:36.682456 17978 consensus_queue.cc:1059] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.082s
I20260709 13:05:36.762913 15936 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:36.767832 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 68736.3us
I20260709 13:05:36.617271 17992 consensus_queue.cc:1059] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.082s
I20260709 13:05:36.693199 17999 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:05:36.772671 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:36.692832 17987 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:36.700896 17784 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:36.670974 17876 consensus_queue.cc:1048] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.102s
I20260709 13:05:36.746560 17957 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:36.790639 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:36.773723 17985 raft_consensus.cc:2749] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:36.769549 17881 raft_consensus.cc:2804] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:36.762596 17968 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:36.679945 17724 consensus_queue.cc:260] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:36.850457 15936 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:05:36.851613 15934 raft_consensus.cc:2376] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2. Re-sending same reply.
I20260709 13:05:36.683142 17982 consensus_queue.cc:1059] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.101s
I20260709 13:05:36.742527 17973 consensus_queue.cc:1059] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.090s
I20260709 13:05:36.702042 17769 consensus_queue.cc:1048] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.091s
I20260709 13:05:36.784471 17993 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.859877 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:36.680994 17885 consensus_queue.cc:260] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:36.864073 15825 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:36.846742 17988 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.866919 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:36.873051 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:36.852133 17792 raft_consensus.cc:697] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:05:36.848680 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 184071us
I20260709 13:05:36.845885 17987 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:36.861120 17881 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:36.772341 17986 consensus_queue.cc:237] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:36.770675 17880 consensus_queue.cc:260] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:36.864534 17979 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:36.877900 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:36.867254 17784 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.881512 15936 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
I20260709 13:05:36.872025 17724 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:36.875303 18000 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:36.933346 18017 log.cc:927] Time spent T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 13:05:36.933844 18000 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:36.936302 18000 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:36.883577 15827 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:36.887121 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:36.937367 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:05:36.885511 17932 consensus_meta.cc:287] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.789s	user 0.009s	sys 0.001s
I20260709 13:05:36.879490 17931 raft_consensus.cc:3055] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:36.880720 17978 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:36.937556 15935 raft_consensus.cc:2368] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 3. Current term is 4.
W20260709 13:05:36.886992 17865 consensus_meta.cc:287] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 1.284s	user 0.010s	sys 0.003s
I20260709 13:05:36.884749 17880 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:36.948107 17932 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:36.884181 17822 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.074s
I20260709 13:05:36.962684 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:36.963423 16008 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:36.875028 17988 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:36.878875 17987 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:36.877808 15928 raft_consensus.cc:2376] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5. Re-sending same reply.
I20260709 13:05:36.970942 15858 raft_consensus.cc:2410] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:05:36.940424 16533 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:36.884500 17784 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:36.882475 17881 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:36.953344 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 70871.5us
I20260709 13:05:36.999192 15827 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:37.000259 15827 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:36.888664 17991 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:36.954276 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:36.954957 17979 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:36.954561 17792 consensus_queue.cc:237] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:36.958413 16002 raft_consensus.cc:1240] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:36.966699 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:37.007007 15932 raft_consensus.cc:2376] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3. Re-sending same reply.
W20260709 13:05:36.884233 18015 log.cc:927] Time spent T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
I20260709 13:05:36.977691 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:37.009397 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:05:36.980014 17584 consensus_meta.cc:287] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 1.245s	user 0.010s	sys 0.002s
I20260709 13:05:36.981448 17913 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:36.969622 18011 log.cc:927] Time spent T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.331s	user 0.000s	sys 0.001s
I20260709 13:05:37.011898 15933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1606) took 1108 ms. Trace:
I20260709 13:05:37.012229 15933 rpcz_store.cc:276] 0709 13:05:35.903783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.907037 (+  3254us) service_pool.cc:224] Handling call
0709 13:05:35.907430 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:36.244724 (+337294us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:36.244770 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:36.295799 (+ 51029us) log.cc:844] Serialized 26 byte log entry
0709 13:05:36.497469 (+201670us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:36.497505 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:36.497576 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:37.011694 (+514118us) raft_consensus.cc:1685] finished
0709 13:05:37.011727 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:37.011846 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":88787,"raft.queue_time_us":120753,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":138,"thread_start_us":367754,"threads_started":2,"wal-append.queue_time_us":229339}
I20260709 13:05:37.014065 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:37.015112 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:05:37.015280 17871 consensus_meta.cc:287] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.517s	user 0.006s	sys 0.002s
W20260709 13:05:36.984910 18024 log.cc:927] Time spent T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:05:36.985116 17988 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:36.985522 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:36.985859 17961 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:36.948388 15853 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:05:36.994453 15974 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:36.885144 17981 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:36.996899 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:37.001089 16008 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:37.088894 15937 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 4.
I20260709 13:05:37.089526 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:37.090411 16008 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
I20260709 13:05:36.965354 17958 raft_consensus.cc:3055] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:36.948524 17931 raft_consensus.cc:740] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:37.004012 17979 raft_consensus.cc:697] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:36.983951 17820 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:37.015942 18019 log.cc:927] Time spent T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:05:36.996374 17976 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.017009 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:37.019775 15933 raft_consensus.cc:2393] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 3: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:37.025179 17928 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:05:37.047473 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:36.876459 17993 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:37.079432 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:36.972879 17978 raft_consensus.cc:697] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 4 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
W20260709 13:05:37.084357 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 89997.8us
I20260709 13:05:37.093490 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:37.115314 16006 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5.
I20260709 13:05:37.116505 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:37.114319 15974 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:37.027629 17865 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:37.094388 15900 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:37.104369 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:37.119351 15974 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:37.107613 15934 raft_consensus.cc:2410] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:05:37.080832 17925 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:37.094810 17961 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.118788 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:37.119741 15937 raft_consensus.cc:2393] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 6: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:37.099920 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:37.122609 15936 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 3: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:37.120546 15900 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:37.123042 16008 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:05:37.123296 16006 raft_consensus.cc:2393] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 3: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:37.095536 17958 raft_consensus.cc:740] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:37.125929 15974 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:37.127200 16569 raft_consensus.cc:1275] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:05:37.104619 17966 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:37.127643 15902 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:37.121521 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:37.131440 15974 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:37.131992 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:37.133251 15900 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:37.094496 18031 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:37.136546 15902 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:37.137010 15936 raft_consensus.cc:1240] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:37.141003 17969 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:37.141465 17969 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:05:37.141690 17949 log.cc:927] Time spent T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 1.464s	user 0.004s	sys 0.012s
I20260709 13:05:37.141862 17969 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:37.094835 18025 consensus_queue.cc:1048] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:05:37.145120 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:37.145318 17993 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:37.145844 16562 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 4.
I20260709 13:05:37.146687 17925 consensus_queue.cc:1048] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 13:05:37.134651 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:37.148874 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:37.137773 17988 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:37.149576 15861 raft_consensus.cc:2393] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 4: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:37.113061 17976 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:37.150283 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:37.150767 15861 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:37.151293 17976 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:37.105119 17876 consensus_queue.cc:1048] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 13:05:37.152045 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:37.152842 16008 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:37.138860 18031 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.153920 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:37.154376 18031 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:37.154862 16008 raft_consensus.cc:2393] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 3: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:37.138207 17967 raft_consensus.cc:2749] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:37.135097 17966 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:37.159821 17966 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:37.161278 17784 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:37.162495 17958 consensus_queue.cc:260] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:37.122151 17769 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:37.164276 16003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1691) took 1020 ms. Trace:
I20260709 13:05:37.128379 17627 raft_consensus.cc:2749] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:37.164706 16003 rpcz_store.cc:276] 0709 13:05:36.143383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.434988 (+291605us) service_pool.cc:224] Handling call
0709 13:05:36.480822 (+ 45834us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:36.481253 (+   431us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:36.481292 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:36.485380 (+  4088us) log.cc:844] Serialized 26 byte log entry
0709 13:05:36.614577 (+129197us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:36.614658 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:36.614726 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:37.164051 (+549325us) raft_consensus.cc:1685] finished
0709 13:05:37.164072 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:37.164210 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":738,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":11081,"thread_start_us":98265,"threads_started":1,"wal-append.queue_time_us":268736}
I20260709 13:05:37.165846 17979 consensus_queue.cc:237] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:37.164095 17958 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:37.185532 15933 raft_consensus.cc:2410] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:05:37.174614 15936 raft_consensus.cc:1240] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:37.187307 15936 raft_consensus.cc:1240] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:37.174675 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:37.187928 16008 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:37.188074 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 56712.6us
I20260709 13:05:37.175719 15976 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:37.177119 15937 raft_consensus.cc:1275] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 13:05:37.190905 15937 pending_rounds.cc:85] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 3
I20260709 13:05:37.177713 17931 consensus_queue.cc:260] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:37.131469 17961 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:37.184474 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 48002.4us
I20260709 13:05:37.192714 15902 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:37.193637 15902 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:37.188833 15974 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:37.216341 15936 raft_consensus.cc:1275] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:05:37.191205 15937 pending_rounds.cc:107] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:05:37.217062 15937 raft_consensus.cc:2889] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:37.122646 17938 raft_consensus.cc:2749] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:37.224341 15933 raft_consensus.cc:1275] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:05:37.227469 17961 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:37.216081 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:37.228083 17978 consensus_queue.cc:237] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:37.228859 16563 raft_consensus.cc:2393] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 4: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
W20260709 13:05:37.229290 16563 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45246 (request call id 553) took 1515 ms (client timeout 1509 ms). Trace:
W20260709 13:05:37.229674 16563 rpcz_store.cc:269] 0709 13:05:35.714282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.724923 (+ 10641us) service_pool.cc:224] Handling call
0709 13:05:37.191753 (+1466830us) spinlock_profiling.cc:243] Waited 6.93 ms on lock 0x7b5800044798. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283eee 0000000000375939 00007f73ef2f71e0 00007f73ef3c00c4 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:05:37.229223 (+ 37470us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15929728}
W20260709 13:05:37.231474 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 55871.9us
I20260709 13:05:37.232326 15937 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 1579) took 1073 ms. Trace:
I20260709 13:05:37.232692 15937 rpcz_store.cc:276] 0709 13:05:36.159161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:37.135551 (+976390us) service_pool.cc:224] Handling call
0709 13:05:37.176747 (+ 41196us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:37.232094 (+ 55347us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:37.232275 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:37.221232 17769 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:37.236836 17769 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.237339 17769 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:37.239764 15934 raft_consensus.cc:1240] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260709 13:05:37.240445 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 46872.6us
I20260709 13:05:37.241153 15902 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:37.242440 17982 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:37.242792 17982 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.243103 17982 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:37.244264 17987 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:37.246716 17987 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:37.129804 17999 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:37.247870 17976 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:37.248250 17976 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.248580 17976 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:37.249010 17999 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:37.189788 17988 raft_consensus.cc:697] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:37.251492 17988 consensus_queue.cc:237] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:37.260490 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 71721.8us
I20260709 13:05:37.261853 15974 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:37.266533 16008 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:05:37.267562 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:37.270918 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 172915us
I20260709 13:05:37.273504 18026 raft_consensus.cc:2804] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:37.272876 15827 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:37.274550 18005 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.275467 18005 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:37.277611 18005 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:37.279558 17976 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:37.281713 17976 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:37.285677 17982 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:37.290231 15827 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:37.293046 15827 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:37.296876 15860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 1692) took 1055 ms. Trace:
I20260709 13:05:37.298478 15860 rpcz_store.cc:276] 0709 13:05:36.241128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.243201 (+  2073us) service_pool.cc:224] Handling call
0709 13:05:37.296662 (+1053461us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b54002486d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:37.296803 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":305309952}
I20260709 13:05:37.255231 17967 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:37.299966 15860 raft_consensus.cc:1240] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:37.300657 15827 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:37.267211 17932 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:37.272217 17938 raft_consensus.cc:2804] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:05:37.315762 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:37.310885 15854 raft_consensus.cc:2393] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 7: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:37.267627 17871 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:37.325810 17871 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:37.326272 17871 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:37.326678 18031 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:37.328845 15855 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:05:37.329028 18031 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:37.221014 17939 consensus_queue.cc:1048] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.081s
I20260709 13:05:37.332722 17939 raft_consensus.cc:2804] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:37.333257 17939 raft_consensus.cc:697] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:37.334470 15859 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2.
I20260709 13:05:37.334218 17939 consensus_queue.cc:237] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:37.316563 17981 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:05:37.319368 18005 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.335976 18005 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:37.337993 18005 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:37.338132 16004 raft_consensus.cc:1275] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:05:37.341760 15827 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:37.342550 18043 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.343261 18043 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:37.346050 15827 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:37.320094 16008 raft_consensus.cc:1275] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:37.298324 17978 raft_consensus.cc:2749] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:37.308985 17986 raft_consensus.cc:2749] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:05:37.368474 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:37.342343 17981 consensus_queue.cc:1059] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:37.313012 17982 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:37.360860 18004 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:37.360998 17979 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:37.361348 18043 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:37.379874 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:37.318427 17932 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.323297 17815 raft_consensus.cc:3055] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:37.364836 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:05:37.371620 15827 leader_election.cc:336] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Timed out: RequestConsensusVote RPC to 127.14.103.4:43643 timed out after 1.509s (SENT)
I20260709 13:05:37.383389 15827 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:37.383430 15937 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
I20260709 13:05:37.364194 16004 raft_consensus.cc:2955] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:37.384668 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:37.385804 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:37.386510 15933 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:37.391202 15827 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:37.343251 17880 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 13:05:37.375710 17928 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:37.393955 15827 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:37.395224 16008 raft_consensus.cc:1240] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:37.383029 17932 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:37.400660 17928 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.383170 17815 raft_consensus.cc:740] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:37.407848 18038 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:37.408275 17928 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:37.411890 18045 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:37.412442 18045 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:37.413986 15853 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:37.416790 18026 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.417227 18026 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:37.407438 17815 consensus_queue.cc:260] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:37.375692 17881 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:37.404263 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:37.380998 17979 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:37.405684 17982 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:37.420421 15937 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:37.421135 16008 raft_consensus.cc:1275] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:37.394275 17986 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:37.427870 16008 raft_consensus.cc:1275] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:05:37.414628 18045 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:37.433842 18001 raft_consensus.cc:3055] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:37.403052 17978 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:37.415066 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:37.404234 17938 raft_consensus.cc:697] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:37.418385 17981 raft_consensus.cc:2749] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:37.387760 18004 consensus_queue.cc:1059] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
W20260709 13:05:37.428200 18041 log.cc:927] Time spent T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:05:37.436537 17978 raft_consensus.cc:697] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 4 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:37.422806 17979 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:37.420302 17966 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:37.457232 17966 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:37.458885 17928 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:37.462212 15937 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:37.420153 17969 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:37.464064 16533 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:37.441061 18001 raft_consensus.cc:740] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:05:37.474596 15827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:37.484100 15827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:37.430305 17982 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:37.493178 17982 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:37.477131 16533 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:37.462378 17938 consensus_queue.cc:237] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:37.494748 16533 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:37.495860 17784 raft_consensus.cc:2804] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:37.496225 17784 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.441740 15853 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:37.489384 17990 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.489526 17928 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:37.414013 17871 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:37.462767 17939 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:37.506592 15936 raft_consensus.cc:1240] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:37.471654 17865 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:37.511224 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:37.512323 15937 raft_consensus.cc:2368] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 3. Current term is 4.
I20260709 13:05:37.504354 17724 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.527109 17724 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:37.505944 18031 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.529278 18031 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:37.504554 17993 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.531169 17993 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:37.497215 17974 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:37.541034 17974 raft_consensus.cc:697] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:37.502005 18038 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.499660 17966 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.534816 15855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 1564) took 1147 ms. Trace:
I20260709 13:05:37.547094 15855 rpcz_store.cc:276] 0709 13:05:36.387207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.765317 (+378110us) service_pool.cc:224] Handling call
0709 13:05:37.534638 (+769321us) spinlock_profiling.cc:243] Waited 556 ms on lock 0x7b540006ed54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:37.534745 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1277956480}
I20260709 13:05:37.548153 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:37.548840 15855 raft_consensus.cc:2393] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 4: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:37.535712 15859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 1562) took 1149 ms. Trace:
I20260709 13:05:37.539602 15861 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:37.549170 15859 rpcz_store.cc:276] 0709 13:05:36.386630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.508383 (+121753us) service_pool.cc:224] Handling call
0709 13:05:37.535553 (+1027170us) spinlock_profiling.cc:243] Waited 757 ms on lock 0x7b5400219654. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:37.535646 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1741078784}
I20260709 13:05:37.549485 15861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 1566) took 1161 ms. Trace:
I20260709 13:05:37.533582 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:37.549988 15861 rpcz_store.cc:276] 0709 13:05:36.387794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:37.150081 (+762287us) service_pool.cc:224] Handling call
0709 13:05:37.549413 (+399332us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:37.550514 16009 raft_consensus.cc:2393] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 5: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
W20260709 13:05:37.550930 16009 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60156 (request call id 351) took 1382 ms (client timeout 1619 ms). Trace:
W20260709 13:05:37.551280 16009 rpcz_store.cc:269] 0709 13:05:36.168364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.213383 (+ 45019us) service_pool.cc:224] Handling call
0709 13:05:37.533421 (+1320038us) spinlock_profiling.cc:243] Waited 5.27 ms on lock 0x7b5800114398. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:05:37.550876 (+ 17455us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12129152}
I20260709 13:05:37.520746 17939 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:37.496590 17784 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:37.502708 17990 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:37.533516 17978 consensus_queue.cc:237] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:05:37.555351 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:37.556437 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:37.557001 15859 raft_consensus.cc:2376] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2. Re-sending same reply.
I20260709 13:05:37.502544 17985 consensus_queue.cc:1059] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:05:37.565536 17970 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:37.572554 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:37.569088 17974 consensus_queue.cc:237] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:37.578145 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:37.577319 15853 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:37.581259 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:37.582026 15853 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:37.554004 17966 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:37.558625 18038 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:37.594718 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:37.595412 15859 raft_consensus.cc:2376] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2. Re-sending same reply.
I20260709 13:05:37.593326 18047 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:37.596637 18047 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:37.602757 18047 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:37.594826 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:37.605845 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:37.606249 15902 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:37.609194 15861 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 2.
I20260709 13:05:37.612390 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:37.613734 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:37.589300 17970 raft_consensus.cc:697] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:37.584754 17932 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:37.620021 15853 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:05:37.621337 17932 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:37.621291 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:37.622622 15853 raft_consensus.cc:2376] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3. Re-sending same reply.
I20260709 13:05:37.617839 18000 consensus_queue.cc:1059] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 13:05:37.623783 18000 raft_consensus.cc:3055] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:37.624205 18000 raft_consensus.cc:740] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:37.625201 18000 consensus_queue.cc:260] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:37.626657 18000 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:37.616324 17973 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:37.629784 17973 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.630223 17973 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:37.632280 17930 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:37.634562 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:37.636062 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:37.643174 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:37.609844 17881 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:37.645381 15862 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58214 (request call id 398) took 2032 ms (client timeout 1800 ms). Trace:
I20260709 13:05:37.646013 15934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 1572) took 1489 ms. Trace:
I20260709 13:05:37.646389 15934 rpcz_store.cc:276] 0709 13:05:36.157160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:37.134203 (+977043us) service_pool.cc:224] Handling call
0709 13:05:37.140381 (+  6178us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:37.240164 (+ 99783us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:37.645773 (+405609us) spinlock_profiling.cc:243] Waited 94.4 ms on lock 0x7b540044bbd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:37.645953 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":217238272}
I20260709 13:05:37.647881 17885 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:37.645819 15862 rpcz_store.cc:269] 0709 13:05:35.612643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.612794 (+   151us) service_pool.cc:224] Handling call
0709 13:05:35.882446 (+269652us) spinlock_profiling.cc:243] Waited 2.44 ms on lock 0x7b5800005798. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:05:37.645191 (+1762745us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b54003b6854. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:37.645319 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":626129152}
I20260709 13:05:37.614924 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:37.653654 16533 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:37.654352 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:37.654564 16533 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:37.656623 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:37.661355 15862 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:37.661492 15858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 1565) took 1274 ms. Trace:
I20260709 13:05:37.661828 15858 rpcz_store.cc:276] 0709 13:05:36.387502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.605493 (+217991us) service_pool.cc:224] Handling call
0709 13:05:37.661293 (+1055800us) spinlock_profiling.cc:243] Waited 340 ms on lock 0x7b540008cd54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:37.661433 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":781408640}
I20260709 13:05:37.654963 15861 raft_consensus.cc:2376] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4. Re-sending same reply.
I20260709 13:05:37.663472 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:37.664290 15861 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 4.
I20260709 13:05:37.665072 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:37.665824 17985 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:37.666303 17985 raft_consensus.cc:697] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:37.666790 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:37.665081 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:37.668012 16011 raft_consensus.cc:1275] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:05:37.669337 15858 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:05:37.672340 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:37.674229 15934 raft_consensus.cc:1240] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:37.633598 17939 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:05:37.680637 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 390820us
I20260709 13:05:37.683207 15858 raft_consensus.cc:1275] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:37.658052 17987 raft_consensus.cc:2804] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:37.684373 15858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1575) took 1294 ms. Trace:
I20260709 13:05:37.685675 16011 raft_consensus.cc:1275] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:05:37.686753 15858 rpcz_store.cc:276] 0709 13:05:36.390205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:37.670271 (+1280066us) service_pool.cc:224] Handling call
0709 13:05:37.682831 (+ 12560us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:37.683854 (+  1023us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:37.684304 (+   450us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:37.659927 17967 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:37.697479 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:37.702566 15862 raft_consensus.cc:1240] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:37.703544 15862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 406) took 1030 ms. Trace:
I20260709 13:05:37.704025 15862 rpcz_store.cc:276] 0709 13:05:36.672793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:37.680462 (+1007669us) service_pool.cc:224] Handling call
0709 13:05:37.702315 (+ 21853us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:37.703231 (+   916us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:37.703470 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:37.709671 15858 raft_consensus.cc:1240] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:37.710316 15858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 407) took 1036 ms. Trace:
I20260709 13:05:37.711018 15858 rpcz_store.cc:276] 0709 13:05:36.673602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:37.687977 (+1014375us) service_pool.cc:224] Handling call
0709 13:05:37.709492 (+ 21515us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:37.710076 (+   584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:37.710263 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:37.713037 15861 raft_consensus.cc:1240] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:37.713953 15861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 408) took 1027 ms. Trace:
I20260709 13:05:37.714421 15861 rpcz_store.cc:276] 0709 13:05:36.686893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:37.688809 (+1001916us) service_pool.cc:224] Handling call
0709 13:05:37.712816 (+ 24007us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:37.713667 (+   851us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:37.713890 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:37.681156 17971 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:37.716080 16009 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:37.720561 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:37.721176 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:05:37.722337 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:37.723920 15934 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:37.724813 15853 raft_consensus.cc:1240] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:37.698707 17930 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.730363 17930 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:37.730926 18046 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.732968 15861 raft_consensus.cc:1275] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:05:37.733389 15861 pending_rounds.cc:85] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8: Aborting all ops after (but not including) 1
I20260709 13:05:37.733637 15861 pending_rounds.cc:107] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:05:37.734001 15861 raft_consensus.cc:2889] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:37.735760 15861 raft_consensus.cc:1275] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:05:37.729314 15862 raft_consensus.cc:1240] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:37.740399 15862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 409) took 1041 ms. Trace:
I20260709 13:05:37.667142 17985 consensus_queue.cc:237] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:37.707144 17993 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:37.694816 18004 consensus_queue.cc:1048] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 13:05:37.707968 17987 raft_consensus.cc:697] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:37.725446 15853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 405) took 1329 ms. Trace:
I20260709 13:05:37.731457 18046 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:37.719134 18027 consensus_queue.cc:1048] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 13:05:37.728608 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:37.697114 17991 consensus_queue.cc:1048] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:05:37.764333 15858 raft_consensus.cc:1240] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:05:37.729032 17971 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:37.766345 17971 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:37.767364 17987 consensus_queue.cc:237] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:37.755887 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:37.660370 17970 consensus_queue.cc:237] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:37.659300 18001 consensus_queue.cc:260] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:37.741024 15862 rpcz_store.cc:276] 0709 13:05:36.699509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:37.704955 (+1005446us) service_pool.cc:224] Handling call
0709 13:05:37.729104 (+ 24149us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:37.740103 (+ 10999us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:37.740334 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:37.760824 18042 log.cc:927] Time spent T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.194s	user 0.001s	sys 0.000s
I20260709 13:05:37.761145 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:05:37.762676 17769 consensus_meta.cc:287] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.525s	user 0.004s	sys 0.006s
I20260709 13:05:37.754415 17966 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:37.757767 15853 rpcz_store.cc:276] 0709 13:05:36.396097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:37.673504 (+1277407us) service_pool.cc:224] Handling call
0709 13:05:37.724622 (+ 51118us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:37.725204 (+   582us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:37.725393 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:37.752360 18038 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:37.772125 16569 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 4.
I20260709 13:05:37.767210 18026 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:05:37.793162 17784 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:37.804303 15934 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:05:37.801611 16563 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:37.771667 17973 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:37.813344 17784 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:37.790393 17971 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.806331 17885 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.766798 17990 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:37.819401 17885 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:37.821321 18048 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.821777 17983 raft_consensus.cc:3055] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:37.812098 18001 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:37.825860 15976 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:37.857043 18046 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:37.857805 18047 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:37.817548 17966 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:37.860387 18047 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:37.861585 18047 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:37.800988 17769 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:37.862248 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:37.831779 17983 raft_consensus.cc:740] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:37.863960 18038 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.873672 18026 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
I20260709 13:05:37.873500 18038 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:05:37.876647 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:37.877405 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:37.884595 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:37.885759 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:37.886442 16011 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
I20260709 13:05:37.886925 16535 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:37.889148 15933 raft_consensus.cc:2393] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 4: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:37.820766 17971 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:37.892510 18038 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:37.897606 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:37.899770 16011 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:37.900662 16009 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
I20260709 13:05:37.905330 16535 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:37.892688 17885 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:37.865265 17966 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:37.946548 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:37.965549 16570 raft_consensus.cc:1275] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:05:37.964133 16535 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:37.915525 17985 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:37.890535 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 355) took 1505 ms. Trace:
I20260709 13:05:37.915938 17973 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:37.961812 17993 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.863839 18048 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:37.935885 17885 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:37.968981 17985 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:37.973443 16564 raft_consensus.cc:1275] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:05:38.008638 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.036983 17967 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.037705 17967 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:38.045455 17966 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:37.994822 17784 consensus_queue.cc:1059] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:05:38.005252 17985 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:38.001880 17993 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:38.072644 16010 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41674 (request call id 1594) took 2346 ms (client timeout 2432 ms). Trace:
W20260709 13:05:38.073071 16010 rpcz_store.cc:269] 0709 13:05:35.726031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.748506 (+ 22475us) service_pool.cc:224] Handling call
0709 13:05:38.072421 (+2323915us) spinlock_profiling.cc:243] Waited 744 ms on lock 0x7b54000eddd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:38.072569 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1710259072}
I20260709 13:05:38.074175 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:37.972767 16002 rpcz_store.cc:276] 0709 13:05:36.380846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.485157 (+104311us) service_pool.cc:224] Handling call
0709 13:05:36.547960 (+ 62803us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:37.006571 (+458611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:37.863437 (+856866us) spinlock_profiling.cc:243] Waited 407 ms on lock 0x7b54003ece54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:37.886441 (+ 23004us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":936258816}
I20260709 13:05:38.076290 15936 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1625) took 1500 ms. Trace:
I20260709 13:05:37.977586 17983 consensus_queue.cc:260] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:38.076673 15936 rpcz_store.cc:276] 0709 13:05:36.576157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:37.217184 (+641027us) service_pool.cc:224] Handling call
0709 13:05:37.266723 (+ 49539us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:37.507070 (+240347us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:38.076059 (+568989us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b54003b8b54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:38.076236 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":551100800}
I20260709 13:05:38.078050 16005 raft_consensus.cc:1240] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:38.053428 16563 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
I20260709 13:05:38.079509 16005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 356) took 1635 ms. Trace:
I20260709 13:05:38.080000 17967 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:38.071009 17990 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:38.079882 16005 rpcz_store.cc:276] 0709 13:05:36.443802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.667315 (+223513us) service_pool.cc:224] Handling call
0709 13:05:38.077870 (+1410555us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:38.078463 (+   593us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:38.079308 (+   845us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54000eddd0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:38.079450 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3162311936}
I20260709 13:05:38.081398 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:38.081964 16005 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:38.083065 15930 raft_consensus.cc:1240] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:38.083608 15930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1621) took 1935 ms. Trace:
W20260709 13:05:38.084107 15856 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58214 (request call id 404) took 1690 ms (client timeout 1617 ms). Trace:
W20260709 13:05:38.084491 15856 rpcz_store.cc:269] 0709 13:05:36.393444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.433524 (+ 40080us) service_pool.cc:224] Handling call
0709 13:05:38.083932 (+1650408us) spinlock_profiling.cc:243] Waited 53.6 ms on lock 0x7b5400218254. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:38.084045 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":123350784}
I20260709 13:05:38.075543 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:38.087455 16010 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:38.088107 17769 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:38.061481 17979 consensus_meta.cc:287] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.602s	user 0.004s	sys 0.006s
I20260709 13:05:38.089867 17979 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:38.091758 17979 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:38.094189 17971 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:38.094791 16010 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:38.096482 17993 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:38.096844 18048 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:38.097506 15860 raft_consensus.cc:1275] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:05:38.097561 16008 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.009s	sys 0.028s
W20260709 13:05:38.097991 16008 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.009s	sys 0.028s
I20260709 13:05:38.083913 15930 rpcz_store.cc:276] 0709 13:05:36.148171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.148679 (+   508us) service_pool.cc:224] Handling call
0709 13:05:36.149072 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:38.082938 (+1933866us) spinlock_profiling.cc:243] Waited 6.49 ms on lock 0x7b54000d4ba0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea31fdc0 00007f73ea3aa1c0 00007f73ea3ab231 00007f73ea3a7d8f 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c
0709 13:05:38.083406 (+   468us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:38.083563 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14924032}
W20260709 13:05:38.062265 17961 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.015s	sys 0.048s
I20260709 13:05:38.076558 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:38.119776 16011 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
W20260709 13:05:38.106974 17961 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.741s	user 0.015s	sys 0.048s
W20260709 13:05:38.124737 15931 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.014s	sys 0.016s
W20260709 13:05:38.129889 15931 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.015s	sys 0.016s
I20260709 13:05:38.106635 17819 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:05:38.130050 16534 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.045s	sys 0.064s
I20260709 13:05:38.061939 17885 raft_consensus.cc:697] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:05:38.130688 16534 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.046s	sys 0.064s
W20260709 13:05:38.136510 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.135286 17819 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:05:38.136431 16004 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.014s	sys 0.023s
I20260709 13:05:38.168865 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:38.174024 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:38.174886 16567 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
W20260709 13:05:38.168458 16004 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.015s	sys 0.023s
I20260709 13:05:38.164925 17819 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:38.152242 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:38.181684 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:05:38.181114 16007 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.019s	sys 0.022s
W20260709 13:05:38.182780 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:38.182957 16007 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.020s	sys 0.023s
W20260709 13:05:38.167237 17974 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.032s	sys 0.024s
I20260709 13:05:38.118789 18047 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:38.167766 18031 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:05:38.191828 17974 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.032s	sys 0.024s
I20260709 13:05:38.197645 16567 raft_consensus.cc:1240] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 2. Current term is 3. Ops: []
W20260709 13:05:38.202280 15827 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.017s	sys 0.024s
W20260709 13:05:38.229358 15827 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.018s	sys 0.025s
W20260709 13:05:38.229728 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 622016us
W20260709 13:05:38.203735 15974 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.813s	user 0.011s	sys 0.015s
W20260709 13:05:38.231877 15974 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.841s	user 0.012s	sys 0.015s
I20260709 13:05:38.232587 18048 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:38.232874 18047 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:38.194185 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:38.235240 15974 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:38.238077 15974 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:05:38.240397 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 271003us
I20260709 13:05:38.233250 18048 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:38.244549 16535 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:38.253562 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:38.254937 16010 raft_consensus.cc:2410] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:05:38.236714 16004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1698) took 1207 ms. Trace:
I20260709 13:05:38.238641 16562 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:05:38.258322 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:38.241241 17938 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.745s	user 0.017s	sys 0.029s
I20260709 13:05:38.262087 16009 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
I20260709 13:05:38.262248 16011 raft_consensus.cc:1240] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 1. Current term is 2. Ops: []
W20260709 13:05:38.262604 15932 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.009s	sys 0.021s
W20260709 13:05:38.279855 15932 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.010s	sys 0.022s
I20260709 13:05:38.241587 17990 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:05:38.279769 15861 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.011s	sys 0.011s
W20260709 13:05:38.294576 15861 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.011s	sys 0.012s
I20260709 13:05:38.263865 16010 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:38.292991 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:38.255087 18056 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.258170 18047 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:38.291936 16535 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:05:38.327419 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.255417 17970 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.295224 15861 raft_consensus.cc:2955] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:38.299121 15974 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:38.262861 17967 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.235354 17885 consensus_queue.cc:237] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:38.292671 17938 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.796s	user 0.019s	sys 0.031s
W20260709 13:05:38.338966 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 616675us
I20260709 13:05:38.332127 16010 raft_consensus.cc:1275] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:05:38.279909 15929 raft_consensus.cc:1240] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
W20260709 13:05:38.303404 15827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:38.340678 15827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.340798 15929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1608) took 2427 ms. Trace:
I20260709 13:05:38.293658 17974 raft_consensus.cc:2804] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:38.341272 15929 rpcz_store.cc:276] 0709 13:05:35.913117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.924638 (+ 11521us) service_pool.cc:224] Handling call
0709 13:05:38.236089 (+2311451us) spinlock_profiling.cc:243] Waited 6.04 ms on lock 0x7b5800003698. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef381fc0 00007f73ef36c929 00007f73ef36c3d2 00007f73e88e3bd4
0709 13:05:38.241926 (+  5837us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:38.340454 (+ 98528us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:38.340689 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13896448}
I20260709 13:05:38.341886 17974 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:38.293485 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:38.297116 17990 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:38.345402 15827 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:05:38.337517 18051 log.cc:927] Time spent T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:05:38.342486 17974 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:38.333010 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:38.350240 15974 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:38.349041 15827 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:38.339247 17932 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:38.358763 15827 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:38.361579 15974 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:38.343560 18031 raft_consensus.cc:2804] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:05:38.407440 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 46363us
I20260709 13:05:38.411806 15976 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:38.351972 17653 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:38.377230 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:38.336956 17970 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:38.339761 17967 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:38.336483 18056 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:38.349327 17990 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:38.417421 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 55949.4us
I20260709 13:05:38.453795 17961 consensus_queue.cc:1048] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 13:05:38.486840 15974 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:38.348059 16005 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:38.348455 17979 consensus_queue.cc:1048] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.093s
I20260709 13:05:38.406503 17939 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:38.405362 18038 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:05:38.496258 16568 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45244 (request call id 653) took 2314 ms (client timeout 1802 ms). Trace:
I20260709 13:05:38.454308 17966 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:05:38.490847 18054 log.cc:927] Time spent T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.194s	user 0.001s	sys 0.002s
I20260709 13:05:38.436060 17866 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:38.508589 16568 rpcz_store.cc:269] 0709 13:05:36.176404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.176560 (+   156us) service_pool.cc:224] Handling call
0709 13:05:38.490272 (+2313712us) spinlock_profiling.cc:243] Waited 435 ms on lock 0x7b54000cdad4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:38.490377 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":999428864}
W20260709 13:05:38.501433 15854 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 1576) took 2106 ms (client timeout 1883 ms). Trace:
I20260709 13:05:38.487296 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:38.487186 18070 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.489684 17902 raft_consensus.cc:3055] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:38.411152 17932 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:38.493818 17938 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:38.510033 18055 log.cc:927] Time spent T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.263s	user 0.000s	sys 0.006s
I20260709 13:05:38.356243 17881 consensus_queue.cc:1048] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.153s
I20260709 13:05:38.499951 17653 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:38.540557 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:05:38.501500 16561 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45246 (request call id 554) took 2771 ms (client timeout 1762 ms). Trace:
I20260709 13:05:38.413851 18049 consensus_queue.cc:1048] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.104s
I20260709 13:05:38.501444 18061 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:38.512058 18018 log.cc:927] Time spent T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 1.738s	user 0.004s	sys 0.008s
I20260709 13:05:38.552846 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:38.551414 18048 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:38.549508 16002 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:38.549983 18053 log.cc:927] Time spent T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.294s	user 0.001s	sys 0.005s
W20260709 13:05:38.524267 15854 rpcz_store.cc:269] 0709 13:05:36.390466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.579710 (+189244us) service_pool.cc:224] Handling call
0709 13:05:38.465231 (+1885521us) spinlock_profiling.cc:243] Waited 574 ms on lock 0x7b5400426dd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:38.496494 (+ 31263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1321300352}
I20260709 13:05:38.524399 18038 raft_consensus.cc:697] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:38.614326 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:38.629464 16568 raft_consensus.cc:2393] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 3: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
W20260709 13:05:38.618921 18058 log.cc:927] Time spent T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.079s	user 0.000s	sys 0.002s
I20260709 13:05:38.571695 17999 raft_consensus.cc:3055] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:05:38.607978 18062 log.cc:927] Time spent T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 13:05:38.633248 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:05:38.607728 18065 log.cc:927] Time spent T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.064s	user 0.000s	sys 0.002s
I20260709 13:05:38.530877 17939 raft_consensus.cc:2749] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:38.553035 18070 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:38.606154 17938 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:38.626137 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.609318 17653 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:38.571780 17932 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:38.259545 16004 rpcz_store.cc:276] 0709 13:05:37.027865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:37.339392 (+311527us) service_pool.cc:224] Handling call
0709 13:05:37.363824 (+ 24432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:37.364073 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:05:37.552553 (+188480us) log.cc:844] Serialized 14 byte log entry
0709 13:05:38.192692 (+640139us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:38.196093 (+  3401us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:38.198345 (+  2252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:38.202936 (+  4591us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:38.235381 (+ 32445us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7553,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":5682,"thread_start_us":625444,"threads_started":1,"wal-append.queue_time_us":699597}
I20260709 13:05:38.635754 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:38.621379 17987 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:38.550271 17902 raft_consensus.cc:740] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:38.525187 17966 raft_consensus.cc:697] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:38.648077 15931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1609) took 2734 ms. Trace:
W20260709 13:05:38.699410 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.636817 17784 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.633314 17999 raft_consensus.cc:740] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:38.692894 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:38.710074 16568 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
I20260709 13:05:38.711226 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:38.696400 16008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1601) took 1315 ms. Trace:
W20260709 13:05:38.609843 16561 rpcz_store.cc:269] 0709 13:05:35.725236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.740177 (+ 14941us) service_pool.cc:224] Handling call
0709 13:05:38.465505 (+2725328us) spinlock_profiling.cc:243] Waited 619 ms on lock 0x7b5400545e54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:38.496563 (+ 31058us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1422607616}
I20260709 13:05:38.718266 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:05:38.718412 17819 consensus_meta.cc:287] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.521s	user 0.008s	sys 0.002s
I20260709 13:05:38.718868 16561 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:38.721402 15932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1635) took 1043 ms. Trace:
I20260709 13:05:38.721680 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:38.721765 15932 rpcz_store.cc:276] 0709 13:05:37.678298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:37.678949 (+   651us) service_pool.cc:224] Handling call
0709 13:05:37.679308 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:37.679563 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:37.679599 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:37.684815 (+  5216us) log.cc:844] Serialized 26 byte log entry
0709 13:05:38.294142 (+609327us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:38.294179 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:38.300985 (+  6806us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:38.721211 (+420226us) raft_consensus.cc:1685] finished
0709 13:05:38.721232 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:38.721350 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":565383,"threads_started":1,"wal-append.queue_time_us":695597}
I20260709 13:05:38.722884 16009 raft_consensus.cc:2410] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:05:38.698904 15931 rpcz_store.cc:276] 0709 13:05:35.913528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.914968 (+  1440us) service_pool.cc:224] Handling call
0709 13:05:37.493766 (+1578798us) spinlock_profiling.cc:243] Waited 2.79 ms on lock 0x7b5800003698. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283e9c 00007f73e7283a5b 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef381fc0 00007f73ef36c929 00007f73ef36c3d2 00007f73e88e3bd4
0709 13:05:37.493927 (+   161us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:37.500000 (+  6073us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:37.500027 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:37.505169 (+  5142us) log.cc:844] Serialized 26 byte log entry
0709 13:05:38.130303 (+625134us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:38.130339 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:38.130409 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:38.643753 (+513344us) raft_consensus.cc:1685] finished
0709 13:05:38.643784 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:38.648011 (+  4227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5740,"raft.queue_time_us":6237,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":7563,"spinlock_wait_cycles":6439040,"thread_start_us":575657,"threads_started":1,"wal-append.queue_time_us":682651}
I20260709 13:05:38.716281 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:05:38.725667 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 367038us
I20260709 13:05:38.726873 15827 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:38.728076 16009 raft_consensus.cc:1240] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:38.726974 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:38.729961 16010 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:38.712077 16008 rpcz_store.cc:276] 0709 13:05:37.380556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:37.439924 (+ 59368us) service_pool.cc:224] Handling call
0709 13:05:37.440190 (+   266us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:37.441228 (+  1038us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:37.441260 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:37.441647 (+   387us) log.cc:844] Serialized 26 byte log entry
0709 13:05:38.098366 (+656719us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:38.098406 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:38.098467 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:38.696142 (+597675us) raft_consensus.cc:1685] finished
0709 13:05:38.696169 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:38.696332 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":32555,"raft.run_cpu_time_us":601,"raft.run_wall_time_us":28764,"thread_start_us":601529,"threads_started":1,"wal-append.queue_time_us":743648}
I20260709 13:05:38.732263 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:38.692711 17653 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:38.735069 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 248294us
I20260709 13:05:38.736150 15974 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:38.736369 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:38.740121 15974 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:38.720160 17784 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:38.742751 17871 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:38.744123 17902 consensus_queue.cc:260] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:38.745178 17902 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:38.745462 15934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1638) took 1044 ms. Trace:
I20260709 13:05:38.745841 15934 rpcz_store.cc:276] 0709 13:05:37.700780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:37.814034 (+113254us) service_pool.cc:224] Handling call
0709 13:05:37.814425 (+   391us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:38.136770 (+322345us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:38.143997 (+  7227us) log.cc:844] Serialized 14 byte log entry
0709 13:05:38.491120 (+347123us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:38.491628 (+   508us) log.cc:844] Serialized 26 byte log entry
0709 13:05:38.491806 (+   178us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:38.491851 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:38.491939 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:38.745220 (+253281us) raft_consensus.cc:1685] finished
0709 13:05:38.745242 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:38.745383 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":340391,"prepare.queue_time_us":5858,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"raft.queue_time_us":321464,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":10118,"thread_start_us":320170,"threads_started":1,"wal-append.queue_time_us":590769}
I20260709 13:05:38.747483 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:38.748297 18048 raft_consensus.cc:2749] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:38.752542 16561 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
I20260709 13:05:38.753660 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:38.754236 16561 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:38.705832 17987 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:38.756095 17967 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:38.755527 15974 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:38.756608 18038 consensus_queue.cc:237] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:05:38.760891 17985 consensus_meta.cc:287] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.672s	user 0.011s	sys 0.001s
I20260709 13:05:38.761327 17985 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:38.738118 16562 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:05:38.775998 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:38.777045 16568 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:05:38.718992 17819 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:38.780031 15976 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:38.781828 15861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1590) took 1295 ms. Trace:
I20260709 13:05:38.730104 18056 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:38.781140 17957 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.765038 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:38.828629 15861 rpcz_store.cc:276] 0709 13:05:37.486465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:37.737217 (+250752us) service_pool.cc:224] Handling call
0709 13:05:37.739690 (+  2473us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:37.740118 (+   428us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:37.740156 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:37.749730 (+  9574us) log.cc:844] Serialized 556 byte log entry
0709 13:05:38.295051 (+545321us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:38.434565 (+139514us) log.cc:844] Serialized 14 byte log entry
0709 13:05:38.781441 (+346876us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:38.781517 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:38.781560 (+    43us) raft_consensus.cc:1685] finished
0709 13:05:38.781581 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:38.781753 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2494,"raft.queue_time_us":61904,"raft.run_cpu_time_us":422,"raft.run_wall_time_us":13225,"thread_start_us":882746,"threads_started":2,"wal-append.queue_time_us":743419}
I20260709 13:05:38.881712 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:38.778126 17881 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:38.783155 18061 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:38.890618 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:38.779161 17784 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:38.778467 17979 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:38.896396 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 372946us
I20260709 13:05:38.909730 16007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1602) took 1439 ms. Trace:
I20260709 13:05:38.831274 17985 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:38.831370 17939 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:38.781936 18031 raft_consensus.cc:697] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:38.764014 18043 consensus_queue.cc:1048] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.131s
I20260709 13:05:38.830206 17970 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:38.765866 17932 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:38.858582 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:38.879586 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:38.910550 17881 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:38.928514 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:38.784471 18049 consensus_queue.cc:1048] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.132s
I20260709 13:05:38.763614 17966 consensus_queue.cc:237] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:38.918651 17979 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:38.932574 18031 consensus_queue.cc:237] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:38.934226 15861 raft_consensus.cc:2393] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 2: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:38.925187 16007 rpcz_store.cc:276] 0709 13:05:37.470344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:37.489649 (+ 19305us) service_pool.cc:224] Handling call
0709 13:05:37.489962 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:37.499775 (+  9813us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:37.499805 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:37.505418 (+  5613us) log.cc:844] Serialized 26 byte log entry
0709 13:05:37.603272 (+ 97854us) spinlock_profiling.cc:243] Waited 6.19 ms on lock 0x7b18002fc7a0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f480d1 00007f73ea363a0f
0709 13:05:38.183551 (+580279us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:38.542400 (+358849us) log.cc:844] Serialized 14 byte log entry
0709 13:05:38.828781 (+286381us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:38.828858 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:38.828898 (+    40us) raft_consensus.cc:1685] finished
0709 13:05:38.909517 (+ 80619us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:38.909670 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":135309,"prepare.queue_time_us":564051,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"raft.queue_time_us":108889,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":17630,"spinlock_wait_cycles":14226816,"thread_start_us":1206495,"threads_started":3,"wal-append.queue_time_us":779307}
I20260709 13:05:38.936097 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:38.858031 17957 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:38.921494 17784 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:38.935653 16562 raft_consensus.cc:2410] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:05:38.933881 17881 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:38.963635 16002 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
W20260709 13:05:38.952958 18047 consensus_meta.cc:287] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.618s	user 0.010s	sys 0.000s
I20260709 13:05:38.932044 17970 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.970105 18047 raft_consensus.cc:2749] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:38.934288 18085 consensus_queue.cc:1048] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 13:05:38.970957 17819 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:38.967769 15853 raft_consensus.cc:2393] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 4: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:38.965047 16004 raft_consensus.cc:2368] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5.
I20260709 13:05:38.939647 18089 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:38.963317 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:38.917333 17999 consensus_queue.cc:260] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:38.969152 17967 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:38.981451 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:38.982863 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:38.982895 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:38.987254 15859 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:38.989622 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:38.990212 15859 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:38.990599 17969 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:38.991001 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:38.953001 17784 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:38.985574 17974 consensus_meta.cc:287] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.578s	user 0.010s	sys 0.000s
I20260709 13:05:38.974112 17938 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:38.997575 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:39.000841 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:39.002406 16569 raft_consensus.cc:2393] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 3: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:38.985819 18070 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:38.992297 17904 raft_consensus.cc:3055] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:38.992183 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:38.974574 17970 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:38.988046 17866 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.007756 15974 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:39.007992 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:38.988636 18089 raft_consensus.cc:697] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 5 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:39.010488 16568 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
W20260709 13:05:39.001106 18080 log.cc:927] Time spent T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:05:39.016067 15974 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:38.993206 17979 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:39.001065 18091 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.022832 17938 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:39.001049 18084 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.023366 17938 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:39.023895 17938 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:39.023589 18084 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:39.026069 18084 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:39.023001 18091 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:39.028995 18091 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:39.029304 17987 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:39.009407 18066 log.cc:927] Time spent T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.338s	user 0.001s	sys 0.000s
I20260709 13:05:39.031282 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:39.032069 15862 raft_consensus.cc:2393] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 3: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:39.013716 17904 raft_consensus.cc:740] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:39.000700 17974 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:39.032927 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:39.033583 17904 consensus_queue.cc:260] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:39.034559 17866 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:39.035006 17974 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:39.036334 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:39.036633 17784 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:39.036712 17866 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:39.037729 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:39.038448 17970 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:39.038938 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:39.041462 16561 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:39.042578 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:39.043205 15862 raft_consensus.cc:2393] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 6: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:39.020228 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:39.000620 18081 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.021471 15827 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:39.000927 18088 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.009550 17999 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:39.063580 15974 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:39.035063 15856 raft_consensus.cc:1275] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:05:39.056354 15825 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:39.045728 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:39.081954 15862 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 2.
I20260709 13:05:39.082556 15859 raft_consensus.cc:2410] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:05:39.079094 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 57691.6us
W20260709 13:05:39.055312 17990 consensus_meta.cc:287] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.555s	user 0.004s	sys 0.005s
I20260709 13:05:39.081257 18047 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:39.050905 18085 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:39.061787 16004 raft_consensus.cc:1240] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:39.085862 17981 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:39.087399 18091 consensus_queue.cc:1048] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:39.087965 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:39.088120 18084 consensus_queue.cc:1059] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:39.094439 16003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41662 (request call id 1703) took 1982 ms. Trace:
I20260709 13:05:39.082628 17866 raft_consensus.cc:2749] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:39.062320 17932 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:39.061268 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:39.089499 15825 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:39.093555 15827 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:05:39.060429 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.094885 16003 rpcz_store.cc:276] 0709 13:05:37.111774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:37.166764 (+ 54990us) service_pool.cc:224] Handling call
0709 13:05:39.087804 (+1921040us) spinlock_profiling.cc:243] Waited 8.06 ms on lock 0x7b5800114398. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:05:39.094357 (+  6553us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18529536}
W20260709 13:05:39.100250 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.090220 17959 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:39.105319 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:39.108062 16569 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
I20260709 13:05:39.096262 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:39.103564 15827 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:39.108918 15859 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:39.104435 17815 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:39.110133 17925 raft_consensus.cc:2749] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:39.111892 15827 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:39.092561 18085 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:39.118978 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:39.122835 16003 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:05:39.124083 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:39.125470 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:39.126142 16003 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
I20260709 13:05:39.127110 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:39.060330 17957 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:39.080462 18088 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:39.118289 18084 raft_consensus.cc:2749] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:39.120966 17979 raft_consensus.cc:2804] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:39.117547 15853 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:39.119555 17986 raft_consensus.cc:2804] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:39.129730 18085 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:39.079937 18081 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:39.143908 16004 raft_consensus.cc:1240] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:39.148391 17881 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:39.095682 17653 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:39.149071 18088 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:39.096796 17990 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:39.051637 18089 consensus_queue.cc:237] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:39.159840 16003 raft_consensus.cc:2410] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:39.121157 18082 consensus_queue.cc:1048] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:05:39.154304 17979 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:39.158535 17986 raft_consensus.cc:697] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 4 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:39.174728 16007 raft_consensus.cc:1240] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:05:39.138406 17957 consensus_queue.cc:1059] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 13:05:39.156095 17819 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.180864 15856 raft_consensus.cc:2393] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 4: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:05:39.186388 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.186311 15859 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
I20260709 13:05:39.189091 17881 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:39.190043 17881 raft_consensus.cc:3055] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:39.190513 17881 raft_consensus.cc:740] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:39.191553 17881 consensus_queue.cc:260] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:39.175967 17979 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:39.196421 15932 raft_consensus.cc:1240] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:39.197121 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:39.182183 17988 raft_consensus.cc:3055] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:05:39.202476 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 89258us
I20260709 13:05:39.179287 16561 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:05:39.202328 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:39.179531 17966 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:39.181617 18004 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:39.216641 15932 raft_consensus.cc:2393] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 5: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:39.185156 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:39.216856 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:39.217679 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:39.226089 15931 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:39.224236 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:39.233928 15930 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:39.235183 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:39.243009 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:39.184990 17819 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:39.213143 17988 raft_consensus.cc:740] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:39.263136 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:39.248410 18016 consensus_queue.cc:1059] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 13:05:39.221210 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:39.277817 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:39.278002 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:39.278446 15932 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
W20260709 13:05:39.282617 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.284462 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:05:39.286598 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 169089us
W20260709 13:05:39.328298 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 49775us
W20260709 13:05:39.329036 16533 leader_election.cc:336] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 1.800s (SENT)
I20260709 13:05:39.329479 16533 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:39.328320 15932 raft_consensus.cc:1240] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:05:39.261850 17958 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.236788 18004 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:39.223248 17966 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:39.273520 16563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 572) took 1079 ms. Trace:
I20260709 13:05:39.343966 15928 raft_consensus.cc:2393] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 2: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:39.345520 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:39.348246 18081 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:39.349406 15932 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 2.
I20260709 13:05:39.330235 17990 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:05:39.355752 17822 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.338s	user 0.003s	sys 0.012s
W20260709 13:05:39.356148 17822 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.339s	user 0.003s	sys 0.012s
I20260709 13:05:39.363915 17763 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:39.341110 16004 raft_consensus.cc:1275] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:39.352749 15936 raft_consensus.cc:1275] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:05:39.337888 17902 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:39.219892 17986 consensus_queue.cc:237] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:39.398684 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:39.400137 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:05:39.352070 18098 log.cc:927] Time spent T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.215s	user 0.000s	sys 0.001s
I20260709 13:05:39.338582 17815 consensus_queue.cc:1059] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
W20260709 13:05:39.406415 18108 log.cc:927] Time spent T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.069s	user 0.000s	sys 0.004s
I20260709 13:05:39.406574 17815 raft_consensus.cc:3055] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:39.406980 17815 raft_consensus.cc:740] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:39.407981 16567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 574) took 1181 ms. Trace:
I20260709 13:05:39.407859 17815 consensus_queue.cc:260] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:39.343259 17958 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:39.408595 16567 rpcz_store.cc:276] 0709 13:05:38.226795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.054765 (+827970us) service_pool.cc:224] Handling call
0709 13:05:39.055118 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:39.055342 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:39.055368 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:39.055798 (+   430us) log.cc:844] Serialized 26 byte log entry
0709 13:05:39.230358 (+174560us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:39.230410 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:39.230499 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:39.407667 (+177168us) raft_consensus.cc:1685] finished
0709 13:05:39.407714 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:39.407904 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":166964,"threads_started":1,"wal-append.queue_time_us":346320}
I20260709 13:05:39.410653 15857 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:05:39.410966 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:39.384075 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:39.412645 15853 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:39.413082 17966 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:39.405017 15902 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:39.419659 15902 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:39.420473 18104 raft_consensus.cc:2749] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:39.421571 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:39.421620 15902 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:39.422374 16561 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
I20260709 13:05:39.422569 18072 raft_consensus.cc:2749] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:39.424037 15902 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:39.427958 15974 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:39.427429 16567 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:39.430365 15902 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:39.432608 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:39.433264 16566 raft_consensus.cc:2410] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:05:39.433635 16566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45232 (request call id 519) took 1563 ms. Trace:
I20260709 13:05:39.433869 16566 rpcz_store.cc:276] 0709 13:05:37.870299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:37.870466 (+   167us) service_pool.cc:224] Handling call
0709 13:05:39.432462 (+1561996us) spinlock_profiling.cc:243] Waited 1.07 ms on lock 0x7b5800044798. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:05:39.433590 (+  1128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2456320}
W20260709 13:05:39.418687 18106 log.cc:927] Time spent T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:05:39.439620 15974 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:39.409144 17815 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:39.443185 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:39.442414 15974 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:39.445529 18099 raft_consensus.cc:2804] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:39.446010 18099 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:05:39.431879 18105 log.cc:927] Time spent T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 13:05:39.449187 15976 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:39.441526 17966 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:39.447923 17990 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:39.454435 16009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1711) took 1019 ms. Trace:
I20260709 13:05:39.448184 17871 raft_consensus.cc:2804] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:39.461256 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:39.451225 18049 raft_consensus.cc:2749] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:39.462572 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:39.444162 15928 raft_consensus.cc:2393] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 4: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:39.463121 18004 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:39.383579 17988 consensus_queue.cc:260] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:39.464048 17990 raft_consensus.cc:697] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:39.464188 17988 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:39.465023 17990 consensus_queue.cc:237] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:39.436543 18072 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:39.454937 16009 rpcz_store.cc:276] 0709 13:05:38.435258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:38.731214 (+295956us) service_pool.cc:224] Handling call
0709 13:05:38.731579 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:38.733380 (+  1801us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:38.733431 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:38.765363 (+ 31932us) log.cc:844] Serialized 26 byte log entry
0709 13:05:39.005483 (+240120us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:05:39.005527 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:39.005596 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:39.454165 (+448569us) raft_consensus.cc:1685] finished
0709 13:05:39.454192 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:39.454356 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2302,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":25624,"thread_start_us":148077,"threads_started":1,"wal-append.queue_time_us":266353}
I20260709 13:05:39.467873 18072 raft_consensus.cc:697] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:39.468734 16009 raft_consensus.cc:1240] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:39.436699 17763 consensus_queue.cc:1059] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:05:39.469449 17763 raft_consensus.cc:3055] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:39.469089 18072 consensus_queue.cc:237] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:39.445816 15974 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:39.435185 18104 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.461444 17871 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:39.469815 17763 raft_consensus.cc:740] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:39.485445 18104 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:39.485880 17763 consensus_queue.cc:260] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:39.486728 16566 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:39.486586 18085 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:39.487128 17763 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:39.488006 17871 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:39.474254 15974 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:39.491096 15974 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:39.424786 17653 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:39.435256 16561 raft_consensus.cc:1275] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:05:39.441082 18012 raft_consensus.cc:2804] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:39.492635 18047 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:39.492859 18012 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:39.493157 18047 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:39.492667 16561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 578) took 1158 ms. Trace:
W20260709 13:05:39.476599 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.493878 17938 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:39.493880 16561 rpcz_store.cc:276] 0709 13:05:38.334019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.423067 (+1089048us) service_pool.cc:224] Handling call
0709 13:05:39.423386 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:39.492376 (+ 68990us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:39.492609 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:39.495319 17979 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:39.496037 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:39.496698 16561 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:39.497004 17967 consensus_queue.cc:1059] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:39.350831 16563 rpcz_store.cc:276] 0709 13:05:38.190710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:38.191694 (+   984us) service_pool.cc:224] Handling call
0709 13:05:38.234815 (+ 43121us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:38.290401 (+ 55586us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:38.294411 (+  4010us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:38.299783 (+  5372us) log.cc:844] Serialized 26 byte log entry
0709 13:05:38.650812 (+351029us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:39.200214 (+549402us) log.cc:844] Serialized 14 byte log entry
0709 13:05:39.216233 (+ 16019us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:39.223535 (+  7302us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:39.238763 (+ 15228us) raft_consensus.cc:1685] finished
0709 13:05:39.242194 (+  3431us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:39.269848 (+ 27654us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2424,"prepare.queue_time_us":528935,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":3313,"raft.queue_time_us":40154,"raft.run_cpu_time_us":52,"raft.run_wall_time_us":4982,"spinlock_wait_cycles":4291968,"thread_start_us":738759,"threads_started":2,"wal-append.queue_time_us":579077}
I20260709 13:05:39.482527 16010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1712) took 1047 ms. Trace:
I20260709 13:05:39.498941 16010 rpcz_store.cc:276] 0709 13:05:38.435540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:38.732493 (+296953us) service_pool.cc:224] Handling call
0709 13:05:38.732913 (+   420us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:38.734362 (+  1449us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:05:39.011253 (+276891us) log.cc:844] Serialized 14 byte log entry
0709 13:05:39.200435 (+189182us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:39.201007 (+   572us) log.cc:844] Serialized 26 byte log entry
0709 13:05:39.201167 (+   160us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:39.201212 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:39.201300 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:39.482248 (+280948us) raft_consensus.cc:1685] finished
0709 13:05:39.482275 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:39.482451 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38482,"prepare.queue_time_us":13634,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":8614,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":86,"spinlock_wait_cycles":12293504,"thread_start_us":450989,"threads_started":2,"wal-append.queue_time_us":257885}
I20260709 13:05:39.450529 18099 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:39.493629 18047 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:39.484336 18049 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:39.493330 18012 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:39.519025 18067 log.cc:927] Time spent T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.995s	user 0.001s	sys 0.002s
I20260709 13:05:39.520233 16562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 576) took 1265 ms. Trace:
I20260709 13:05:39.520642 16562 rpcz_store.cc:276] 0709 13:05:38.255158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.092130 (+836972us) service_pool.cc:224] Handling call
0709 13:05:39.518546 (+426416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:39.518766 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:39.518806 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:39.518859 (+    53us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:39.518892 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:39.518956 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:39.520051 (+  1095us) spinlock_profiling.cc:243] Waited 423 ms on lock 0x7b5400502150. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:39.520173 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":973846272}
I20260709 13:05:39.522744 17871 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:39.513346 17938 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:39.516887 18112 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:39.525738 18112 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:39.526093 18112 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:39.489611 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:39.528378 17966 raft_consensus.cc:2749] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:39.512710 17979 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:39.524407 18085 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:39.528403 17958 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:39.525245 17930 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:39.530078 16009 raft_consensus.cc:1240] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:39.533993 18104 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:39.497740 16561 raft_consensus.cc:3055] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:39.534826 16561 raft_consensus.cc:740] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:39.535645 16561 consensus_queue.cc:260] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:39.536681 17983 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:39.537372 16561 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:39.535853 18112 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:39.537009 16567 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5.
I20260709 13:05:39.539165 17871 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:39.540614 16567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 523) took 1127 ms. Trace:
I20260709 13:05:39.541205 16567 rpcz_store.cc:276] 0709 13:05:38.412836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.539358 (+1126522us) service_pool.cc:224] Handling call
0709 13:05:39.539929 (+   571us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:39.540152 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:39.540183 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:39.540358 (+   175us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:39.540405 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:39.540460 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:39.540555 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:39.544094 16567 raft_consensus.cc:1240] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:39.545009 16567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 664) took 1056 ms. Trace:
I20260709 13:05:39.545423 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:39.545502 16567 rpcz_store.cc:276] 0709 13:05:38.489050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.542806 (+1053756us) service_pool.cc:224] Handling call
0709 13:05:39.543859 (+  1053us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:39.544717 (+   858us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:39.544944 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:39.546233 16564 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
I20260709 13:05:39.550390 17866 raft_consensus.cc:3055] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:39.552619 18099 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:39.559926 17866 raft_consensus.cc:740] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:39.563233 18112 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:39.563499 17866 consensus_queue.cc:260] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:39.568053 18012 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:39.570432 17904 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:39.572508 18099 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:39.570807 18047 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:39.724916 18047 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:39.618261 15827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:39.621066 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.622969 15859 raft_consensus.cc:1275] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:05:37.657354 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:39.637530 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:39.639436 15860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1567) took 3251 ms. Trace:
I20260709 13:05:39.642278 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:05:39.637614 15857 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 1570) took 3248 ms (client timeout 1995 ms). Trace:
I20260709 13:05:39.643976 15902 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:05:39.650475 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 159301us
I20260709 13:05:39.653546 16007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1714) took 1204 ms. Trace:
I20260709 13:05:39.654165 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:39.656467 16002 raft_consensus.cc:1240] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 5 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:05:39.657423 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:05:39.660880 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.664448 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:39.547039 16564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45246 (request call id 565) took 1409 ms. Trace:
I20260709 13:05:39.553155 16563 raft_consensus.cc:1240] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:05:39.677321 16567 raft_consensus.cc:1240] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:39.705492 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:39.707201 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:39.712428 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:39.713922 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:39.715574 16570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 518) took 1847 ms. Trace:
I20260709 13:05:39.717465 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:39.711479 17819 leader_election.cc:290] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:39.717811 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:39.743610 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.720307 17988 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.764694 17988 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:39.767331 17988 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:39.721681 18012 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:39.582947 18079 log.cc:927] Time spent T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.636s	user 0.006s	sys 0.000s
W20260709 13:05:39.784672 18064 log.cc:927] Time spent T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 1.286s	user 0.002s	sys 0.002s
I20260709 13:05:39.585373 18084 raft_consensus.cc:3055] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:39.786356 18084 raft_consensus.cc:740] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:39.788058 16006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1605) took 2126 ms. Trace:
I20260709 13:05:39.728472 15860 rpcz_store.cc:276] 0709 13:05:36.388061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:37.319329 (+931268us) service_pool.cc:224] Handling call
0709 13:05:38.097152 (+777823us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:38.097898 (+   746us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:39.638914 (+1541016us) spinlock_profiling.cc:243] Waited 777 ms on lock 0x7b5400218250. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:39.639201 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1788040064}
W20260709 13:05:39.728993 15857 rpcz_store.cc:269] 0709 13:05:36.388862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.658537 (+269675us) service_pool.cc:224] Handling call
0709 13:05:39.637138 (+2978601us) spinlock_profiling.cc:243] Waited 2.73 s on lock 0x7b54004263d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:39.637380 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6288342784}
I20260709 13:05:39.739972 16570 rpcz_store.cc:276] 0709 13:05:37.868145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:37.868275 (+   130us) service_pool.cc:224] Handling call
0709 13:05:37.868654 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:37.966086 (+ 97432us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:39.714873 (+1748787us) spinlock_profiling.cc:243] Waited 93.9 ms on lock 0x7b5400102554. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:39.715336 (+   463us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":216072960}
I20260709 13:05:39.789803 16567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 674) took 1101 ms. Trace:
I20260709 13:05:39.732898 15757 catalog_manager.cc:5697] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 1 to 2, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 2 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:39.750118 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:39.752302 15856 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:39.746394 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:05:39.755573 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.756939 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:39.761451 17724 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:39.721508 17858 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:39.770871 16011 raft_consensus.cc:2410] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:05:39.772243 16566 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 4: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:39.777184 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:39.778527 18048 raft_consensus.cc:3055] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:39.779132 17981 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:05:39.779791 17993 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 13:05:39.779727 16561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 520) took 1414 ms. Trace:
I20260709 13:05:39.729471 16007 rpcz_store.cc:276] 0709 13:05:38.448786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:38.937066 (+488280us) service_pool.cc:224] Handling call
0709 13:05:38.937469 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:39.175225 (+237756us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:39.652712 (+477487us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b54000ed8d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:39.653230 (+   518us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":545005824}
I20260709 13:05:39.748597 18038 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:39.796347 15933 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 1630) took 2162 ms (client timeout 2206 ms). Trace:
I20260709 13:05:39.797402 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:39.790161 16567 rpcz_store.cc:276] 0709 13:05:38.643274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.550290 (+907016us) service_pool.cc:224] Handling call
0709 13:05:39.676750 (+126460us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:39.740298 (+ 63548us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:39.744227 (+  3929us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:40.075974 15935 raft_consensus.cc:1240] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:40.074791 15756 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58434 (request call id 341) took 7565 ms. Trace:
W20260709 13:05:40.131323 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.856735 16568 raft_consensus.cc:2393] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 3: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:40.122982 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:39.849814 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:05:39.974668 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.788532 16006 rpcz_store.cc:276] 0709 13:05:37.661824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:37.662350 (+   526us) service_pool.cc:224] Handling call
0709 13:05:37.662725 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:37.675408 (+ 12683us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:37.675450 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:37.676079 (+   629us) log.cc:844] Serialized 26 byte log entry
0709 13:05:38.178475 (+502396us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:38.574237 (+395762us) log.cc:844] Serialized 14 byte log entry
0709 13:05:38.914161 (+339924us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:38.914301 (+   140us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:39.585705 (+671404us) spinlock_profiling.cc:243] Waited 9.8 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:05:39.787374 (+201669us) raft_consensus.cc:1685] finished
0709 13:05:39.787535 (+   161us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:39.787755 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12187,"prepare.queue_time_us":546165,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":87376,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":45580,"spinlock_wait_cycles":22541184,"thread_start_us":1200903,"threads_started":3,"wal-append.queue_time_us":707740}
I20260709 13:05:39.856048 16565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 511) took 2159 ms. Trace:
I20260709 13:05:39.938971 16009 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:05:39.797884 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:40.239300 15856 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
W20260709 13:05:40.227286 15933 rpcz_store.cc:269] 0709 13:05:37.633869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:37.635858 (+  1989us) service_pool.cc:224] Handling call
0709 13:05:39.795936 (+2160078us) spinlock_profiling.cc:243] Waited 252 ms on lock 0x7b54003c50d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:39.796169 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":580103936}
I20260709 13:05:39.788089 18084 consensus_queue.cc:260] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:40.006688 15862 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
I20260709 13:05:40.005882 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1717) took 1556 ms. Trace:
I20260709 13:05:40.070819 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:39.849320 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:05:40.243103 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 77065.2us
I20260709 13:05:40.075656 15858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 1700) took 2582 ms. Trace:
I20260709 13:05:40.123872 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:40.086885 16563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 661) took 1680 ms. Trace:
I20260709 13:05:40.123167 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:39.970854 18012 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:40.002625 18004 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.239326 15859 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
I20260709 13:05:40.079361 17881 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:39.999697 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:05:39.999460 15853 consensus_meta.cc:287] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.586s	user 0.011s	sys 0.011s
I20260709 13:05:40.229751 18048 raft_consensus.cc:740] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:40.074766 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:40.252678 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:40.252918 17866 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:40.253216 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:40.005276 15827 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:40.255659 16569 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:40.073872 17931 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:40.258428 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 144209us
W20260709 13:05:40.255203 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 615668us
I20260709 13:05:40.248522 16563 rpcz_store.cc:276] 0709 13:05:38.406145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.533338 (+1127193us) service_pool.cc:224] Handling call
0709 13:05:39.541745 (+  8407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:39.963325 (+421580us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:40.086470 (+123145us) spinlock_profiling.cc:243] Waited 6.71 ms on lock 0x7b5400550ad0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:40.086693 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15436160}
I20260709 13:05:40.090672 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:40.276446 16570 raft_consensus.cc:2393] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 3: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:39.938601 16564 rpcz_store.cc:276] 0709 13:05:38.138060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:38.139088 (+  1028us) service_pool.cc:224] Handling call
0709 13:05:39.545268 (+1406180us) spinlock_profiling.cc:243] Waited 8.5 ms on lock 0x7b5800044798. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:05:39.546955 (+  1687us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19547520}
I20260709 13:05:40.260133 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:40.072687 15757 catalog_manager.cc:5697] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 3 to 4, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 4 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:40.250358 18012 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:40.272754 15853 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:40.301110 15853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 1720) took 1082 ms. Trace:
I20260709 13:05:40.301469 15853 rpcz_store.cc:276] 0709 13:05:39.218825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.343463 (+124638us) service_pool.cc:224] Handling call
0709 13:05:40.301025 (+957562us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:40.302604 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:40.302431 18004 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:40.272949 15856 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:40.268146 17981 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:40.273320 17932 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:40.274019 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:40.278126 18084 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:40.278791 15935 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:40.280090 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:40.280809 15757 catalog_manager.cc:5697] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 1 to 2, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 2 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:40.281787 15936 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:40.207727 16561 rpcz_store.cc:276] 0709 13:05:38.365494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.497283 (+1131789us) service_pool.cc:224] Handling call
0709 13:05:39.497588 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:39.778834 (+281246us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:39.778909 (+    75us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:39.779027 (+   118us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:39.779095 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:39.779215 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:39.779436 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":138,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":217}
I20260709 13:05:40.307585 17932 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:40.310515 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:40.311245 15936 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:40.284350 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:40.312075 16567 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:40.312526 17932 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:40.286521 15859 raft_consensus.cc:1240] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:40.316944 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:40.317667 15859 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:40.319205 15933 raft_consensus.cc:1240] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:05:40.291844 17969 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.291870 18048 consensus_queue.cc:260] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:05:40.295908 18089 consensus_queue.cc:1048] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:40.325178 15856 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
I20260709 13:05:40.325496 15937 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 4.
I20260709 13:05:40.297544 17957 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.297981 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:05:40.299316 17904 consensus_meta.cc:287] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.728s	user 0.005s	sys 0.009s
W20260709 13:05:40.304296 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.012439 17763 consensus_queue.cc:1048] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.253s
I20260709 13:05:40.307555 15930 raft_consensus.cc:1275] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:40.309770 18086 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.310051 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:40.311580 16561 raft_consensus.cc:1240] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:40.236347 16565 rpcz_store.cc:276] 0709 13:05:37.696544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:37.698893 (+  2349us) service_pool.cc:224] Handling call
0709 13:05:37.732447 (+ 33554us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:38.185269 (+452822us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:38.185314 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:38.233903 (+ 48589us) log.cc:844] Serialized 26 byte log entry
0709 13:05:38.619617 (+385714us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:38.619651 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:38.619731 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:39.582496 (+962765us) spinlock_profiling.cc:243] Waited 2.74 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:05:39.788184 (+205688us) raft_consensus.cc:1685] finished
0709 13:05:39.788222 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:39.855806 (+ 67584us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4572,"raft.queue_time_us":43348,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":8367,"spinlock_wait_cycles":6304128,"thread_start_us":816315,"threads_started":2,"wal-append.queue_time_us":675811}
I20260709 13:05:40.296868 15976 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:40.246707 15858 rpcz_store.cc:276] 0709 13:05:37.492762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:37.765126 (+272364us) service_pool.cc:224] Handling call
0709 13:05:37.765465 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:39.058838 (+1293373us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:39.058866 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:39.059294 (+   428us) log.cc:844] Serialized 26 byte log entry
0709 13:05:39.193637 (+134343us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:39.193677 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:40.074793 (+881116us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54001009d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:40.074872 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:40.074990 (+   118us) raft_consensus.cc:1685] finished
0709 13:05:40.075005 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:40.075421 (+   416us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":914,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":148,"spinlock_wait_cycles":2973537152,"thread_start_us":133988,"threads_started":1,"wal-append.queue_time_us":177433}
I20260709 13:05:40.306424 18012 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:40.320451 15900 connection.cc:256] RPC call timeout handler was delayed by -1.7826603879999539s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18830000000000002s for a response
I20260709 13:05:40.334143 15937 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 1605) took 1134 ms. Trace:
I20260709 13:05:40.335335 17803 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.335819 17932 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:40.335983 18048 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:40.336453 18086 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:40.245548 16002 rpcz_store.cc:276] 0709 13:05:38.449693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:38.980349 (+530656us) service_pool.cc:224] Handling call
0709 13:05:38.998917 (+ 18568us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:39.837986 (+839069us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:40.005404 (+167418us) spinlock_profiling.cc:243] Waited 654 ms on lock 0x7b5400031454. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:40.005774 (+   370us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1503996800}
I20260709 13:05:40.326709 17969 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:40.263643 18038 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:40.345676 15861 raft_consensus.cc:1240] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 5. Current term is 6. Ops: []
W20260709 13:05:40.347623 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 342541us
I20260709 13:05:40.348809 15936 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:05:40.352948 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:40.356521 16570 raft_consensus.cc:1275] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:40.355876 15757 catalog_manager.cc:5697] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 1 to 2, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 2 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:40.356698 16564 raft_consensus.cc:1240] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:40.306664 17858 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:40.345419 15937 rpcz_store.cc:276] 0709 13:05:39.196124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.196221 (+    97us) service_pool.cc:224] Handling call
0709 13:05:40.070412 (+874191us) spinlock_profiling.cc:243] Waited 1.13 ms on lock 0x7b5800003698. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:05:40.330329 (+259917us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2603904}
I20260709 13:05:40.358382 15931 raft_consensus.cc:2410] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:05:40.363629 15900 connection.cc:256] RPC call timeout handler was delayed by -1.469942532999994s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1695s for a response
I20260709 13:05:40.362671 15856 raft_consensus.cc:1275] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:05:40.375633 15900 connection.cc:256] RPC call timeout handler was delayed by -1.3844425329999694s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17900000000000002s for a response
W20260709 13:05:40.376005 15900 connection.cc:256] RPC call timeout handler was delayed by -1.3646425329999374s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18120000000000003s for a response
W20260709 13:05:40.376394 15900 connection.cc:256] RPC call timeout handler was delayed by -1.3530777400000034s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1623s for a response
W20260709 13:05:40.376688 15900 connection.cc:256] RPC call timeout handler was delayed by -1.3034425329999522s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.188s for a response
W20260709 13:05:40.377009 15900 connection.cc:256] RPC call timeout handler was delayed by -1.1317420360001051s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1995s for a response
I20260709 13:05:40.377110 15856 raft_consensus.cc:1275] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:40.344126 15854 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:40.378064 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:40.378669 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:40.379302 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.380040 17999 raft_consensus.cc:2804] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:40.380453 17999 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.380816 17999 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:40.381644 16564 raft_consensus.cc:1240] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:40.368155 17964 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.368611 17803 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:40.396390 18089 raft_consensus.cc:2749] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:40.396824 16567 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:40.397787 17803 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:40.399722 16567 raft_consensus.cc:1275] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:05:40.401345 16567 raft_consensus.cc:1275] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:05:40.401659 15853 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:05:40.232167 15756 rpcz_store.cc:276] 0709 13:05:32.527498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.535296 (+  7798us) service_pool.cc:224] Handling call
0709 13:05:40.074484 (+7539188us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:35.519441 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:35.548285 (+ 28844us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:35.548334 (+    49us) write_op.cc:435] Acquired schema lock
0709 13:05:35.548355 (+    21us) tablet.cc:662] Decoding operations
0709 13:05:35.549548 (+  1193us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:35.549607 (+    59us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:35.549634 (+    27us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:05:35.550819 (+  1185us) tablet.cc:701] Row locks acquired
0709 13:05:35.550845 (+    26us) write_op.cc:260] PREPARE: finished
0709 13:05:35.551008 (+   163us) write_op.cc:270] Start()
0709 13:05:35.551143 (+   135us) write_op.cc:276] Timestamp: P: 1783602335550955 usec, L: 0
0709 13:05:35.551171 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 13:05:35.704611 (+153440us) log.cc:844] Serialized 4725 byte log entry
0709 13:05:36.449848 (+745237us) op_driver.cc:464] REPLICATION: finished
0709 13:05:36.765307 (+315459us) write_op.cc:301] APPLY: starting
0709 13:05:36.765386 (+    79us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:36.765630 (+   244us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:36.765769 (+   139us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:36.934878 (+169109us) spinlock_profiling.cc:243] Waited 24.6 ms on lock 0x7b54003f3c54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a3e28 00007f73eddde581 00007f73eddddd58 00007f73edde29f1 00007f73edde29a9 00007f73edde2939 00007f73edde2901 00007f73edde1b2d 000000000038f354 000000000038f199
0709 13:05:37.005319 (+ 70441us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:37.005651 (+   332us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=12
0709 13:05:37.005719 (+    68us) write_op.cc:312] APPLY: finished
0709 13:05:37.006100 (+   381us) log.cc:844] Serialized 89 byte log entry
0709 13:05:37.006326 (+   226us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:37.006497 (+   171us) write_op.cc:454] Released schema lock
0709 13:05:37.006555 (+    58us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:37.007003 (+   448us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:37.007039 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1994,"reported_tablets":17,"child_traces":[["op",{"apply.queue_time_us":292793,"apply.run_cpu_time_us":5243,"apply.run_wall_time_us":241681,"num_ops":12,"prepare.queue_time_us":438355,"prepare.run_cpu_time_us":53468,"prepare.run_wall_time_us":1448680,"raft.queue_time_us":188673,"raft.run_cpu_time_us":25480,"raft.run_wall_time_us":159745,"replication_time_us":880317,"spinlock_wait_cycles":56609024,"thread_start_us":497079,"threads_started":4,"wal-append.queue_time_us":348936,"wal-append.run_cpu_time_us":26088,"wal-append.run_wall_time_us":1970979}]]}
I20260709 13:05:40.369350 17932 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:40.405989 17932 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:40.369802 15936 raft_consensus.cc:1275] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:05:40.370621 15862 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:40.383345 16570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 585) took 1081 ms. Trace:
I20260709 13:05:40.408193 16570 rpcz_store.cc:276] 0709 13:05:39.289856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.327878 (+1038022us) service_pool.cc:224] Handling call
0709 13:05:40.356084 (+ 28206us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:40.371305 (+ 15221us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:40.371486 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:40.379580 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.386432 17964 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:40.387370 18004 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:40.389191 17858 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:40.389820 15757 catalog_manager.cc:5697] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 1 to 2, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 2 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:40.366956 17957 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:40.429236 18086 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:40.429895 18086 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.430424 18086 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:40.432561 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:40.433534 15933 raft_consensus.cc:2393] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 6: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:40.393214 17969 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:40.395663 17904 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:40.395803 17763 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.411448 16569 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
I20260709 13:05:40.414502 18038 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:40.367324 16562 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:40.416762 15827 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:40.422206 17981 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:40.422611 18012 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:40.423156 15825 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 2.206s (SENT) [suppressed 37 similar messages]
I20260709 13:05:40.434894 15859 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:40.441910 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:40.449846 17904 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:40.452309 17904 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:40.453459 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:40.452947 15757 catalog_manager.cc:5697] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 4 to 5, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 5 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:40.454886 15935 raft_consensus.cc:3055] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:40.455210 15935 raft_consensus.cc:740] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:40.429898 17957 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:40.461228 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 93859.8us
I20260709 13:05:40.455374 15757 catalog_manager.cc:5697] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 1 to 2, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 2 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:05:40.461791 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:40.462139 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.463109 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:40.463797 15931 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:40.459964 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:40.446071 16564 raft_consensus.cc:1275] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:05:40.464377 17981 raft_consensus.cc:2804] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:40.479854 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:40.444186 17763 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:40.484898 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:40.481105 17981 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:40.490779 16011 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41662 (request call id 1713) took 2039 ms (client timeout 1611 ms). Trace:
I20260709 13:05:40.507513 15933 raft_consensus.cc:1275] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:40.477535 17957 consensus_queue.cc:1059] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 13:05:40.525274 15933 raft_consensus.cc:1240] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:40.518275 15825 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:05:40.513057 16011 rpcz_store.cc:269] 0709 13:05:38.448428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.655490 (+1207062us) service_pool.cc:224] Handling call
0709 13:05:40.480805 (+825315us) spinlock_profiling.cc:243] Waited 23.2 ms on lock 0x7b5400305a54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:40.487226 (+  6421us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":53306624}
I20260709 13:05:40.502942 16562 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:40.519411 18132 log.cc:927] Time spent T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:05:40.516022 17981 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:40.549862 15854 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:40.554111 15855 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 4.
I20260709 13:05:40.551067 15935 consensus_queue.cc:260] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:40.561618 15935 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:40.564059 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:40.564826 16569 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:05:40.571300 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:40.574213 15934 raft_consensus.cc:1240] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:40.521413 17866 consensus_queue.cc:1059] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:05:40.586117 15933 raft_consensus.cc:1240] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:40.593924 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:40.594817 15934 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
I20260709 13:05:40.600637 17763 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:05:40.603937 18129 log.cc:927] Time spent T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 13:05:40.605644 16003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1622) took 1316 ms. Trace:
I20260709 13:05:40.606009 16003 rpcz_store.cc:276] 0709 13:05:39.289015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.496664 (+207649us) service_pool.cc:224] Handling call
0709 13:05:39.496993 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:39.497292 (+   299us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:39.497319 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:39.497813 (+   494us) log.cc:844] Serialized 26 byte log entry
0709 13:05:40.243627 (+745814us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:40.251900 (+  8273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:40.256356 (+  4456us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:40.605298 (+348942us) raft_consensus.cc:1685] finished
0709 13:05:40.605316 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:40.605459 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5636,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":7622,"thread_start_us":167436,"threads_started":1,"wal-append.queue_time_us":673752}
I20260709 13:05:40.612087 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:40.613658 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:40.614524 15933 raft_consensus.cc:2393] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 6: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
W20260709 13:05:40.618142 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.584347 18086 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:40.615578 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:40.573761 17932 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.625865 17932 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:40.628001 15900 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:40.628618 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:40.628685 17932 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:40.629431 15930 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
I20260709 13:05:40.629601 15900 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:40.630352 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:40.630697 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:40.632030 15935 raft_consensus.cc:1275] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:05:40.632241 17858 raft_consensus.cc:2804] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:40.585718 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:40.632401 15935 pending_rounds.cc:85] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 1
I20260709 13:05:40.632793 15935 pending_rounds.cc:107] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:05:40.632761 17858 raft_consensus.cc:697] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:40.633278 15935 raft_consensus.cc:2889] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:40.624632 18012 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:40.633967 17858 consensus_queue.cc:237] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:40.634415 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:40.634572 18012 raft_consensus.cc:2749] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:40.634958 15935 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:40.631515 16565 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:40.631490 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:40.645855 18038 raft_consensus.cc:2749] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:05:40.646483 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.647329 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:40.631526 15900 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:40.668058 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:40.684307 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:40.685788 15931 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:40.687091 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:40.696278 15931 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:40.650115 18038 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.687850 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:40.705312 15935 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:40.700728 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:40.702728 15902 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:05:40.713914 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 82446.6us
I20260709 13:05:40.718676 15930 raft_consensus.cc:2393] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 4: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
W20260709 13:05:40.721410 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.723801 18038 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:40.726075 18038 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:40.727180 15902 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:40.728955 15902 leader_election.cc:400] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5.
I20260709 13:05:40.729274 15902 leader_election.cc:403] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:05:40.735260 15931 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
I20260709 13:05:40.731865 17969 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:40.737494 17969 raft_consensus.cc:697] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:40.738612 17969 consensus_queue.cc:237] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:40.743933 15934 raft_consensus.cc:1240] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:40.745736 15934 raft_consensus.cc:1275] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:40.747287 15934 raft_consensus.cc:1275] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:05:40.712105 17904 raft_consensus.cc:2749] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:40.747797 18004 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:40.750674 17803 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:40.711876 17964 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:40.753180 16570 raft_consensus.cc:1275] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:05:40.761411 15933 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:40.766907 15934 raft_consensus.cc:1275] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:05:40.765887 15933 raft_consensus.cc:1275] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:40.743494 16562 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:05:40.787101 15930 raft_consensus.cc:3055] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:40.765204 16564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 532) took 1071 ms. Trace:
I20260709 13:05:40.765352 15853 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
I20260709 13:05:40.787583 15930 raft_consensus.cc:740] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:40.819326 15934 raft_consensus.cc:1275] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:05:40.822250 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 101306us
I20260709 13:05:40.825116 15935 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
I20260709 13:05:40.828596 15933 raft_consensus.cc:1240] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260709 13:05:40.836203 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.816524 17931 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.819190 18012 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.833756 15930 consensus_queue.cc:260] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:40.844254 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 547468us
I20260709 13:05:40.806591 17999 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:40.849373 15930 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:40.853180 15976 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:40.804625 16564 rpcz_store.cc:276] 0709 13:05:39.715788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.382479 (+666691us) service_pool.cc:224] Handling call
0709 13:05:40.382857 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:40.470400 (+ 87543us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:40.748692 (+278292us) spinlock_profiling.cc:243] Waited 24.5 ms on lock 0x7b54005654d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:40.762842 (+ 14150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":56445696}
I20260709 13:05:40.850135 17763 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.837092 17932 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:40.854538 18089 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.843968 17931 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:05:40.869303 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:40.875975 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 409696us
I20260709 13:05:40.876683 17961 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:40.931521 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.859458 18012 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:40.863523 17763 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:40.908912 15974 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:41.001413 15930 raft_consensus.cc:1275] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:40.913483 16533 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:40.915993 16535 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:40.881590 18048 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:41.013754 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 484244us
I20260709 13:05:40.917698 17931 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:41.022285 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:41.022387 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 504188us
I20260709 13:05:40.882752 17932 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:05:41.018086 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 102227us
I20260709 13:05:40.910706 17957 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:41.025506 15825 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:05:41.030033 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 191540us
W20260709 13:05:41.043624 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 130302us
I20260709 13:05:41.024075 17993 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:41.049650 17999 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:41.052958 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:41.053632 16002 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
I20260709 13:05:41.054030 17932 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:41.051333 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:41.054994 16003 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:41.057205 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:41.057741 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:41.057852 16007 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 1.
I20260709 13:05:41.059259 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:41.059883 16002 raft_consensus.cc:1275] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:05:41.061367 15825 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:41.062477 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:41.063015 16011 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:41.009829 17961 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:41.065507 17763 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:41.067745 16003 raft_consensus.cc:2410] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:05:41.068017 15929 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 4.
I20260709 13:05:41.012279 18089 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:41.076768 15757 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.011s	sys 0.042s
W20260709 13:05:41.077348 15757 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.011s	sys 0.042s
I20260709 13:05:41.080473 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:41.081882 16011 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:05:41.047304 17993 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.027573 17957 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:05:41.085915 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:41.085659 16570 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 1 LEARNER]: Advancing to term 4
I20260709 13:05:41.088536 16533 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:41.093555 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:41.077711 18086 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:41.098287 18140 raft_consensus.cc:2804] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:05:41.098482 15929 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 1606) took 1902 ms (client timeout 1562 ms). Trace:
I20260709 13:05:41.099937 16566 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
I20260709 13:05:41.079630 17763 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:41.139223 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:41.140179 16569 raft_consensus.cc:2393] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 4: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
W20260709 13:05:41.112262 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 64447.9us
I20260709 13:05:41.087512 17993 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:41.093335 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:41.105968 18086 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.109447 16562 raft_consensus.cc:1240] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:41.108676 18084 raft_consensus.cc:3055] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:41.108807 18140 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.152523 17866 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.166862 18086 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:41.256688 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.267868 18141 consensus_queue.cc:1059] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.159s
W20260709 13:05:41.269502 18138 log.cc:927] Time spent T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
W20260709 13:05:41.151790 15929 rpcz_store.cc:269] 0709 13:05:39.196439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.200037 (+  3598us) service_pool.cc:224] Handling call
0709 13:05:41.098189 (+1898152us) spinlock_profiling.cc:243] Waited 1.79 s on lock 0x7b540009a1d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:41.098393 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4112801024}
I20260709 13:05:41.280036 18149 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.280887 18149 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:41.283854 17881 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.284560 17881 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:41.286993 17881 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:41.178680 18084 raft_consensus.cc:740] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:41.292248 15900 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:05:41.300427 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.223529 18140 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:41.179082 18012 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:41.160780 17763 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:41.256386 16561 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 4: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:41.301122 16565 raft_consensus.cc:1275] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:05:41.329542 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.283619 18149 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:41.303552 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 420291us
I20260709 13:05:41.257815 18146 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:41.302649 17881 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:05:41.305876 18004 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.011s	sys 0.040s
W20260709 13:05:41.328824 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 418820us
I20260709 13:05:41.336790 17999 raft_consensus.cc:3055] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:41.250063 17866 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:41.350886 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 278311us
W20260709 13:05:41.360205 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 298896us
W20260709 13:05:41.348989 18004 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.012s	sys 0.040s
W20260709 13:05:41.373500 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 296004us
W20260709 13:05:41.379208 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 470360us
W20260709 13:05:41.379719 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:41.380331 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.340828 17881 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.346320 17969 raft_consensus.cc:2804] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:05:41.407992 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 232853us
I20260709 13:05:41.349252 17999 raft_consensus.cc:740] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:41.381568 15902 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:05:41.452107 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.451251 15860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 1713) took 2495 ms. Trace:
W20260709 13:05:41.449517 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 79116.8us
I20260709 13:05:41.372637 18004 raft_consensus.cc:697] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:41.417881 17881 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:41.469259 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 55345.1us
W20260709 13:05:41.471467 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 112231us
I20260709 13:05:41.450730 18146 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.457901 17957 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:41.366633 18084 consensus_queue.cc:260] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:41.453557 15860 rpcz_store.cc:276] 0709 13:05:38.928290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.229997 (+1301707us) service_pool.cc:224] Handling call
0709 13:05:40.268210 (+ 38213us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:40.272910 (+  4700us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:40.763594 (+490684us) log.cc:844] Serialized 14 byte log entry
0709 13:05:41.423000 (+659406us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:41.423067 (+    67us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:41.423132 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:41.423235 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:41.423384 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":777,"prepare.queue_time_us":634741,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":47,"raft.queue_time_us":124501,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":106,"thread_start_us":1104010,"threads_started":3}
I20260709 13:05:41.480204 18048 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:41.483626 17957 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.508785 15976 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:05:41.521662 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 44917.1us
W20260709 13:05:41.531252 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 66146.6us
W20260709 13:05:41.541258 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 56124.2us
I20260709 13:05:41.543326 15825 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:05:41.545543 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 95332.2us
I20260709 13:05:41.553956 16570 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEARNER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:05:41.554577 16006 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:05:41.565718 15902 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
W20260709 13:05:41.568630 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 90105.1us
I20260709 13:05:41.573546 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:41.542286 18141 raft_consensus.cc:2804] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:05:41.581694 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.544008 18038 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.540230 18089 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:41.544785 18084 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:41.592551 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:41.616029 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 46643.4us
W20260709 13:05:41.662158 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 118954us
I20260709 13:05:41.661983 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:41.663591 15862 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:41.581102 17763 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:41.675659 15974 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:05:41.598271 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 89581.5us
I20260709 13:05:41.540038 18146 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:41.593600 18147 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:41.681280 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 108482us
W20260709 13:05:41.615451 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.592854 18149 raft_consensus.cc:2749] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:41.570060 16563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 583) took 2576 ms. Trace:
I20260709 13:05:41.688980 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:41.544394 17932 consensus_queue.cc:1048] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.128s
I20260709 13:05:41.662602 17866 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:41.593881 18160 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.662469 18141 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.593743 18153 raft_consensus.cc:2764] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
W20260709 13:05:41.706571 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.592891 18163 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.569443 17957 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:41.553392 17969 raft_consensus.cc:697] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:05:41.700503 18157 log.cc:927] Time spent T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.230s	user 0.000s	sys 0.001s
I20260709 13:05:41.648699 17904 consensus_queue.cc:1059] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 13:05:41.593475 18145 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:41.592094 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:41.689775 18089 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:41.561659 17999 consensus_queue.cc:260] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:41.705153 18137 log.cc:927] Time spent T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.321s	user 0.001s	sys 0.000s
I20260709 13:05:41.671816 18038 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:41.592684 18169 consensus_queue.cc:1048] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.222s
I20260709 13:05:41.724108 15859 raft_consensus.cc:1240] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 2. Current term is 3. Ops: []
W20260709 13:05:41.723991 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 164944us
I20260709 13:05:41.669348 18004 consensus_queue.cc:237] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:41.723259 15900 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:41.707540 18141 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:41.728425 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 60416.5us
I20260709 13:05:41.729815 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:41.745445 16570 raft_consensus.cc:2393] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 4: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:41.745349 18172 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.745965 18172 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:41.746520 18169 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.746696 15858 raft_consensus.cc:1240] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 1. Current term is 2. Ops: []
W20260709 13:05:41.747997 18154 log.cc:927] Time spent T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
W20260709 13:05:41.749541 18180 log.cc:927] Time spent T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:05:41.750494 18165 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.751001 18165 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:41.732530 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:41.753427 16569 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
W20260709 13:05:41.752367 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 56258.5us
I20260709 13:05:41.751791 16568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 525) took 3017 ms. Trace:
W20260709 13:05:41.735256 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:41.739156 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 124484us
W20260709 13:05:41.726331 17803 consensus_meta.cc:287] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.972s	user 0.013s	sys 0.001s
I20260709 13:05:41.711103 18161 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:41.747080 18169 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:41.734192 18167 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:41.756106 16568 rpcz_store.cc:276] 0709 13:05:38.741551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.259435 (+1517884us) service_pool.cc:224] Handling call
0709 13:05:40.297834 (+ 38399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:40.326928 (+ 29094us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:40.326964 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:40.357813 (+ 30849us) log.cc:844] Serialized 26 byte log entry
0709 13:05:40.821541 (+463728us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:40.821595 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:40.821684 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:41.751540 (+929856us) raft_consensus.cc:1685] finished
0709 13:05:41.751566 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:41.751735 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10198,"raft.queue_time_us":24947,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":10274,"thread_start_us":445180,"threads_started":1,"wal-append.queue_time_us":753029}
I20260709 13:05:41.760208 18156 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:41.760917 17969 consensus_queue.cc:237] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:41.762979 18038 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:41.763056 15862 raft_consensus.cc:2410] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:05:41.763855 15854 raft_consensus.cc:1240] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:41.763948 17999 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:41.726400 18181 raft_consensus.cc:3055] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:41.766635 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:41.764556 18175 consensus_queue.cc:1059] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 6.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 13:05:41.764350 18186 consensus_queue.cc:1048] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:05:41.768463 17866 consensus_queue.cc:1048] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:41.769434 18183 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.770668 18182 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:41.771636 15902 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:41.853860 18183 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:41.854723 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 83148.1us
I20260709 13:05:41.731737 18177 consensus_queue.cc:1048] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.167s
I20260709 13:05:41.856201 18183 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:41.772380 18166 consensus_queue.cc:1059] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:05:41.856576 15902 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:41.857079 18166 raft_consensus.cc:2764] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
W20260709 13:05:41.857954 18155 log.cc:927] Time spent T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.354s	user 0.000s	sys 0.003s
I20260709 13:05:41.857870 18166 consensus_queue.cc:1059] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:41.858456 15902 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:41.859905 18182 raft_consensus.cc:3055] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:41.772687 18161 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:41.868552 17858 raft_consensus.cc:3055] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:41.868762 18161 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:41.869021 17858 raft_consensus.cc:740] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:41.869931 17858 consensus_queue.cc:260] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:41.870522 18175 raft_consensus.cc:3055] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:41.870904 18175 raft_consensus.cc:740] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:41.755086 18145 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:41.860846 15902 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:41.871850 18175 consensus_queue.cc:260] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 6.9, Last appended by leader: 9, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:41.872687 15902 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:05:41.812160 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 106127us
I20260709 13:05:41.871385 17858 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:41.694159 16563 rpcz_store.cc:276] 0709 13:05:38.993718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.284906 (+1291188us) service_pool.cc:224] Handling call
0709 13:05:40.285267 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:40.285646 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:05:40.767583 (+481937us) log.cc:844] Serialized 14 byte log entry
0709 13:05:41.503235 (+735652us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:41.508383 (+  5148us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:41.536052 (+ 27669us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:41.545298 (+  9246us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:41.563543 (+ 18245us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11867,"prepare.queue_time_us":735472,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":462246,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":8111,"spinlock_wait_cycles":23679744,"thread_start_us":1041107,"threads_started":3,"wal-append.queue_time_us":643501}
W20260709 13:05:41.814460 18144 log.cc:927] Time spent T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.515s	user 0.002s	sys 0.002s
I20260709 13:05:41.814702 17957 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:41.815054 18178 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:41.815771 15974 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:05:41.789259 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.706597 18147 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:41.910226 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:41.734472 17961 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:41.820546 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 65726.6us
W20260709 13:05:41.821626 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:41.823694 18151 log.cc:927] Time spent T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.313s	user 0.001s	sys 0.001s
I20260709 13:05:41.816429 18149 consensus_queue.cc:1048] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.008s
W20260709 13:05:41.824628 18174 log.cc:927] Time spent T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:05:41.825111 18172 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:41.825139 18167 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:41.925541 18169 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:41.926187 15827 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:41.926378 18038 consensus_queue.cc:1048] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:41.926932 18167 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:41.716156 18160 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:41.927057 15859 raft_consensus.cc:1275] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:05:41.928534 18038 raft_consensus.cc:2764] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:05:41.836417 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:41.929219 18160 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:41.840171 18165 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:41.929594 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:05:41.930377 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 154564us
I20260709 13:05:41.930989 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:41.931187 15974 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:41.931447 18147 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:41.842859 18086 consensus_meta.cc:287] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.535s	user 0.007s	sys 0.000s
I20260709 13:05:41.931927 16003 raft_consensus.cc:2410] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:05:41.932214 18086 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:41.932731 15974 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:41.934684 18086 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:41.847815 18150 consensus_queue.cc:1048] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 13:05:41.848506 18156 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:41.938651 18156 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:41.864765 15860 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:41.859843 18012 consensus_queue.cc:1059] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:41.860407 18182 raft_consensus.cc:740] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:41.939991 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:41.940135 18012 consensus_queue.cc:1048] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:41.940500 18183 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:41.941124 16570 raft_consensus.cc:2393] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 3: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:41.772615 18153 raft_consensus.cc:3055] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:41.817229 17803 raft_consensus.cc:2749] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5.
I20260709 13:05:41.941874 18153 raft_consensus.cc:740] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:41.942312 17803 raft_consensus.cc:3055] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:41.942737 17803 raft_consensus.cc:740] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:41.942925 18153 consensus_queue.cc:260] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:41.943868 17803 consensus_queue.cc:260] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:41.930409 16568 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:05:41.945408 17803 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:41.939919 17961 raft_consensus.cc:2804] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:41.859196 18166 consensus_queue.cc:1059] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:41.947834 17961 raft_consensus.cc:697] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:41.948006 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:41.948597 18166 raft_consensus.cc:3055] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:41.948872 16569 raft_consensus.cc:2368] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5.
I20260709 13:05:41.949015 18166 raft_consensus.cc:740] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:41.949126 17961 consensus_queue.cc:237] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:41.950181 16567 raft_consensus.cc:1275] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:41.950058 18166 consensus_queue.cc:260] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:41.773949 18089 raft_consensus.cc:697] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:41.875891 17904 raft_consensus.cc:2804] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:41.873114 18175 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:41.876151 18190 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:05:41.880453 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.884577 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:41.885432 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:42.006809 15858 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:41.835915 18181 raft_consensus.cc:740] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:41.824673 18146 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:41.731256 18163 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:41.909430 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:41.873378 18145 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:41.907567 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 94239us
I20260709 13:05:41.728165 18048 raft_consensus.cc:2749] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:41.918640 16006 raft_consensus.cc:2393] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 4: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:41.880240 17931 consensus_queue.cc:1048] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 13:05:41.924737 15857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 1719) took 2771 ms. Trace:
I20260709 13:05:41.929713 15862 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 4: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:41.940340 18182 consensus_queue.cc:260] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 5, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:41.868260 17964 raft_consensus.cc:2804] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:41.951478 18166 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:41.927603 15827 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:42.001009 17904 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.952558 18089 consensus_queue.cc:237] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:41.911659 17981 consensus_meta.cc:287] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 1.349s	user 0.013s	sys 0.007s
I20260709 13:05:42.001768 15974 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:05:42.002050 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.002367 18160 consensus_queue.cc:1048] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:42.006767 16008 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
W20260709 13:05:42.034623 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 158486us
I20260709 13:05:42.051584 15855 raft_consensus.cc:2368] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5.
I20260709 13:05:42.037707 16003 raft_consensus.cc:1240] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:41.920969 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:42.041723 15857 rpcz_store.cc:276] 0709 13:05:39.153877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.237094 (+1083217us) service_pool.cc:224] Handling call
0709 13:05:40.451198 (+214104us) spinlock_profiling.cc:243] Waited 3.22 ms on lock 0x7b5800005798. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef381fc0 00007f73ef36c929 00007f73ef36c3d2 00007f73e88e3bd4
0709 13:05:40.451414 (+   216us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:40.472980 (+ 21566us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:40.473022 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:40.479390 (+  6368us) log.cc:844] Serialized 26 byte log entry
0709 13:05:40.935000 (+455610us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:40.935039 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:40.935118 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:41.814124 (+879006us) spinlock_profiling.cc:243] Waited 5.13 ms on lock 0x7b24001956c8. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71419c4 00007f73e713facb 00007f73e72fadc9 00007f73e72faa55 00007f73e72fa09d 00007f73e9f2c10e 00007f73e9f4388c 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221
0709 13:05:41.924512 (+110388us) raft_consensus.cc:1685] finished
0709 13:05:41.924536 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:41.924679 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15357,"raft.queue_time_us":19105,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":7621,"spinlock_wait_cycles":19200000,"thread_start_us":402771,"threads_started":1,"wal-append.queue_time_us":675722}
I20260709 13:05:42.045894 18159 raft_consensus.cc:2749] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:42.046322 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:42.046470 15861 raft_consensus.cc:1240] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:42.005388 18190 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.893487 18178 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:05:42.056046 18162 log.cc:927] Time spent T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.331s	user 0.000s	sys 0.012s
I20260709 13:05:42.056244 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:42.077966 15936 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 2.
I20260709 13:05:42.057370 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:42.078934 15937 raft_consensus.cc:2368] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 4. Current term is 5.
I20260709 13:05:42.057684 18153 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:42.058144 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:42.058285 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:42.058367 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:05:42.058866 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.058905 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:42.059024 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:42.088639 18145 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:42.094842 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:42.095182 18178 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:42.096418 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:42.097064 15928 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5.
I20260709 13:05:42.060091 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:42.098302 15934 raft_consensus.cc:2393] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 6: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:42.060441 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:42.061863 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:42.090181 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:05:42.042891 18140 consensus_meta.cc:287] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.692s	user 0.007s	sys 0.003s
I20260709 13:05:41.982272 17881 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:42.063527 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:42.041313 16564 raft_consensus.cc:1240] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 3. Current term is 4. Ops: [3.6-3.6]
W20260709 13:05:42.053054 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 125535us
I20260709 13:05:42.067278 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:42.070094 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:42.070680 18141 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:42.071344 15900 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:42.071970 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:42.072070 16562 raft_consensus.cc:2393] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 3: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:42.122025 18183 raft_consensus.cc:2749] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:42.049378 17904 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:42.123548 15853 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
W20260709 13:05:42.124429 16533 leader_election.cc:336] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.506s (SENT)
I20260709 13:05:42.124645 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:05:42.125039 16533 leader_election.cc:336] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 1.506s (SENT)
I20260709 13:05:42.125454 15853 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:05:42.125799 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:42.126957 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:42.122025 16002 raft_consensus.cc:2410] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:05:42.128238 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:42.128844 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:42.129014 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:42.129583 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:42.129971 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:42.130272 16005 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
I20260709 13:05:42.131078 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:42.074523 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 72848.2us
I20260709 13:05:42.051223 18156 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:42.075961 18160 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:42.063138 17981 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:42.082576 15931 raft_consensus.cc:1240] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:42.088099 17964 raft_consensus.cc:697] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:42.088404 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:42.088773 18198 consensus_queue.cc:1048] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:42.093248 15854 raft_consensus.cc:1240] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:42.093468 15935 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5.
I20260709 13:05:42.099503 16003 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:42.089423 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:42.090806 18181 consensus_queue.cc:260] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:42.114341 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:42.120077 15857 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:42.103266 18012 consensus_queue.cc:1048] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 13:05:42.120992 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:05:42.175833 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.189133 15936 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5.
I20260709 13:05:42.118683 18140 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:42.215198 18140 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:42.125454 16533 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:42.076507 18190 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:42.160722 15827 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:05:42.217900 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 57273.7us
I20260709 13:05:42.127513 16007 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:42.122085 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:42.067075 18169 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:05:42.219022 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.098734 18084 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.191615 15861 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
I20260709 13:05:42.133605 16010 raft_consensus.cc:1275] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:42.192988 15858 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:42.132133 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:42.132984 18195 log.cc:927] Time spent T 00000000000000000000000000000000 P d33cebb189b14a989836214fdbafb898: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:05:42.208884 15825 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:42.068084 18167 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:42.134262 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:42.090426 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:42.200346 15976 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:42.118395 18145 raft_consensus.cc:697] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:42.224821 16011 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:42.130312 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:05:42.225641 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 184084us
I20260709 13:05:42.208709 15974 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:42.229586 16009 raft_consensus.cc:2393] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 4: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:42.233886 15825 leader_election.cc:400] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 election: Vote denied by peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) with higher term. Message: Invalid argument: T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 4. Current term is 5.
I20260709 13:05:42.234905 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:42.264967 15827 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:42.236121 17981 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:42.226588 18169 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:42.238946 15931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 424) took 1630 ms. Trace:
I20260709 13:05:42.208263 18149 consensus_queue.cc:1048] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:05:42.131598 16005 raft_consensus.cc:1275] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 13:05:42.139787 15929 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:05:42.138463 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:42.137697 18141 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:42.278402 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 88503.8us
I20260709 13:05:42.144236 18182 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:42.256603 16003 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
W20260709 13:05:42.262490 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 137458us
I20260709 13:05:42.092249 18086 consensus_queue.cc:1048] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.135s
W20260709 13:05:42.267297 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 71569.6us
I20260709 13:05:42.265600 15855 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
W20260709 13:05:42.270375 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.270435 16008 raft_consensus.cc:1240] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 4. Current term is 5. Ops: []
W20260709 13:05:42.271302 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 230726us
W20260709 13:05:42.283514 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 212264us
I20260709 13:05:42.273586 15825 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:42.272496 15931 rpcz_store.cc:276] 0709 13:05:40.608708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.760558 (+151850us) service_pool.cc:224] Handling call
0709 13:05:40.760918 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:42.190389 (+1429471us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.238690 (+ 48301us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54002c2754. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:42.238859 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3039053184}
I20260709 13:05:42.230002 17866 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:42.133301 17904 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:42.288977 15974 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
W20260709 13:05:42.289732 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.226842 17803 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:42.290498 15825 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:42.227945 18156 raft_consensus.cc:697] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:42.136330 18172 consensus_queue.cc:1048] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.125s
I20260709 13:05:42.133344 18163 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:42.287320 16010 raft_consensus.cc:1275] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:42.293745 15862 raft_consensus.cc:2410] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 6 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:05:42.277052 16005 pending_rounds.cc:85] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting all ops after (but not including) 4
I20260709 13:05:42.260939 18160 raft_consensus.cc:697] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:42.271922 18089 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:42.296572 15825 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:42.310189 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:42.324061 15937 raft_consensus.cc:2410] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:05:42.255865 18084 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:05:42.305680 16533 leader_election.cc:336] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.617s (SENT)
I20260709 13:05:42.268888 18150 consensus_queue.cc:1048] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:05:42.328032 15902 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:42.356684 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:42.295473 18147 raft_consensus.cc:2804] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:42.293113 17866 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:42.354812 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:42.358743 16533 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:42.303869 16005 pending_rounds.cc:107] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:05:42.371054 16009 raft_consensus.cc:1240] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 3. Current term is 4. Ops: []
W20260709 13:05:42.296554 18196 log.cc:927] Time spent T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.080s	user 0.001s	sys 0.001s
I20260709 13:05:42.367185 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:05:42.368862 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 79970.4us
I20260709 13:05:42.372992 15974 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:42.269236 18167 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:42.370085 15931 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:05:42.297289 17961 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.303932 18165 raft_consensus.cc:2804] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:42.375864 15902 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:42.375998 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:42.376600 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:42.305069 17999 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:42.326454 16004 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
I20260709 13:05:42.380904 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:42.371237 16005 raft_consensus.cc:2889] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:42.381408 18168 raft_consensus.cc:2749] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:42.381126 18160 consensus_queue.cc:237] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:42.382519 18156 consensus_queue.cc:237] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:42.385814 18145 consensus_queue.cc:237] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:42.316239 18089 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:42.355441 17932 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.326472 18163 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:42.327535 18186 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:05:42.422593 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 49673.2us
I20260709 13:05:42.326570 18159 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:42.371737 17964 consensus_queue.cc:237] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:42.381096 18194 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:42.539299 18194 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.539690 18194 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:42.539918 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:42.427305 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:42.372135 17866 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:42.375133 18198 raft_consensus.cc:2749] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:42.445248 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:42.548588 16003 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:05:42.449713 18167 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:42.457407 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 63054.4us
I20260709 13:05:42.531001 18163 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.461105 15931 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:05:42.462483 16006 raft_consensus.cc:2410] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:05:42.465121 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.424767 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:42.475960 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:42.354586 18191 consensus_queue.cc:1059] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.206s
I20260709 13:05:42.460045 17999 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:42.530970 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 238422us
I20260709 13:05:42.534204 18186 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:42.459064 18086 raft_consensus.cc:2749] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:42.460282 18165 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.468170 17961 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:42.485433 18153 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:42.518661 18183 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:42.472141 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:42.568658 16002 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 1.
I20260709 13:05:42.569909 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:42.531400 15900 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:42.528913 18089 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:42.571820 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:42.573158 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:42.573994 16004 raft_consensus.cc:2410] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:42.574797 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:42.576047 18197 raft_consensus.cc:2749] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:05:42.458912 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 83137.6us
W20260709 13:05:42.543222 17993 consensus_meta.cc:287] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 1.366s	user 0.007s	sys 0.005s
W20260709 13:05:42.555471 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.555842 16010 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:42.556236 16009 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:05:42.557596 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:42.457055 18147 raft_consensus.cc:697] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 4 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:42.558130 18163 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:42.565107 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:42.591384 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:42.591663 16003 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5.
W20260709 13:05:42.571847 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 108730us
W20260709 13:05:42.592201 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:42.592653 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.534844 17932 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:42.500888 18188 consensus_queue.cc:1059] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:05:42.578572 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:42.536464 18190 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:42.570928 16002 raft_consensus.cc:2393] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 4: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:42.548056 16561 raft_consensus.cc:2468] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:05:42.586045 16007 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
I20260709 13:05:42.586728 15827 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:42.602319 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:42.575433 16004 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
W20260709 13:05:42.563907 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.588608 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:42.589079 17993 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:42.579239 18153 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.589726 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:42.611774 16009 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5.
I20260709 13:05:42.586580 18183 raft_consensus.cc:697] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:42.614051 18183 consensus_queue.cc:237] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:42.615000 17961 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:42.565379 17881 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:42.611491 18146 consensus_queue.cc:1048] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:05:42.581489 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:42.617498 15935 raft_consensus.cc:1240] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:05:42.592825 15900 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:42.593627 15902 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:42.618973 15902 leader_election.cc:400] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5.
I20260709 13:05:42.619313 15902 leader_election.cc:403] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:05:42.594789 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:42.595062 15855 raft_consensus.cc:2393] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 4: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:42.562888 18186 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:42.561442 17999 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:42.598299 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 56423.8us
I20260709 13:05:42.599226 16570 raft_consensus.cc:2393] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 4: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:42.577610 18165 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:42.603665 17904 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:42.604508 15827 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:42.605144 18038 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:05:42.606856 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:42.649405 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 56664.3us
W20260709 13:05:42.649590 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.621923 15902 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:42.620594 15859 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
I20260709 13:05:42.610262 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:42.651722 16007 raft_consensus.cc:2393] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 4: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:42.624681 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:42.653689 16003 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:05:42.638093 16010 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
I20260709 13:05:42.617229 18153 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:42.623898 17961 raft_consensus.cc:2804] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:42.616142 18147 consensus_queue.cc:237] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:42.645587 18182 raft_consensus.cc:2749] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5.
I20260709 13:05:42.620630 18197 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:42.649065 18004 raft_consensus.cc:2749] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:42.679287 15859 raft_consensus.cc:1275] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:05:42.679812 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 75396.2us
I20260709 13:05:42.649623 18169 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:42.683727 15900 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:05:42.690946 15974 connection.cc:256] RPC call timeout handler was delayed by -2.4860594340000262s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1562s for a response
I20260709 13:05:42.692027 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:42.697669 16003 raft_consensus.cc:1240] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:42.698151 15859 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:42.622254 16005 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:05:42.700906 16010 raft_consensus.cc:1240] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:42.645478 18166 consensus_queue.cc:1048] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:05:42.703253 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:42.663254 17961 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:05:42.696724 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 74895us
I20260709 13:05:42.699378 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:42.700948 15861 raft_consensus.cc:1275] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:05:42.659235 18159 raft_consensus.cc:697] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:42.683113 17904 raft_consensus.cc:2749] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:05:42.692983 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 148260us
I20260709 13:05:42.708690 18167 consensus_queue.cc:1048] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:42.668489 17931 consensus_queue.cc:1048] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
W20260709 13:05:42.709962 18215 log.cc:927] Time spent T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 13:05:42.710320 18168 consensus_queue.cc:1048] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:05:42.715209 18161 consensus_meta.cc:287] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.846s	user 0.003s	sys 0.010s
I20260709 13:05:42.751627 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:42.753737 16003 raft_consensus.cc:2410] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:05:42.716374 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:42.754654 15855 raft_consensus.cc:2393] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 4: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:42.683722 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:42.695384 15827 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:42.761101 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:42.730634 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:05:42.734579 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 50990.6us
I20260709 13:05:42.699111 17981 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:42.709647 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.737360 16535 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:42.701539 17881 raft_consensus.cc:2749] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:42.750613 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:42.767783 15936 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 4.
I20260709 13:05:42.768023 15900 leader_election.cc:400] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5.
I20260709 13:05:42.751096 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:42.756048 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:42.719043 17961 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:42.733574 17969 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:05:42.761914 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 66606.3us
I20260709 13:05:42.763932 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:42.698182 18197 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:42.705000 18188 consensus_queue.cc:1059] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
W20260709 13:05:42.770937 15974 connection.cc:256] RPC call timeout handler was delayed by -2.1188594340000009s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.197s for a response
I20260709 13:05:42.702746 18169 consensus_queue.cc:1048] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 13:05:42.774703 15932 raft_consensus.cc:2393] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 4: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:42.744041 18089 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:42.763695 17957 raft_consensus.cc:2764] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:05:42.767212 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:42.837387 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:42.837715 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:42.774219 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:42.836901 16004 raft_consensus.cc:2393] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 3: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
W20260709 13:05:42.877178 17803 consensus_meta.cc:287] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.581s	user 0.009s	sys 0.001s
I20260709 13:05:42.877355 15853 raft_consensus.cc:2393] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 3: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:42.763175 18161 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:42.771946 17981 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:42.881844 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:42.959414 15825 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:42.961920 16003 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
I20260709 13:05:42.924613 16009 raft_consensus.cc:1240] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:42.959805 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:42.881057 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:05:43.031461 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 72127.1us
I20260709 13:05:43.035152 15862 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 4.
I20260709 13:05:42.877568 17932 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:43.033821 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 50373.5us
I20260709 13:05:43.042085 15827 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:05:43.032091 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 264180us
I20260709 13:05:43.043527 15900 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:05:42.956096 18206 log.cc:927] Time spent T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.215s	user 0.001s	sys 0.000s
I20260709 13:05:43.040870 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:43.047798 16003 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
I20260709 13:05:43.050984 15827 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:05:43.029721 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 292417us
I20260709 13:05:42.845515 17969 raft_consensus.cc:697] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:05:42.957453 18223 log.cc:927] Time spent T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:05:42.782843 18217 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:42.906164 18224 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.085001 18224 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:43.087149 18224 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:42.877744 16005 raft_consensus.cc:1275] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:05:43.076992 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:42.873194 18172 raft_consensus.cc:2804] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:43.092307 15933 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 2.
I20260709 13:05:43.077425 17981 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:43.079712 18197 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:43.079967 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:42.880504 16562 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
I20260709 13:05:43.044749 17803 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:43.081101 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:43.103168 15862 raft_consensus.cc:2393] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 4: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:43.035763 17993 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:43.084354 17904 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.086992 17999 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:43.039335 18145 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:05:43.094923 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 51504.7us
I20260709 13:05:43.106791 15902 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:43.055502 18150 raft_consensus.cc:2749] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:43.108945 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:43.074374 17964 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:43.112139 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:43.115531 15902 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:43.092952 17881 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:42.928622 18194 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:43.143707 15933 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:05:43.144419 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:43.116077 17904 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:43.149909 18161 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:43.150676 17904 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:43.150902 18181 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:43.151782 18012 raft_consensus.cc:2804] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:43.152331 18012 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.152769 18012 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:43.156931 18225 log.cc:927] Time spent T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:05:43.157243 17999 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:43.145695 18145 raft_consensus.cc:697] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 4 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:43.109345 18172 raft_consensus.cc:697] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:05:43.164052 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 254993us
I20260709 13:05:43.178546 15858 raft_consensus.cc:2368] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 2. Current term is 3.
I20260709 13:05:42.930058 18159 consensus_queue.cc:237] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:43.125766 17931 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:43.183223 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:43.212061 15933 raft_consensus.cc:2393] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 3: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:43.161099 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:05:43.212682 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 161797us
I20260709 13:05:43.234338 15827 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:43.235565 15827 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:43.238194 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:43.092573 18217 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:43.182123 17964 raft_consensus.cc:697] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 5 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:43.242946 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:43.211591 17931 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.210992 17957 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:43.245632 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 85791.2us
W20260709 13:05:43.235882 18165 consensus_meta.cc:287] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.613s	user 0.003s	sys 0.007s
I20260709 13:05:43.250591 15902 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:43.246871 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:43.165653 17904 consensus_queue.cc:1059] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.066s
I20260709 13:05:43.249050 16535 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:43.231803 17881 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.252763 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:43.254132 15933 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
I20260709 13:05:43.158931 18145 consensus_queue.cc:237] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:43.258345 16003 raft_consensus.cc:2410] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:05:43.258665 17993 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:43.259001 17803 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:43.253829 18165 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:43.253947 18038 raft_consensus.cc:2804] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:43.242449 18150 raft_consensus.cc:2749] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:43.237632 18175 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:43.249343 17931 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:43.264863 15974 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:43.245685 18089 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:43.241537 16004 raft_consensus.cc:2376] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4. Re-sending same reply.
I20260709 13:05:43.269016 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:43.260070 18038 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.266171 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:43.280781 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:43.282192 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:43.283236 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:43.285240 18217 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:43.268054 16535 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:43.286688 16535 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:43.287609 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:43.287879 17993 raft_consensus.cc:2804] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:43.288290 17993 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.288352 15854 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:43.288762 17993 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:43.289367 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:43.289914 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:43.290342 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.288364 18172 consensus_queue.cc:237] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:43.291057 18183 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:43.291613 18183 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.291960 18038 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:43.292052 18183 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:43.289721 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:43.252983 17957 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:43.307016 16565 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:05:43.294515 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:43.307459 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:43.291060 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:43.256649 18197 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:43.264333 18186 consensus_meta.cc:287] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.607s	user 0.008s	sys 0.002s
I20260709 13:05:43.239843 17969 consensus_queue.cc:237] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:43.260330 17881 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:43.296614 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:43.409061 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:43.503610 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:05:43.815565 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:44.011456 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.654507 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:05:43.546172 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 292557us
I20260709 13:05:43.305872 15862 raft_consensus.cc:2410] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 6 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
W20260709 13:05:44.012121 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.738375 15858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 1730) took 1035 ms. Trace:
I20260709 13:05:44.014051 15858 rpcz_store.cc:276] 0709 13:05:42.702743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.702939 (+   196us) service_pool.cc:224] Handling call
0709 13:05:43.738145 (+1035206us) spinlock_profiling.cc:243] Waited 474 ms on lock 0x7b5400248bd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:43.738309 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1090393472}
W20260709 13:05:43.753995 15860 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58214 (request call id 429) took 1675 ms (client timeout 1617 ms). Trace:
W20260709 13:05:44.015056 15860 rpcz_store.cc:269] 0709 13:05:42.041486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.045991 (+  4505us) service_pool.cc:224] Handling call
0709 13:05:43.656774 (+1610783us) spinlock_profiling.cc:243] Waited 697 ms on lock 0x7b5400218254. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:43.717311 (+ 60537us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1603276288}
W20260709 13:05:43.268242 18232 log.cc:927] Time spent T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 13:05:44.040657 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:44.041335 15936 raft_consensus.cc:2376] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4. Re-sending same reply.
W20260709 13:05:43.294313 18163 consensus_meta.cc:287] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.692s	user 0.009s	sys 0.000s
W20260709 13:05:43.294471 17866 consensus_meta.cc:287] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.734s	user 0.007s	sys 0.003s
W20260709 13:05:43.439540 18227 log.cc:927] Time spent T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
W20260709 13:05:43.934211 18201 log.cc:927] Time spent T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.511s	user 0.001s	sys 0.000s
I20260709 13:05:44.046646 15858 raft_consensus.cc:1240] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:43.297806 18165 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:44.052948 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:05:43.499429 18218 log.cc:927] Time spent T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.626s	user 0.001s	sys 0.001s
I20260709 13:05:43.561010 18181 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:43.267918 18175 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:44.081545 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:43.296178 18161 consensus_queue.cc:1048] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.779s
I20260709 13:05:43.294844 15854 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:05:44.096601 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:44.097196 17993 consensus_meta.cc:287] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.808s	user 0.006s	sys 0.005s
I20260709 13:05:43.515592 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:44.097666 16005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60156 (request call id 399) took 1316 ms. Trace:
I20260709 13:05:44.097630 17993 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:44.098320 16005 rpcz_store.cc:276] 0709 13:05:42.781483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.115466 (+333983us) service_pool.cc:224] Handling call
0709 13:05:44.096432 (+980966us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b5800114398. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:05:44.097590 (+  1158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33368320}
I20260709 13:05:43.628804 18170 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:44.099982 18170 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:44.101317 18221 log.cc:927] Time spent T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 1.005s	user 0.006s	sys 0.004s
I20260709 13:05:44.086350 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:44.102465 18170 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:44.103600 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:44.105100 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:44.105991 15936 raft_consensus.cc:2393] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 4: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:44.087023 15974 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:44.032807 17981 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:44.080164 18235 log.cc:927] Time spent T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.676s	user 0.001s	sys 0.001s
I20260709 13:05:44.083730 15861 raft_consensus.cc:1240] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:43.736063 18150 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.433308 18194 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:44.085556 15860 raft_consensus.cc:1240] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:05:44.075733 18158 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.297762 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:43.295964 18228 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:44.126915 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:44.111775 15857 raft_consensus.cc:2393] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 3: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
W20260709 13:05:44.128089 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 194703us
I20260709 13:05:44.099077 16004 raft_consensus.cc:1240] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:44.047485 18197 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:43.305541 18183 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:44.132267 15936 raft_consensus.cc:2410] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:05:43.646368 18038 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:44.083003 17866 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:44.154356 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 80049.1us
I20260709 13:05:44.019274 18186 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:44.159391 15861 raft_consensus.cc:1275] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:05:44.126245 18228 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:44.140945 18089 raft_consensus.cc:2804] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:44.163530 16567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 550) took 1614 ms. Trace:
I20260709 13:05:43.296521 17964 consensus_queue.cc:237] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:44.089418 18163 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:44.163960 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:44.121696 18150 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:44.094799 18181 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:44.165508 16567 rpcz_store.cc:276] 0709 13:05:42.548695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.620754 (+ 72059us) service_pool.cc:224] Handling call
0709 13:05:42.652527 (+ 31773us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:42.678717 (+ 26190us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:42.678744 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:42.694042 (+ 15298us) log.cc:844] Serialized 26 byte log entry
0709 13:05:43.054701 (+360659us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:43.054756 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:43.054829 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:44.163273 (+1108444us) raft_consensus.cc:1685] finished
0709 13:05:44.163315 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:44.163463 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":20769,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":5097,"thread_start_us":287461,"threads_started":1,"wal-append.queue_time_us":385535}
I20260709 13:05:44.111552 17981 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:44.165380 15853 raft_consensus.cc:2393] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 6: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:44.159049 18241 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.158684 18194 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.130684 18158 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:44.172504 15936 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
I20260709 13:05:44.178386 16570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 551) took 1618 ms. Trace:
I20260709 13:05:44.183524 15859 raft_consensus.cc:2393] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 6: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:44.167966 16004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1743) took 1421 ms. Trace:
I20260709 13:05:44.185312 15856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 443) took 1626 ms. Trace:
I20260709 13:05:44.184244 16570 rpcz_store.cc:276] 0709 13:05:42.561519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.624020 (+ 62501us) service_pool.cc:224] Handling call
0709 13:05:42.655503 (+ 31483us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:42.659476 (+  3973us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:42.659514 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:42.683151 (+ 23637us) log.cc:844] Serialized 26 byte log entry
0709 13:05:42.982822 (+299671us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:42.982882 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.982987 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:44.178153 (+1195166us) raft_consensus.cc:1685] finished
0709 13:05:44.178170 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:44.178313 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2636,"raft.queue_time_us":92,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":10303,"thread_start_us":247688,"threads_started":1,"wal-append.queue_time_us":422336}
I20260709 13:05:44.166080 18228 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:44.161919 18177 consensus_queue.cc:1048] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:05:44.188304 15856 rpcz_store.cc:276] 0709 13:05:42.558890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.559039 (+   149us) service_pool.cc:224] Handling call
0709 13:05:42.576019 (+ 16980us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:42.576446 (+   427us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:42.576476 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:42.577033 (+   557us) log.cc:844] Serialized 52 byte log entry
0709 13:05:42.660385 (+ 83352us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:42.971918 (+311533us) log.cc:844] Serialized 14 byte log entry
0709 13:05:43.033977 (+ 62059us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:43.034057 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:43.498665 (+464608us) spinlock_profiling.cc:243] Waited 22.9 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:05:44.180076 (+681411us) raft_consensus.cc:1685] finished
0709 13:05:44.185112 (+  5036us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:44.185247 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59331,"prepare.queue_time_us":414389,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":69464,"raft.run_cpu_time_us":368,"raft.run_wall_time_us":27812,"spinlock_wait_cycles":52759424,"thread_start_us":351896,"threads_started":2,"wal-append.queue_time_us":188576}
I20260709 13:05:44.179628 16005 raft_consensus.cc:1240] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:05:44.214466 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:44.182902 18161 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:05:44.187386 15929 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 1661) took 2128 ms (client timeout 1853 ms). Trace:
I20260709 13:05:44.190515 18246 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:44.230099 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 72469.6us
I20260709 13:05:44.246457 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:05:44.226962 17961 consensus_meta.cc:287] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 1.442s	user 0.013s	sys 0.001s
I20260709 13:05:44.216027 18089 raft_consensus.cc:697] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:44.251593 17881 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:44.218915 17993 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:44.187932 18194 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:44.186393 18241 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:44.187117 16004 rpcz_store.cc:276] 0709 13:05:42.741695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.098594 (+1356899us) service_pool.cc:224] Handling call
0709 13:05:44.098890 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:44.142552 (+ 43662us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.163000 (+ 20448us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:44.232034 16005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 402) took 1125 ms. Trace:
I20260709 13:05:44.233374 18242 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.267875 15974 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:44.228603 18161 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:44.280275 17981 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:44.280978 18086 raft_consensus.cc:3055] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:05:44.281317 18153 consensus_meta.cc:287] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 1.588s	user 0.012s	sys 0.000s
I20260709 13:05:44.281502 18086 raft_consensus.cc:740] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:05:44.281836 18244 log.cc:927] Time spent T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
W20260709 13:05:44.282788 18243 log.cc:927] Time spent T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:05:44.282527 18086 consensus_queue.cc:260] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:44.284256 18245 log.cc:927] Time spent T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:05:44.260071 17881 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:44.226636 15929 rpcz_store.cc:269] 0709 13:05:42.056151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.058040 (+  1889us) service_pool.cc:224] Handling call
0709 13:05:44.177263 (+2119223us) spinlock_profiling.cc:243] Waited 49.2 ms on lock 0x7b54001a81d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:44.184817 (+  7554us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":113051392}
W20260709 13:05:44.281178 18240 log.cc:927] Time spent T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 13:05:44.303419 16535 leader_election.cc:336] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 1.751s (SENT)
I20260709 13:05:44.256527 17961 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:44.310081 15936 raft_consensus.cc:2410] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:05:44.306681 16535 leader_election.cc:304] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:44.244336 18246 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:44.278229 18175 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:44.305840 18159 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.001s	sys 0.004s
W20260709 13:05:44.318578 15900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.002s	sys 0.005s
I20260709 13:05:44.321875 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:05:44.322228 15900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.002s	sys 0.005s
W20260709 13:05:44.320851 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 65975.8us
I20260709 13:05:44.300992 18177 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:44.305006 18181 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:44.269790 18256 consensus_queue.cc:1048] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 13:05:44.315368 18086 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:44.265043 16005 rpcz_store.cc:276] 0709 13:05:43.107054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.111612 (+1004558us) service_pool.cc:224] Handling call
0709 13:05:44.165604 (+ 53992us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:44.206237 (+ 40633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.229864 (+ 23627us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:44.313191 18038 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:44.281858 18153 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:44.335449 15974 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:05:44.321961 18159 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.001s	sys 0.004s
I20260709 13:05:44.313408 18183 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:44.342466 15974 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:05:44.347373 16535 leader_election.cc:336] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 1.793s (SENT)
I20260709 13:05:44.326614 18251 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.328367 18175 raft_consensus.cc:2749] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:44.325888 18177 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:44.305672 17881 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:44.344947 17866 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:44.344877 18259 raft_consensus.cc:2764] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:05:44.371256 18194 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:44.339316 18255 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.352825 18158 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:44.352964 18183 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:05:44.353901 18150 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:44.354121 18153 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:44.354704 15933 raft_consensus.cc:2393] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 5: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:44.356012 18246 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:44.390508 16008 raft_consensus.cc:1240] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:44.391160 18241 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:44.391929 16566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 598) took 2046 ms. Trace:
I20260709 13:05:44.392882 17961 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:44.369223 18251 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:44.372219 18012 consensus_meta.cc:287] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 1.219s	user 0.014s	sys 0.001s
I20260709 13:05:44.359889 15854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1620) took 1075 ms. Trace:
I20260709 13:05:44.350503 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:44.344839 18089 consensus_queue.cc:237] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:44.330590 16563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 599) took 1985 ms. Trace:
I20260709 13:05:44.330018 18197 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:44.273550 18242 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
W20260709 13:05:44.373108 16535 leader_election.cc:336] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 1.822s (SENT)
W20260709 13:05:44.370033 17931 consensus_meta.cc:287] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 1.057s	user 0.003s	sys 0.009s
I20260709 13:05:44.339612 18186 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:05:44.389103 15930 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 1663) took 2331 ms (client timeout 2858 ms). Trace:
I20260709 13:05:44.389823 17803 raft_consensus.cc:3055] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:44.397773 18161 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:44.399166 15860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1623) took 1113 ms. Trace:
W20260709 13:05:44.514566 18253 log.cc:927] Time spent T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.199s	user 0.002s	sys 0.000s
I20260709 13:05:44.520061 15928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1672) took 2459 ms. Trace:
I20260709 13:05:44.520932 15860 rpcz_store.cc:276] 0709 13:05:43.285355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.110881 (+825526us) service_pool.cc:224] Handling call
0709 13:05:44.133281 (+ 22400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:44.133689 (+   408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:44.133722 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:44.158306 (+ 24584us) log.cc:844] Serialized 26 byte log entry
0709 13:05:44.227173 (+ 68867us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:44.227219 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.227285 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:44.398950 (+171665us) raft_consensus.cc:1685] finished
0709 13:05:44.398971 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:44.399107 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":539,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":15428,"thread_start_us":53057,"threads_started":1,"wal-append.queue_time_us":197993}
I20260709 13:05:44.508370 15935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1675) took 2300 ms. Trace:
I20260709 13:05:44.399631 17881 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:44.359908 18163 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:44.509629 15859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 1734) took 1746 ms. Trace:
I20260709 13:05:44.532276 15859 rpcz_store.cc:276] 0709 13:05:42.763155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.764109 (+   954us) service_pool.cc:224] Handling call
0709 13:05:43.503402 (+739293us) spinlock_profiling.cc:243] Waited 1.41 ms on lock 0x7b5800005798. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:05:44.509393 (+1005991us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b5400217d54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:44.509563 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":398748416}
I20260709 13:05:44.401942 18251 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:44.400624 18255 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:44.516754 18012 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:05:44.516812 15930 rpcz_store.cc:269] 0709 13:05:42.057385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.057907 (+   522us) service_pool.cc:224] Handling call
0709 13:05:44.388865 (+2330958us) spinlock_profiling.cc:243] Waited 677 ms on lock 0x7b54000992d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:44.389034 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1557216512}
I20260709 13:05:44.537034 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:44.539228 15930 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:44.369387 18177 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:44.518308 15900 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:05:44.523470 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 150370us
I20260709 13:05:44.559322 15858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1619) took 1275 ms. Trace:
W20260709 13:05:44.567134 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:44.567708 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:44.566992 15858 rpcz_store.cc:276] 0709 13:05:43.284355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.088308 (+803953us) service_pool.cc:224] Handling call
0709 13:05:44.088653 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:44.088997 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:44.089028 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:44.093186 (+  4158us) log.cc:844] Serialized 26 byte log entry
0709 13:05:44.230925 (+137739us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:44.313858 (+ 82933us) log.cc:844] Serialized 14 byte log entry
0709 13:05:44.314171 (+   313us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.314282 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:44.513985 (+199703us) spinlock_profiling.cc:243] Waited 3.74 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:05:44.520572 (+  6587us) raft_consensus.cc:1685] finished
0709 13:05:44.559054 (+ 38482us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:44.559252 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":169223,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":16447,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":6373,"spinlock_wait_cycles":8598784,"thread_start_us":203030,"threads_started":2,"wal-append.queue_time_us":180045}
I20260709 13:05:44.515483 15854 rpcz_store.cc:276] 0709 13:05:43.284618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.096299 (+811681us) service_pool.cc:224] Handling call
0709 13:05:44.096642 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:44.096877 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:44.096918 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:44.097440 (+   522us) log.cc:844] Serialized 26 byte log entry
0709 13:05:44.179519 (+ 82079us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:44.269265 (+ 89746us) log.cc:844] Serialized 14 byte log entry
0709 13:05:44.309191 (+ 39926us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.318388 (+  9197us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:44.359622 (+ 41234us) raft_consensus.cc:1685] finished
0709 13:05:44.359651 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:44.359827 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23323,"prepare.queue_time_us":217454,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":29936,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":11926,"thread_start_us":119806,"threads_started":2,"wal-append.queue_time_us":213235}
I20260709 13:05:44.392463 16566 rpcz_store.cc:276] 0709 13:05:42.345064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.345543 (+   479us) service_pool.cc:224] Handling call
0709 13:05:44.176154 (+1830611us) spinlock_profiling.cc:243] Waited 4.4 ms on lock 0x7b5800044798. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef381fc0 00007f73ef36c929 00007f73ef36c3d2 00007f73e88e3bd4
0709 13:05:44.176385 (+   231us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:44.176659 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:05:44.259142 (+ 82483us) log.cc:844] Serialized 14 byte log entry
0709 13:05:44.391475 (+132333us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:44.391529 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:44.391586 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.391698 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:44.391847 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":268,"prepare.queue_time_us":49322,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":24,"raft.queue_time_us":5898,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":84,"spinlock_wait_cycles":10126080,"thread_start_us":190685,"threads_started":2,"wal-append.queue_time_us":246963}
W20260709 13:05:44.576270 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 58049.4us
I20260709 13:05:44.524928 15935 rpcz_store.cc:276] 0709 13:05:42.208233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.208357 (+   124us) service_pool.cc:224] Handling call
0709 13:05:42.234997 (+ 26640us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:42.617785 (+382788us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.507915 (+1890130us) spinlock_profiling.cc:243] Waited 382 ms on lock 0x7b540034ba54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:44.508237 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":879096576}
I20260709 13:05:44.565163 15825 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:44.493382 18150 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.578214 15900 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:44.520534 15928 rpcz_store.cc:276] 0709 13:05:42.060733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.097694 (+ 36961us) service_pool.cc:224] Handling call
0709 13:05:42.102866 (+  5172us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:42.211849 (+108983us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:42.219507 (+  7658us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:42.294206 (+ 74699us) log.cc:844] Serialized 26 byte log entry
0709 13:05:42.544436 (+250230us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:42.549388 (+  4952us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.551847 (+  2459us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:44.397017 (+1845170us) spinlock_profiling.cc:243] Waited 1.52 ms on lock 0x7b18003255a0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb3f3 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd 000000000038f354 000000000038f199 00007f73e729e83a
0709 13:05:44.454069 (+ 57052us) raft_consensus.cc:1685] finished
0709 13:05:44.454097 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:44.519929 (+ 65832us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38621,"raft.queue_time_us":14878,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":35340,"spinlock_wait_cycles":3500800,"thread_start_us":88607,"threads_started":1,"wal-append.queue_time_us":173124}
I20260709 13:05:44.581064 18242 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
I20260709 13:05:44.524106 17931 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:44.582657 15937 raft_consensus.cc:2393] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 3: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:44.583633 17931 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:44.402271 18183 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:44.584451 18183 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:44.587853 18150 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:44.589946 18150 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:44.590039 18012 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:44.517709 17803 raft_consensus.cc:740] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:05:44.591315 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:44.594250 18190 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:44.594964 18242 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:44.595368 18242 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:44.595741 18242 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:44.518747 16563 rpcz_store.cc:276] 0709 13:05:42.345402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.354981 (+  9579us) service_pool.cc:224] Handling call
0709 13:05:43.444572 (+1089591us) spinlock_profiling.cc:243] Waited 34.8 ms on lock 0x7b5800044798. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef381fc0 00007f73ef36c929 00007f73ef36c3d2 00007f73e88e3bd4
0709 13:05:43.444918 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:43.445600 (+   682us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:43.445656 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:43.490102 (+ 44446us) log.cc:844] Serialized 26 byte log entry
0709 13:05:43.884146 (+394044us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:44.214716 (+330570us) log.cc:844] Serialized 14 byte log entry
0709 13:05:44.230796 (+ 16080us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.245638 (+ 14842us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:44.313517 (+ 67879us) raft_consensus.cc:1685] finished
0709 13:05:44.318959 (+  5442us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:44.328538 (+  9579us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1005,"prepare.queue_time_us":228052,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":25,"raft.queue_time_us":6268,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":6164,"spinlock_wait_cycles":80136960,"thread_start_us":318829,"threads_started":2,"wal-append.queue_time_us":588602}
I20260709 13:05:44.604426 17803 consensus_queue.cc:260] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:44.593567 15900 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:44.605540 17803 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:44.611088 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 268680us
I20260709 13:05:44.608512 15929 raft_consensus.cc:2393] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 4: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:44.616470 15930 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:05:44.627120 15974 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:44.607616 18150 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:44.592833 17969 raft_consensus.cc:2749] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:44.630582 15974 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:44.607410 18194 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:44.610313 18190 raft_consensus.cc:2749] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:44.632131 15974 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:44.632383 18140 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:44.632694 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:44.638356 15929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53636 (request call id 442) took 1058 ms. Trace:
I20260709 13:05:44.645370 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:44.631537 15825 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:44.648078 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:44.655364 15974 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:44.654749 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:44.656468 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:44.656140 18272 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:44.656322 16562 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:44.642719 18181 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.669193 15860 raft_consensus.cc:1275] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:05:44.639389 18089 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:05:44.653986 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 133802us
I20260709 13:05:44.657981 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:44.642859 18175 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:44.656553 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:44.672326 15935 raft_consensus.cc:2393] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 6: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:44.667042 15974 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:44.668360 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:44.657673 17981 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:44.673871 17981 raft_consensus.cc:697] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 6 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
W20260709 13:05:44.657054 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 63548.8us
I20260709 13:05:44.661947 18272 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:44.672262 16533 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:44.676258 15974 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:44.655189 18217 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.657856 18186 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.644660 18167 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.640162 18258 consensus_queue.cc:1059] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:05:44.680334 16567 raft_consensus.cc:2410] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 7 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:05:44.644721 15929 rpcz_store.cc:276] 0709 13:05:43.579552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.336238 (+756686us) service_pool.cc:224] Handling call
0709 13:05:44.638274 (+302036us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:44.677138 15900 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:44.678136 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:44.692457 15930 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:44.643704 18194 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:44.668545 18159 raft_consensus.cc:2804] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:44.660292 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:44.690858 16533 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:44.662766 16570 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0b3676041725488fb6ffad37889b386b"
dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
mode: GRACEFUL
new_leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:42622
I20260709 13:05:44.693748 18161 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:44.709259 18161 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:44.709762 18161 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:44.696038 17957 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:44.698985 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:44.712884 15937 raft_consensus.cc:2410] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:05:44.713732 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:44.674861 18216 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.674700 17981 consensus_queue.cc:237] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:44.700346 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:44.670351 18089 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:44.670550 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:44.654639 18228 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:44.714344 15937 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:44.660471 18156 consensus_queue.cc:1048] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 13:05:44.715783 16533 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:44.685997 18272 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:44.690069 18256 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:44.717013 15935 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5.
W20260709 13:05:44.760388 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 44700.3us
I20260709 13:05:44.691846 17993 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:44.757887 18089 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:44.685844 18175 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:44.757714 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:44.774092 16562 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 4.
I20260709 13:05:44.702580 18159 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:44.690588 17931 consensus_queue.cc:1059] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.073s
W20260709 13:05:44.775465 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 75777.3us
I20260709 13:05:44.670423 18259 consensus_queue.cc:1048] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.089s
I20260709 13:05:44.778327 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:44.756669 18140 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:44.772655 15930 raft_consensus.cc:2468] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:05:44.756031 18086 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:44.716993 18012 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:44.714215 17803 consensus_queue.cc:1059] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.060s
I20260709 13:05:44.691660 18217 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:44.692087 18167 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:44.759083 18216 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:44.785420 16533 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:44.692410 18186 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:05:44.690201 18181 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:44.762250 18197 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.756433 16564 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 4.
I20260709 13:05:44.775353 18251 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.775074 18271 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.777961 17993 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:44.792163 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:44.788861 15853 raft_consensus.cc:2393] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 4: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:44.798653 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:44.798806 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:44.787292 18159 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:44.714633 18145 raft_consensus.cc:3055] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:44.786365 15929 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:05:44.803156 18086 raft_consensus.cc:2804] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:05:44.807986 18280 log.cc:927] Time spent T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 13:05:44.810953 15937 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
I20260709 13:05:44.797860 18183 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:44.826057 18285 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.826706 17803 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:44.828378 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 152174us
I20260709 13:05:44.879443 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:44.880270 15930 raft_consensus.cc:2410] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:05:44.880507 15974 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:44.803328 18084 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:44.797567 18012 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:44.816334 16533 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:44.846853 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:44.859154 18267 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:44.845067 18086 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:44.861001 18217 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:44.862006 18140 raft_consensus.cc:2749] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:44.846226 18145 raft_consensus.cc:740] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:44.864449 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:44.834900 18197 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:44.866806 16003 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60156 (request call id 396) took 2124 ms (client timeout 1751 ms). Trace:
I20260709 13:05:44.856431 18258 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.870328 18228 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:44.870250 18273 log.cc:927] Time spent T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.281s	user 0.001s	sys 0.000s
I20260709 13:05:44.871155 18194 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:44.871977 18255 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:44.873215 18242 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:05:44.877177 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:44.905954 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:44.906298 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:44.906644 18242 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:44.908244 16569 raft_consensus.cc:1240] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:05:44.909226 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:44.909799 16569 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:05:44.910422 15929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1679) took 1951 ms. Trace:
I20260709 13:05:44.910718 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:44.884733 15935 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
I20260709 13:05:44.888373 15974 leader_election.cc:400] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 pre-election: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 2. Current term is 3.
I20260709 13:05:44.889407 16007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1736) took 2454 ms. Trace:
I20260709 13:05:44.914222 15935 raft_consensus.cc:1240] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 3. Current term is 4. Ops: []
W20260709 13:05:44.865768 15862 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 1607) took 1640 ms (client timeout 1985 ms). Trace:
I20260709 13:05:44.839223 18251 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:44.869104 18181 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:44.917752 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:44.918340 18181 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:44.918536 18167 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:44.918783 18181 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:44.919055 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:44.919226 18181 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:44.919541 18167 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:44.919987 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:44.921263 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:44.921442 18251 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:44.914700 15862 rpcz_store.cc:269] 0709 13:05:43.225141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.225356 (+   215us) service_pool.cc:224] Handling call
0709 13:05:44.865495 (+1640139us) spinlock_profiling.cc:243] Waited 61.7 ms on lock 0x7b54000e9cd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:44.865680 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":141933440}
I20260709 13:05:44.904605 15932 raft_consensus.cc:1240] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:44.840965 17993 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:44.913369 16007 rpcz_store.cc:276] 0709 13:05:42.435223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.792760 (+2357537us) service_pool.cc:224] Handling call
0709 13:05:44.801748 (+  8988us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:44.801956 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:05:44.839037 (+ 37081us) log.cc:844] Serialized 14 byte log entry
0709 13:05:44.889078 (+ 50041us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:44.889128 (+    50us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:44.889174 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.889250 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:44.889355 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":27369,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":1399,"raft.queue_time_us":3849,"thread_start_us":40270,"threads_started":1}
W20260709 13:05:44.898900 16003 rpcz_store.cc:269] 0709 13:05:42.742819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.102071 (+359252us) service_pool.cc:224] Handling call
0709 13:05:44.866520 (+1764449us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b54001f86d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:44.866681 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":271685888}
I20260709 13:05:44.895975 18086 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:44.917042 18217 consensus_queue.cc:1048] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:44.917289 18175 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:44.919584 16568 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:44.925920 15974 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:44.916227 16569 raft_consensus.cc:2393] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 4: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:44.846699 18177 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:44.912900 16562 raft_consensus.cc:1275] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:44.884420 17803 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:44.956661 16566 raft_consensus.cc:1275] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:05:44.956826 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:44.895646 18256 raft_consensus.cc:2804] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:05:44.926915 18188 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.044s	user 0.023s	sys 0.015s
I20260709 13:05:44.955796 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:44.957553 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:44.962147 16007 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:44.959805 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:44.957343 16003 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:05:44.962806 15932 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 4.
I20260709 13:05:44.845254 18271 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:44.957296 15974 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:44.912928 18216 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:44.964988 16569 raft_consensus.cc:2393] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 4: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:44.911000 15929 rpcz_store.cc:276] 0709 13:05:42.959074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.890765 (+1931691us) service_pool.cc:224] Handling call
0709 13:05:44.909700 (+ 18935us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:44.909940 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:44.909986 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:44.910043 (+    57us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:44.910118 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.910202 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:44.910349 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:44.968196 15933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 1637) took 2165 ms. Trace:
I20260709 13:05:44.968608 15933 rpcz_store.cc:276] 0709 13:05:42.802734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.386206 (+583472us) service_pool.cc:224] Handling call
0709 13:05:44.968017 (+1581811us) spinlock_profiling.cc:243] Waited 351 ms on lock 0x7b540019c3d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:44.968143 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":806342400}
I20260709 13:05:44.884446 18285 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
W20260709 13:05:44.972484 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:44.961388 18188 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.079s	user 0.023s	sys 0.015s
I20260709 13:05:44.924587 18258 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:05:44.968058 18290 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:44.984316 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:44.993764 18290 raft_consensus.cc:697] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:44.994436 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:44.995123 16562 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:44.985800 18183 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:44.996119 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:44.998265 16569 raft_consensus.cc:1275] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:44.998682 16009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1636) took 2624 ms. Trace:
I20260709 13:05:44.999189 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:44.999024 16009 rpcz_store.cc:276] 0709 13:05:42.374142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.652525 (+278383us) service_pool.cc:224] Handling call
0709 13:05:42.665430 (+ 12905us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:43.045065 (+379635us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.998485 (+1953420us) spinlock_profiling.cc:243] Waited 248 ms on lock 0x7b54001f8bd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:44.998631 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":570108672}
I20260709 13:05:44.999902 16569 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
I20260709 13:05:45.000286 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:45.001294 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:45.001837 16009 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:45.002059 18269 log.cc:927] Time spent T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.417s	user 0.001s	sys 0.003s
I20260709 13:05:44.988029 18012 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:44.988874 17803 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:44.989065 15929 raft_consensus.cc:1275] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:44.989540 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:44.990280 16006 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:05:45.006021 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:45.006409 15929 raft_consensus.cc:1275] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 13:05:45.006794 15929 pending_rounds.cc:85] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 6
I20260709 13:05:44.985751 18285 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:45.007078 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:44.994686 18290 consensus_queue.cc:237] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:45.007663 16566 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
I20260709 13:05:45.008621 15934 raft_consensus.cc:1275] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:05:45.007019 15929 pending_rounds.cc:107] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 13:05:45.009291 15929 raft_consensus.cc:2889] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:45.008982 15934 pending_rounds.cc:85] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 1
I20260709 13:05:45.010162 16008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1742) took 2268 ms. Trace:
I20260709 13:05:45.010516 15934 pending_rounds.cc:107] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:05:45.011030 15934 raft_consensus.cc:2889] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:45.006922 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:45.012710 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 196446us
I20260709 13:05:45.015812 16533 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:45.000667 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:45.017226 16569 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:05:44.988220 16007 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
I20260709 13:05:45.017381 16568 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:45.010592 16008 rpcz_store.cc:276] 0709 13:05:42.741410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.443204 (+701794us) service_pool.cc:224] Handling call
0709 13:05:44.298691 (+855487us) spinlock_profiling.cc:243] Waited 7.81 ms on lock 0x7b5800114398. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef381fc0 00007f73ef36c929 00007f73ef36c3d2 00007f73e88e3bd4
0709 13:05:44.298910 (+   219us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:44.390915 (+ 92005us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.009936 (+619021us) spinlock_profiling.cc:243] Waited 91.4 ms on lock 0x7b54000f0354. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:45.010103 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":228164736}
I20260709 13:05:45.025781 16010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 401) took 1919 ms. Trace:
I20260709 13:05:44.964685 18259 raft_consensus.cc:2749] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:45.032887 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:45.033584 16008 raft_consensus.cc:2376] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4. Re-sending same reply.
I20260709 13:05:45.026194 16010 rpcz_store.cc:276] 0709 13:05:43.106772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.104128 (+997356us) service_pool.cc:224] Handling call
0709 13:05:44.104479 (+   351us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:44.104916 (+   437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:44.104950 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:44.105509 (+   559us) log.cc:844] Serialized 52 byte log entry
0709 13:05:44.325697 (+220188us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:44.523229 (+197532us) log.cc:844] Serialized 14 byte log entry
0709 13:05:44.524990 (+  1761us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.525064 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:45.001334 (+476270us) spinlock_profiling.cc:243] Waited 35.5 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:05:45.012770 (+ 11436us) raft_consensus.cc:1685] finished
0709 13:05:45.025551 (+ 12781us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:45.025718 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1509,"prepare.queue_time_us":45592,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"raft.queue_time_us":3050,"raft.run_cpu_time_us":314,"raft.run_wall_time_us":24709,"spinlock_wait_cycles":82745984,"thread_start_us":402079,"threads_started":2,"wal-append.queue_time_us":417321}
W20260709 13:05:45.033751 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 56229.7us
I20260709 13:05:45.035588 15934 raft_consensus.cc:3055] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:45.037441 15974 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:45.036830 15934 raft_consensus.cc:740] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:44.899318 18267 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:45.044076 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:45.045583 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:45.050113 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:45.050064 15902 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:45.038888 18293 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:45.052994 16008 raft_consensus.cc:2393] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 4: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:45.056840 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:45.057435 15862 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:45.030802 17803 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.060058 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:45.060751 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:45.066253 16010 raft_consensus.cc:2410] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:05:45.056627 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:45.066553 16569 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:45.039016 18216 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:45.059456 16003 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:05:45.067493 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:45.070173 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:45.071003 15858 raft_consensus.cc:2410] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 7 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:05:45.073591 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:45.081357 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:45.070652 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:45.048072 17981 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:45.085005 16533 connection.cc:256] RPC call timeout handler was delayed by -1.7217334079999773s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1517s for a response
W20260709 13:05:45.085460 16533 connection.cc:256] RPC call timeout handler was delayed by -1.6299334080000563s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16190000000000002s for a response
W20260709 13:05:45.085793 16533 connection.cc:256] RPC call timeout handler was delayed by -1.6164334080000344s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1634s for a response
W20260709 13:05:45.086095 16533 connection.cc:256] RPC call timeout handler was delayed by -1.4967334080000683s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1767s for a response
W20260709 13:05:45.086369 16533 connection.cc:256] RPC call timeout handler was delayed by -1.1466334080000706s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2156s for a response
W20260709 13:05:45.086632 16533 connection.cc:256] RPC call timeout handler was delayed by -1.1235737149999068s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1513s for a response
W20260709 13:05:45.086906 16533 connection.cc:256] RPC call timeout handler was delayed by -1.0452737149998939s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16000000000000003s for a response
I20260709 13:05:45.088271 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:45.088886 16569 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:05:45.083124 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:45.090163 16007 raft_consensus.cc:2393] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 3: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:45.093142 15855 raft_consensus.cc:1275] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:05:45.093444 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:45.094172 15860 raft_consensus.cc:2393] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 8: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:45.098366 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:45.099361 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:45.100278 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:45.102114 16565 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45246 (request call id 596) took 2748 ms (client timeout 1642 ms). Trace:
I20260709 13:05:45.055780 18242 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:45.102973 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:45.102558 16565 rpcz_store.cc:269] 0709 13:05:42.337731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.337901 (+   170us) service_pool.cc:224] Handling call
0709 13:05:45.086158 (+2748257us) spinlock_profiling.cc:243] Waited 851 ms on lock 0x7b5400744854. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:45.086300 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1956275456}
I20260709 13:05:45.102885 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:45.103893 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:45.106312 16010 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:44.968938 18145 consensus_queue.cc:260] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:45.042448 18140 consensus_queue.cc:1048] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:05:45.068985 18216 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:45.051374 18012 consensus_queue.cc:1059] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.052s
I20260709 13:05:45.111752 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:45.074071 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:45.061956 18188 consensus_queue.cc:1059] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:05:45.039063 18284 consensus_queue.cc:1059] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.076s
I20260709 13:05:45.171797 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:45.172437 15858 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:05:45.172734 15860 raft_consensus.cc:1275] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:05:45.078460 17803 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:45.102456 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:45.179968 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:45.064414 18246 consensus_queue.cc:1048] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.074s
I20260709 13:05:45.061825 18255 consensus_queue.cc:1048] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:05:45.117784 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:45.180567 15858 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
W20260709 13:05:45.181098 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:45.181736 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:45.182789 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:45.187160 16007 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:45.188066 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:45.055964 18293 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:45.142149 18216 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:45.186272 15860 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 4.
I20260709 13:05:45.104308 18259 consensus_queue.cc:1048] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.076s
I20260709 13:05:45.224839 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:45.221613 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:45.235769 15862 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:45.264923 15855 raft_consensus.cc:1240] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:45.105837 17981 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:45.179438 15854 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:45.267156 15861 raft_consensus.cc:2368] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 2. Current term is 3.
I20260709 13:05:45.269984 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:45.271068 16008 raft_consensus.cc:2410] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:05:45.219192 18293 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:45.216327 15853 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 2.
I20260709 13:05:45.351593 16010 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
W20260709 13:05:45.374652 15825 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 1.928s (SENT) [suppressed 38 similar messages]
I20260709 13:05:45.380054 16009 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:05:45.289503 18188 raft_consensus.cc:3055] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:05:45.336364 16535 leader_election.cc:336] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 2.047s (SENT)
W20260709 13:05:45.394405 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 58082.6us
I20260709 13:05:45.394659 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:45.395433 16011 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:45.417800 16011 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
I20260709 13:05:45.419929 18169 raft_consensus.cc:3055] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:45.420341 18169 raft_consensus.cc:740] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:05:45.421171 18299 log.cc:927] Time spent T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 13:05:45.421283 18169 consensus_queue.cc:260] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:45.424029 18302 log.cc:927] Time spent T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
I20260709 13:05:45.432714 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:05:45.453482 16535 leader_election.cc:336] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 2.164s (SENT)
I20260709 13:05:45.463368 16009 raft_consensus.cc:1275] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:05:45.464028 16009 pending_rounds.cc:85] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting all ops after (but not including) 5
I20260709 13:05:45.464363 16009 pending_rounds.cc:107] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:05:45.464710 16009 raft_consensus.cc:2889] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:45.465211 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:45.472888 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:45.610757 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:45.611344 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 390903us
W20260709 13:05:45.626302 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:45.626669 15902 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.000s	sys 0.015s
W20260709 13:05:45.627023 15902 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.000s	sys 0.015s
W20260709 13:05:45.627238 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:45.627297 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 572317us
I20260709 13:05:45.628082 15902 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:45.401242 18188 raft_consensus.cc:740] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:05:45.666159 15974 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.012s	sys 0.006s
I20260709 13:05:45.668172 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:45.692603 15974 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.013s	sys 0.007s
W20260709 13:05:45.692970 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 654331us
I20260709 13:05:45.661418 15902 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:05:45.695981 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 66297.5us
I20260709 13:05:45.696673 15902 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:05:45.697310 16006 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.014s	sys 0.007s
I20260709 13:05:45.703020 15932 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:45.705502 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:45.394055 17993 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:45.628134 18156 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:45.752434 16006 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.014s	sys 0.007s
I20260709 13:05:45.754505 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:45.780716 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:05:45.776153 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 104960us
I20260709 13:05:45.661984 18190 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:45.779649 15974 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:05:45.723151 18301 log.cc:927] Time spent T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.330s	user 0.001s	sys 0.000s
I20260709 13:05:45.794000 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:45.785789 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:45.781870 15933 raft_consensus.cc:2393] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 8: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:05:45.782393 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 328944us
W20260709 13:05:45.803905 16535 leader_election.cc:336] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 2.165s (SENT)
W20260709 13:05:45.804554 16535 leader_election.cc:336] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 2.244s (SENT)
I20260709 13:05:45.806667 16535 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:45.809824 16008 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
I20260709 13:05:45.810161 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:45.809140 15974 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:45.722647 17961 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:45.816375 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:45.817193 15933 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 4.
I20260709 13:05:45.779893 18169 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:45.786981 18190 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:45.783531 18145 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:45.790863 18208 raft_consensus.cc:2804] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:45.818112 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:45.777599 18177 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:05:45.824687 18281 log.cc:927] Time spent T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 1.060s	user 0.002s	sys 0.000s
I20260709 13:05:45.787981 18156 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:45.828291 18275 log.cc:927] Time spent T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 1.265s	user 0.000s	sys 0.010s
I20260709 13:05:45.853160 16004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 403) took 2745 ms. Trace:
I20260709 13:05:45.854362 18156 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:45.853542 16004 rpcz_store.cc:276] 0709 13:05:43.107295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.299494 (+1192199us) service_pool.cc:224] Handling call
0709 13:05:44.328164 (+ 28670us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:44.364290 (+ 36126us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:44.364329 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:44.365008 (+   679us) log.cc:844] Serialized 52 byte log entry
0709 13:05:44.561219 (+196211us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:44.561289 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.561372 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:45.684658 (+1123286us) spinlock_profiling.cc:243] Waited 55.8 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:05:45.852857 (+168199us) raft_consensus.cc:1685] finished
0709 13:05:45.852884 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:45.853099 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3686,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":7731,"spinlock_wait_cycles":128263936,"thread_start_us":155699,"threads_started":1,"wal-append.queue_time_us":149642}
I20260709 13:05:45.830127 18291 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:45.855913 18258 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:45.856447 18258 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:45.856942 18258 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:45.858856 18258 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:45.816397 18286 consensus_queue.cc:1048] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:05:45.815402 17961 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:45.861446 18161 consensus_meta.cc:287] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 1.151s	user 0.011s	sys 0.001s
I20260709 13:05:45.861863 18161 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:45.864059 18258 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:45.864253 18308 raft_consensus.cc:2804] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:45.864710 18308 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:45.865091 18308 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:05:45.865235 15854 consensus_meta.cc:287] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.576s	user 0.007s	sys 0.002s
I20260709 13:05:45.865628 15854 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:45.864593 18258 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:45.869244 17961 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:45.869935 18258 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:45.819988 18234 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:45.871170 18234 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:45.871843 18089 consensus_meta.cc:287] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 1.088s	user 0.009s	sys 0.002s
I20260709 13:05:45.832947 17993 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:45.872773 17993 consensus_queue.cc:1048] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:45.873248 18234 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:45.842085 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:45.874218 16005 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:45.843585 18190 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:45.884779 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:45.885636 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:45.886201 16569 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:45.885406 16008 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
W20260709 13:05:45.841833 18159 consensus_meta.cc:287] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.994s	user 0.008s	sys 0.000s
I20260709 13:05:45.890352 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:45.890445 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:45.890895 16008 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5.
I20260709 13:05:45.891052 16007 raft_consensus.cc:2368] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 3. Current term is 4.
I20260709 13:05:45.851133 15933 raft_consensus.cc:2368] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 2. Current term is 3.
W20260709 13:05:45.839457 18181 consensus_meta.cc:287] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.913s	user 0.005s	sys 0.000s
I20260709 13:05:45.891722 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:45.892273 16011 raft_consensus.cc:2376] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3. Re-sending same reply.
W20260709 13:05:45.833870 18272 consensus_meta.cc:287] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 1.059s	user 0.001s	sys 0.007s
I20260709 13:05:45.892421 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:45.812301 18290 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:05:45.855621 18216 consensus_meta.cc:287] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.635s	user 0.007s	sys 0.002s
I20260709 13:05:45.679014 18188 consensus_queue.cc:260] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:45.863956 17981 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:45.898839 15974 leader_election.cc:400] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 election: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 2. Current term is 3.
I20260709 13:05:45.900535 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:45.827597 18267 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:45.898170 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:45.901893 15974 leader_election.cc:403] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:05:45.895222 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:45.904628 16567 raft_consensus.cc:1240] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 3. Current term is 4. Ops: []
W20260709 13:05:45.900928 15827 connection.cc:256] RPC call timeout handler was delayed by -1.588281729000073s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21730000000000002s for a response
W20260709 13:05:45.905397 15827 connection.cc:256] RPC call timeout handler was delayed by -1.5387817290001067s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22280000000000003s for a response
I20260709 13:05:45.905472 16565 raft_consensus.cc:2393] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 4: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:45.850205 18208 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:05:45.905941 15827 connection.cc:256] RPC call timeout handler was delayed by -1.5225817290000805s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22460000000000002s for a response
I20260709 13:05:45.910408 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:45.912273 15860 raft_consensus.cc:2393] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 4: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:05:45.913059 15827 connection.cc:256] RPC call timeout handler was delayed by -1.4586817290000909s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23170000000000002s for a response
W20260709 13:05:45.921028 15827 connection.cc:256] RPC call timeout handler was delayed by -1.14548054699992s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1678s for a response
W20260709 13:05:45.926662 15827 connection.cc:256] RPC call timeout handler was delayed by -1.1364805470000192s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1688s for a response
W20260709 13:05:45.926997 15827 connection.cc:256] RPC call timeout handler was delayed by -1.0652416710000807s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1533s for a response
I20260709 13:05:45.895799 18272 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:45.932379 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:45.932893 18272 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:45.933907 15827 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:45.934715 17981 raft_consensus.cc:2804] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:45.935105 17981 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:45.935462 17981 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:45.927970 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:45.936947 16565 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:05:45.929503 18197 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:45.932981 15928 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:45.910811 18239 log.cc:927] Time spent T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 2.157s	user 0.007s	sys 0.000s
I20260709 13:05:45.900203 18213 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:45.948740 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:45.940026 16569 raft_consensus.cc:2468] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:05:45.940245 16011 raft_consensus.cc:2393] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 6: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:45.895697 18290 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:45.942879 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:45.952987 15858 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 4.
I20260709 13:05:45.944046 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:45.944711 15827 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:05:45.946504 18086 consensus_meta.cc:287] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.982s	user 0.011s	sys 0.000s
I20260709 13:05:45.919587 15861 raft_consensus.cc:1275] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:05:45.912477 18259 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 2. Current term is 3.
I20260709 13:05:45.887928 18089 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:45.911870 18161 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:45.914402 18208 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:45.955529 18084 raft_consensus.cc:2804] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:45.959514 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:45.936937 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:45.914089 17993 consensus_queue.cc:1048] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:05:45.904125 18216 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:45.919919 17961 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:45.896279 18181 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:45.895251 18159 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:45.919027 18177 consensus_queue.cc:1059] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:05:45.963070 16561 raft_consensus.cc:1275] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEARNER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:45.965960 18084 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:45.971324 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:45.971735 18089 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:45.956310 18290 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:46.012318 16566 raft_consensus.cc:1240] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:45.964524 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:45.979310 16010 raft_consensus.cc:1240] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 3. Current term is 5. Ops: []
I20260709 13:05:45.961063 16004 raft_consensus.cc:1275] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:45.966159 16006 raft_consensus.cc:1240] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:45.949820 18234 consensus_queue.cc:1059] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:05:45.959110 18086 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:46.006821 15932 raft_consensus.cc:2368] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 5. Current term is 6.
I20260709 13:05:46.019191 15931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 1632) took 3281 ms. Trace:
I20260709 13:05:46.014983 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:46.000106 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:46.019606 15931 rpcz_store.cc:276] 0709 13:05:42.737664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.737769 (+   105us) service_pool.cc:224] Handling call
0709 13:05:42.738079 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:42.763352 (+ 25273us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:42.763384 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:42.773042 (+  9658us) log.cc:844] Serialized 26 byte log entry
0709 13:05:43.250599 (+477557us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:43.250643 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:43.250718 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:45.879405 (+2628687us) spinlock_profiling.cc:243] Waited 7.88 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:05:46.012019 (+132614us) raft_consensus.cc:1685] finished
0709 13:05:46.012038 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.019134 (+  7096us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5553,"raft.queue_time_us":14688,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":3677,"spinlock_wait_cycles":18113408,"thread_start_us":367524,"threads_started":1,"wal-append.queue_time_us":586931}
I20260709 13:05:46.000413 18272 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.022771 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:46.005834 18084 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:45.957890 18213 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:46.026225 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:46.000028 18259 consensus_queue.cc:1048] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 13:05:46.042501 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:46.022859 16002 raft_consensus.cc:1240] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 3. Current term is 4. Ops: []
W20260709 13:05:46.036451 18293 consensus_meta.cc:287] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.713s	user 0.006s	sys 0.003s
W20260709 13:05:46.047600 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 240998us
W20260709 13:05:46.047901 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 245364us
W20260709 13:05:46.063460 15976 connection.cc:256] RPC call timeout handler was delayed by -1.0111285689998795s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1663s for a response
I20260709 13:05:46.064541 15976 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:46.030855 18086 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:46.071630 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:46.031070 16562 raft_consensus.cc:1240] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:46.030846 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:46.044277 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:46.075529 15931 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
W20260709 13:05:46.086901 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 236402us
I20260709 13:05:46.083732 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:05:46.087416 15936 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53636 (request call id 441) took 2519 ms (client timeout 2092 ms). Trace:
I20260709 13:05:46.088037 15861 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:46.030881 18213 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:46.087366 15931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53636 (request call id 455) took 1034 ms. Trace:
I20260709 13:05:46.089270 15931 rpcz_store.cc:276] 0709 13:05:45.053031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.047198 (+994167us) service_pool.cc:224] Handling call
0709 13:05:46.087274 (+ 40076us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:46.088084 15936 rpcz_store.cc:269] 0709 13:05:43.567935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.225116 (+657181us) service_pool.cc:224] Handling call
0709 13:05:46.082732 (+1857616us) spinlock_profiling.cc:243] Waited 60.4 ms on lock 0x7b54003b9054. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:46.087352 (+  4620us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":138940672}
I20260709 13:05:46.025915 18190 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:46.062448 18089 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:46.099448 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 145038us
I20260709 13:05:46.027159 18272 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:46.094916 15976 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:46.101141 16533 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:46.101506 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:46.073565 18310 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:46.102226 15827 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:46.104200 15931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53636 (request call id 451) took 1052 ms. Trace:
I20260709 13:05:46.105656 15933 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:46.106161 15933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53636 (request call id 449) took 1058 ms. Trace:
I20260709 13:05:46.106005 15931 rpcz_store.cc:276] 0709 13:05:45.051915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.090402 (+1038487us) service_pool.cc:224] Handling call
0709 13:05:46.104144 (+ 13742us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:46.106302 15976 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:46.106570 15933 rpcz_store.cc:276] 0709 13:05:45.047431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.892144 (+844713us) service_pool.cc:224] Handling call
0709 13:05:46.106110 (+213966us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:46.107618 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:46.108335 15933 raft_consensus.cc:2376] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3. Re-sending same reply.
I20260709 13:05:46.108960 15933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53636 (request call id 456) took 1046 ms. Trace:
I20260709 13:05:46.110170 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:46.109421 15933 rpcz_store.cc:276] 0709 13:05:45.062854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.107290 (+1044436us) service_pool.cc:224] Handling call
0709 13:05:46.108900 (+  1610us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:46.111757 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:46.114290 16533 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:46.116016 15933 raft_consensus.cc:2393] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 6: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:46.116791 15936 raft_consensus.cc:2393] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 4: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
W20260709 13:05:46.099311 18160 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.575s	user 0.005s	sys 0.013s
I20260709 13:05:46.091790 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:46.129129 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:46.131757 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:46.073848 18293 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:46.116297 18216 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:46.137297 18293 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:46.137976 15934 consensus_queue.cc:260] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:46.139132 15934 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:46.142915 15757 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58966 (request call id 242) took 12769 ms (client timeout 15000 ms). Trace:
I20260709 13:05:46.134111 16561 raft_consensus.cc:2393] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 6: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:46.156517 15934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 1647) took 1503 ms. Trace:
I20260709 13:05:46.157829 15934 rpcz_store.cc:276] 0709 13:05:44.652891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.011950 (+359059us) service_pool.cc:224] Handling call
0709 13:05:45.035486 (+ 23536us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.137623 (+1102137us) spinlock_profiling.cc:243] Waited 1.38 ms on lock 0x7b4c004d1c00. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e8674a20 00007f73e8685a29 00007f73e864b644 00007f73e8672f2d 00007f73e8672b93 00007f73ea3a318b 00007f73ea3a3cb5 00007f73ea39ef30 00007f73ea3a07de 00007f73ea3aab49 00007f73ea3ab416
0709 13:05:46.155489 (+ 17866us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 13:05:46.155518 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:46.155557 (+    39us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:05:46.155585 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.155646 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.156465 (+   819us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2450,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":79,"spinlock_wait_cycles":3175296}
I20260709 13:05:46.160979 18272 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:46.161537 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:46.162309 16007 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
I20260709 13:05:46.165364 15934 raft_consensus.cc:1240] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:05:46.167438 18256 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:46.171737 15931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53636 (request call id 459) took 1106 ms. Trace:
I20260709 13:05:46.178673 15931 rpcz_store.cc:276] 0709 13:05:45.064958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.106958 (+1042000us) service_pool.cc:224] Handling call
0709 13:05:46.171637 (+ 64679us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:46.172966 15825 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:46.119782 15936 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53636 (request call id 452) took 1067 ms. Trace:
I20260709 13:05:46.086248 18177 consensus_queue.cc:1059] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
W20260709 13:05:46.177549 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 75408.4us
I20260709 13:05:46.180936 15827 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:46.181520 15936 rpcz_store.cc:276] 0709 13:05:45.052157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.091386 (+1039229us) service_pool.cc:224] Handling call
0709 13:05:46.119716 (+ 28330us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:46.108685 18086 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.095471 18291 raft_consensus.cc:697] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:46.115031 18307 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:46.117460 18140 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:46.189522 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:46.111009 18213 consensus_queue.cc:1048] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:05:46.193778 15934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 1649) took 1532 ms. Trace:
W20260709 13:05:46.175711 18160 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.615s	user 0.006s	sys 0.014s
I20260709 13:05:46.145632 18186 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:05:46.228732 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 55868.6us
I20260709 13:05:46.227311 18167 raft_consensus.cc:2749] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:05:46.226687 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 112455us
I20260709 13:05:46.218290 15861 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
I20260709 13:05:46.109130 18089 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:46.186932 18234 raft_consensus.cc:2804] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:05:46.202988 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:46.186303 18084 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:46.190402 18322 consensus_queue.cc:1048] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.116s
I20260709 13:05:46.233596 15931 raft_consensus.cc:1240] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:05:46.237744 15860 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:46.185081 18256 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:46.240262 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 59406.4us
I20260709 13:05:46.263118 15934 rpcz_store.cc:276] 0709 13:05:44.656371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.162419 (+1506048us) service_pool.cc:224] Handling call
0709 13:05:46.165054 (+  2635us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:46.188598 (+ 23544us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.188796 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:46.143328 15757 rpcz_store.cc:269] 0709 13:05:33.373805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.374065 (+   260us) service_pool.cc:224] Handling call
0709 13:05:46.142835 (+12768770us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:41.330471 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:41.568323 (+237852us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:41.597000 (+ 28677us) write_op.cc:435] Acquired schema lock
0709 13:05:41.613918 (+ 16918us) tablet.cc:662] Decoding operations
0709 13:05:41.749999 (+136081us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:41.750081 (+    82us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:41.750110 (+    29us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:05:41.750843 (+   733us) tablet.cc:701] Row locks acquired
0709 13:05:41.750885 (+    42us) write_op.cc:260] PREPARE: finished
0709 13:05:41.751058 (+   173us) write_op.cc:270] Start()
0709 13:05:41.751197 (+   139us) write_op.cc:276] Timestamp: P: 1783602341750996 usec, L: 0
0709 13:05:41.751216 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:05:41.757430 (+  6214us) log.cc:844] Serialized 2814 byte log entry
0709 13:05:42.565037 (+807607us) op_driver.cc:464] REPLICATION: finished
0709 13:05:42.704063 (+139026us) write_op.cc:301] APPLY: starting
0709 13:05:42.704166 (+   103us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:42.758478 (+ 54312us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:42.776855 (+ 18377us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:43.082427 (+305572us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:43.082864 (+   437us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 13:05:43.082927 (+    63us) write_op.cc:312] APPLY: finished
0709 13:05:43.083369 (+   442us) log.cc:844] Serialized 59 byte log entry
0709 13:05:43.083614 (+   245us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:43.097217 (+ 13603us) write_op.cc:454] Released schema lock
0709 13:05:43.109928 (+ 12711us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:43.183756 (+ 73828us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:43.210229 (+ 26473us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":834383,"reported_tablets":19,"child_traces":[["op",{"apply.queue_time_us":138756,"apply.run_cpu_time_us":7165,"apply.run_wall_time_us":456007,"num_ops":7,"prepare.queue_time_us":690567,"prepare.run_cpu_time_us":66986,"prepare.run_wall_time_us":760446,"raft.queue_time_us":113899,"raft.run_cpu_time_us":47892,"raft.run_wall_time_us":182333,"replication_time_us":813667,"thread_start_us":976151,"threads_started":4,"wal-append.queue_time_us":299016,"wal-append.run_cpu_time_us":21225,"wal-append.run_wall_time_us":2111344}]]}
I20260709 13:05:46.134653 18318 consensus_queue.cc:1048] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.109s
I20260709 13:05:46.317677 15930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1680) took 2872 ms. Trace:
I20260709 13:05:46.285000 16533 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:46.265196 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:46.234782 17881 raft_consensus.cc:2764] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:05:46.389487 15930 rpcz_store.cc:276] 0709 13:05:43.445378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.891166 (+1445788us) service_pool.cc:224] Handling call
0709 13:05:44.915468 (+ 24302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.316895 (+1401427us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:46.316925 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:46.316962 (+    37us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:46.316995 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.317484 (+   489us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b540034b554. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:46.317519 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.317625 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3222463744}
W20260709 13:05:46.390573 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 280015us
I20260709 13:05:46.391389 15976 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:46.291085 15856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 1747) took 1120 ms. Trace:
I20260709 13:05:46.404826 15936 raft_consensus.cc:2393] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 3: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:46.406160 15931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 1651) took 1749 ms. Trace:
I20260709 13:05:46.408349 15931 rpcz_store.cc:276] 0709 13:05:44.656916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.179289 (+1522373us) service_pool.cc:224] Handling call
0709 13:05:46.233295 (+ 54006us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:46.405889 (+172594us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.406105 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:46.406677 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 164665us
W20260709 13:05:46.411341 15900 connection.cc:256] RPC call timeout handler was delayed by -1.09019713400005s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22130000000000002s for a response
I20260709 13:05:46.393957 15856 rpcz_store.cc:276] 0709 13:05:45.170963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.171086 (+   123us) service_pool.cc:224] Handling call
0709 13:05:45.187241 (+ 16155us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:45.187844 (+   603us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:45.187886 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:45.632644 (+444758us) log.cc:844] Serialized 52 byte log entry
0709 13:05:46.093060 (+460416us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:46.263992 (+170932us) log.cc:844] Serialized 14 byte log entry
0709 13:05:46.290649 (+ 26657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.290759 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:46.290814 (+    55us) raft_consensus.cc:1685] finished
0709 13:05:46.290832 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.291015 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":42656,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":56292,"raft.run_cpu_time_us":304,"raft.run_wall_time_us":20674,"thread_start_us":574547,"threads_started":2,"wal-append.queue_time_us":446018}
W20260709 13:05:46.237473 18314 log.cc:927] Time spent T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:05:46.235433 17961 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:46.238197 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:46.237310 18140 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:46.263072 18181 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:46.234027 18295 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:05:46.426926 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 142033us
I20260709 13:05:46.426898 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:05:46.281484 18297 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.004s	sys 0.007s
I20260709 13:05:46.261984 18086 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:46.312631 15929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 1646) took 1660 ms. Trace:
I20260709 13:05:46.305495 18159 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:46.266834 18327 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:46.403950 18234 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:46.308825 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:46.444828 15860 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 4.
W20260709 13:05:46.445807 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 52034.1us
I20260709 13:05:46.380908 18307 raft_consensus.cc:697] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:46.285758 18309 raft_consensus.cc:3055] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:46.442798 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:46.416383 18177 raft_consensus.cc:2804] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:05:46.438109 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 141199us
I20260709 13:05:46.453270 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:46.453765 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:46.477520 15930 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:05:46.442345 18140 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:46.420976 18167 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:46.461973 16533 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:46.351819 17981 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:46.492631 18181 raft_consensus.cc:2749] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:46.492899 18312 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:05:46.446830 18297 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.769s	user 0.006s	sys 0.011s
I20260709 13:05:46.468039 15976 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:46.300376 18249 consensus_queue.cc:1059] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.155s
I20260709 13:05:46.301819 18258 consensus_queue.cc:1059] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.140s
I20260709 13:05:46.503389 18258 raft_consensus.cc:3055] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:46.451314 18170 consensus_queue.cc:1059] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:05:46.503762 18258 raft_consensus.cc:740] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:46.504534 18258 consensus_queue.cc:260] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:46.504963 18084 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:46.467188 18166 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 13:05:46.469282 18234 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:46.505702 18258 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:46.508560 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 56957.4us
I20260709 13:05:46.508410 18140 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:46.510233 16533 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:46.510556 18140 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:46.425879 17964 consensus_queue.cc:1048] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
W20260709 13:05:46.512290 18336 log.cc:927] Time spent T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:05:46.512912 18186 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:05:46.513109 18324 log.cc:927] Time spent T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
W20260709 13:05:46.513797 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:46.514376 16568 raft_consensus.cc:1240] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:46.514958 18089 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:05:46.515154 18329 log.cc:927] Time spent T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:05:46.514416 16009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1747) took 1466 ms. Trace:
I20260709 13:05:46.515928 16569 raft_consensus.cc:1275] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:05:46.516031 16561 raft_consensus.cc:1240] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:46.516398 16564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 604) took 1609 ms. Trace:
I20260709 13:05:46.516754 16564 rpcz_store.cc:276] 0709 13:05:44.906983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.911242 (+  4259us) service_pool.cc:224] Handling call
0709 13:05:44.911587 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:45.081964 (+170377us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:45.188842 (+106878us) log.cc:844] Serialized 14 byte log entry
0709 13:05:46.044915 (+856073us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:46.109860 (+ 64945us) log.cc:844] Serialized 26 byte log entry
0709 13:05:46.176761 (+ 66901us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:46.183830 (+  7069us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.194842 (+ 11012us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:46.494807 (+299965us) raft_consensus.cc:1685] finished
0709 13:05:46.496629 (+  1822us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.516334 (+ 19705us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":79732,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":71738,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":168,"thread_start_us":867298,"threads_started":3,"wal-append.queue_time_us":678793}
W20260709 13:05:46.518505 18328 log.cc:927] Time spent T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
I20260709 13:05:46.519165 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:46.519809 16569 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 4.
I20260709 13:05:46.520865 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:46.515849 16009 rpcz_store.cc:276] 0709 13:05:45.048687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.602073 (+553386us) service_pool.cc:224] Handling call
0709 13:05:45.666955 (+ 64882us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:45.791549 (+124594us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:45.791621 (+    72us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:45.792635 (+  1014us) log.cc:844] Serialized 52 byte log entry
0709 13:05:46.104055 (+311420us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:46.281907 (+177852us) log.cc:844] Serialized 14 byte log entry
0709 13:05:46.449690 (+167783us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.449770 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:46.514229 (+ 64459us) raft_consensus.cc:1685] finished
0709 13:05:46.514244 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.514366 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":213108,"prepare.queue_time_us":199304,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":9176,"raft.queue_time_us":194674,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":29940,"thread_start_us":373042,"threads_started":2,"wal-append.queue_time_us":416396}
W20260709 13:05:46.496184 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 107251us
I20260709 13:05:46.498835 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:46.522423 15931 raft_consensus.cc:2393] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 6: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:46.501968 17981 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:46.502528 18249 raft_consensus.cc:3055] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:46.523168 15827 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:46.524099 18307 consensus_queue.cc:237] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:46.525888 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:46.525889 15859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 1749) took 1354 ms. Trace:
I20260709 13:05:46.535490 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:05:46.500792 15928 consensus_meta.cc:287] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.537s	user 0.009s	sys 0.000s
I20260709 13:05:46.479298 18177 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:46.472914 18190 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:46.475572 18309 raft_consensus.cc:740] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:46.539345 16569 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:46.535360 18290 consensus_meta.cc:287] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.515s	user 0.005s	sys 0.004s
I20260709 13:05:46.539125 15859 rpcz_store.cc:276] 0709 13:05:45.171767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.171877 (+   110us) service_pool.cc:224] Handling call
0709 13:05:45.172162 (+   285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:45.318824 (+146662us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:45.318866 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:45.813941 (+495075us) log.cc:844] Serialized 26 byte log entry
0709 13:05:46.086602 (+272661us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:46.086641 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.525374 (+438733us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b54002481d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:46.525437 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:46.525498 (+    61us) raft_consensus.cc:1685] finished
0709 13:05:46.525518 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.525818 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9023,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":30185,"spinlock_wait_cycles":282647296,"thread_start_us":240936,"threads_started":1,"wal-append.queue_time_us":452330}
I20260709 13:05:46.541697 16533 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:46.543164 15827 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:46.436919 18291 consensus_queue.cc:237] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:46.540263 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:46.549291 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:46.549528 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:46.550838 15931 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
I20260709 13:05:46.551942 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:46.552868 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:46.553458 15931 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:05:46.554418 15827 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:46.519098 18327 consensus_queue.cc:1048] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:05:46.537424 17881 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:46.535483 18249 raft_consensus.cc:740] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:46.537066 18297 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:46.559790 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:46.561020 15931 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5.
I20260709 13:05:46.555509 17981 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:46.562760 18086 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:46.563898 16009 raft_consensus.cc:1275] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:05:46.565490 18309 consensus_queue.cc:260] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:46.566808 18309 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:46.493507 18312 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:46.568397 18249 consensus_queue.cc:260] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:46.569475 18312 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:46.570253 16563 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.767s	user 0.012s	sys 0.032s
W20260709 13:05:46.570688 16563 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.771s	user 0.012s	sys 0.032s
I20260709 13:05:46.571378 16563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 554) took 1930 ms. Trace:
I20260709 13:05:46.478425 18295 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:46.571725 16563 rpcz_store.cc:276] 0709 13:05:44.641318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.642757 (+  1439us) service_pool.cc:224] Handling call
0709 13:05:44.675318 (+ 32561us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:44.683916 (+  8598us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:05:44.684280 (+   364us) log.cc:844] Serialized 14 byte log entry
0709 13:05:46.571006 (+1886726us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:46.571067 (+    61us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:46.571110 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.571193 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.571324 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":152,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":126006,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":82,"thread_start_us":1876364,"threads_started":2,"wal-append.queue_time_us":175414,"wal-append.run_cpu_time_us":1056,"wal-append.run_wall_time_us":1001088}
I20260709 13:05:46.573194 18271 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:46.574664 15935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1685) took 2398 ms. Trace:
I20260709 13:05:46.575023 15935 rpcz_store.cc:276] 0709 13:05:44.176389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.915081 (+738692us) service_pool.cc:224] Handling call
0709 13:05:44.922256 (+  7175us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:44.922445 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:05:45.020365 (+ 97920us) log.cc:844] Serialized 14 byte log entry
0709 13:05:45.973740 (+953375us) spinlock_profiling.cc:243] Waited 2.9 ms on lock 0x7b18002ead40. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb3f3 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd 000000000038f354 000000000038f199 00007f73e729e83a
0709 13:05:46.574042 (+600302us) spinlock_profiling.cc:243] Waited 2.49 ms on lock 0x7b18002ead40. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e9f504bc 00007f73e729e48b 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f4823a
0709 13:05:46.574335 (+   293us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:46.574387 (+    52us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:46.574433 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.574503 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.574611 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16456,"prepare.queue_time_us":161823,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"spinlock_wait_cycles":12391936,"thread_start_us":178600,"threads_started":2,"wal-append.queue_time_us":152530}
I20260709 13:05:46.576133 16533 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:46.577461 16533 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:46.578428 16533 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:46.579352 16533 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:46.580200 18181 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:46.581365 18181 raft_consensus.cc:697] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:46.582226 18181 consensus_queue.cc:237] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:46.582778 18293 raft_consensus.cc:2749] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:46.583873 18323 raft_consensus.cc:2749] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:46.583544 18293 raft_consensus.cc:2749] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:46.585525 16570 raft_consensus.cc:606] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Received request to transfer leadership to TS 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:46.585891 16570 raft_consensus.cc:614] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:05:46.587188 18208 consensus_meta.cc:287] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.615s	user 0.009s	sys 0.004s
I20260709 13:05:46.588064 16570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:42622 (request call id 0) took 1934 ms. Trace:
I20260709 13:05:46.588407 16570 rpcz_store.cc:276] 0709 13:05:44.658916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.662603 (+  3687us) service_pool.cc:224] Handling call
0709 13:05:46.587874 (+1925271us) spinlock_profiling.cc:243] Waited 1.88 s on lock 0x7b5400724554. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a2853 00007f73ef37060a 00007f73e88f67d4 00007f73e88f6762 00007f73e88f6691 00007f73e88f660c 00007f73e88f58c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:46.587983 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4314947840}
W20260709 13:05:46.590724 15932 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53636 (request call id 448) took 1543 ms (client timeout 1822 ms). Trace:
W20260709 13:05:46.591077 15932 rpcz_store.cc:269] 0709 13:05:45.047174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.709544 (+662370us) service_pool.cc:224] Handling call
0709 13:05:46.590541 (+880997us) spinlock_profiling.cc:243] Waited 196 ms on lock 0x7b540008a7d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:46.590658 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":450077952}
I20260709 13:05:46.591549 16570 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589"
dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
mode: GRACEFUL
new_leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:42622
I20260709 13:05:46.592133 18293 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:46.592362 16570 raft_consensus.cc:606] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Received request to transfer leadership to TS 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:46.587643 18208 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:46.592892 16570 raft_consensus.cc:614] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:46.544978 18177 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:46.594753 17881 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:46.586040 16533 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:46.542518 15928 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
W20260709 13:05:46.556052 18308 consensus_meta.cc:287] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.683s	user 0.009s	sys 0.002s
I20260709 13:05:46.596158 16533 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:46.592610 16564 raft_consensus.cc:1240] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:46.597505 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:46.600901 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:46.601581 15931 raft_consensus.cc:2376] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8. Re-sending same reply.
I20260709 13:05:46.592576 18293 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:05:46.603583 15928 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53636 (request call id 454) took 1550 ms (client timeout 2065 ms). Trace:
I20260709 13:05:46.608047 18342 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:46.608050 17981 raft_consensus.cc:697] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
W20260709 13:05:46.608811 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 54488.9us
W20260709 13:05:46.583506 18169 consensus_meta.cc:287] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.739s	user 0.009s	sys 0.000s
I20260709 13:05:46.608888 18327 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.609550 18318 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:46.627521 15931 raft_consensus.cc:1240] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:46.631019 16533 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:46.639279 15827 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:46.624185 18293 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:46.643224 15932 raft_consensus.cc:1275] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:05:46.647919 15932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 453) took 1595 ms. Trace:
I20260709 13:05:46.648258 15932 rpcz_store.cc:276] 0709 13:05:45.052414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.629957 (+1577543us) service_pool.cc:224] Handling call
0709 13:05:46.642926 (+ 12969us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.647576 (+  4650us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.647865 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:46.555930 18290 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:46.645551 15827 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:46.626614 16569 raft_consensus.cc:2410] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:05:46.643167 16567 raft_consensus.cc:1275] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 13:05:46.652194 16567 pending_rounds.cc:85] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting all ops after (but not including) 6
I20260709 13:05:46.652446 16567 pending_rounds.cc:107] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 13:05:46.652667 16567 raft_consensus.cc:2889] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:46.653194 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:46.639034 18323 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:46.653726 16533 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:46.644304 15931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 1652) took 1718 ms. Trace:
I20260709 13:05:46.655158 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:46.642967 17931 raft_consensus.cc:3055] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:46.660899 17931 raft_consensus.cc:740] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:46.662101 17931 consensus_queue.cc:260] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:46.663113 17931 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:05:46.624775 15928 rpcz_store.cc:269] 0709 13:05:45.052723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.932124 (+879401us) service_pool.cc:224] Handling call
0709 13:05:46.603510 (+671386us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:46.664525 15928 raft_consensus.cc:1240] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:46.609365 18308 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:05:46.657172 17881 raft_consensus.cc:2804] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:46.666283 17881 raft_consensus.cc:697] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 6 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:46.666486 18308 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:46.656482 18322 raft_consensus.cc:2749] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:46.646883 18327 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:46.659546 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:46.673887 15862 raft_consensus.cc:2410] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:46.656665 18169 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:46.675096 18169 raft_consensus.cc:697] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:46.647986 18342 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:46.676041 16008 raft_consensus.cc:2393] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 7: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:46.676254 18342 raft_consensus.cc:697] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 6 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:46.676945 18290 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:46.677459 18342 consensus_queue.cc:237] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:46.455825 15929 rpcz_store.cc:276] 0709 13:05:44.652645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.010308 (+357663us) service_pool.cc:224] Handling call
0709 13:05:45.068318 (+ 58010us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:45.105447 (+ 37129us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:05:45.216537 (+111090us) log.cc:844] Serialized 14 byte log entry
0709 13:05:45.812712 (+596175us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:45.819171 (+  6459us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:45.820667 (+  1496us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.312301 (+491634us) spinlock_profiling.cc:243] Waited 9.07 ms on lock 0x7b540034bf54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:46.312381 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.312551 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":66579,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":97732,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":11887,"spinlock_wait_cycles":20852480,"thread_start_us":494308,"threads_started":1,"wal-append.queue_time_us":626407}
I20260709 13:05:46.680101 18317 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.680650 18317 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:46.683291 18317 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:46.685513 18327 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:46.658903 15853 raft_consensus.cc:1240] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:46.667423 17881 consensus_queue.cc:237] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:46.693687 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:46.696929 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:46.654848 15931 rpcz_store.cc:276] 0709 13:05:44.926302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.626964 (+1700662us) service_pool.cc:224] Handling call
0709 13:05:46.627318 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.644062 (+ 16744us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.644250 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:46.700691 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:46.702062 15860 raft_consensus.cc:2410] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:05:46.703112 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:46.716666 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:46.716725 15860 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:46.718040 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:05:46.718887 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:46.718755 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:46.718973 15862 raft_consensus.cc:1275] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:46.720695 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:46.692845 18291 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:46.721287 16533 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:46.721565 15862 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:46.723263 15860 raft_consensus.cc:2393] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 4: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:46.724483 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:46.723479 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:46.700095 18318 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.728492 15860 raft_consensus.cc:1275] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:05:46.687278 18169 consensus_queue.cc:237] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:46.702355 18208 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:46.733879 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:46.737598 15856 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:46.733632 16533 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:46.735167 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:46.736761 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:46.739099 15937 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5.
I20260709 13:05:46.741842 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:46.748405 18234 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:46.750694 18234 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:46.751435 15857 raft_consensus.cc:1240] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:46.753603 18256 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:46.737500 18086 consensus_queue.cc:1048] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 13:05:46.759989 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:46.761196 18086 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:46.762624 16570 raft_consensus.cc:1275] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:05:46.764541 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:46.722914 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:46.767190 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:46.770203 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:46.772502 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:46.774791 15860 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:46.716866 18177 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:46.734118 18181 raft_consensus.cc:2749] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:46.778101 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:46.780934 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:46.781741 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:46.783984 16570 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:05:46.704034 17981 consensus_queue.cc:237] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:46.790431 16569 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:46.795799 15862 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
I20260709 13:05:46.796772 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:46.798054 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:46.794924 15827 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:46.800160 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:46.801404 15862 raft_consensus.cc:2368] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 6. Current term is 7.
I20260709 13:05:46.800211 15827 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:05:46.802469 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:46.802186 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:46.774948 18208 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.803536 18346 raft_consensus.cc:2804] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:46.806739 18208 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:46.781808 18256 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:46.799413 16005 raft_consensus.cc:2393] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 5: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:46.815963 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:46.806038 15860 raft_consensus.cc:2410] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:46.814584 15859 raft_consensus.cc:1240] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:05:46.827314 15859 raft_consensus.cc:1240] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:05:46.830385 15856 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:46.831959 15860 raft_consensus.cc:1240] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:46.835570 18297 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:46.815953 18317 consensus_queue.cc:1048] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:05:46.838686 18293 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:46.840850 18353 log.cc:927] Time spent T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:05:46.841528 18293 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:46.844012 18208 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:46.811156 18346 raft_consensus.cc:697] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 8 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:46.737983 18318 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:46.845608 18346 consensus_queue.cc:237] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 6.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:46.847641 18318 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:46.815099 15862 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7c0fef632f9a4485838855b553a79163"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:05:46.873811 15862 raft_consensus.cc:606] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Received request to transfer leadership to TS 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:46.874313 15862 raft_consensus.cc:614] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:46.799657 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:46.807811 18258 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:46.875412 16569 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:05:46.880283 15827 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:46.887001 15827 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:46.887847 15759 catalog_manager.cc:5697] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 2 to 3, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 3 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:46.890072 18038 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:46.889734 15759 catalog_manager.cc:5697] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 1 to 3, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 3 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:46.890468 18038 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:46.890825 18038 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:46.898062 18038 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:46.900367 15929 raft_consensus.cc:1275] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:05:46.812901 18147 consensus_queue.cc:1048] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.090s
I20260709 13:05:46.900417 15827 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:46.905071 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:46.912317 15827 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:46.845628 18177 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:46.919822 15827 leader_election.cc:400] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 3. Current term is 4.
I20260709 13:05:46.920334 15827 leader_election.cc:403] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:05:46.905735 18317 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:46.887528 18234 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:46.886679 16004 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:46.905128 18038 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:46.882684 18258 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:46.922183 15827 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:46.923943 18343 raft_consensus.cc:2749] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 3. Current term is 4.
I20260709 13:05:46.917188 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:46.919817 18147 raft_consensus.cc:2804] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:46.922027 18317 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:46.929231 17881 raft_consensus.cc:2749] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:46.932233 18343 consensus_queue.cc:1048] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:46.930879 16564 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:05:46.925402 18234 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:46.935112 18234 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:46.928604 18258 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:46.931236 15827 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:46.938333 18343 raft_consensus.cc:2749] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:46.940024 18343 consensus_queue.cc:1059] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:46.941380 17881 raft_consensus.cc:3055] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:46.891250 15759 catalog_manager.cc:5697] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 2 to 3, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 3 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:46.941733 17881 raft_consensus.cc:740] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:46.942502 17881 consensus_queue.cc:260] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:46.942663 15759 catalog_manager.cc:5697] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 4 to 5, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 5 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:46.944106 15827 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:05:46.946969 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 213405us
I20260709 13:05:46.947048 15827 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:46.947790 16533 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:46.948674 15827 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:46.948912 16533 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:46.931581 18317 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:46.957616 15827 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:46.970775 16533 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:46.936897 18147 raft_consensus.cc:697] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:46.972015 15827 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:46.945912 17881 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:47.010843 18284 consensus_queue.cc:1059] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 13:05:46.956872 18318 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:47.118356 18323 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:47.113791 18297 raft_consensus.cc:2804] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:05:47.118441 17981 consensus_queue.cc:1048] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:05:47.125561 17803 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:47.121796 18291 raft_consensus.cc:2749] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:47.151615 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:47.151952 16568 raft_consensus.cc:1275] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 8. (term mismatch)
I20260709 13:05:47.152488 16568 pending_rounds.cc:85] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting all ops after (but not including) 7
I20260709 13:05:47.152778 16568 pending_rounds.cc:107] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting uncommitted NO_OP operation due to leader change: 5.8
I20260709 13:05:47.153146 16568 raft_consensus.cc:2889] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:47.154568 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:47.156625 16567 raft_consensus.cc:1240] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 6. Current term is 7. Ops: [6.8-6.8]
W20260709 13:05:47.067621 18359 log.cc:927] Time spent T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.234s	user 0.002s	sys 0.000s
I20260709 13:05:46.952643 15759 catalog_manager.cc:5697] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: config changed from index 2 to 3, term changed from 1 to 2, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to <none>, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 2 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:46.954958 18194 raft_consensus.cc:2804] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:47.164458 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:47.149549 18147 consensus_queue.cc:237] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:47.164872 18194 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:47.165002 16567 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:47.165303 18194 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:47.149781 18342 raft_consensus.cc:2749] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:47.004177 18038 raft_consensus.cc:2749] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:05:47.116539 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 107659us
W20260709 13:05:47.168991 16533 leader_election.cc:336] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 1.822s (SENT)
I20260709 13:05:47.169377 16533 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:47.169489 15974 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
W20260709 13:05:47.136703 16008 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41674 (request call id 1659) took 1239 ms (client timeout 1642 ms). Trace:
W20260709 13:05:47.170179 16008 rpcz_store.cc:269] 0709 13:05:45.897629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.897968 (+   339us) service_pool.cc:224] Handling call
0709 13:05:47.136431 (+1238463us) spinlock_profiling.cc:243] Waited 772 ms on lock 0x7b54001dbd54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:47.136628 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1775222528}
I20260709 13:05:47.157961 15825 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:47.116294 18145 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:47.131395 18357 log.cc:927] Time spent T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
I20260709 13:05:47.128468 18323 raft_consensus.cc:697] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:47.132431 18297 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:47.187341 18297 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:47.194336 15933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53636 (request call id 457) took 2131 ms. Trace:
I20260709 13:05:47.194805 15933 rpcz_store.cc:276] 0709 13:05:45.063122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.111426 (+1048304us) service_pool.cc:224] Handling call
0709 13:05:47.194078 (+1082652us) spinlock_profiling.cc:243] Waited 3.25 ms on lock 0x7b540022cc54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:47.194267 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7482112}
I20260709 13:05:47.152849 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:47.196442 16561 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:05:47.199993 16566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 710) took 2217 ms. Trace:
I20260709 13:05:47.200477 16566 rpcz_store.cc:276] 0709 13:05:44.982565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.008500 (+ 25935us) service_pool.cc:224] Handling call
0709 13:05:45.008739 (+   239us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.012999 (+1004260us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:47.199754 (+1186755us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b54006a1654. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:47.199939 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2307467520}
I20260709 13:05:47.160953 15759 catalog_manager.cc:5697] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:47.163822 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:46.955958 18343 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:05:47.112896 18293 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.203372 15759 catalog_manager.cc:5697] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:47.171463 15974 leader_election.cc:400] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 2 election: Vote denied by peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) with higher term. Message: Invalid argument: T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 2. Current term is 3.
W20260709 13:05:47.217945 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 46657.8us
I20260709 13:05:47.142088 16004 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5.
I20260709 13:05:47.149251 16010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1749) took 2099 ms. Trace:
I20260709 13:05:47.220386 16010 rpcz_store.cc:276] 0709 13:05:45.049399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.602415 (+553016us) service_pool.cc:224] Handling call
0709 13:05:45.789481 (+187066us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.013587 (+224106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:47.149022 (+1135435us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b54000eecd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:47.149186 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":436086144}
I20260709 13:05:47.179133 16567 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:05:47.179744 18216 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:47.223654 18216 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:47.226151 18216 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:47.179818 18318 raft_consensus.cc:697] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:47.228439 18318 consensus_queue.cc:237] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:47.185142 17881 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:47.186199 18316 raft_consensus.cc:3055] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:47.231822 18316 raft_consensus.cc:740] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:47.232744 18316 consensus_queue.cc:260] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:47.204181 16569 raft_consensus.cc:2393] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 4: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
W20260709 13:05:47.203389 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:47.214532 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:47.181653 18256 raft_consensus.cc:2804] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:47.234833 18256 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:47.234953 17881 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:47.235230 18256 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:47.170665 18177 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.235957 18177 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:47.237927 18177 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:47.219540 15759 catalog_manager.cc:5697] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 2 to 3, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 3 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:47.240578 15759 catalog_manager.cc:5697] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 1 to 2, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 2 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:47.241648 18145 raft_consensus.cc:2749] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:47.231858 18181 raft_consensus.cc:2749] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:47.235491 17881 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:47.244084 18323 consensus_queue.cc:237] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:47.244748 18317 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:47.246673 18317 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:47.184242 18038 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.242878 15759 catalog_manager.cc:5697] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 2 to 3, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 3 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:47.139605 18258 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:47.248780 15928 raft_consensus.cc:1240] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:47.257347 15759 catalog_manager.cc:5697] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 2 to 3, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 3 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:47.222378 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:05:47.275103 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 303242us
I20260709 13:05:47.144893 18234 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:47.219754 18293 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:47.283416 17803 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.287530 16564 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:47.257356 18038 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:05:47.296075 18354 log.cc:927] Time spent T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
I20260709 13:05:47.285964 18297 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:47.297647 17803 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:05:47.330090 16011 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41674 (request call id 1658) took 1435 ms (client timeout 1517 ms). Trace:
W20260709 13:05:47.330552 16011 rpcz_store.cc:269] 0709 13:05:45.894705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.894971 (+   266us) service_pool.cc:224] Handling call
0709 13:05:47.329889 (+1434918us) spinlock_profiling.cc:243] Waited 34.8 ms on lock 0x7b54001ba154. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:47.330030 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":80127360}
I20260709 13:05:47.331843 16009 raft_consensus.cc:1275] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:05:47.332971 16009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1669) took 1221 ms. Trace:
I20260709 13:05:47.333164 18297 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:47.333282 16009 rpcz_store.cc:276] 0709 13:05:46.111593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.564651 (+453058us) service_pool.cc:224] Handling call
0709 13:05:47.331480 (+766829us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:47.332239 (+   759us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:47.332800 (+   561us) spinlock_profiling.cc:243] Waited 766 ms on lock 0x7b54001ba150. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:47.332918 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1762840832}
W20260709 13:05:47.339840 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 181988us
I20260709 13:05:47.342295 15825 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:47.343981 16533 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:47.345834 16533 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:05:47.349750 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 131161us
I20260709 13:05:47.351531 15825 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:47.354123 15825 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:05:47.359158 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:47.362115 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:47.364638 15825 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:47.366922 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:47.349619 18297 raft_consensus.cc:2804] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:47.347636 18246 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:47.374037 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:47.374104 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:47.377377 16010 raft_consensus.cc:2410] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 8.
I20260709 13:05:47.380712 18343 consensus_queue.cc:1059] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:47.382223 18343 raft_consensus.cc:3055] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:47.382561 18343 raft_consensus.cc:740] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:47.383273 18343 consensus_queue.cc:260] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:47.384241 18343 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:47.385812 18346 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:47.376302 18145 raft_consensus.cc:2749] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:47.351897 18258 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:47.373797 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:47.356705 18156 raft_consensus.cc:3055] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:47.349359 17881 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:47.385537 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 202273us
I20260709 13:05:47.395344 16003 raft_consensus.cc:1275] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:05:47.395457 16009 raft_consensus.cc:1240] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:47.397464 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:47.402189 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:47.402983 16011 raft_consensus.cc:1275] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:05:47.403570 15902 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:05:47.397758 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:47.380170 18161 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.901s	user 0.010s	sys 0.005s
I20260709 13:05:47.399570 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:05:47.404958 18161 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.935s	user 0.010s	sys 0.005s
I20260709 13:05:47.362555 18368 raft_consensus.cc:2749] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:47.368121 18342 raft_consensus.cc:2804] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:47.409157 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:47.353615 17931 consensus_queue.cc:1059] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
W20260709 13:05:47.410487 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 59926.8us
I20260709 13:05:47.349619 17993 consensus_queue.cc:1048] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 13:05:47.414129 18161 consensus_queue.cc:1048] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:47.415962 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:47.418538 16561 raft_consensus.cc:1240] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:47.418769 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:47.420842 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:47.421993 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:47.425200 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:47.427199 15902 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:47.429190 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:47.430058 16011 raft_consensus.cc:1275] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:05:47.433259 16566 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:47.436436 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:47.361848 18308 consensus_queue.cc:1059] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:05:47.437441 15933 raft_consensus.cc:1240] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:05:47.437754 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:47.440537 15855 raft_consensus.cc:1240] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:47.440791 18293 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:47.443949 18370 consensus_queue.cc:1048] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:47.444533 17803 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:47.700788 18316 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:47.567718 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:47.715528 15856 raft_consensus.cc:2393] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 7: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:47.715431 15933 raft_consensus.cc:1275] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:05:47.650358 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:47.567369 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:47.446841 16009 raft_consensus.cc:1275] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:05:47.573611 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 153895us
I20260709 13:05:47.714049 15859 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:47.714350 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:47.541257 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:47.722388 15937 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:47.435925 18318 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:47.699751 18338 log.cc:927] Time spent T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 1.183s	user 0.001s	sys 0.008s
I20260709 13:05:47.728595 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:47.729437 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:05:47.680583 18366 log.cc:927] Time spent T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 13:05:47.735610 15935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 1706) took 1553 ms. Trace:
I20260709 13:05:47.740911 15935 rpcz_store.cc:276] 0709 13:05:46.181676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:47.681732 (+1500056us) service_pool.cc:224] Handling call
0709 13:05:47.735535 (+ 53803us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:47.432974 18342 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:47.738344 15933 raft_consensus.cc:1275] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:05:47.739988 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:47.652065 18375 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.744848 16533 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:47.540851 18038 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:47.753755 15858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 1765) took 1613 ms. Trace:
I20260709 13:05:47.444687 18297 raft_consensus.cc:697] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:47.727087 18188 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:47.399833 18156 raft_consensus.cc:740] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:47.745784 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:47.767551 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:47.768265 16009 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:47.747004 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:47.747416 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:47.769181 15931 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:47.541427 17931 consensus_queue.cc:1243] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer 31882606e87447fb9a36bbf625129fe8 log is divergent from this leader: its last log entry 5.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:05:47.607895 18290 consensus_queue.cc:1059] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.145s
I20260709 13:05:47.710354 18219 consensus_queue.cc:1059] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:05:47.788579 16008 raft_consensus.cc:1240] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:47.431797 18234 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:47.754101 15858 rpcz_store.cc:276] 0709 13:05:46.140418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.140587 (+   169us) service_pool.cc:224] Handling call
0709 13:05:46.140960 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:46.231860 (+ 90900us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:46.231904 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:46.232474 (+   570us) log.cc:844] Serialized 26 byte log entry
0709 13:05:46.453968 (+221494us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:05:46.454011 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.454092 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:47.699112 (+1245020us) spinlock_profiling.cc:243] Waited 19.1 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:05:47.748036 (+ 48924us) raft_consensus.cc:1685] finished
0709 13:05:47.748057 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:47.753696 (+  5639us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":60115,"raft.queue_time_us":53271,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":134,"spinlock_wait_cycles":44040960,"thread_start_us":171618,"threads_started":1,"wal-append.queue_time_us":234123}
I20260709 13:05:47.762431 15937 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
W20260709 13:05:47.762979 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 406262us
I20260709 13:05:47.754129 16003 raft_consensus.cc:2393] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 5: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:47.755618 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:47.765700 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:47.743702 18386 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.773674 18228 raft_consensus.cc:3055] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:47.775514 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:47.776520 18346 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:47.777554 18249 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:47.718607 18322 consensus_queue.cc:1059] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:05:47.431154 18246 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:47.769263 15935 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:05:47.741075 18318 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:47.786753 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:47.717271 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:47.751139 18342 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:47.791777 17881 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:47.792409 18295 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:05:47.792807 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:47.793972 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:47.817710 15858 raft_consensus.cc:2410] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:05:47.727494 17981 consensus_queue.cc:1048] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 13:05:47.794598 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:47.796046 16533 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:47.567857 18194 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:47.824800 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:47.825646 15858 raft_consensus.cc:2368] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 5. Current term is 6.
I20260709 13:05:47.825892 18194 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:47.826431 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:47.826606 18323 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:47.827109 15858 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:47.827196 18323 raft_consensus.cc:697] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:47.828372 18323 consensus_queue.cc:237] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:47.806074 18346 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:47.832815 15825 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:05:47.804497 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 377321us
I20260709 13:05:47.744284 18293 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:05:47.725023 18256 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:47.770418 18308 raft_consensus.cc:3055] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:47.808081 15937 raft_consensus.cc:1275] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:05:47.840544 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 407059us
W20260709 13:05:47.839006 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:47.809723 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:47.717964 18208 raft_consensus.cc:2749] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:47.811116 15931 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:05:47.811116 15930 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:47.841393 15856 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:47.805739 18386 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:47.803448 18228 raft_consensus.cc:740] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:05:47.820503 18369 log.cc:927] Time spent T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:05:47.760802 18367 consensus_queue.cc:1048] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 13:05:47.784392 16564 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:05:47.763988 17993 raft_consensus.cc:2749] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:47.833766 16533 leader_election.cc:400] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 5. Current term is 6.
I20260709 13:05:47.876260 16533 leader_election.cc:403] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:47.876955 16533 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:47.803759 16006 raft_consensus.cc:2393] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 7: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
W20260709 13:05:47.877895 18332 log.cc:927] Time spent T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 1.353s	user 0.006s	sys 0.003s
W20260709 13:05:47.878262 16006 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41674 (request call id 1666) took 1856 ms (client timeout 2044 ms). Trace:
W20260709 13:05:47.878609 16006 rpcz_store.cc:269] 0709 13:05:46.021709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:47.775293 (+1753584us) service_pool.cc:224] Handling call
0709 13:05:47.878207 (+102914us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:47.836824 15902 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:47.880177 16567 raft_consensus.cc:1240] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:47.880577 18208 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:05:47.880720 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 43970.7us
I20260709 13:05:47.881500 15902 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:47.881955 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:47.882212 18208 raft_consensus.cc:2749] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:47.882550 15902 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:47.883267 18208 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:47.883333 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:47.884186 16010 raft_consensus.cc:2393] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 7: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:47.884303 18208 raft_consensus.cc:697] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 5 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:47.885144 15902 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:47.885224 18208 consensus_queue.cc:237] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:47.885946 17993 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:05:47.888485 15902 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:47.855289 15976 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:47.889444 16566 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:05:47.835238 15974 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:47.810185 18381 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:47.845229 18234 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:47.859252 18256 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:47.860066 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:47.894037 15937 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:47.832125 18385 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:47.865079 18386 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:47.764880 18375 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:47.867002 15935 raft_consensus.cc:1275] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 6 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:05:47.868614 18194 consensus_queue.cc:1048] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:47.869184 18228 consensus_queue.cc:260] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:47.835170 18390 raft_consensus.cc:3055] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:47.872301 15930 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:05:47.839897 15858 raft_consensus.cc:1240] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:05:47.887516 18181 raft_consensus.cc:2804] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:05:47.895824 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 60673.8us
I20260709 13:05:47.943580 15974 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:05:47.918947 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 63765.7us
I20260709 13:05:47.945147 15976 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:47.924897 15902 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:47.928905 15859 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
I20260709 13:05:47.889995 18325 raft_consensus.cc:2749] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:47.938657 15937 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
W20260709 13:05:47.974781 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 49992.7us
I20260709 13:05:47.976011 15902 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:47.846060 18308 raft_consensus.cc:740] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:47.929282 15856 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
I20260709 13:05:47.947737 18181 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:47.948386 15859 raft_consensus.cc:1240] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:47.844075 18377 consensus_queue.cc:1059] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 13:05:47.893790 18234 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:47.954505 18371 log.cc:927] Time spent T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 13:05:47.939494 18392 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:05:47.957526 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:47.959789 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1734) took 5580 ms. Trace:
I20260709 13:05:47.965057 18375 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:47.965682 16568 raft_consensus.cc:1275] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 13:05:47.968243 18379 log.cc:927] Time spent T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:05:47.969374 18391 raft_consensus.cc:2804] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:05:47.969488 16565 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45246 (request call id 616) took 2084 ms (client timeout 1781 ms). Trace:
I20260709 13:05:47.941957 18310 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:47.930681 18387 consensus_queue.cc:1059] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 13:05:47.932286 18390 raft_consensus.cc:740] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:47.977640 15934 raft_consensus.cc:1240] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 3. Current term is 4. Ops: [3.4-3.4]
I20260709 13:05:47.978582 16533 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:05:47.952191 18378 log.cc:927] Time spent T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
W20260709 13:05:48.037231 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 58746.5us
I20260709 13:05:47.953248 15858 raft_consensus.cc:1240] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 3. Current term is 4. Ops: [3.4-3.4]
I20260709 13:05:48.039214 15858 raft_consensus.cc:1275] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:47.896452 17931 raft_consensus.cc:2749] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:47.892211 18381 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:47.986543 15902 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:48.046726 15859 raft_consensus.cc:1275] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:05:47.884305 18370 consensus_queue.cc:1048] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.090s
I20260709 13:05:47.939922 17961 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:05:48.030963 16565 rpcz_store.cc:269] 0709 13:05:45.884972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.937730 (+ 52758us) service_pool.cc:224] Handling call
0709 13:05:47.969205 (+2031475us) spinlock_profiling.cc:243] Waited 695 ms on lock 0x7b54007017d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:47.969382 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1598630656}
I20260709 13:05:48.031586 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:47.979830 18228 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:47.983511 18181 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:47.987527 18234 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:47.983139 18386 raft_consensus.cc:2749] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:48.033561 18310 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:47.929605 18385 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:48.053651 15856 raft_consensus.cc:2410] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:05:47.857401 18297 consensus_queue.cc:237] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:48.058091 15900 leader_election.cc:336] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.529s (SENT)
I20260709 13:05:48.092621 15900 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:48.032321 18161 raft_consensus.cc:2764] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
W20260709 13:05:48.048815 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 62397.8us
I20260709 13:05:48.034533 17803 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:48.070957 18167 consensus_queue.cc:1048] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:48.072018 18323 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:48.050549 18256 raft_consensus.cc:2749] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:48.050386 18381 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:48.046623 18392 raft_consensus.cc:697] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:48.049742 18363 raft_consensus.cc:2804] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:05:47.989481 16002 rpcz_store.cc:276] 0709 13:05:42.379367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.894873 (+2515506us) service_pool.cc:224] Handling call
0709 13:05:44.930582 (+ 35709us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.074180 (+1143598us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:47.959459 (+1885279us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54001b9754. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:47.959702 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2460057856}
I20260709 13:05:48.051645 18295 consensus_queue.cc:1048] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
W20260709 13:05:48.108561 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 165074us
I20260709 13:05:48.050786 18375 consensus_queue.cc:1059] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.054s
W20260709 13:05:48.108781 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 50726.2us
W20260709 13:05:48.133219 15900 leader_election.cc:336] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.557s (SENT)
I20260709 13:05:48.133625 15900 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:48.067919 18391 raft_consensus.cc:697] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:48.069516 18310 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:48.128521 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 155261us
I20260709 13:05:48.105849 15854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 1754) took 2202 ms. Trace:
W20260709 13:05:48.148655 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 91994.8us
I20260709 13:05:48.119885 16562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 706) took 3126 ms. Trace:
I20260709 13:05:48.131700 17931 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:05:48.139232 18343 consensus_meta.cc:287] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.755s	user 0.010s	sys 0.001s
I20260709 13:05:48.156888 15860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1645) took 1404 ms. Trace:
I20260709 13:05:48.139974 18386 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:48.164609 15860 rpcz_store.cc:276] 0709 13:05:46.752846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.825377 (+ 72531us) service_pool.cc:224] Handling call
0709 13:05:46.831709 (+  6332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.832417 (+   708us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:48.156561 (+1324144us) spinlock_profiling.cc:243] Waited 5.93 ms on lock 0x7b54002ebb50. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:48.156783 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13637376}
I20260709 13:05:48.153121 16562 rpcz_store.cc:276] 0709 13:05:44.981458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.007918 (+ 26460us) service_pool.cc:224] Handling call
0709 13:05:45.008192 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.090717 (+1082525us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:48.067692 (+1976975us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54000f6ed4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:48.107990 (+ 40298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2345219456}
I20260709 13:05:48.168414 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:48.169313 15856 raft_consensus.cc:2410] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:05:48.171064 18156 consensus_queue.cc:260] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:48.172109 18156 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:48.151825 15854 rpcz_store.cc:276] 0709 13:05:45.900623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.907854 (+  7231us) service_pool.cc:224] Handling call
0709 13:05:45.929922 (+ 22068us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:45.930351 (+   429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:45.930388 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:47.141161 (+1210773us) spinlock_profiling.cc:243] Waited 17.6 ms on lock 0x7b2c00000f48. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e8ca1a11 00007f73e8cb1372 00007f73ea3e3a01 00007f73ea3a843a 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2
0709 13:05:47.141772 (+   611us) log.cc:844] Serialized 26 byte log entry
0709 13:05:47.363726 (+221954us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:05:47.374479 (+ 10753us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:47.391007 (+ 16528us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:48.030853 (+639846us) raft_consensus.cc:1685] finished
0709 13:05:48.034336 (+  3483us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:48.103339 (+ 69003us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":470,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":103,"spinlock_wait_cycles":40556800,"thread_start_us":211611,"threads_started":1,"wal-append.queue_time_us":605962}
I20260709 13:05:48.070011 18308 consensus_queue.cc:260] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:48.174082 18308 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:48.177798 18388 log.cc:927] Time spent T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.349s	user 0.002s	sys 0.000s
I20260709 13:05:48.177608 18391 consensus_queue.cc:237] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:48.145609 18390 consensus_queue.cc:260] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:48.180712 18390 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:48.180584 18346 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:48.183270 18346 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:48.166733 18399 raft_consensus.cc:3055] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:48.184170 18399 raft_consensus.cc:740] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:48.185245 18399 consensus_queue.cc:260] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:48.186797 18399 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:48.189869 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:48.150596 18318 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:48.107393 17803 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:48.105468 18323 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:48.153330 18363 raft_consensus.cc:697] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:48.134830 18177 consensus_queue.cc:1059] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:05:48.167078 18375 raft_consensus.cc:3055] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:48.222126 15976 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:05:48.222735 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 91968.6us
I20260709 13:05:48.209046 18375 raft_consensus.cc:740] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:48.214461 18394 raft_consensus.cc:2804] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:48.201885 18156 consensus_queue.cc:1048] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:05:48.200801 18290 consensus_queue.cc:1048] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:05:48.235353 16570 raft_consensus.cc:1240] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:05:48.232865 15974 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:48.244199 18404 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:48.246166 16565 raft_consensus.cc:1240] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:48.249414 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:48.250521 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:48.251016 15932 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:48.254035 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:48.254632 15937 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 4.
I20260709 13:05:48.255438 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:48.256412 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:48.257253 15937 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:05:48.258304 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:05:48.264968 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:48.269250 15974 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:48.251497 17993 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 5. Current term is 6.
I20260709 13:05:48.274098 15932 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
I20260709 13:05:48.281404 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:48.270366 15853 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
W20260709 13:05:48.266304 18396 log.cc:927] Time spent T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:05:48.228680 18377 consensus_queue.cc:1059] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:05:48.288098 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:48.290082 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:48.270591 18167 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:48.289512 15937 raft_consensus.cc:2410] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:05:48.300494 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:48.213006 18392 consensus_queue.cc:237] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:48.286984 18403 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.310554 16570 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEARNER]: Advancing to term 5
I20260709 13:05:48.306264 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:05:48.307334 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:48.281266 18385 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:48.284775 17993 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:48.314256 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:48.312911 15932 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:48.316936 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:48.289232 18407 raft_consensus.cc:2749] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:48.313759 18374 raft_consensus.cc:2804] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:48.303440 18167 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:48.318439 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:05:48.331147 15974 connection.cc:256] RPC call timeout handler was delayed by -1.1333590909999884s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1521s for a response
W20260709 13:05:48.331947 15974 connection.cc:256] RPC call timeout handler was delayed by -1.028074026000013s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17800000000000002s for a response
I20260709 13:05:48.267217 18363 consensus_queue.cc:237] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:48.340238 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:48.343209 16566 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:05:48.341216 16570 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 LEARNER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:05:48.289490 18404 raft_consensus.cc:697] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:48.333810 18373 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:48.338444 18188 consensus_meta.cc:287] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.568s	user 0.008s	sys 0.002s
I20260709 13:05:48.287215 18375 consensus_queue.cc:260] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:48.308663 18342 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:05:48.322217 17993 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:48.333884 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:48.355784 18323 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:48.356597 18234 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:48.360064 18181 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:48.365576 15932 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:05:48.366319 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:48.368316 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:48.369045 15934 raft_consensus.cc:2393] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 5: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:48.369115 18167 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:48.354681 18322 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.369963 18322 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:48.371506 16561 raft_consensus.cc:1275] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 5. (term mismatch)
I20260709 13:05:48.371870 16561 pending_rounds.cc:85] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting all ops after (but not including) 4
I20260709 13:05:48.372099 16561 pending_rounds.cc:107] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting uncommitted NO_OP operation due to leader change: 4.5
I20260709 13:05:48.372341 17803 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:48.372699 18404 consensus_queue.cc:237] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:48.374941 15935 raft_consensus.cc:1240] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:48.376852 18183 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:48.372346 16561 raft_consensus.cc:2889] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:48.377389 18183 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:48.382694 16565 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:05:48.382750 15862 raft_consensus.cc:2393] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 7: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:48.316619 18403 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:48.384891 15935 raft_consensus.cc:1240] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:48.363881 18372 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:48.408596 18389 log.cc:927] Time spent T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.579s	user 0.001s	sys 0.001s
I20260709 13:05:48.399315 18403 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:48.353938 18373 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:48.399525 15853 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 1630) took 1707 ms (client timeout 1529 ms). Trace:
I20260709 13:05:48.400903 15936 raft_consensus.cc:3055] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:48.417928 15936 raft_consensus.cc:740] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:48.408254 18372 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:48.388578 15859 raft_consensus.cc:1240] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 3. Current term is 5. Ops: []
I20260709 13:05:48.416260 16569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 722) took 1048 ms. Trace:
I20260709 13:05:48.378775 16562 raft_consensus.cc:2468] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:05:48.386178 18375 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:48.432253 18342 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:48.420203 16569 rpcz_store.cc:276] 0709 13:05:47.367563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:47.368789 (+  1226us) service_pool.cc:224] Handling call
0709 13:05:47.369053 (+   264us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:47.388459 (+ 19406us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:47.388488 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:47.412333 (+ 23845us) log.cc:844] Serialized 26 byte log entry
0709 13:05:47.806432 (+394099us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:47.806472 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:47.806537 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:48.407994 (+601457us) spinlock_profiling.cc:243] Waited 65.1 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:05:48.410026 (+  2032us) raft_consensus.cc:1685] finished
0709 13:05:48.410048 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:48.416173 (+  6125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19058,"raft.queue_time_us":26520,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":39476,"spinlock_wait_cycles":149820160,"thread_start_us":229201,"threads_started":1,"wal-append.queue_time_us":349814}
I20260709 13:05:48.444305 16007 raft_consensus.cc:1275] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:48.438488 15936 consensus_queue.cc:260] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:48.460505 15936 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:48.462936 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:48.463826 16010 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:48.465934 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:48.466840 16010 raft_consensus.cc:2410] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:05:48.382277 18381 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:48.472221 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:48.473001 16007 raft_consensus.cc:1275] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:05:48.473417 16007 pending_rounds.cc:85] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting all ops after (but not including) 5
W20260709 13:05:48.471935 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:48.496572 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:48.474196 16010 raft_consensus.cc:1240] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 13:05:48.417717 15853 rpcz_store.cc:269] 0709 13:05:46.692275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.696301 (+  4026us) service_pool.cc:224] Handling call
0709 13:05:48.399300 (+1702999us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400218c54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:48.399440 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2531842944}
W20260709 13:05:48.514088 15900 leader_election.cc:336] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.985s (SENT)
I20260709 13:05:48.784966 18234 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:48.794420 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 280373us
I20260709 13:05:48.710172 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:05:48.796170 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:48.473637 16007 pending_rounds.cc:107] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:05:48.561283 15936 raft_consensus.cc:1275] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 13:05:48.649647 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:48.518819 15930 raft_consensus.cc:1240] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:48.797962 16007 raft_consensus.cc:2889] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:48.844367 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:48.467478 18310 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:48.848415 15974 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.006s	sys 0.016s
W20260709 13:05:48.850011 15974 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.007s	sys 0.016s
I20260709 13:05:48.850914 15974 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:48.478205 18208 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.854043 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:48.854638 15856 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:48.747496 18338 log.cc:927] Time spent T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.216s	user 0.001s	sys 0.000s
W20260709 13:05:48.476485 18285 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.002s	sys 0.013s
I20260709 13:05:48.543097 18297 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.858800 15974 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:48.860630 15936 raft_consensus.cc:1240] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:05:48.547988 18372 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:48.531647 18374 raft_consensus.cc:697] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 5 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:48.842837 18183 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:48.868157 18297 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:48.868542 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:05:48.860533 18285 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.040s	user 0.002s	sys 0.028s
I20260709 13:05:48.841955 18404 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:48.869514 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:48.846863 18181 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:48.867822 15856 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:05:48.849267 18291 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:48.849821 18323 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.530439 18399 consensus_queue.cc:1059] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.337s
I20260709 13:05:48.857525 18392 raft_consensus.cc:2749] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:48.882498 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:48.619804 18411 consensus_queue.cc:1048] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.273s
I20260709 13:05:48.892693 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:48.894769 18177 raft_consensus.cc:3055] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:05:48.897423 16005 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41674 (request call id 1670) took 2761 ms (client timeout 2112 ms). Trace:
W20260709 13:05:48.912326 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:48.907172 15936 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
W20260709 13:05:48.912251 16005 rpcz_store.cc:269] 0709 13:05:46.136210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.303004 (+166794us) service_pool.cc:224] Handling call
0709 13:05:48.897166 (+2594162us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b54003ed854. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:48.897326 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":663559040}
I20260709 13:05:48.899828 18404 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:48.917676 18241 raft_consensus.cc:3055] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:48.918198 18241 raft_consensus.cc:740] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:48.896958 18323 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:48.895754 18291 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:48.913686 18177 raft_consensus.cc:740] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:48.859982 18208 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:05:48.922833 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 64349.5us
I20260709 13:05:48.930604 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:48.941640 16566 raft_consensus.cc:2393] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 5: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:48.932934 15930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 468) took 1147 ms. Trace:
I20260709 13:05:48.938979 16570 raft_consensus.cc:1240] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:48.943018 15974 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:48.929788 15934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 471) took 1049 ms. Trace:
I20260709 13:05:48.944198 15934 rpcz_store.cc:276] 0709 13:05:47.880118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:48.369632 (+489514us) service_pool.cc:224] Handling call
0709 13:05:48.369925 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:48.378511 (+  8586us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:48.378545 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:48.378943 (+   398us) log.cc:844] Serialized 26 byte log entry
0709 13:05:48.468467 (+ 89524us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:48.468522 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:48.468610 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:48.929574 (+460964us) raft_consensus.cc:1685] finished
0709 13:05:48.929595 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:48.929731 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8214,"raft.queue_time_us":8521,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":2928,"thread_start_us":81261,"threads_started":1,"wal-append.queue_time_us":541141}
I20260709 13:05:48.942117 15930 rpcz_store.cc:276] 0709 13:05:47.786008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:47.933022 (+147014us) service_pool.cc:224] Handling call
0709 13:05:47.933342 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:48.805127 (+871785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:48.932647 (+127520us) spinlock_profiling.cc:243] Waited 585 ms on lock 0x7b54001a81d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:48.932866 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1345970432}
W20260709 13:05:48.932895 18418 log.cc:927] Time spent T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 13:05:48.928660 18219 raft_consensus.cc:3055] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:48.905048 18167 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.900405 16562 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
W20260709 13:05:48.925633 18355 log.cc:927] Time spent T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.976s	user 0.008s	sys 0.008s
I20260709 13:05:48.948863 15974 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:48.923743 18256 raft_consensus.cc:3055] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:48.923234 18417 raft_consensus.cc:2749] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:48.924322 18291 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:48.922161 18323 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:48.919610 18404 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:48.913544 17993 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:48.932435 16563 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:48.925756 18297 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:48.962177 15974 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:48.951279 18373 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:48.958176 18375 consensus_meta.cc:287] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.526s	user 0.008s	sys 0.003s
I20260709 13:05:48.919175 18241 consensus_queue.cc:260] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:48.949182 18219 raft_consensus.cc:740] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:48.954507 18167 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:48.949288 18363 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.009s	sys 0.031s
I20260709 13:05:48.980146 15976 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:48.951751 18424 raft_consensus.cc:2749] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:48.951558 18390 consensus_queue.cc:1059] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:05:48.986147 15858 raft_consensus.cc:1240] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:48.972723 18322 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:48.955191 18256 raft_consensus.cc:740] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:48.984004 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:48.984966 15857 raft_consensus.cc:1240] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:48.978017 18381 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:48.989192 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:48.936049 18177 consensus_queue.cc:260] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:48.994874 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:48.981137 15856 raft_consensus.cc:1275] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:05:48.989490 18399 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:49.003209 15861 raft_consensus.cc:2393] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 5: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:48.922590 18374 consensus_queue.cc:237] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:48.973469 18421 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:05:48.983237 18363 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.009s	sys 0.031s
I20260709 13:05:48.979826 18404 raft_consensus.cc:2749] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:49.004218 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:05:49.005453 15862 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 1769) took 1588 ms (client timeout 1790 ms). Trace:
I20260709 13:05:49.009109 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:49.010267 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:49.009987 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:49.012758 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:49.032697 16010 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:49.013475 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:49.010816 18415 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.039037 18415 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:05:49.040405 18425 log.cc:927] Time spent T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:05:49.040545 15932 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:46118
I20260709 13:05:49.041352 15932 raft_consensus.cc:606] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Received request to transfer leadership to TS 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:49.041527 18415 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:49.041760 15932 raft_consensus.cc:614] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:49.011988 18390 consensus_queue.cc:1243] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer eb1e0493af1e46dbb6dec7e25525bab5 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:05:49.043426 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:05:49.045132 16008 raft_consensus.cc:1240] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 4. Current term is 5. Ops: [1.3-4.4]
I20260709 13:05:49.045485 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:49.046425 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:49.046981 16008 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:49.011147 18399 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:49.047986 18399 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:49.048418 18208 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:49.048226 18256 consensus_queue.cc:260] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:49.008018 18310 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:49.050086 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:49.015437 18394 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:49.016057 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:49.016676 15857 raft_consensus.cc:1275] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:05:49.018464 15861 raft_consensus.cc:1275] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:05:49.019542 15858 raft_consensus.cc:1275] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:48.988575 18241 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:49.060845 15928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1703) took 2879 ms. Trace:
I20260709 13:05:49.063788 15928 rpcz_store.cc:276] 0709 13:05:46.180981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.665460 (+484479us) service_pool.cc:224] Handling call
0709 13:05:47.216211 (+550751us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:47.263369 (+ 47158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:49.060525 (+1797156us) spinlock_profiling.cc:243] Waited 536 ms on lock 0x7b540022cc50. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:49.060783 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1232800128}
I20260709 13:05:49.033888 16569 raft_consensus.cc:1240] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 2. Current term is 3. Ops: []
I20260709 13:05:49.052912 16010 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:05:49.021040 18246 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:49.037225 18167 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:49.037299 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:49.070237 16003 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:48.984357 18219 consensus_queue.cc:260] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:49.077710 18219 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:49.078114 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:49.078747 16011 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:05:49.081933 16010 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:05:49.082762 18392 consensus_queue.cc:1175] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "31882606e87447fb9a36bbf625129fe8" responder_term: 3 status { last_received { term: 3 index: 5 } last_committed_idx: 5 last_received_current_leader { term: 3 index: 5 } }
I20260709 13:05:49.083873 18363 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.084440 18363 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:49.085158 18395 log.cc:927] Time spent T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.892s	user 0.016s	sys 0.000s
I20260709 13:05:49.038326 16567 raft_consensus.cc:1240] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 3. Current term is 4. Ops: []
W20260709 13:05:49.030274 15862 rpcz_store.cc:269] 0709 13:05:47.418282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:47.418535 (+   253us) service_pool.cc:224] Handling call
0709 13:05:49.005181 (+1586646us) spinlock_profiling.cc:243] Waited 666 ms on lock 0x7b54001009d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:49.005383 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1530958720}
I20260709 13:05:49.058881 16005 raft_consensus.cc:2410] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:05:49.069593 18310 consensus_queue.cc:1059] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:05:49.068225 18394 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:49.074832 18421 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:49.098171 18376 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:49.098786 18376 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:49.100979 18376 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:49.088779 15929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 464) took 1650 ms. Trace:
I20260709 13:05:49.052063 18381 consensus_queue.cc:1059] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:05:49.103322 18310 consensus_queue.cc:1243] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer eb1e0493af1e46dbb6dec7e25525bab5 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:05:49.103484 18381 raft_consensus.cc:3055] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:49.103953 18310 raft_consensus.cc:3055] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:05:49.104501 18310 raft_consensus.cc:740] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:49.104879 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:49.105458 16006 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:49.105337 18310 consensus_queue.cc:260] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:49.106216 18381 raft_consensus.cc:740] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:49.106513 18310 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:05:49.102912 15929 rpcz_store.cc:276] 0709 13:05:47.438606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:47.444931 (+  6325us) service_pool.cc:224] Handling call
0709 13:05:47.643966 (+199035us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:47.752113 (+108147us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:47.756228 (+  4115us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:47.812828 (+ 56600us) log.cc:844] Serialized 26 byte log entry
0709 13:05:48.095454 (+282626us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:48.270583 (+175129us) log.cc:844] Serialized 14 byte log entry
0709 13:05:48.286358 (+ 15775us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:48.292153 (+  5795us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:49.084557 (+792404us) spinlock_profiling.cc:243] Waited 2.35 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:05:49.086562 (+  2005us) raft_consensus.cc:1685] finished
0709 13:05:49.088584 (+  2022us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:49.088724 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65690,"prepare.queue_time_us":119017,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":1730,"raft.queue_time_us":96828,"raft.run_cpu_time_us":259,"raft.run_wall_time_us":13506,"spinlock_wait_cycles":5405568,"thread_start_us":317568,"threads_started":2,"wal-append.queue_time_us":300925}
I20260709 13:05:49.108942 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:49.097493 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:49.110721 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:49.098402 18421 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:49.112699 18290 consensus_queue.cc:1059] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:49.113736 18414 raft_consensus.cc:3055] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:49.114076 18414 raft_consensus.cc:740] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:49.114514 18421 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:49.097879 18086 raft_consensus.cc:3055] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:49.115129 18086 raft_consensus.cc:740] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:49.114890 18414 consensus_queue.cc:260] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:49.058250 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:49.107903 17866 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.110723 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:49.096302 16535 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:49.114090 16566 raft_consensus.cc:1240] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:05:49.124321 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:49.126859 16566 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
W20260709 13:05:49.116338 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 92933.7us
I20260709 13:05:49.127843 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:49.128093 15974 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:49.128512 16566 raft_consensus.cc:2393] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 7: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:49.118960 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:49.121209 16008 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:05:49.129199 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:49.130628 16566 raft_consensus.cc:2410] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 4 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:05:49.128522 16007 raft_consensus.cc:2393] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 7: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:49.116375 16003 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5.
I20260709 13:05:49.133042 16535 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:05:49.122267 18435 log.cc:927] Time spent T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.208s	user 0.003s	sys 0.000s
I20260709 13:05:49.131904 16010 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:05:49.134358 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:49.136016 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:49.136735 16567 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:05:49.146106 15827 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:49.141885 16006 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:05:49.143556 16009 raft_consensus.cc:1240] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 5 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:49.130360 18376 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:49.112744 16561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 580) took 1685 ms. Trace:
I20260709 13:05:49.174860 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:49.126390 18414 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:49.146664 16566 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:05:49.102089 18394 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:49.176213 15862 raft_consensus.cc:2393] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 5: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:49.086606 18363 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:49.176925 17993 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:49.172874 15827 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:49.127784 16002 raft_consensus.cc:2376] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5. Re-sending same reply.
I20260709 13:05:49.126695 16563 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:49.187498 16566 raft_consensus.cc:1275] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 6 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:05:49.137457 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:49.193120 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:49.137290 18234 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:49.198705 16566 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 LEARNER]: Advancing to term 6
I20260709 13:05:49.203285 16535 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:49.116024 18086 consensus_queue.cc:260] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:49.207952 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:49.209110 15860 raft_consensus.cc:2410] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 8 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:05:49.179032 18367 raft_consensus.cc:2804] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:49.127663 17866 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:49.214329 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:49.215333 15937 raft_consensus.cc:2393] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 4: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:49.216327 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:49.215132 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:49.213415 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.148118 18167 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:05:49.225713 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 52963.6us
I20260709 13:05:49.225699 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:49.188639 18216 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:49.148079 18325 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 13:05:49.227284 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:49.147339 18322 consensus_queue.cc:1048] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 13:05:49.226608 15860 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:49.235596 18445 raft_consensus.cc:2804] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:49.224911 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:49.254369 15934 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:49.243207 16566 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 LEARNER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:05:49.238517 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:49.240882 16533 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:49.218760 18367 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:49.212322 16005 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:49.219214 18375 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.267464 15860 raft_consensus.cc:1240] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 4. Current term is 5. Ops: []
W20260709 13:05:49.268842 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 96451.8us
I20260709 13:05:49.209877 18323 consensus_queue.cc:1048] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:05:49.238534 18376 raft_consensus.cc:2804] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:49.224532 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:49.257697 16533 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:49.238305 18086 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:49.228196 18297 consensus_queue.cc:1175] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "85e997ba5003439781ec4f1f8db0c3ee" responder_term: 4 status { last_received { term: 4 index: 6 } last_committed_idx: 6 last_received_current_leader { term: 4 index: 6 } }
I20260709 13:05:49.259042 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:49.252571 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:49.294166 15862 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:49.273578 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:49.273929 18374 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.150022 18381 consensus_queue.cc:260] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:49.274915 15854 raft_consensus.cc:1275] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:49.266450 18256 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:49.265779 18367 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:49.175967 16561 rpcz_store.cc:276] 0709 13:05:47.427033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:47.783507 (+356474us) service_pool.cc:224] Handling call
0709 13:05:47.817060 (+ 33553us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:48.377742 (+560682us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:49.112475 (+734733us) spinlock_profiling.cc:243] Waited 537 ms on lock 0x7b54001018d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:49.112655 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1234263424}
I20260709 13:05:49.252866 18284 consensus_queue.cc:1048] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:05:49.280827 15933 raft_consensus.cc:2410] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:05:49.272320 18373 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:49.281977 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.295761 15928 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:49.303150 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:49.282721 15860 raft_consensus.cc:1275] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:05:49.218019 18234 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:49.296411 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 67684.2us
I20260709 13:05:49.264899 18167 raft_consensus.cc:2749] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:49.281378 16569 raft_consensus.cc:1217] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Deduplicated request from leader. Original: 2.5->[3.6-3.6]   Dedup: 3.6->[]
I20260709 13:05:49.320811 18381 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:49.322325 16569 raft_consensus.cc:1240] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:05:49.293543 18322 raft_consensus.cc:2804] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:49.348384 18322 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:49.348822 18322 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:49.283234 18392 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:49.372282 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 51415.9us
I20260709 13:05:49.373557 15827 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:49.375195 15827 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:05:49.377120 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.297216 18376 raft_consensus.cc:697] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:49.385227 15937 raft_consensus.cc:3055] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:49.387920 15937 raft_consensus.cc:740] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:49.388904 15937 consensus_queue.cc:260] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:49.390298 15937 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:49.391220 16003 raft_consensus.cc:1275] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:05:49.391760 16003 pending_rounds.cc:85] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee: Aborting all ops after (but not including) 5
I20260709 13:05:49.392061 16003 pending_rounds.cc:107] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:05:49.392350 16003 raft_consensus.cc:2889] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:49.385474 15827 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:05:49.393666 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.394515 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:49.271978 18375 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:49.299721 18363 consensus_queue.cc:1059] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.084s
W20260709 13:05:49.406064 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.402041 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:49.407451 15935 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
W20260709 13:05:49.403623 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:49.409274 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 147752us
I20260709 13:05:49.304934 15929 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
I20260709 13:05:49.409924 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:49.371692 18290 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.412066 15827 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:49.352881 18373 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:49.299789 18399 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:49.301743 18374 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:49.415462 16003 raft_consensus.cc:3055] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:49.414310 15935 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:49.386327 18372 raft_consensus.cc:2749] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:49.419513 16003 raft_consensus.cc:740] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:49.391462 18444 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.420305 16003 consensus_queue.cc:260] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:49.423633 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 128268us
I20260709 13:05:49.425031 16533 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:49.425832 18452 raft_consensus.cc:2804] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:49.425957 16533 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:49.426323 18452 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:49.428496 16533 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:49.404852 18445 raft_consensus.cc:697] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 7 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:49.381058 18392 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:49.411384 18258 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:49.415047 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:49.387108 18404 consensus_queue.cc:1048] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:05:49.426715 18452 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:49.474195 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:49.474611 15902 connection.cc:256] RPC call timeout handler was delayed by -1.2056332029999339s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1519s for a response
W20260709 13:05:49.480034 15902 connection.cc:256] RPC call timeout handler was delayed by -1.2056332029999339s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1519s for a response
W20260709 13:05:49.480331 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:49.480388 15902 connection.cc:256] RPC call timeout handler was delayed by -1.0616332029999285s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16790000000000002s for a response
W20260709 13:05:49.480904 15902 connection.cc:256] RPC call timeout handler was delayed by -1.05005590199994s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.248s for a response
I20260709 13:05:49.481622 15902 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:49.426739 18284 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:49.488987 15937 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260709 13:05:49.429271 17993 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.489514 15937 pending_rounds.cc:85] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 4
I20260709 13:05:49.500901 15937 pending_rounds.cc:107] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260709 13:05:49.501288 15937 raft_consensus.cc:2889] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:49.503189 15937 raft_consensus.cc:1275] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:05:49.503633 15937 pending_rounds.cc:85] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 1
I20260709 13:05:49.504050 15937 pending_rounds.cc:107] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:05:49.504330 15937 raft_consensus.cc:2889] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:49.496713 15935 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5.
I20260709 13:05:49.472352 18194 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.473328 18258 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:49.418951 18290 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:49.506686 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 78235.2us
I20260709 13:05:49.417199 18446 consensus_queue.cc:1059] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 13:05:49.507817 15935 raft_consensus.cc:1275] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:05:49.478188 18241 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:05:49.519448 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 44602.9us
I20260709 13:05:49.486819 18208 raft_consensus.cc:2749] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:49.520702 16533 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:49.486548 18284 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:49.516652 16002 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:49.525797 18293 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.506125 18450 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:49.518339 18258 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:49.518416 18394 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:49.521690 18363 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:05:49.537317 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.519620 17866 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:49.527444 15854 raft_consensus.cc:2393] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 7: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:49.534813 18284 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:49.543977 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 60490us
I20260709 13:05:49.532495 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:49.513607 17993 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:49.519937 18194 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:49.488708 18376 consensus_queue.cc:237] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:49.542019 18446 raft_consensus.cc:3055] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:49.564455 15902 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:49.527268 18241 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:49.547627 18447 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:49.571631 18194 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:49.545215 18450 raft_consensus.cc:697] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 6 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:49.547771 18455 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:49.589701 15902 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:49.541127 18293 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:49.590634 15900 leader_election.cc:336] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.803s (SENT)
W20260709 13:05:49.594316 15900 leader_election.cc:336] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.814s (SENT)
I20260709 13:05:49.594875 15900 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:05:49.598603 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 60826.8us
I20260709 13:05:49.572504 18446 raft_consensus.cc:740] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:49.601593 15929 raft_consensus.cc:2393] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 7: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:49.587982 18447 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:49.420920 18444 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:05:49.616551 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.594121 18375 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:49.639221 18381 raft_consensus.cc:2749] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:49.536177 18445 consensus_queue.cc:237] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:49.622742 18208 raft_consensus.cc:2804] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:49.575297 18458 consensus_queue.cc:1048] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 13:05:49.605845 18457 raft_consensus.cc:2749] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:49.675876 15859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 1777) took 1345 ms. Trace:
W20260709 13:05:49.676729 18219 consensus_meta.cc:287] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.599s	user 0.008s	sys 0.002s
W20260709 13:05:49.676993 16565 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45244 (request call id 725) took 1379 ms (client timeout 1543 ms). Trace:
I20260709 13:05:49.627996 18447 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:49.682287 15859 rpcz_store.cc:276] 0709 13:05:48.330219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:48.330357 (+   138us) service_pool.cc:224] Handling call
0709 13:05:48.333706 (+  3349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:48.419959 (+ 86253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:49.672791 (+1252832us) spinlock_profiling.cc:243] Waited 54.7 ms on lock 0x7b540008e654. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:49.675806 (+  3015us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":125815680}
W20260709 13:05:49.682574 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 82355.9us
I20260709 13:05:49.665810 18374 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:49.665427 18392 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:49.683846 16565 rpcz_store.cc:269] 0709 13:05:48.297900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:48.298067 (+   167us) service_pool.cc:224] Handling call
0709 13:05:49.676841 (+1378774us) spinlock_profiling.cc:243] Waited 61 ms on lock 0x7b5400724554. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:49.676945 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":140221696}
I20260709 13:05:49.686697 16533 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:49.670289 18399 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:49.670095 18401 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:05:49.690814 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 278840us
I20260709 13:05:49.682564 18208 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:49.685660 18373 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:49.696372 16011 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:49.679921 18322 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:49.690604 18375 raft_consensus.cc:2764] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:05:49.683625 18367 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:49.723489 18462 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.724315 16005 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5.
I20260709 13:05:49.724924 18367 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:49.725893 16005 raft_consensus.cc:1240] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:05:49.711540 17993 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:49.716470 18086 consensus_queue.cc:1059] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:49.717684 16008 raft_consensus.cc:2410] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:05:49.685477 18446 consensus_queue.cc:260] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:49.693418 18290 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:49.686422 18450 consensus_queue.cc:237] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:49.729327 18234 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:49.730033 16564 raft_consensus.cc:1240] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:49.716059 18322 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:49.724079 18462 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:49.727438 17993 raft_consensus.cc:3055] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:49.722620 18444 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:49.706283 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 111988us
I20260709 13:05:49.707875 18208 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:49.758957 15900 leader_election.cc:336] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.869s (SENT)
I20260709 13:05:49.759394 15900 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:05:49.739563 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 206844us
I20260709 13:05:49.760113 15827 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:05:49.761148 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.690852 18241 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:49.730392 18284 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:49.723366 18373 consensus_queue.cc:1048] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:05:49.765980 18468 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
W20260709 13:05:49.722697 16563 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45244 (request call id 720) took 2364 ms (client timeout 2227 ms). Trace:
I20260709 13:05:49.769676 18452 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:49.744149 17993 raft_consensus.cc:740] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:49.717680 18293 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:49.772360 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 80032.4us
I20260709 13:05:49.763165 18471 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:05:49.805790 16533 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:05:49.806687 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.694375 18401 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:49.798048 18464 raft_consensus.cc:2804] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:49.793138 18145 consensus_queue.cc:1059] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:05:49.813510 18464 raft_consensus.cc:697] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:49.813963 18145 consensus_queue.cc:1243] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Peer 8430db80a39d4868bcf7641c16de35ea log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:05:49.814240 18258 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:49.814488 18464 consensus_queue.cc:237] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:49.806501 18445 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.009s
I20260709 13:05:49.797739 17993 consensus_queue.cc:260] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:49.791250 18444 consensus_queue.cc:1048] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
W20260709 13:05:49.784003 16563 rpcz_store.cc:269] 0709 13:05:47.358034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:47.367886 (+  9852us) service_pool.cc:224] Handling call
0709 13:05:48.892545 (+1524659us) spinlock_profiling.cc:243] Waited 4.71 ms on lock 0x7b5800044798. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283eee 0000000000375939 00007f73ef2f71e0 00007f73ef3c00c4 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:05:49.722499 (+829954us) spinlock_profiling.cc:243] Waited 22.6 ms on lock 0x7b5400500354. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:49.722626 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":62759936}
I20260709 13:05:49.822956 17993 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:49.824357 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 234703us
I20260709 13:05:49.800575 18167 raft_consensus.cc:3055] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:49.825078 18181 consensus_queue.cc:1048] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:49.827903 18293 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:49.830139 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:49.831383 15937 raft_consensus.cc:1275] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:49.783591 18241 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:49.824743 18376 raft_consensus.cc:2749] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:49.835889 15902 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:49.836493 18462 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:49.798632 18452 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:49.837450 18167 raft_consensus.cc:740] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:05:49.837606 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 77583.3us
I20260709 13:05:49.857273 15902 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:05:49.843798 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 68939.3us
I20260709 13:05:49.840806 16533 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:49.799983 18468 raft_consensus.cc:2955] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 4 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:49.854903 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:49.859071 15902 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:49.860935 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:49.861709 15937 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:49.855510 18241 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:49.855748 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:49.866863 15902 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:49.856671 18322 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.856974 18471 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:49.867441 15934 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:49.872678 16533 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:49.866340 18376 raft_consensus.cc:2804] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:49.878221 18394 raft_consensus.cc:2804] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:49.878773 18452 raft_consensus.cc:2804] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:49.880539 15937 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:05:49.871354 18381 raft_consensus.cc:2804] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:49.885232 16533 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:49.871611 18469 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:49.881830 18376 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:49.871939 18322 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:49.883224 18452 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:49.885458 18381 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:49.875691 18474 consensus_queue.cc:1048] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:05:49.890470 18461 raft_consensus.cc:2804] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:49.888090 18258 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:49.885962 18394 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:49.882227 18462 consensus_queue.cc:1048] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 13:05:49.897908 18376 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:49.896185 18452 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:49.900718 18381 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:49.912927 18394 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:49.977049 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 117018us
I20260709 13:05:49.876912 15827 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:05:49.979318 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:49.981135 15900 leader_election.cc:336] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 2.064s (SENT)
W20260709 13:05:49.981864 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 104909us
W20260709 13:05:49.985075 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 118169us
I20260709 13:05:49.986285 15902 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:49.979393 15825 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:49.989732 15902 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:05:49.998201 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.999491 15902 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:50.000777 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:50.002344 15937 raft_consensus.cc:2410] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:05:50.002799 15902 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:49.997027 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:50.004117 15934 raft_consensus.cc:2376] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6. Re-sending same reply.
I20260709 13:05:50.005148 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:50.008379 15827 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:50.017259 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:50.021013 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:50.021996 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:50.022279 15859 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:50.023068 15937 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:50.027145 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.031802 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:49.887495 18167 consensus_queue.cc:260] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:50.040846 18241 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:50.041728 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:50.042855 15932 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:05:50.040875 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:05:50.066290 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.077059 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:50.041790 18241 raft_consensus.cc:2804] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:05:50.090152 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 77093.8us
I20260709 13:05:50.084689 18241 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:49.916512 18293 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.104331 18471 raft_consensus.cc:697] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:50.110383 18474 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:49.928543 18447 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:50.111433 18474 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:49.916709 18284 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:50.113637 18284 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.114116 18447 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:50.114310 18284 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:50.114588 18367 consensus_queue.cc:1059] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 13:05:50.126199 16003 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:50.130517 16565 raft_consensus.cc:1240] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:50.131268 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:50.131929 18460 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:50.132134 18246 raft_consensus.cc:2749] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:50.132488 16009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1771) took 2700 ms. Trace:
I20260709 13:05:50.132684 18460 raft_consensus.cc:697] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:50.132993 15902 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:50.132992 16009 rpcz_store.cc:276] 0709 13:05:47.432332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:47.770022 (+337690us) service_pool.cc:224] Handling call
0709 13:05:48.897977 (+1127955us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:49.147217 (+249240us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:49.388393 (+241176us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b54003ed854. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:50.132062 (+743669us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54003ed850. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:50.132259 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3157679232}
I20260709 13:05:50.136124 16533 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:05:50.137717 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 47826.6us
I20260709 13:05:50.103543 18241 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:50.115116 18447 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:50.101099 18258 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:50.027231 18234 consensus_queue.cc:1048] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.112s
I20260709 13:05:50.171926 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:50.106607 18458 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:05:50.179003 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 46191.9us
I20260709 13:05:50.114159 18474 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:05:50.185097 15900 leader_election.cc:336] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 2.249s (SENT)
I20260709 13:05:50.196681 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:50.197649 16565 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:50.197885 15827 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:05:50.200764 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.202207 18471 consensus_queue.cc:237] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:50.202554 18394 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:50.203814 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.205499 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:50.209235 15902 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:50.230516 18485 raft_consensus.cc:2749] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:05:50.243050 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 139058us
I20260709 13:05:50.243616 16568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 587) took 1392 ms. Trace:
I20260709 13:05:50.244167 16568 rpcz_store.cc:276] 0709 13:05:48.850644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:48.850782 (+   138us) service_pool.cc:224] Handling call
0709 13:05:49.677354 (+826572us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:49.677549 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:49.677572 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:49.677614 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:49.677643 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:49.677701 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:50.243182 (+565481us) spinlock_profiling.cc:243] Waited 826 ms on lock 0x7b5400724550. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:50.243425 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1900254976}
I20260709 13:05:50.246186 18480 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:50.269917 18489 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:50.269796 18490 consensus_queue.cc:1059] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:50.244192 15827 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:50.271874 15827 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:50.272789 18497 raft_consensus.cc:3055] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:50.273114 18496 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:50.273252 18497 raft_consensus.cc:740] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:50.273531 15827 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:50.273721 18496 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:50.270581 18489 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:50.271962 18393 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.274199 18496 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:50.278618 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 96827.1us
I20260709 13:05:50.218396 18461 raft_consensus.cc:697] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:50.221760 18455 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.209159 18293 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:50.290885 15937 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5.
I20260709 13:05:50.307353 16565 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:05:50.278242 18489 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:50.230757 18475 consensus_queue.cc:1048] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.167s
I20260709 13:05:50.318287 18393 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:50.319056 18322 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:50.319253 18475 raft_consensus.cc:2804] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:50.319782 18464 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:50.319880 18475 raft_consensus.cc:697] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 4 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:50.320415 18464 raft_consensus.cc:697] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 6 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:50.321100 18393 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:50.321303 18475 consensus_queue.cc:237] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:50.321635 18464 consensus_queue.cc:237] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:50.324298 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:50.324148 18461 consensus_queue.cc:237] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:50.324817 18394 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:50.325310 15855 raft_consensus.cc:2393] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 5: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:05:50.327503 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:50.328333 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.332437 18460 consensus_queue.cc:237] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:50.317996 18455 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:50.223435 18458 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:50.286836 18469 raft_consensus.cc:697] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:50.372107 18393 consensus_queue.cc:1048] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:50.360920 18234 raft_consensus.cc:2749] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:50.376713 18458 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:50.378427 18399 consensus_queue.cc:1048] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:05:50.381451 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.411312 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:05:50.412317 18208 consensus_meta.cc:287] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.647s	user 0.018s	sys 0.003s
I20260709 13:05:50.381645 18381 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:50.301409 18457 consensus_queue.cc:1048] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.120s
I20260709 13:05:50.452634 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:50.437613 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:50.360666 18488 raft_consensus.cc:2804] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:50.465807 16003 raft_consensus.cc:1275] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:50.278472 18497 consensus_queue.cc:260] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:50.378471 16563 raft_consensus.cc:1275] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
W20260709 13:05:50.497699 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 224319us
I20260709 13:05:50.498175 18241 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:50.498966 15827 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:50.503237 18469 consensus_queue.cc:237] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:50.504550 18156 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.505363 18156 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:05:50.515749 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 317158us
I20260709 13:05:50.517259 18455 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:50.479929 18208 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:50.520272 18497 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:50.520903 18293 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:50.524108 18491 log.cc:927] Time spent T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.292s	user 0.000s	sys 0.001s
I20260709 13:05:50.524781 18208 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:50.523339 18381 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:50.498009 18393 consensus_queue.cc:1048] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
W20260709 13:05:50.534368 18308 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 13:05:50.536417 16003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 441) took 1215 ms. Trace:
I20260709 13:05:50.508865 18156 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:50.536825 16003 rpcz_store.cc:276] 0709 13:05:49.333040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:49.415066 (+ 82026us) service_pool.cc:224] Handling call
0709 13:05:49.415352 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:50.125608 (+710256us) spinlock_profiling.cc:243] Waited 14.9 ms on lock 0x7b5800671cd0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3040aa 00007f73ea386447 00007f73ea39efe7 00007f73ea3a07de 00007f73ea3aab49 00007f73ea3ab416 00007f73ea3a7d8f 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4
0709 13:05:50.482848 (+357240us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:50.528161 (+ 45313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34324224}
I20260709 13:05:50.412724 18507 consensus_queue.cc:1048] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.128s
W20260709 13:05:50.534409 18376 consensus_meta.cc:287] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.617s	user 0.011s	sys 0.002s
I20260709 13:05:50.554255 18316 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:50.554538 18393 consensus_queue.cc:1048] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:50.555814 18291 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:05:50.556910 17993 consensus_meta.cc:287] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.721s	user 0.008s	sys 0.007s
I20260709 13:05:50.556572 18291 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:50.532099 18488 raft_consensus.cc:697] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:50.562592 15862 raft_consensus.cc:2393] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 5: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:50.565588 18376 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:50.585984 15855 raft_consensus.cc:1275] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:05:50.582958 18316 raft_consensus.cc:697] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:50.590636 18316 consensus_queue.cc:237] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:05:50.592388 18492 log.cc:927] Time spent T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:05:50.561698 18291 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:50.566322 18381 consensus_queue.cc:1059] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:05:50.608906 18509 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:50.599828 18241 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:50.620924 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 122132us
I20260709 13:05:50.579530 18363 consensus_queue.cc:1059] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
W20260709 13:05:50.624023 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 111695us
I20260709 13:05:50.625506 18509 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:50.637526 16006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 439) took 1288 ms. Trace:
I20260709 13:05:50.638777 18509 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:50.639544 18508 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.640159 18508 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:50.638087 16006 rpcz_store.cc:276] 0709 13:05:49.305903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:49.387016 (+ 81113us) service_pool.cc:224] Handling call
0709 13:05:49.387393 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:49.394204 (+  6811us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:49.394247 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:50.129516 (+735269us) spinlock_profiling.cc:243] Waited 2.7 ms on lock 0x7b2c000e5768. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e8ca1a11 00007f73e8cb1372 00007f73ea3e3a01 00007f73ea3a843a 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2
0709 13:05:50.130302 (+   786us) log.cc:844] Serialized 26 byte log entry
0709 13:05:50.172851 (+ 42549us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:50.222922 (+ 50071us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:50.289227 (+ 66305us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:50.583077 (+293850us) raft_consensus.cc:1685] finished
0709 13:05:50.583105 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:50.591934 (+  8829us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6865,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":9538,"spinlock_wait_cycles":6220032,"thread_start_us":34416,"threads_started":1,"wal-append.queue_time_us":449367}
W20260709 13:05:50.641925 18452 consensus_meta.cc:287] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.723s	user 0.011s	sys 0.000s
I20260709 13:05:50.642611 18508 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:50.642386 18452 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:50.643764 16533 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:50.626921 18376 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:50.657860 16564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 635) took 1623 ms. Trace:
I20260709 13:05:50.658430 16564 rpcz_store.cc:276] 0709 13:05:49.034679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:49.034847 (+   168us) service_pool.cc:224] Handling call
0709 13:05:49.729845 (+694998us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:49.730436 (+   591us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:50.657464 (+927028us) spinlock_profiling.cc:243] Waited 692 ms on lock 0x7b5400500350. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:50.657689 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1591660800}
I20260709 13:05:50.660987 15825 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:50.662281 18234 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:50.662920 18234 raft_consensus.cc:697] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:50.666620 16569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 637) took 1629 ms. Trace:
I20260709 13:05:50.667145 16569 rpcz_store.cc:276] 0709 13:05:49.036828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:49.067287 (+ 30459us) service_pool.cc:224] Handling call
0709 13:05:49.281194 (+213907us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:49.323759 (+ 42565us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:50.666108 (+1342349us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b54006a22d0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:50.666418 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":491284224}
I20260709 13:05:50.664276 18234 consensus_queue.cc:237] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:50.673681 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.592391 18488 consensus_queue.cc:237] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:50.706705 18376 raft_consensus.cc:2749] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:50.707649 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60156 (request call id 438) took 1492 ms. Trace:
W20260709 13:05:50.707238 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 63627.4us
I20260709 13:05:50.708137 16002 rpcz_store.cc:276] 0709 13:05:49.214689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:49.241463 (+ 26774us) service_pool.cc:224] Handling call
0709 13:05:50.707231 (+1465768us) spinlock_profiling.cc:243] Waited 207 ms on lock 0x7b54000aa5d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:50.707421 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":476900736}
I20260709 13:05:50.646734 18496 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:50.720340 18291 consensus_queue.cc:1048] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:05:50.726737 18494 log.cc:927] Time spent T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.426s	user 0.003s	sys 0.000s
I20260709 13:05:50.721047 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:50.728173 15860 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:50.662925 18177 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:50.729877 15933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53636 (request call id 477) took 1537 ms. Trace:
I20260709 13:05:50.730381 15933 rpcz_store.cc:276] 0709 13:05:49.192672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:49.192822 (+   150us) service_pool.cc:224] Handling call
0709 13:05:50.729417 (+1536595us) spinlock_profiling.cc:243] Waited 77.8 ms on lock 0x7b5400197b54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:50.729675 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":178953600}
I20260709 13:05:50.714545 18363 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:50.731704 16535 leader_election.cc:336] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 1.513s (SENT)
I20260709 13:05:50.732283 16535 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:50.732441 18363 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:50.733819 18446 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:50.742682 18375 raft_consensus.cc:2749] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:50.725728 18496 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:50.744431 15933 raft_consensus.cc:1240] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:50.747197 15933 raft_consensus.cc:1275] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:05:50.752282 15933 raft_consensus.cc:1275] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:05:50.735316 18363 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:50.756739 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:50.758064 18381 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.759482 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:05:50.760349 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.763543 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:50.764436 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:50.766132 16533 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:50.766489 18249 consensus_queue.cc:1059] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:50.767833 18145 consensus_queue.cc:1048] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:05:50.774557 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.775749 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:50.776136 18381 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:50.784103 18342 consensus_queue.cc:1048] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 13:05:50.785933 18381 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:50.787004 18452 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:50.782696 15933 raft_consensus.cc:2410] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:05:50.779731 16562 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
W20260709 13:05:50.789134 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.714694 18489 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:50.783623 15930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53636 (request call id 481) took 1583 ms. Trace:
I20260709 13:05:50.783846 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:50.791663 15930 rpcz_store.cc:276] 0709 13:05:49.200572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:49.200721 (+   149us) service_pool.cc:224] Handling call
0709 13:05:50.759103 (+1558382us) spinlock_profiling.cc:243] Waited 6.04 ms on lock 0x7b5800003698. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:05:50.783568 (+ 24465us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13888768}
I20260709 13:05:50.793224 15860 raft_consensus.cc:1275] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:05:50.793407 15930 raft_consensus.cc:1240] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 8. Current term is 9. Ops: [8.10-8.10]
I20260709 13:05:50.794446 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:50.794736 18506 raft_consensus.cc:3055] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:50.795167 18506 raft_consensus.cc:740] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:50.797037 16533 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:50.792064 18145 consensus_queue.cc:1243] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer 8430db80a39d4868bcf7641c16de35ea log is divergent from this leader: its last log entry 6.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:05:50.838541 18485 consensus_queue.cc:1059] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 6.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 13:05:50.807806 18452 raft_consensus.cc:2804] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:05:50.840209 15854 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.018s	sys 0.022s
I20260709 13:05:50.841375 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:50.837707 18363 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:05:50.843648 15854 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.018s	sys 0.022s
I20260709 13:05:50.842607 18452 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:50.845183 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:50.796121 18506 consensus_queue.cc:260] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:50.862954 15931 raft_consensus.cc:1240] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 4. Current term is 5. Ops: []
W20260709 13:05:50.885515 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 96994.9us
W20260709 13:05:50.862970 18468 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.015s	sys 0.009s
I20260709 13:05:50.910465 18452 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:50.913421 16562 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:50.916639 15900 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.583s (SENT) [suppressed 28 similar messages]
I20260709 13:05:50.919905 15933 pending_rounds.cc:85] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 7
I20260709 13:05:50.914599 18496 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:50.918380 18468 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.017s	sys 0.015s
W20260709 13:05:50.935731 18475 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.009s	sys 0.018s
W20260709 13:05:50.945501 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 476784us
I20260709 13:05:50.936784 15933 pending_rounds.cc:107] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 6.8
W20260709 13:05:50.951944 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:50.953008 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:50.953809 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.952603 15933 raft_consensus.cc:2889] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:50.952816 16533 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:05:50.976627 16535 leader_election.cc:336] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 1.732s (SENT)
W20260709 13:05:50.952468 18475 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.010s	sys 0.018s
W20260709 13:05:50.971217 15828 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.766s	user 0.008s	sys 0.038s
W20260709 13:05:50.980191 15828 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.775s	user 0.008s	sys 0.039s
I20260709 13:05:50.982049 16535 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:50.946285 18496 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:50.985661 18496 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:50.988161 16561 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.752s	user 0.015s	sys 0.019s
W20260709 13:05:50.988655 16561 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.850s	user 0.015s	sys 0.020s
I20260709 13:05:50.989439 18514 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.989662 15935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 486) took 1561 ms. Trace:
I20260709 13:05:50.990069 18514 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:50.990175 15935 rpcz_store.cc:276] 0709 13:05:49.428119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:49.524113 (+ 95994us) service_pool.cc:224] Handling call
0709 13:05:50.591232 (+1067119us) spinlock_profiling.cc:243] Waited 2.44 ms on lock 0x7b5800003698. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef381fc0 00007f73ef36c929 00007f73ef36c3d2 00007f73e88e3bd4
0709 13:05:50.591612 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:50.591980 (+   368us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:05:50.592327 (+   347us) log.cc:844] Serialized 14 byte log entry
0709 13:05:50.983758 (+391431us) log.cc:844] Serialized 14 byte log entry
0709 13:05:50.989181 (+  5423us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:50.989227 (+    46us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:50.989267 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:50.989343 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:50.989492 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5227,"prepare.queue_time_us":2815,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":82,"raft.queue_time_us":3844,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":89,"spinlock_wait_cycles":5618048,"thread_start_us":387316,"threads_started":1}
I20260709 13:05:50.991794 18474 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:50.977555 18489 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:50.988099 18373 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.994046 18373 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:50.996227 18373 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:50.994215 18474 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:50.955555 18506 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:51.001750 18363 raft_consensus.cc:2749] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:50.990482 18461 raft_consensus.cc:3055] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:51.006493 16570 raft_consensus.cc:1240] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 3. Current term is 4. Ops: []
W20260709 13:05:51.008747 15900 leader_election.cc:336] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.679s (SENT)
I20260709 13:05:51.015094 15900 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:51.009676 18461 raft_consensus.cc:740] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:51.001122 18474 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:51.028434 18489 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:51.041029 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:51.043154 15859 raft_consensus.cc:2410] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:05:51.045198 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 98722.2us
I20260709 13:05:51.065198 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:51.019941 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:51.024015 18376 raft_consensus.cc:2749] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:51.041460 18461 consensus_queue.cc:260] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:51.073746 16570 raft_consensus.cc:2393] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 5: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:51.075773 16533 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:05:51.095667 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 133973us
W20260709 13:05:51.097975 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 89266.2us
I20260709 13:05:51.103017 16007 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6.
W20260709 13:05:51.080432 15929 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.007s	sys 0.019s
I20260709 13:05:51.044571 18308 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:05:51.103711 15937 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.014s	sys 0.013s
W20260709 13:05:51.124778 15937 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.014s	sys 0.013s
W20260709 13:05:51.121368 15976 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.012s	sys 0.031s
W20260709 13:05:51.125470 15976 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.012s	sys 0.031s
I20260709 13:05:51.125279 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:05:51.130503 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.103957 18397 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:51.103926 18489 raft_consensus.cc:2749] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:05:51.110249 15929 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.007s	sys 0.020s
I20260709 13:05:51.131203 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:51.103916 18514 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:51.136796 15935 raft_consensus.cc:2410] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:05:51.138391 16568 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:51.103571 18322 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.026s	sys 0.026s
I20260709 13:05:51.139318 18468 raft_consensus.cc:3055] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:51.132771 18399 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:51.147753 18322 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.029s	sys 0.026s
I20260709 13:05:51.141528 18163 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:51.150624 18461 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:51.169970 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:51.149720 18468 raft_consensus.cc:740] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:51.151082 18177 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:51.172487 15935 raft_consensus.cc:1240] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 8. Current term is 9. Ops: [8.10-8.10]
I20260709 13:05:51.158185 18490 raft_consensus.cc:2749] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:05:51.175755 15856 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.003s	sys 0.024s
I20260709 13:05:51.180413 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:05:51.184571 15856 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.824s	user 0.004s	sys 0.024s
I20260709 13:05:51.143805 18397 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:51.199935 15900 leader_election.cc:336] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.889s (SENT)
I20260709 13:05:51.201092 15862 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 7.
W20260709 13:05:51.176491 18464 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.005s	sys 0.037s
I20260709 13:05:51.210729 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:51.194592 18363 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:51.216704 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.194288 18524 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:51.171284 18163 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:51.222976 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:51.223937 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:51.202574 18387 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:51.218691 18464 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.006s	sys 0.037s
I20260709 13:05:51.229935 18216 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:05:51.219512 18517 log.cc:927] Time spent T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.153s	user 0.001s	sys 0.001s
I20260709 13:05:51.213536 18316 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:51.219398 18470 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:05:51.244719 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.182459 18308 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:51.227666 16564 raft_consensus.cc:1275] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
W20260709 13:05:51.252909 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 45243.8us
I20260709 13:05:51.204187 18399 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:51.235119 18376 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:51.226097 18373 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:51.263016 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:51.263756 18522 log.cc:927] Time spent T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:05:51.266010 16003 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:05:51.242849 18234 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:51.228034 18177 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:51.266945 16005 raft_consensus.cc:2410] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 6 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:51.263021 16566 raft_consensus.cc:2955] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:51.257872 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:51.277879 16533 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:51.309340 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:51.276492 18464 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:51.266708 18460 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:51.356645 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.241207 18363 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:51.255501 18387 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:51.373653 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 187392us
W20260709 13:05:51.385854 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 108047us
I20260709 13:05:51.307557 18446 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:51.282886 18500 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:51.389405 16568 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5.
I20260709 13:05:51.397964 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:51.264545 18524 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:51.366689 18291 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:51.401970 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:51.384024 16002 raft_consensus.cc:1275] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:05:51.398717 18322 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:51.277220 18470 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:51.405260 18322 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:51.411403 18322 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:51.411521 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:51.408843 18521 log.cc:927] Time spent T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.269s	user 0.001s	sys 0.000s
I20260709 13:05:51.413587 16002 raft_consensus.cc:1275] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:05:51.414522 15900 leader_election.cc:336] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 2.018s (SENT)
I20260709 13:05:51.414956 15900 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:51.415655 18086 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:51.416275 18086 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:51.416700 18246 raft_consensus.cc:2749] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:51.417400 18387 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:51.418062 18470 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:51.418339 18086 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:51.418557 18446 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:51.420881 18446 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:51.421212 15929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 495) took 1396 ms. Trace:
I20260709 13:05:51.421627 15929 rpcz_store.cc:276] 0709 13:05:50.025145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:50.197542 (+172397us) service_pool.cc:224] Handling call
0709 13:05:50.199750 (+  2208us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:50.298039 (+ 98289us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:50.303170 (+  5131us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:50.377299 (+ 74129us) log.cc:844] Serialized 26 byte log entry
0709 13:05:51.143729 (+766430us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:51.147986 (+  4257us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:51.157925 (+  9939us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:51.421003 (+263078us) raft_consensus.cc:1685] finished
0709 13:05:51.421030 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:51.421155 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6295,"raft.queue_time_us":71015,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":150,"thread_start_us":816039,"threads_started":2,"wal-append.queue_time_us":1041729}
I20260709 13:05:51.405671 16009 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:05:51.404974 18373 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:51.424477 16009 raft_consensus.cc:1275] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:05:51.426600 18373 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:51.428000 18363 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:51.427727 18500 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:51.429909 18308 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:51.430053 18500 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:51.430161 15854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 486) took 1324 ms. Trace:
I20260709 13:05:51.431839 15854 rpcz_store.cc:276] 0709 13:05:50.142652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:50.181212 (+ 38560us) service_pool.cc:224] Handling call
0709 13:05:50.184913 (+  3701us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:50.191789 (+  6876us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:50.191867 (+    78us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:50.245918 (+ 54051us) log.cc:844] Serialized 26 byte log entry
0709 13:05:50.916997 (+671079us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:50.918120 (+  1123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:50.920800 (+  2680us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:51.429926 (+509126us) raft_consensus.cc:1685] finished
0709 13:05:51.429949 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:51.430084 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17912,"raft.queue_time_us":16389,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":17618,"thread_start_us":659825,"threads_started":1,"wal-append.queue_time_us":730410}
I20260709 13:05:51.415099 16002 raft_consensus.cc:1275] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:05:51.434115 18532 log.cc:927] Time spent T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.170s	user 0.000s	sys 0.000s
I20260709 13:05:51.434284 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:51.434806 16006 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:05:51.435055 15854 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 4.
W20260709 13:05:51.435985 18533 log.cc:927] Time spent T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 13:05:51.436419 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:51.437248 15854 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
I20260709 13:05:51.438453 16006 raft_consensus.cc:1275] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:05:51.432920 18524 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:51.441124 18284 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:51.443339 16563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 596) took 1315 ms. Trace:
I20260709 13:05:51.271126 18216 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:05:51.445250 16007 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60156 (request call id 435) took 2233 ms (client timeout 1513 ms). Trace:
W20260709 13:05:51.445353 18538 log.cc:927] Time spent T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
W20260709 13:05:51.445654 16007 rpcz_store.cc:269] 0709 13:05:49.212207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:49.212384 (+   177us) service_pool.cc:224] Handling call
0709 13:05:51.445009 (+2232625us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b5400259ed4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:51.445154 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3541588224}
I20260709 13:05:51.446323 18216 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:05:51.444293 18531 log.cc:927] Time spent T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
I20260709 13:05:51.446740 18460 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:51.432935 18399 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:51.449667 18460 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:51.443853 16563 rpcz_store.cc:276] 0709 13:05:50.128582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:50.128906 (+   324us) service_pool.cc:224] Handling call
0709 13:05:50.129563 (+   657us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:50.495662 (+366099us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:51.443002 (+947340us) spinlock_profiling.cc:243] Waited 236 ms on lock 0x7b5400744854. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:51.443261 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":543604096}
I20260709 13:05:51.446983 16007 raft_consensus.cc:1240] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:51.364404 18376 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:51.453649 18397 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:51.453991 18376 leader_election.cc:290] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:51.455451 16011 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60156 (request call id 437) took 2241 ms (client timeout 1732 ms). Trace:
W20260709 13:05:51.456025 16011 rpcz_store.cc:269] 0709 13:05:49.214358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:49.216040 (+  1682us) service_pool.cc:224] Handling call
0709 13:05:51.455062 (+2239022us) spinlock_profiling.cc:243] Waited 469 ms on lock 0x7b540037b9d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:51.455242 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1079193984}
I20260709 13:05:51.458531 16567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 593) took 2421 ms. Trace:
I20260709 13:05:51.467065 18167 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:51.467252 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.470368 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:51.308122 18316 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:51.441047 18177 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:51.473335 18316 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:51.458927 16567 rpcz_store.cc:276] 0709 13:05:49.037468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:49.146556 (+109088us) service_pool.cc:224] Handling call
0709 13:05:49.185554 (+ 38998us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:49.242209 (+ 56655us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:49.242237 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:49.242670 (+   433us) log.cc:844] Serialized 26 byte log entry
0709 13:05:49.408964 (+166294us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:49.707236 (+298272us) log.cc:844] Serialized 14 byte log entry
0709 13:05:49.731667 (+ 24431us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:50.945030 (+1213363us) spinlock_profiling.cc:243] Waited 7.42 ms on lock 0x7b1800351a60. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72892a6 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419
0709 13:05:51.458055 (+513025us) spinlock_profiling.cc:243] Waited 56.3 ms on lock 0x7b5400695ad4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:51.458106 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:51.458167 (+    61us) raft_consensus.cc:1685] finished
0709 13:05:51.458186 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:51.458333 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24069,"prepare.queue_time_us":302310,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":40,"raft.queue_time_us":24312,"raft.run_cpu_time_us":310,"raft.run_wall_time_us":31769,"spinlock_wait_cycles":146452224,"thread_start_us":416636,"threads_started":2,"wal-append.queue_time_us":163234,"wal-append.run_cpu_time_us":2439,"wal-append.run_wall_time_us":1297591}
W20260709 13:05:51.474236 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.474031 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:51.474749 15857 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:51.475186 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.476209 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:51.476887 16569 raft_consensus.cc:2368] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5.
W20260709 13:05:51.478138 18515 log.cc:927] Time spent T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.379s	user 0.002s	sys 0.000s
I20260709 13:05:51.478328 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:51.480106 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:51.481798 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:51.482571 15857 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:05:51.483134 16563 raft_consensus.cc:1240] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:51.486166 16008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60156 (request call id 436) took 2272 ms. Trace:
I20260709 13:05:51.486523 16008 rpcz_store.cc:276] 0709 13:05:49.214015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:49.214859 (+   844us) service_pool.cc:224] Handling call
0709 13:05:51.485845 (+2270986us) spinlock_profiling.cc:243] Waited 500 ms on lock 0x7b54000ca654. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:51.486007 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1149757312}
I20260709 13:05:51.488618 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:51.489197 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:51.490494 15934 raft_consensus.cc:1275] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:05:51.491619 16563 raft_consensus.cc:1240] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:05:51.433035 15936 raft_consensus.cc:1275] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:05:51.455533 18534 log.cc:927] Time spent T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:05:51.418363 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:51.497004 15932 pending_rounds.cc:85] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 1
I20260709 13:05:51.518054 15932 pending_rounds.cc:107] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:05:51.308601 18468 consensus_queue.cc:260] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:51.518379 15932 raft_consensus.cc:2889] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:51.530288 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:51.531072 15931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1751) took 1148 ms. Trace:
I20260709 13:05:51.531440 15931 rpcz_store.cc:276] 0709 13:05:50.383047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:50.383211 (+   164us) service_pool.cc:224] Handling call
0709 13:05:50.436837 (+ 53626us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:50.907652 (+470815us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:51.530815 (+623163us) spinlock_profiling.cc:243] Waited 389 ms on lock 0x7b54001a68d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:51.531019 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":894595968}
I20260709 13:05:51.532711 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:51.533500 15934 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 5: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:51.530412 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:51.403734 18464 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:51.399865 18234 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:51.423460 16561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 599) took 1293 ms. Trace:
W20260709 13:05:51.756116 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 341600us
W20260709 13:05:51.759685 16570 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.014s	sys 0.010s
W20260709 13:05:51.760126 16570 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.014s	sys 0.010s
I20260709 13:05:51.576854 15936 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 483) took 2375 ms. Trace:
W20260709 13:05:51.614781 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 117778us
I20260709 13:05:51.733911 15937 raft_consensus.cc:1275] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:51.731559 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:51.782243 16565 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
I20260709 13:05:51.786042 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:51.787794 16011 raft_consensus.cc:1240] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:05:51.788231 15976 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:51.788899 18284 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:51.494048 18291 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:51.789505 18284 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:51.790772 18523 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:51.791119 18291 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:51.645911 15856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 485) took 1564 ms. Trace:
W20260709 13:05:51.802109 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.812484 16569 raft_consensus.cc:2410] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:05:51.813040 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:51.813952 15937 raft_consensus.cc:2393] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 5: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:51.695540 15931 raft_consensus.cc:1240] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:51.538260 18474 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:51.527374 18452 consensus_meta.cc:287] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.571s	user 0.007s	sys 0.004s
W20260709 13:05:51.725104 18503 log.cc:927] Time spent T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:05:51.816619 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:05:51.816952 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.817474 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:51.824759 16533 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:51.826448 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:51.812696 15976 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:05:51.826900 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:51.540446 16562 consensus_meta.cc:287] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.575s	user 0.009s	sys 0.001s
I20260709 13:05:51.829761 15937 raft_consensus.cc:1275] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:05:51.761044 15936 rpcz_store.cc:276] 0709 13:05:49.201179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:49.201291 (+   112us) service_pool.cc:224] Handling call
0709 13:05:50.761269 (+1559978us) spinlock_profiling.cc:243] Waited 6.28 ms on lock 0x7b5800003698. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef381fc0 00007f73ef36c929 00007f73ef36c3d2 00007f73e88e3bd4
0709 13:05:50.761566 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:51.494789 (+733223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:51.540025 (+ 45236us) spinlock_profiling.cc:243] Waited 503 ms on lock 0x7b5400328554. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:51.576742 (+ 36717us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1170297472}
I20260709 13:05:51.677160 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:51.791373 18523 raft_consensus.cc:697] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:51.832860 16567 raft_consensus.cc:2393] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 4: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:51.613266 18177 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:51.537065 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:05:51.827428 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:51.838443 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.792610 15856 rpcz_store.cc:276] 0709 13:05:50.085738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:50.098636 (+ 12898us) service_pool.cc:224] Handling call
0709 13:05:50.209753 (+111117us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:50.283577 (+ 73824us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:05:51.191562 (+907985us) log.cc:844] Serialized 14 byte log entry
0709 13:05:51.645454 (+453892us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:51.645525 (+    71us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:51.645585 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:51.645672 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:51.645836 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2927,"prepare.queue_time_us":54644,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":56,"raft.queue_time_us":33522,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":15055,"thread_start_us":1217262,"threads_started":2}
I20260709 13:05:51.524784 18514 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:51.728910 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:51.418057 18387 consensus_queue.cc:1048] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.402s
I20260709 13:05:51.834607 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:51.811913 18397 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:51.841007 16565 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:05:51.842553 15934 raft_consensus.cc:2410] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:05:51.842573 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:51.844019 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:51.845131 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:51.858063 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 46267.5us
I20260709 13:05:51.858397 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:51.539885 18322 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:51.860571 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:51.864877 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:51.865558 16565 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:51.869921 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:51.839913 18376 raft_consensus.cc:2804] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:51.870733 15937 raft_consensus.cc:2393] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 4: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:51.870415 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:51.826097 18452 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:51.879122 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:51.880014 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:51.881855 15931 raft_consensus.cc:1240] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:05:51.886464 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:51.887217 15860 raft_consensus.cc:2410] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:05:51.888559 15859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58214 (request call id 490) took 1508 ms. Trace:
I20260709 13:05:51.875859 16562 raft_consensus.cc:2410] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:05:51.840075 18177 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:51.892237 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:51.890890 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:51.889025 15859 rpcz_store.cc:276] 0709 13:05:50.372959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:50.383206 (+ 10247us) service_pool.cc:224] Handling call
0709 13:05:51.873985 (+1490779us) spinlock_profiling.cc:243] Waited 556 ms on lock 0x7b5400218254. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:51.880849 (+  6864us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1277963136}
I20260709 13:05:51.884495 16563 raft_consensus.cc:2393] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 4: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:51.906824 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:51.907892 15856 raft_consensus.cc:2410] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:05:51.882956 18376 raft_consensus.cc:697] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:51.900861 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:51.903976 15928 raft_consensus.cc:1240] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:51.896349 18493 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:51.876348 18397 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:51.920562 18145 raft_consensus.cc:3055] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:51.881287 18514 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:51.931260 16009 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.014s	sys 0.036s
W20260709 13:05:51.932811 16009 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.014s	sys 0.036s
I20260709 13:05:51.927603 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:51.928416 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:51.885887 18524 consensus_queue.cc:1048] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:05:51.925869 15934 raft_consensus.cc:1275] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:05:51.938376 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:51.935350 15856 raft_consensus.cc:2376] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5. Re-sending same reply.
I20260709 13:05:51.948796 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:51.913085 16562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45246 (request call id 649) took 1067 ms. Trace:
W20260709 13:05:51.932591 18540 log.cc:927] Time spent T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.296s	user 0.000s	sys 0.001s
I20260709 13:05:51.949563 15931 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:05:51.995033 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:51.934194 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:51.995560 15930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1759) took 1206 ms. Trace:
I20260709 13:05:51.996178 15929 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:51.955477 18397 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:51.996397 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:51.930861 18145 raft_consensus.cc:740] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:51.997395 15931 raft_consensus.cc:2393] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 5: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:51.998402 18145 consensus_queue.cc:260] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:51.956066 18376 consensus_queue.cc:237] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:52.003027 15937 raft_consensus.cc:2376] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7. Re-sending same reply.
I20260709 13:05:52.004083 16533 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:52.004307 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:51.919180 18284 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:51.959071 18363 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:52.004938 18291 consensus_queue.cc:1048] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:52.005189 15937 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:05:52.005474 18363 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:52.006434 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:52.007133 15937 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:52.005187 18284 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:51.960319 16003 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:51.961424 18514 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:51.962919 15856 raft_consensus.cc:1240] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:05:51.963644 15860 raft_consensus.cc:1240] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:52.011054 16567 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
I20260709 13:05:52.013751 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:51.995976 15930 rpcz_store.cc:276] 0709 13:05:50.789205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:50.806395 (+ 17190us) service_pool.cc:224] Handling call
0709 13:05:50.842280 (+ 35885us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:50.860008 (+ 17728us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:05:50.890716 (+ 30708us) log.cc:844] Serialized 14 byte log entry
0709 13:05:51.193929 (+303213us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:51.205305 (+ 11376us) log.cc:844] Serialized 26 byte log entry
0709 13:05:51.205466 (+   161us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:51.205508 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:51.205582 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:51.431837 (+226255us) spinlock_profiling.cc:243] Waited 19.4 ms on lock 0x7b1800325ba0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72892a6 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ef2d1aa1 00007f73ef2c8b37 00007f73ef2c89ca 00007f73ef3c026f
0709 13:05:51.433069 (+  1232us) spinlock_profiling.cc:243] Waited 3.51 ms on lock 0x7b4800002dd8. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ef2d1aa1 00007f73ef2c8b37 00007f73ef2c89ca 00007f73ef3c026f 00007f73ef3bff37 00007f73ef3d455c 00007f73ef3d4508 00007f73ef3d4488 00007f73ef3d4443 00007f73ef3d36ec 00007f73ee2fda96
0709 13:05:51.995360 (+562291us) raft_consensus.cc:1685] finished
0709 13:05:51.995385 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:51.995510 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10015,"prepare.queue_time_us":21277,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":75,"raft.queue_time_us":15162,"raft.run_cpu_time_us":3380,"raft.run_wall_time_us":578111,"spinlock_wait_cycles":52744192,"thread_start_us":273142,"threads_started":1,"wal-append.queue_time_us":461254}
I20260709 13:05:52.007663 18363 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:52.012674 18145 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:52.014904 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:52.015403 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:51.968465 18474 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:51.969514 18234 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:52.016250 16570 raft_consensus.cc:2410] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:05:51.757210 16561 rpcz_store.cc:276] 0709 13:05:50.130059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:50.136763 (+  6704us) service_pool.cc:224] Handling call
0709 13:05:50.137233 (+   470us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:50.137880 (+   647us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:50.137914 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:50.138554 (+   640us) log.cc:844] Serialized 26 byte log entry
0709 13:05:50.989043 (+850489us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:50.989088 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:50.989158 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:51.423225 (+434067us) raft_consensus.cc:1685] finished
0709 13:05:51.423245 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:51.423401 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":66746,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":158,"thread_start_us":850020,"threads_started":1,"wal-append.queue_time_us":927474}
I20260709 13:05:51.970643 16564 raft_consensus.cc:1240] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:52.017413 16570 raft_consensus.cc:1275] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:05:51.972551 18322 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:51.953513 16562 rpcz_store.cc:276] 0709 13:05:50.834035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:50.840592 (+  6557us) service_pool.cc:224] Handling call
0709 13:05:51.901790 (+1061198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:51.973802 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:52.019873 16563 raft_consensus.cc:2393] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 5: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:51.973907 18452 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:51.975437 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:52.020509 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:52.021541 15856 raft_consensus.cc:2410] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:52.022367 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:52.022936 15856 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:05:52.023000 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:52.024034 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:52.025607 16005 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:52.025688 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:52.026674 15856 raft_consensus.cc:2410] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:05:51.982499 18523 consensus_queue.cc:237] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:51.927112 18401 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:51.989410 18177 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:51.993350 18464 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:51.954351 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:52.013054 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:05:52.034122 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.036727 16565 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:52.027809 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:52.047973 15859 raft_consensus.cc:1240] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:05:52.034056 16003 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5.
I20260709 13:05:52.055744 15862 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:52.058009 16533 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:52.060655 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:52.064128 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:52.066699 15934 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:52.070000 16533 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:52.072299 18322 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:52.072537 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:52.072806 18322 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:52.073256 18322 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:52.073282 16004 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:52.077740 15935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1766) took 1012 ms. Trace:
I20260709 13:05:52.078116 15935 rpcz_store.cc:276] 0709 13:05:51.065549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.198641 (+133092us) service_pool.cc:224] Handling call
0709 13:05:51.209875 (+ 11234us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:51.210066 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:51.210090 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:51.233164 (+ 23074us) log.cc:844] Serialized 26 byte log entry
0709 13:05:51.839512 (+606348us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:51.839565 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:51.839755 (+   190us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:52.077544 (+237789us) raft_consensus.cc:1685] finished
0709 13:05:52.077561 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:52.077688 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":475106,"threads_started":1,"wal-append.queue_time_us":804315}
I20260709 13:05:52.079710 18493 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:52.069049 18524 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.080868 18524 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:52.082291 18322 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:52.083058 18524 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:52.062026 18401 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.072146 18452 consensus_queue.cc:1048] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:52.083761 15936 raft_consensus.cc:2393] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 6: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:52.085660 16533 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:52.088941 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:52.092559 16567 raft_consensus.cc:1275] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 13:05:52.093843 15900 leader_election.cc:336] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.919s (SENT)
I20260709 13:05:52.094365 15900 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:05:52.097843 16535 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.044s	sys 0.065s
W20260709 13:05:52.089990 18506 consensus_meta.cc:287] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 1.085s	user 0.009s	sys 0.004s
W20260709 13:05:52.102602 16535 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.697s	user 0.044s	sys 0.065s
I20260709 13:05:52.102440 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:52.104439 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:52.106976 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 739560us
I20260709 13:05:52.104483 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:52.102416 18401 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:52.109254 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:52.110800 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:05:52.102140 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 496222us
I20260709 13:05:52.088497 18514 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.110236 16011 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:05:52.111917 15930 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 5: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:52.110435 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:52.109922 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:52.113649 15936 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:05:52.099274 18493 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:52.113504 18514 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:52.118260 16005 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 5: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:52.107345 18490 raft_consensus.cc:2749] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:52.122910 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:52.102516 18524 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.142102 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:52.121831 15928 raft_consensus.cc:3055] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:52.143699 15935 raft_consensus.cc:3055] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:52.148746 15935 raft_consensus.cc:740] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:52.117532 18493 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:52.121028 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:52.122300 18506 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:05:52.150058 15974 leader_election.cc:336] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 3.102s (SENT)
I20260709 13:05:52.152935 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:52.155404 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:52.157518 15936 raft_consensus.cc:1275] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:05:52.154488 15974 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:52.163257 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:52.158830 15935 consensus_queue.cc:260] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:52.164049 15859 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:52.169245 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:52.150780 18493 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:52.148552 15928 raft_consensus.cc:740] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:52.171867 15935 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:52.160651 16002 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:52.174609 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:05:52.176160 18552 log.cc:927] Time spent T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 13:05:52.179873 18468 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:52.181428 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:52.183032 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:52.184145 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:52.184769 16005 raft_consensus.cc:2393] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 5: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:52.185444 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:52.185796 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:52.186045 16005 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 5: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:52.186329 15932 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:52.186858 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:52.187886 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:52.190730 16568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 659) took 1024 ms. Trace:
I20260709 13:05:52.191237 16568 rpcz_store.cc:276] 0709 13:05:51.165874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.430643 (+264769us) service_pool.cc:224] Handling call
0709 13:05:51.430957 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:51.518929 (+ 87972us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:51.518976 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:51.524497 (+  5521us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:51.524553 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:52.190451 (+665898us) spinlock_profiling.cc:243] Waited 64.1 ms on lock 0x7b54004790d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:52.190511 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:52.190669 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":147440512}
I20260709 13:05:52.194991 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:05:52.201879 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.204957 15859 raft_consensus.cc:2393] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 4: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:05:52.220506 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.218868 16533 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:52.227605 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:52.229146 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:52.230587 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:52.231242 15854 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:05:52.232298 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:52.233309 18490 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.233397 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:52.234025 15854 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:52.233843 18490 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:05:52.238646 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.162827 18524 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:52.242851 15932 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6.
I20260709 13:05:52.258522 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:52.261358 15934 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:05:52.243592 15935 raft_consensus.cc:1275] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:05:52.262517 15935 pending_rounds.cc:85] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 2
I20260709 13:05:52.262843 15935 pending_rounds.cc:107] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:05:52.263134 15935 raft_consensus.cc:2889] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:52.244176 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.258883 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:52.259093 16565 raft_consensus.cc:1240] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:52.258986 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:52.265271 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 46581.1us
I20260709 13:05:52.165824 18506 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:52.267012 16533 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:52.260074 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:52.267791 15931 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:52.273551 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:52.273874 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.274262 16005 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:52.275846 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:52.276127 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:52.276288 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:52.276453 16565 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
I20260709 13:05:52.276643 15934 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:52.277122 16567 raft_consensus.cc:2410] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:05:52.277279 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:52.277657 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:52.277987 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:52.279527 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:52.225835 18145 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:52.278288 15859 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:05:52.226276 18177 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:52.284431 18514 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:52.284632 15928 consensus_queue.cc:260] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:52.285885 15928 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:52.288241 18401 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:52.271822 18167 raft_consensus.cc:2749] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:05:52.289539 15974 leader_election.cc:336] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 1.790s (SENT)
I20260709 13:05:52.277014 18474 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:52.290109 15974 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:52.290679 18145 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:52.291296 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:52.292104 18177 raft_consensus.cc:2749] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:52.292666 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:52.292860 16011 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:52.293753 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:52.295231 16003 raft_consensus.cc:1240] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:05:52.300904 18452 consensus_queue.cc:1048] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:52.301222 16535 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:52.302966 18452 raft_consensus.cc:2749] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:52.306116 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:52.312361 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.314980 16005 raft_consensus.cc:2393] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 4: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:52.313547 15855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 496) took 1156 ms. Trace:
I20260709 13:05:51.307582 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:05:52.327659 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:52.328557 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:52.329196 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.334698 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:52.335814 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:52.337201 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:52.337908 16570 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5.
I20260709 13:05:52.339757 16568 raft_consensus.cc:1275] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:05:52.340654 15862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 1651) took 1036 ms. Trace:
I20260709 13:05:52.343055 16567 raft_consensus.cc:1275] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 4. (term mismatch)
I20260709 13:05:52.305116 18322 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:52.314061 18474 raft_consensus.cc:697] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:52.341156 15862 rpcz_store.cc:276] 0709 13:05:51.287038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.369146 (+ 82108us) service_pool.cc:224] Handling call
0709 13:05:52.322988 (+953842us) spinlock_profiling.cc:243] Waited 426 ms on lock 0x7b540008dc54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:52.323116 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":979996416}
I20260709 13:05:52.348982 16565 raft_consensus.cc:1275] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:05:52.343864 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:52.346330 15933 raft_consensus.cc:1275] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:05:52.345038 16569 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:05:52.347959 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:05:52.351742 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.355731 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:05:52.359218 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.319191 18513 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.343453 16567 pending_rounds.cc:85] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5: Aborting all ops after (but not including) 3
I20260709 13:05:52.381438 16567 pending_rounds.cc:107] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5: Aborting uncommitted NO_OP operation due to leader change: 3.4
I20260709 13:05:52.381830 16567 raft_consensus.cc:2889] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:52.318320 15855 rpcz_store.cc:276] 0709 13:05:51.157404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.157551 (+   147us) service_pool.cc:224] Handling call
0709 13:05:51.166742 (+  9191us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:51.210865 (+ 44123us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:51.482666 (+271801us) spinlock_profiling.cc:243] Waited 2.99 ms on lock 0x7b180020c980. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb3f3 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd 000000000038f354 000000000038f199 00007f73e729e83a
0709 13:05:51.620743 (+138077us) log.cc:844] Serialized 14 byte log entry
0709 13:05:52.112108 (+491365us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:52.121412 (+  9304us) log.cc:844] Serialized 26 byte log entry
0709 13:05:52.121661 (+   249us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:52.121708 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:52.121791 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:52.280028 (+158237us) raft_consensus.cc:1685] finished
0709 13:05:52.313360 (+ 33332us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:52.313490 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23754,"prepare.queue_time_us":38896,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":32935,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":9952,"spinlock_wait_cycles":6874496,"thread_start_us":819610,"threads_started":2,"wal-append.queue_time_us":529443}
I20260709 13:05:52.384222 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:05:52.393301 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.316651 18234 consensus_queue.cc:1059] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:05:52.393812 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:52.393445 16535 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:52.394583 18163 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:52.396055 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:52.396827 15862 raft_consensus.cc:2393] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 5: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:52.397658 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:52.398675 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:05:52.400091 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:52.400794 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.401566 15974 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:52.402324 15974 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:05:52.405437 18553 log.cc:927] Time spent T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
I20260709 13:05:52.319341 18284 consensus_queue.cc:1048] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 13:05:52.412075 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:52.413081 16563 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5.
I20260709 13:05:52.416405 18376 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:52.416906 18376 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:52.417304 18376 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:52.418061 16569 raft_consensus.cc:1240] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 13:05:52.419127 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:52.420238 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:52.420886 16569 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:05:52.421902 16569 raft_consensus.cc:1240] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:52.422662 16005 raft_consensus.cc:1275] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:05:52.423010 16005 pending_rounds.cc:85] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting all ops after (but not including) 6
I20260709 13:05:52.423210 16005 pending_rounds.cc:107] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:05:52.423460 16005 raft_consensus.cc:2889] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:52.426096 15857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 1799) took 1003 ms. Trace:
I20260709 13:05:52.426461 15974 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:52.393491 15859 raft_consensus.cc:2410] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:52.395368 16003 raft_consensus.cc:1240] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:05:52.438647 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:52.440516 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:52.426524 15857 rpcz_store.cc:276] 0709 13:05:51.422656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.483375 (+ 60719us) service_pool.cc:224] Handling call
0709 13:05:51.483703 (+   328us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:51.484089 (+   386us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:51.484120 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:51.488579 (+  4459us) log.cc:844] Serialized 52 byte log entry
0709 13:05:51.891300 (+402721us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:05:51.891355 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:51.891442 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:52.425863 (+534421us) raft_consensus.cc:1685] finished
0709 13:05:52.425888 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:52.426034 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1446,"raft.run_cpu_time_us":73,"raft.run_wall_time_us":74,"thread_start_us":402212,"threads_started":1,"wal-append.queue_time_us":659453}
I20260709 13:05:52.442914 15976 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:52.321132 18167 consensus_queue.cc:1048] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:05:52.450497 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:05:52.463696 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 133869us
I20260709 13:05:52.346602 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:05:52.465322 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.356704 18506 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:52.451429 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:52.477136 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:52.481588 15855 raft_consensus.cc:2410] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 9 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 9.
W20260709 13:05:52.478071 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 51734.2us
I20260709 13:05:52.480827 15856 raft_consensus.cc:2376] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4. Re-sending same reply.
I20260709 13:05:52.384593 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:05:52.480552 16010 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.016s	sys 0.028s
W20260709 13:05:52.515341 16010 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.016s	sys 0.028s
W20260709 13:05:52.475664 18461 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.019s	sys 0.023s
I20260709 13:05:52.529541 16535 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:52.530634 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:52.535012 15855 raft_consensus.cc:2410] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 9 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 9.
I20260709 13:05:52.448302 18387 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:52.448159 18397 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:52.537451 16566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 655) took 1489 ms. Trace:
I20260709 13:05:52.544605 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:52.557556 15859 raft_consensus.cc:1240] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 7 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 6. Current term is 7. Ops: [6.8-6.8]
W20260709 13:05:52.541872 18461 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.020s	sys 0.023s
I20260709 13:05:52.521181 18401 raft_consensus.cc:2749] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:52.446262 18513 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:05:52.568979 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 255267us
I20260709 13:05:52.563889 15855 raft_consensus.cc:1275] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:05:52.565038 15856 raft_consensus.cc:1275] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:05:52.571905 15856 pending_rounds.cc:85] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8: Aborting all ops after (but not including) 5
I20260709 13:05:52.572181 15856 pending_rounds.cc:107] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:05:52.572397 15856 raft_consensus.cc:2889] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:52.464910 18177 consensus_queue.cc:1059] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.108s
I20260709 13:05:52.543512 18397 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:52.563210 16535 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:52.543107 18387 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:52.574697 15825 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:52.561124 18322 raft_consensus.cc:2749] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:52.559855 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:52.574726 18506 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.570900 18393 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:52.577142 18397 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:52.578065 18387 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:52.692224 16004 consensus_meta.cc:287] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.618s	user 0.001s	sys 0.013s
I20260709 13:05:52.692695 16004 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:05:52.693358 16004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60156 (request call id 447) took 1507 ms. Trace:
I20260709 13:05:52.693660 16006 raft_consensus.cc:1275] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:05:52.693740 16004 rpcz_store.cc:276] 0709 13:05:51.188180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.220251 (+ 32071us) service_pool.cc:224] Handling call
0709 13:05:52.072251 (+852000us) spinlock_profiling.cc:243] Waited 3.69 ms on lock 0x7b5800114398. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:05:52.693280 (+621029us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8496512}
I20260709 13:05:52.596998 18291 raft_consensus.cc:2749] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:52.622254 18393 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:52.672714 18490 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:52.715387 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 232946us
I20260709 13:05:52.688700 18399 raft_consensus.cc:2749] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:52.748497 15974 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:52.742858 18393 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:52.764446 15974 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:52.577474 18474 consensus_queue.cc:237] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:52.701167 18145 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:52.778093 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.671872 18506 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:52.781634 15853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 1798) took 1181 ms. Trace:
W20260709 13:05:52.782399 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.779019 15902 leader_election.cc:400] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5.
I20260709 13:05:52.791518 15902 leader_election.cc:403] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:05:52.788884 15858 raft_consensus.cc:1240] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 3. Current term is 4. Ops: []
W20260709 13:05:52.764873 18468 consensus_meta.cc:287] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.584s	user 0.013s	sys 0.000s
W20260709 13:05:52.793584 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.743777 18524 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:52.801196 18468 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:52.801664 18468 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:52.802027 18468 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:52.797415 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 222773us
I20260709 13:05:52.791355 15853 rpcz_store.cc:276] 0709 13:05:51.422418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.457884 (+ 35466us) service_pool.cc:224] Handling call
0709 13:05:51.458441 (+   557us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:51.459133 (+   692us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:51.459177 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:51.520864 (+ 61687us) log.cc:844] Serialized 26 byte log entry
0709 13:05:52.128676 (+607812us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:52.128722 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:52.128802 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:52.603499 (+474697us) raft_consensus.cc:1685] finished
0709 13:05:52.603528 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:52.603708 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1169,"raft.run_cpu_time_us":67,"raft.run_wall_time_us":67,"thread_start_us":423971,"threads_started":1,"wal-append.queue_time_us":687559}
I20260709 13:05:52.764114 18490 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:52.809662 16005 raft_consensus.cc:1240] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:52.782689 18514 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:52.782049 18177 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:52.803597 18363 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:05:52.825301 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 382317us
I20260709 13:05:52.826928 15976 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:05:52.816514 18560 log.cc:927] Time spent T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.351s	user 0.001s	sys 0.000s
W20260709 13:05:52.846422 18566 log.cc:927] Time spent T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.240s	user 0.000s	sys 0.001s
I20260709 13:05:52.814870 18514 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:52.846609 18506 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:52.847563 15854 consensus_meta.cc:287] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.613s	user 0.011s	sys 0.003s
I20260709 13:05:52.878062 16010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 454) took 1396 ms. Trace:
W20260709 13:05:52.850023 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 142737us
I20260709 13:05:52.850975 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:52.815059 18490 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:52.852380 18446 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:05:52.854014 15929 consensus_meta.cc:287] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.857s	user 0.011s	sys 0.000s
I20260709 13:05:52.855163 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:52.861428 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:52.862555 15934 consensus_meta.cc:287] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.559s	user 0.010s	sys 0.000s
I20260709 13:05:52.859596 15860 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
I20260709 13:05:52.873965 16009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1691) took 1515 ms. Trace:
I20260709 13:05:52.816090 18577 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5.
W20260709 13:05:52.882042 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.863333 18145 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:05:52.872587 18561 log.cc:927] Time spent T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.318s	user 0.000s	sys 0.001s
I20260709 13:05:52.886310 15934 raft_consensus.cc:2410] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:05:52.885785 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 58958.8us
I20260709 13:05:52.914554 16008 raft_consensus.cc:1240] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 4. Current term is 5. Ops: [1.3-4.4]
W20260709 13:05:52.915760 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.882581 16009 rpcz_store.cc:276] 0709 13:05:51.358488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.425195 (+ 66707us) service_pool.cc:224] Handling call
0709 13:05:51.425532 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:51.426042 (+   510us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:05:51.936366 (+510324us) log.cc:844] Serialized 14 byte log entry
0709 13:05:52.326242 (+389876us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:52.811321 (+485079us) spinlock_profiling.cc:243] Waited 1.09 ms on lock 0x7b2c000e5768. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e8ca1a11 00007f73e8cb1372 00007f73ea3e3a01 00007f73ea3a843a 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2
0709 13:05:52.811722 (+   401us) log.cc:844] Serialized 26 byte log entry
0709 13:05:52.811861 (+   139us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:52.813868 (+  2007us) log.cc:844] Serialized 14 byte log entry
0709 13:05:52.822095 (+  8227us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:52.822178 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:52.873656 (+ 51478us) raft_consensus.cc:1685] finished
0709 13:05:52.873701 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:52.873891 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7968,"prepare.queue_time_us":692266,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":71,"raft.queue_time_us":13048,"raft.run_cpu_time_us":333,"raft.run_wall_time_us":10765,"spinlock_wait_cycles":2503936,"thread_start_us":853601,"threads_started":2,"wal-append.queue_time_us":723028}
I20260709 13:05:52.864250 18513 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:52.875451 18514 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:52.922116 15931 consensus_meta.cc:287] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.643s	user 0.009s	sys 0.000s
I20260709 13:05:52.922722 15931 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6.
I20260709 13:05:52.935806 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:52.899120 18523 raft_consensus.cc:2749] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:52.885200 15929 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:05:52.939239 15902 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:52.942094 15902 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:52.866087 18363 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:52.944957 15902 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:52.945940 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:52.940259 15854 raft_consensus.cc:2410] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:05:52.912328 18446 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:52.924242 18145 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:52.994064 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 184753us
I20260709 13:05:52.946151 18577 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:52.997592 16007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 457) took 1211 ms. Trace:
I20260709 13:05:52.998809 15825 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:53.007596 16007 rpcz_store.cc:276] 0709 13:05:51.786131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.787413 (+  1282us) service_pool.cc:224] Handling call
0709 13:05:51.787768 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:51.788150 (+   382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:51.788179 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:51.788655 (+   476us) log.cc:844] Serialized 26 byte log entry
0709 13:05:52.130178 (+341523us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:52.130245 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:52.130342 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:52.997331 (+866989us) raft_consensus.cc:1685] finished
0709 13:05:52.997352 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:52.997502 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2408,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":115,"thread_start_us":341095,"threads_started":1,"wal-append.queue_time_us":547980}
W20260709 13:05:52.993156 18376 consensus_meta.cc:287] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.575s	user 0.011s	sys 0.000s
I20260709 13:05:52.946581 15929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53636 (request call id 503) took 1064 ms. Trace:
W20260709 13:05:52.993235 18576 log.cc:927] Time spent T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:05:52.932071 18579 consensus_queue.cc:1048] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.068s
W20260709 13:05:53.012960 18572 log.cc:927] Time spent T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:05:53.014158 18490 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:53.015261 15928 consensus_meta.cc:287] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.729s	user 0.011s	sys 0.002s
I20260709 13:05:53.018412 18363 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:52.945609 18590 consensus_queue.cc:1059] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.079s
W20260709 13:05:52.992808 18561 log.cc:927] Time spent T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:05:52.954599 18571 raft_consensus.cc:2804] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:53.036816 16007 raft_consensus.cc:1275] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:05:52.995541 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:52.942723 18581 consensus_queue.cc:1059] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.086s
I20260709 13:05:52.997956 15854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 1671) took 1030 ms. Trace:
W20260709 13:05:53.045960 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.023947 18506 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:53.020004 18549 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:53.007500 18577 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:53.067888 18577 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:53.068536 18468 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:53.070703 18468 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:53.072785 18590 consensus_queue.cc:1048] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:05:53.074371 18564 log.cc:927] Time spent T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.385s	user 0.001s	sys 0.002s
I20260709 13:05:53.075479 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:53.075891 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:53.076711 16561 raft_consensus.cc:2393] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 5: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:53.076601 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:53.077411 15859 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:05:53.077957 16009 raft_consensus.cc:1275] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:05:53.078210 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:53.078874 15859 raft_consensus.cc:2393] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 6: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:53.079239 16009 raft_consensus.cc:1275] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:05:53.079974 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:53.080513 15855 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:53.081302 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:53.082239 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:53.083150 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:53.083231 15858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1653) took 1561 ms. Trace:
I20260709 13:05:53.084146 18574 consensus_queue.cc:1048] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:05:53.086344 18562 log.cc:927] Time spent T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.484s	user 0.000s	sys 0.001s
I20260709 13:05:53.086566 18464 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:53.086822 15827 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:53.087266 18464 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:53.087822 18578 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.088330 18578 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:53.089603 18464 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:53.089735 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:53.090289 18464 consensus_queue.cc:1048] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:53.063933 18549 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:53.067339 16002 consensus_meta.cc:287] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.862s	user 0.014s	sys 0.004s
I20260709 13:05:53.090955 18549 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:53.091409 16002 raft_consensus.cc:2410] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:05:53.062296 15854 rpcz_store.cc:276] 0709 13:05:51.965305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:52.233160 (+267855us) service_pool.cc:224] Handling call
0709 13:05:52.995446 (+762286us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:53.092257 16533 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:53.092811 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:53.093312 16533 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:53.094594 16533 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:53.095127 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:53.095204 15827 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:53.083598 15855 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:53.095791 15934 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
I20260709 13:05:53.079599 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:53.066475 16007 raft_consensus.cc:1275] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:05:53.090492 18513 raft_consensus.cc:2749] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:53.083580 15858 rpcz_store.cc:276] 0709 13:05:51.521805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.732503 (+210698us) service_pool.cc:224] Handling call
0709 13:05:52.603824 (+871321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:52.797298 (+193474us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.083050 (+285752us) spinlock_profiling.cc:243] Waited 812 ms on lock 0x7b540006ed50. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:53.083176 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1867515776}
I20260709 13:05:53.074340 18590 raft_consensus.cc:3055] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:53.099020 15827 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:05:53.101521 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 184898us
W20260709 13:05:53.062000 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 123927us
I20260709 13:05:53.100468 15861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 1801) took 1580 ms. Trace:
I20260709 13:05:53.100589 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:53.096635 16533 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:53.024267 18397 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:53.095715 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:53.020789 18376 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:53.099655 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 173548us
I20260709 13:05:53.105170 15853 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:05:53.028746 15929 rpcz_store.cc:276] 0709 13:05:51.882177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.892766 (+ 10589us) service_pool.cc:224] Handling call
0709 13:05:52.943206 (+1050440us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:53.111552 15931 raft_consensus.cc:1275] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:05:53.109479 15976 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:53.094061 15854 raft_consensus.cc:1275] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:05:53.113101 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:05:53.106567 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 161598us
I20260709 13:05:53.102771 15861 rpcz_store.cc:276] 0709 13:05:51.519862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.520001 (+   139us) service_pool.cc:224] Handling call
0709 13:05:51.535971 (+ 15970us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:51.536272 (+   301us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:51.536302 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:51.757908 (+221606us) log.cc:844] Serialized 26 byte log entry
0709 13:05:52.165393 (+407485us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:52.165450 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:52.165524 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:53.085769 (+920245us) spinlock_profiling.cc:243] Waited 65.2 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:05:53.095183 (+  9414us) raft_consensus.cc:1685] finished
0709 13:05:53.095203 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:53.100414 (+  5211us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":287,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":85,"spinlock_wait_cycles":149975808,"thread_start_us":306527,"threads_started":1,"wal-append.queue_time_us":699371}
I20260709 13:05:53.058914 18571 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:53.118062 15931 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:05:53.118332 18571 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:53.118801 18581 consensus_queue.cc:1048] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:53.119117 15902 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:53.115723 15937 raft_consensus.cc:1240] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:53.120921 18581 raft_consensus.cc:2804] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:53.106397 18468 raft_consensus.cc:2749] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:53.144876 15825 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:05:53.145172 18589 log.cc:927] Time spent T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 13:05:53.145691 18468 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:53.146095 18468 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:53.146446 18468 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:53.100522 18506 raft_consensus.cc:697] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 5 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:05:53.148068 18591 log.cc:927] Time spent T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:05:53.148443 18506 consensus_queue.cc:237] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:53.149124 15929 raft_consensus.cc:1240] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:53.150346 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:53.150949 15929 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
I20260709 13:05:53.153811 18468 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:53.136258 15974 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:53.105366 18461 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:53.156759 18461 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:53.139307 16003 raft_consensus.cc:1240] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:53.157243 18461 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:53.117986 18514 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:53.159062 18514 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:53.058516 18592 consensus_queue.cc:1048] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.077s
I20260709 13:05:53.114203 18582 consensus_queue.cc:1048] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:05:53.160342 15931 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:05:53.142306 18513 consensus_queue.cc:1059] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:05:53.161037 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.107800 18163 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:53.161895 18513 raft_consensus.cc:3055] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:53.162145 18163 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:53.109465 15928 raft_consensus.cc:1275] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:05:53.162377 18513 raft_consensus.cc:740] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:53.163003 15928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 1695) took 1186 ms. Trace:
I20260709 13:05:53.101943 18590 raft_consensus.cc:740] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:53.090282 18578 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:53.108620 18291 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:53.141423 15827 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:53.105777 18363 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:53.161893 16004 raft_consensus.cc:2393] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 8: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
W20260709 13:05:53.175223 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.166494 15827 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:53.162420 18468 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:53.182416 15827 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:53.174607 18363 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:53.111049 18596 consensus_queue.cc:1048] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.067s
I20260709 13:05:53.165459 18460 raft_consensus.cc:3055] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:53.164038 18401 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:53.169301 18387 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:53.183696 18163 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:53.184770 18493 consensus_meta.cc:287] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.986s	user 0.011s	sys 0.000s
I20260709 13:05:53.190486 15855 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:05:53.177100 18514 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:53.163415 15928 rpcz_store.cc:276] 0709 13:05:51.976455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:52.068902 (+ 92447us) service_pool.cc:224] Handling call
0709 13:05:52.111762 (+ 42860us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:53.162744 (+1050982us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.162947 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:53.166802 18588 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:53.165163 18291 raft_consensus.cc:2749] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:53.180574 18582 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:53.206625 18586 raft_consensus.cc:2749] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:53.206574 18579 consensus_queue.cc:1059] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:53.206672 18597 consensus_queue.cc:1059] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:53.207829 18468 consensus_queue.cc:1048] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:53.208591 18581 raft_consensus.cc:697] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:53.210006 15857 raft_consensus.cc:1240] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 6 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 5. Current term is 6. Ops: []
W20260709 13:05:53.210875 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 89884.2us
I20260709 13:05:53.215164 18506 consensus_queue.cc:1048] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:53.207756 15825 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:05:53.293530 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 85884.8us
I20260709 13:05:53.163645 18513 consensus_queue.cc:260] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:53.267179 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 153848us
I20260709 13:05:53.297610 15976 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:05:53.268714 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 63491.8us
I20260709 13:05:53.164552 18605 consensus_queue.cc:1059] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:05:53.198310 18363 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:53.180389 18583 consensus_queue.cc:1048] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.088s
I20260709 13:05:53.183621 18592 consensus_queue.cc:1048] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 13:05:53.178483 18577 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:53.263996 18401 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:53.200802 18460 raft_consensus.cc:740] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:53.185745 18574 consensus_queue.cc:1048] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.089s
I20260709 13:05:53.283866 18616 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.294267 15825 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:53.219267 18474 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.164408 18590 consensus_queue.cc:260] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:53.304495 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 207907us
I20260709 13:05:53.305924 15974 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:05:53.305998 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 98404.1us
I20260709 13:05:53.296535 18578 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:53.291483 18588 raft_consensus.cc:697] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:53.278627 18393 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:53.324239 16565 raft_consensus.cc:1240] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:53.277554 18493 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:53.306203 18291 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.374267 18464 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.322989 18401 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:53.379501 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:53.406656 15930 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:53.302227 18399 consensus_queue.cc:1048] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 13:05:53.321534 18513 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:53.368911 18397 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:53.392311 18613 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:53.370442 18582 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:53.299293 18607 consensus_queue.cc:1059] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.094s
I20260709 13:05:53.420295 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:53.370116 16533 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:53.382913 18578 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:53.409031 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 90762.3us
I20260709 13:05:53.422703 18578 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:53.409937 15902 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:53.397158 18485 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.410235 16011 raft_consensus.cc:2393] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 7: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:53.420954 16565 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:05:53.399350 18376 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:53.369079 18571 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:53.374624 18615 consensus_queue.cc:1059] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:05:53.393653 18597 consensus_queue.cc:1048] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 13:05:53.398643 18474 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:53.389499 18580 consensus_queue.cc:1048] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:05:53.404140 18291 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:53.379213 18616 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:53.456245 18393 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:53.647483 18446 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:53.414870 18387 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:53.423079 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:05:53.456683 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 46847.3us
I20260709 13:05:53.714082 16563 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
W20260709 13:05:53.714429 18625 log.cc:927] Time spent T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:05:53.547446 18506 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:05:53.539778 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.419893 18581 consensus_queue.cc:237] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:52.878711 16010 rpcz_store.cc:276] 0709 13:05:51.481189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.749742 (+268553us) service_pool.cc:224] Handling call
0709 13:05:51.811821 (+ 62079us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:51.841695 (+ 29874us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:05:52.515884 (+674189us) log.cc:844] Serialized 14 byte log entry
0709 13:05:52.877558 (+361674us) log.cc:844] Serialized 14 byte log entry
0709 13:05:52.877711 (+   153us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:52.877751 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:52.877800 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:52.877868 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:52.877999 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26160,"prepare.queue_time_us":271489,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":16649,"thread_start_us":896894,"threads_started":2}
I20260709 13:05:53.409157 18590 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:53.716070 15902 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:53.423399 18613 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:53.505868 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.712867 18588 consensus_queue.cc:237] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:52.543725 16566 rpcz_store.cc:276] 0709 13:05:51.048148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.126214 (+ 78066us) service_pool.cc:224] Handling call
0709 13:05:51.157999 (+ 31785us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:51.189748 (+ 31749us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:05:51.695282 (+505534us) log.cc:844] Serialized 14 byte log entry
0709 13:05:52.521572 (+826290us) log.cc:844] Serialized 14 byte log entry
0709 13:05:52.521853 (+   281us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:52.521913 (+    60us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:52.536586 (+ 14673us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:52.536905 (+   319us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:52.537289 (+   384us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":19032,"raft.run_cpu_time_us":325,"raft.run_wall_time_us":24138,"thread_start_us":754782,"threads_started":2,"wal-append.queue_time_us":628698}
I20260709 13:05:53.689306 18401 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:53.709468 15857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1669) took 1744 ms. Trace:
I20260709 13:05:53.726604 15857 rpcz_store.cc:276] 0709 13:05:51.964663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:52.495694 (+531031us) service_pool.cc:224] Handling call
0709 13:05:52.856927 (+361233us) spinlock_profiling.cc:243] Waited 1.41 ms on lock 0x7b5800005798. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283eee 0000000000375939 00007f73ef2f71e0 00007f73ef3c00c4 00007f73ef3c0131 00007f73ef381fc0 00007f73ef36c929 00007f73ef36c3d2 00007f73e88e3bd4
0709 13:05:52.857078 (+   151us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:53.292752 (+435674us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.709156 (+416404us) spinlock_profiling.cc:243] Waited 353 ms on lock 0x7b54003b5e54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:53.709389 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":814130304}
W20260709 13:05:53.623860 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:53.720696 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 296974us
W20260709 13:05:53.728710 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.552946 16562 raft_consensus.cc:2393] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 6: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:53.740784 15827 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:53.736111 15902 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:05:53.736119 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 313125us
I20260709 13:05:53.408592 18619 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:53.743798 18606 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:53.743942 18630 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:53.645619 18614 log.cc:927] Time spent T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.322s	user 0.000s	sys 0.001s
W20260709 13:05:53.646288 18610 log.cc:927] Time spent T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.247s	user 0.000s	sys 0.000s
I20260709 13:05:53.747104 15936 raft_consensus.cc:1240] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:53.716768 18615 raft_consensus.cc:3055] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:53.716439 18506 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:05:53.681944 18611 log.cc:927] Time spent T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.293s	user 0.000s	sys 0.001s
I20260709 13:05:53.684762 18624 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.819054 15930 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:05:53.818849 18624 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:53.458832 18582 raft_consensus.cc:697] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 6 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:53.821435 18506 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:53.822085 18582 consensus_queue.cc:237] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:53.456905 18464 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:05:53.717370 18577 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:53.824512 18577 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:53.692937 18514 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.685581 18167 raft_consensus.cc:2749] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:53.706943 18145 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:53.785934 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 45219.7us
W20260709 13:05:53.646696 18626 log.cc:927] Time spent T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.271s	user 0.000s	sys 0.001s
W20260709 13:05:53.786299 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 45429.5us
W20260709 13:05:53.786857 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 50812.2us
I20260709 13:05:53.725430 18607 raft_consensus.cc:2749] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:53.500695 18485 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:53.781149 18629 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:53.790673 18606 raft_consensus.cc:2749] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:53.680786 18628 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:05:53.793823 18461 consensus_meta.cc:287] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.636s	user 0.009s	sys 0.000s
I20260709 13:05:53.794435 18493 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:05:53.803648 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.808878 18613 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:53.810360 18401 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:53.811710 18571 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:53.421306 18460 consensus_queue.cc:260] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:53.816933 18474 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:53.743886 18376 consensus_queue.cc:1048] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:05:53.744534 18630 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:54.293923 15902 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:05:54.117946 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 464732us
W20260709 13:05:53.990700 16533 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.025s	sys 0.053s
W20260709 13:05:54.131165 15974 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.722s	user 0.042s	sys 0.051s
W20260709 13:05:54.170497 16535 leader_election.cc:336] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 1.582s (SENT)
W20260709 13:05:54.296083 15974 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.887s	user 0.043s	sys 0.064s
W20260709 13:05:54.279770 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:54.296814 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 998182us
I20260709 13:05:54.297513 15827 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
W20260709 13:05:54.264153 18612 log.cc:927] Time spent T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 13:05:54.021111 15976 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.016s	sys 0.064s
W20260709 13:05:54.298769 15976 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.896s	user 0.020s	sys 0.066s
I20260709 13:05:53.941505 18634 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.299705 18634 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:54.296329 16535 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:05:54.302261 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 131815us
I20260709 13:05:54.302817 18634 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:53.828019 18549 consensus_meta.cc:287] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.737s	user 0.010s	sys 0.000s
I20260709 13:05:54.304157 18549 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:54.304738 18581 raft_consensus.cc:2804] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:54.305876 18581 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:54.306442 18549 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:54.306937 18163 consensus_meta.cc:287] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 1.023s	user 0.007s	sys 0.003s
I20260709 13:05:54.322638 18506 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:54.324483 15901 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.011s	sys 0.002s
W20260709 13:05:54.345489 15901 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.012s	sys 0.003s
I20260709 13:05:53.789261 18615 raft_consensus.cc:740] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:54.348318 18615 consensus_queue.cc:260] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:54.349663 18615 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:54.295190 16533 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.873s	user 0.028s	sys 0.069s
W20260709 13:05:54.352406 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 982656us
I20260709 13:05:54.354043 16533 leader_election.cc:304] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:54.296581 18571 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.295184 15902 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:05:54.356264 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 61147.1us
I20260709 13:05:54.356128 18571 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:54.356938 15902 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:54.358433 18571 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:54.329160 16005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1796) took 2164 ms. Trace:
I20260709 13:05:54.359490 18615 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:54.359925 18615 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:54.360486 18615 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:54.359849 16005 rpcz_store.cc:276] 0709 13:05:52.164330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:52.424534 (+260204us) service_pool.cc:224] Handling call
0709 13:05:52.424853 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:52.810108 (+385255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:54.328848 (+1518740us) spinlock_profiling.cc:243] Waited 385 ms on lock 0x7b54001f77d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:54.329078 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":884501376}
I20260709 13:05:54.363263 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:54.364109 15932 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:05:54.366799 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:54.367478 15932 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:54.367764 18615 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:54.370237 18615 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:54.370834 18615 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:54.371476 18615 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.371963 18615 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:54.373907 18615 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:54.355552 16533 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:54.375113 18493 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:54.375605 18493 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:54.376094 18493 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:54.381353 18376 consensus_queue.cc:1048] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:54.385603 16533 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:54.388660 18145 raft_consensus.cc:2764] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260709 13:05:54.388768 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 91344.1us
I20260709 13:05:54.389770 18485 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:53.956313 18578 consensus_meta.cc:287] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.533s	user 0.010s	sys 0.000s
I20260709 13:05:54.333452 18581 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:54.390453 18578 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:54.392544 18578 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:54.040838 18363 consensus_meta.cc:287] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.714s	user 0.008s	sys 0.003s
I20260709 13:05:54.393321 18363 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:54.116736 18322 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.003s	sys 0.007s
I20260709 13:05:53.824736 18464 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:53.941804 18627 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:05:54.389834 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:54.389835 15827 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:53.908838 18500 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.355561 18474 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:54.303521 18634 raft_consensus.cc:2749] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:54.403357 15900 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:05:54.409425 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:54.328502 18624 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:54.417114 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:54.420491 15862 raft_consensus.cc:2468] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 9.
I20260709 13:05:54.414786 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:54.421774 16009 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:54.399940 18322 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.900s	user 0.008s	sys 0.008s
I20260709 13:05:54.427655 15932 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:05:54.428912 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:54.429219 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:54.429663 16007 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 5: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:54.429837 15937 raft_consensus.cc:2468] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:05:54.429090 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:05:54.463332 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 60064.3us
I20260709 13:05:54.404786 18376 raft_consensus.cc:2804] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:54.403312 18616 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:54.469826 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:54.469937 18363 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:54.470284 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:54.470950 18464 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.471236 18490 raft_consensus.cc:3055] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:54.503508 15900 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:05:54.471534 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 50404.8us
I20260709 13:05:54.332113 18577 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:54.472687 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:54.473032 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:54.473605 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:54.474280 16535 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:54.470213 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:54.509018 15856 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 3.
I20260709 13:05:54.478605 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:54.479045 18177 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:54.431352 18446 raft_consensus.cc:2749] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:54.485870 16002 raft_consensus.cc:2376] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5. Re-sending same reply.
I20260709 13:05:54.487327 18638 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.487795 18145 consensus_queue.cc:1059] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:54.488740 18291 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:54.419965 18500 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:54.491362 15860 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 1673) took 2508 ms (client timeout 1595 ms). Trace:
W20260709 13:05:54.420819 16533 connection.cc:256] RPC call timeout handler was delayed by -1.1452695460000086s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2343s for a response
I20260709 13:05:54.534456 16009 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6.
I20260709 13:05:54.494796 15936 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1778) took 1402 ms. Trace:
I20260709 13:05:54.535696 15936 rpcz_store.cc:276] 0709 13:05:53.092211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.100407 (+  8196us) service_pool.cc:224] Handling call
0709 13:05:53.116395 (+ 15988us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:53.786628 (+670233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:54.494567 (+707939us) spinlock_profiling.cc:243] Waited 630 ms on lock 0x7b5400197b54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:54.494740 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1449949952}
I20260709 13:05:54.331513 18461 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:54.538914 18461 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:54.395401 18163 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:54.541023 18163 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:54.542259 16535 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:05:54.542761 18608 log.cc:927] Time spent T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.860s	user 0.001s	sys 0.001s
I20260709 13:05:54.543016 15855 raft_consensus.cc:1240] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:54.545475 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:54.546030 15855 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:54.332376 18628 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:54.500473 18630 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:54.431214 18527 raft_consensus.cc:3055] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:54.548337 18527 raft_consensus.cc:740] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:05:54.565999 16004 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41674 (request call id 1701) took 1719 ms (client timeout 1772 ms). Trace:
W20260709 13:05:54.566568 16004 rpcz_store.cc:269] 0709 13:05:52.847002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:52.850674 (+  3672us) service_pool.cc:224] Handling call
0709 13:05:54.565744 (+1715070us) spinlock_profiling.cc:243] Waited 282 ms on lock 0x7b540025a3d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:54.565924 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":649379584}
W20260709 13:05:54.533531 15860 rpcz_store.cc:269] 0709 13:05:51.965944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:52.014659 (+ 48715us) service_pool.cc:224] Handling call
0709 13:05:54.474335 (+2459676us) spinlock_profiling.cc:243] Waited 785 ms on lock 0x7b54003886d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:54.474493 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1806351232}
I20260709 13:05:54.504030 18376 raft_consensus.cc:697] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:54.508699 15929 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:54.482121 15825 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:54.509954 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:54.466266 18506 raft_consensus.cc:2749] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:54.512559 15827 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:54.513134 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:54.517917 18177 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:54.594331 15856 raft_consensus.cc:2393] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 5: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:54.342042 18514 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:54.520717 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:05:54.521219 18609 log.cc:927] Time spent T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.838s	user 0.001s	sys 0.001s
I20260709 13:05:54.521725 18649 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:54.508255 18490 raft_consensus.cc:740] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:54.513680 18464 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:54.421684 18627 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:54.512262 18401 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.510813 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:54.525605 18577 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:54.528116 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:54.625257 15859 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:54.529106 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:54.549496 18527 consensus_queue.cc:260] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:54.595297 15862 raft_consensus.cc:2393] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 8: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:54.523721 18638 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:05:54.596982 18642 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:54.597393 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:54.648134 16004 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:54.599500 18177 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:54.600096 15860 raft_consensus.cc:1240] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 3. Current term is 4. Ops: []
W20260709 13:05:54.608521 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 96032.7us
I20260709 13:05:54.651032 15827 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:54.566578 18291 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.653424 18578 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:54.653864 18578 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:54.653544 18291 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:54.654286 18578 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:54.654222 18485 consensus_queue.cc:1048] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:54.656172 18291 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:54.656105 18490 consensus_queue.cc:260] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:54.657069 15827 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:54.657476 18490 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:54.657786 18588 consensus_queue.cc:1048] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:54.659080 18588 raft_consensus.cc:2804] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:54.659271 18500 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:54.659466 18588 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:54.659983 18588 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:54.648391 18628 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:54.624512 18401 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:54.661221 15827 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:54.662804 18401 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:54.665824 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:54.666399 16566 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:54.667744 16567 raft_consensus.cc:1275] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:05:54.612514 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 261734us
I20260709 13:05:54.571333 18641 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:54.616899 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 134950us
I20260709 13:05:54.573855 18461 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.510905 18629 consensus_queue.cc:1048] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.096s
W20260709 13:05:54.619553 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 103743us
I20260709 13:05:54.682288 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:54.570781 18647 raft_consensus.cc:2764] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:05:54.622339 16007 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:54.627367 15855 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:05:54.626372 16008 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:05:54.648885 16003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1797) took 2484 ms. Trace:
I20260709 13:05:54.685220 16003 rpcz_store.cc:276] 0709 13:05:52.164618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:52.447528 (+282910us) service_pool.cc:224] Handling call
0709 13:05:52.512542 (+ 65014us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:53.157405 (+644863us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:54.630844 (+1473439us) spinlock_profiling.cc:243] Waited 626 ms on lock 0x7b54002e1b54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:54.648828 (+ 17984us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1440917888}
I20260709 13:05:54.682247 16565 raft_consensus.cc:1240] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:54.680080 18634 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.688215 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:54.688267 18634 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:54.689632 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:54.690335 16570 raft_consensus.cc:2393] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 4: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:54.691109 18634 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:54.623188 18597 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:54.692130 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:54.693434 16567 raft_consensus.cc:1240] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:54.687407 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:54.694424 15825 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:54.694901 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:54.695458 16565 raft_consensus.cc:2468] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:05:54.691080 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:05:54.680071 15974 leader_election.cc:336] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 2.108s (SENT)
W20260709 13:05:54.698112 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 341212us
I20260709 13:05:54.700312 15900 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:54.647698 18363 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.702687 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:54.703581 15825 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:54.706879 15825 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:54.703881 15902 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:54.662863 18578 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:54.679176 16004 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6.
I20260709 13:05:54.709653 15900 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:54.709735 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:54.693022 18527 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:54.717100 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:54.668413 18639 consensus_queue.cc:1059] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 13:05:54.711272 18582 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:54.718672 18639 consensus_queue.cc:1243] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer eb1e0493af1e46dbb6dec7e25525bab5 log is divergent from this leader: its last log entry 6.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:05:54.667917 18643 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:54.719197 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:54.692893 18446 raft_consensus.cc:2764] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
W20260709 13:05:54.720155 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 114113us
I20260709 13:05:54.718243 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:54.721153 15900 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:54.705530 18577 raft_consensus.cc:2749] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:05:54.726987 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:54.697458 18642 raft_consensus.cc:697] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:54.670292 18485 consensus_queue.cc:1048] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:05:54.697721 18574 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:54.709215 18468 raft_consensus.cc:2804] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:54.743086 16566 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:05:54.705318 18628 consensus_queue.cc:1048] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:05:54.718246 18638 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:54.719818 18647 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:54.763032 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:54.698103 18461 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:54.765354 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 85296.6us
I20260709 13:05:54.711687 18363 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:54.705229 18401 consensus_queue.cc:1059] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:05:54.722173 18506 raft_consensus.cc:2749] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:54.743664 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:54.790489 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:54.791082 16004 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:05:54.704082 18597 consensus_queue.cc:1048] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 13:05:54.780938 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:54.780532 18177 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.792003 16008 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:05:54.766566 18630 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:54.771431 18468 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:54.810081 15974 leader_election.cc:336] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 3.030s (SENT)
I20260709 13:05:54.810582 15974 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:54.757239 18655 consensus_queue.cc:1059] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
W20260709 13:05:54.815305 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 108483us
I20260709 13:05:54.767573 18616 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:54.739722 18493 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:54.693418 18641 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:54.814380 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 117266us
I20260709 13:05:54.823568 16533 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:54.786684 16561 raft_consensus.cc:1275] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:05:54.786469 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:54.817906 15930 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 5: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:54.818275 16569 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:54.711402 18376 consensus_queue.cc:237] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:54.847116 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:54.817180 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:54.826627 16533 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:54.828564 15827 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:54.848042 15932 raft_consensus.cc:2393] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 5: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:54.848093 15935 raft_consensus.cc:2393] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 8: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:54.846997 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:54.850718 15937 raft_consensus.cc:2410] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:05:54.846920 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:54.853230 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:54.854028 15936 raft_consensus.cc:2410] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:05:54.814442 18607 raft_consensus.cc:3055] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:54.815955 18401 raft_consensus.cc:2749] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:54.856395 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:54.814805 18468 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:54.859542 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:54.859901 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:54.868432 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:54.813764 18177 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:54.870873 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:05:54.871335 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 56119.5us
I20260709 13:05:54.871915 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:54.872370 18177 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:54.872689 15937 raft_consensus.cc:2410] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:05:54.873466 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:54.874158 15937 raft_consensus.cc:2410] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:05:54.874578 18647 consensus_queue.cc:1048] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:54.874902 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:54.875834 18615 consensus_queue.cc:1048] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:54.876183 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:54.876773 16561 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:54.878319 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:54.878855 15859 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:05:54.879359 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:54.879942 15937 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 3.
I20260709 13:05:54.880990 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:54.882225 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:05:54.883181 18652 log.cc:927] Time spent T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 13:05:54.883242 15937 raft_consensus.cc:2410] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:05:54.884084 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:54.884416 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:54.884889 16002 raft_consensus.cc:2393] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 8: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:54.885911 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:54.887185 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:54.888012 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:54.888585 16004 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:05:54.889397 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:54.889986 16004 raft_consensus.cc:2368] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5.
I20260709 13:05:54.890542 18641 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:54.890666 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:54.892132 18493 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:54.866729 18642 consensus_queue.cc:237] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:54.861855 15935 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 4.
W20260709 13:05:54.895874 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:54.896627 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:54.897637 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:54.898340 15935 raft_consensus.cc:2393] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 8: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:54.899024 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:54.900080 18667 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:54.900671 18667 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:54.826295 18616 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:54.903816 18658 log.cc:927] Time spent T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:05:54.904145 18616 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:54.858479 18607 raft_consensus.cc:740] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:54.905558 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:54.905406 18607 consensus_queue.cc:260] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:54.847215 18167 raft_consensus.cc:2804] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:54.907056 18167 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:54.907456 18167 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:54.907588 18666 consensus_queue.cc:1059] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:54.781294 18574 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:54.912036 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:54.913594 15936 raft_consensus.cc:1240] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:05:54.914739 15936 raft_consensus.cc:1240] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:54.858009 18514 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:54.914783 18581 consensus_meta.cc:287] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.524s	user 0.010s	sys 0.000s
I20260709 13:05:54.917397 18581 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:54.919864 15934 raft_consensus.cc:1240] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:54.864464 18500 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:54.920850 18500 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:54.921175 18500 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:54.906569 18607 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:54.924041 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 133295us
W20260709 13:05:54.924814 15974 connection.cc:256] RPC call timeout handler was delayed by -1.0114391800000249s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2143s for a response
I20260709 13:05:54.925567 18514 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.926030 18514 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:54.927429 18662 log.cc:927] Time spent T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:05:54.928840 18514 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:54.929193 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:54.929806 16010 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:05:54.930655 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:54.931329 16010 raft_consensus.cc:2393] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 5: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:54.864005 18577 raft_consensus.cc:3055] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:54.932928 18577 raft_consensus.cc:740] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:54.926338 15974 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:54.933790 18577 consensus_queue.cc:260] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:54.934916 18577 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:54.936209 18485 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:54.942674 16569 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:05:54.943660 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:54.906625 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:54.956218 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 48929.3us
I20260709 13:05:54.968788 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:54.866261 18549 raft_consensus.cc:2804] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:54.969502 15936 raft_consensus.cc:1275] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:05:54.970789 16004 raft_consensus.cc:2393] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 6: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:54.897709 18474 raft_consensus.cc:2749] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:54.919624 18581 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:54.947345 18401 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:54.977069 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:54.910607 18588 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:54.978682 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:54.980456 15933 raft_consensus.cc:2393] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 6: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:05:54.980940 18664 log.cc:927] Time spent T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:05:54.981316 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:54.981926 15933 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:54.983103 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:54.979607 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:54.992332 15936 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 5: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:54.865852 18322 consensus_queue.cc:1059] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.109s
I20260709 13:05:54.962154 18616 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:05:54.997997 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 169541us
I20260709 13:05:54.974747 18549 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:55.005422 15974 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:55.017158 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:55.022657 15827 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:55.023890 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:55.023890 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:55.020459 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 58247.6us
I20260709 13:05:55.019462 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:55.027139 15827 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:55.031311 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:55.031446 15827 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:55.036342 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:55.036464 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:55.004582 18665 raft_consensus.cc:3055] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:55.005005 18582 raft_consensus.cc:697] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 8 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:55.041308 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:55.048789 16004 raft_consensus.cc:2410] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 6 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:54.994822 18641 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:55.043628 15974 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:55.055661 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:55.055833 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:54.970640 16563 raft_consensus.cc:2955] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:55.056763 15859 raft_consensus.cc:2410] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:05:55.057648 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:55.058221 15859 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:55.058993 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:55.056788 15854 raft_consensus.cc:2410] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:05:55.060672 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:55.059624 15859 raft_consensus.cc:2368] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5.
I20260709 13:05:55.063745 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:05:55.064270 18670 log.cc:927] Time spent T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:05:55.064880 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:55.065529 15930 raft_consensus.cc:2393] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 9: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:55.066182 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:55.024896 18549 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:55.067838 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:55.068742 15930 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:55.071501 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:05:55.076503 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:55.048132 18485 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:55.042577 18665 raft_consensus.cc:740] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:55.030983 18401 consensus_queue.cc:1059] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 13:05:55.088346 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:55.055662 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:55.092631 18393 consensus_queue.cc:1059] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:05:55.095042 18622 log.cc:927] Time spent T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 1.414s	user 0.007s	sys 0.006s
I20260709 13:05:55.052897 18677 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:55.080147 18177 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:55.040963 18616 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:55.107409 16533 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:55.110925 18673 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:55.117151 16533 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:55.121338 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:55.096819 18363 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:55.127096 16004 raft_consensus.cc:2393] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 8: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:55.106065 16002 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:55.080708 18641 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:55.135842 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 89349.3us
I20260709 13:05:55.141254 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:55.124024 18640 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.143514 16004 raft_consensus.cc:2393] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 5: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:55.118709 18677 raft_consensus.cc:697] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:55.120501 18177 raft_consensus.cc:697] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:55.112666 18642 consensus_queue.cc:1048] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:05:55.161410 16010 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:55.175799 18620 raft_consensus.cc:2749] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:55.177273 18177 consensus_queue.cc:237] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:05:55.180415 18674 log.cc:927] Time spent T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:05:55.180267 18167 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:55.182436 18167 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:55.155318 16561 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:05:55.183198 18677 consensus_queue.cc:237] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:55.194485 16002 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5.
I20260709 13:05:55.194949 18468 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:05:55.139205 18582 consensus_queue.cc:237] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:55.145758 18665 consensus_queue.cc:260] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:55.196336 16002 raft_consensus.cc:1275] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:05:55.197438 18616 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:55.199803 18588 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:55.315953 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:55.158923 18640 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:55.201939 18641 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:55.205667 15759 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58448 (request call id 308) took 14931 ms (client timeout 15000 ms). Trace:
I20260709 13:05:55.206568 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:55.207937 16567 raft_consensus.cc:1275] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:05:55.183444 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:55.325150 15932 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:55.211520 15974 leader_election.cc:336] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 1.582s (SENT)
I20260709 13:05:55.211635 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:55.326134 16005 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 6
W20260709 13:05:55.301952 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:55.386081 15974 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:55.388384 18468 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:05:55.215502 18490 consensus_meta.cc:287] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.558s	user 0.007s	sys 0.013s
W20260709 13:05:55.321813 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 204774us
I20260709 13:05:55.404452 15937 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:05:55.399863 16533 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:55.463476 18607 raft_consensus.cc:2749] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:05:55.475040 18500 consensus_meta.cc:287] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.553s	user 0.006s	sys 0.004s
I20260709 13:05:55.481375 15854 raft_consensus.cc:2393] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 5: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:55.486435 15931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1794) took 2158 ms. Trace:
I20260709 13:05:55.560281 15931 rpcz_store.cc:276] 0709 13:05:53.327920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.328348 (+   428us) service_pool.cc:224] Handling call
0709 13:05:53.328610 (+   262us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:53.369180 (+ 40570us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:05:53.397349 (+ 28169us) log.cc:844] Serialized 14 byte log entry
0709 13:05:55.131012 (+1733663us) spinlock_profiling.cc:243] Waited 1.72 ms on lock 0x7b18002a4c60. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f4823a 00007f73ea3b2a2a
0709 13:05:55.172349 (+ 41337us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:55.172792 (+   443us) log.cc:844] Serialized 26 byte log entry
0709 13:05:55.172913 (+   121us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:55.172932 (+    19us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:55.172976 (+    44us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:55.486040 (+313064us) raft_consensus.cc:1685] finished
0709 13:05:55.486166 (+   126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:55.486358 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40176,"prepare.queue_time_us":29204,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":88285,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":6473,"spinlock_wait_cycles":3959040,"thread_start_us":37871,"threads_started":1,"wal-append.queue_time_us":1981340}
W20260709 13:05:55.488127 18649 consensus_meta.cc:287] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.871s	user 0.010s	sys 0.008s
W20260709 13:05:55.492187 15929 consensus_meta.cc:287] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.885s	user 0.014s	sys 0.010s
I20260709 13:05:55.572629 15929 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5.
I20260709 13:05:55.573383 15929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 1803) took 1150 ms. Trace:
I20260709 13:05:55.574604 15929 rpcz_store.cc:276] 0709 13:05:54.423110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.430070 (+  6960us) service_pool.cc:224] Handling call
0709 13:05:55.573313 (+1143243us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:55.526973 15932 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6.
I20260709 13:05:55.601372 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:05:55.615156 15933 consensus_meta.cc:287] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.621s	user 0.009s	sys 0.000s
I20260709 13:05:55.619166 15933 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6.
I20260709 13:05:55.621773 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:05:55.595866 18577 consensus_meta.cc:287] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.661s	user 0.013s	sys 0.000s
I20260709 13:05:55.589982 18649 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:05:55.623418 18549 consensus_meta.cc:287] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.536s	user 0.005s	sys 0.004s
I20260709 13:05:55.631979 16005 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6.
W20260709 13:05:55.646446 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:55.745251 16005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60156 (request call id 467) took 1339 ms. Trace:
I20260709 13:05:55.745651 16005 rpcz_store.cc:276] 0709 13:05:54.405539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.413654 (+  8115us) service_pool.cc:224] Handling call
0709 13:05:55.211459 (+797805us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b5800114398. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:05:55.745180 (+533721us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23907712}
I20260709 13:05:55.729506 15933 raft_consensus.cc:2410] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
W20260709 13:05:55.731400 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:55.625499 16563 consensus_meta.cc:287] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.568s	user 0.009s	sys 0.002s
I20260709 13:05:55.625285 18490 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:55.322706 15759 rpcz_store.cc:269] 0709 13:05:40.274124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.284288 (+ 10164us) service_pool.cc:224] Handling call
0709 13:05:55.205420 (+14921132us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:47.813524 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:47.960060 (+146536us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:47.960099 (+    39us) write_op.cc:435] Acquired schema lock
0709 13:05:47.960117 (+    18us) tablet.cc:662] Decoding operations
0709 13:05:47.961244 (+  1127us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:47.961308 (+    64us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:47.961341 (+    33us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:05:47.962400 (+  1059us) tablet.cc:701] Row locks acquired
0709 13:05:47.962417 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:05:47.962572 (+   155us) write_op.cc:270] Start()
0709 13:05:47.962682 (+   110us) write_op.cc:276] Timestamp: P: 1783602347962521 usec, L: 0
0709 13:05:47.962700 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 13:05:47.963356 (+   656us) log.cc:844] Serialized 4364 byte log entry
0709 13:05:48.530833 (+567477us) op_driver.cc:464] REPLICATION: finished
0709 13:05:49.017877 (+487044us) write_op.cc:301] APPLY: starting
0709 13:05:49.017965 (+    88us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:49.019931 (+  1966us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:49.020187 (+   256us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:49.063812 (+ 43625us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:49.064052 (+   240us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0709 13:05:49.064097 (+    45us) write_op.cc:312] APPLY: finished
0709 13:05:49.128679 (+ 64582us) log.cc:844] Serialized 83 byte log entry
0709 13:05:49.157575 (+ 28896us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:49.182044 (+ 24469us) write_op.cc:454] Released schema lock
0709 13:05:49.188503 (+  6459us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:49.252183 (+ 63680us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:49.258268 (+  6085us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":12591,"reported_tablets":23,"spinlock_wait_cycles":61312,"child_traces":[["op",{"apply.queue_time_us":471464,"apply.run_cpu_time_us":11667,"apply.run_wall_time_us":204961,"num_ops":11,"prepare.queue_time_us":432298,"prepare.run_cpu_time_us":44669,"prepare.run_wall_time_us":478509,"raft.queue_time_us":160531,"raft.run_cpu_time_us":49147,"raft.run_wall_time_us":413555,"replication_time_us":567950,"thread_start_us":869340,"threads_started":4,"wal-append.queue_time_us":357694,"wal-append.run_cpu_time_us":12491,"wal-append.run_wall_time_us":1960770}]]}
I20260709 13:05:55.586268 18500 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:55.784803 18485 consensus_meta.cc:287] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.660s	user 0.005s	sys 0.006s
W20260709 13:05:55.824365 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 424590us
I20260709 13:05:55.784566 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:55.788453 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:55.784317 18490 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:55.836635 18167 raft_consensus.cc:2804] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:55.840649 15935 raft_consensus.cc:2393] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 5: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
W20260709 13:05:55.860343 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 648777us
I20260709 13:05:55.810578 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:55.860433 16567 raft_consensus.cc:1275] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:05:55.825475 16533 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:55.868816 16533 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:55.869854 18493 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.870530 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:55.875224 15933 raft_consensus.cc:2410] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:05:55.871229 16567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 753) took 1085 ms. Trace:
I20260709 13:05:55.876825 16567 rpcz_store.cc:276] 0709 13:05:54.786037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:55.390673 (+604636us) service_pool.cc:224] Handling call
0709 13:05:55.860089 (+469416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:55.863719 (+  3630us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:55.871159 (+  7440us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:55.759706 18549 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:55.872303 15974 leader_election.cc:336] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 1.671s (SENT)
I20260709 13:05:55.826678 18485 raft_consensus.cc:2749] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:55.885491 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:55.861649 18167 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:55.871315 18665 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:55.936035 15974 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:55.870911 18577 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.870386 18363 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:55.884234 18649 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:05:55.934917 18177 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.005s	sys 0.040s
I20260709 13:05:55.943096 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:05:55.943894 18177 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.005s	sys 0.040s
W20260709 13:05:55.946189 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 73926.2us
W20260709 13:05:55.946849 15974 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 1.906s (SENT) [suppressed 27 similar messages]
W20260709 13:05:55.948055 16004 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.013s	sys 0.047s
W20260709 13:05:55.948398 16004 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.013s	sys 0.047s
I20260709 13:05:55.936770 18167 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:55.951809 16568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 748) took 1212 ms. Trace:
I20260709 13:05:55.952221 16568 rpcz_store.cc:276] 0709 13:05:54.738940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:55.077646 (+338706us) service_pool.cc:224] Handling call
0709 13:05:55.093949 (+ 16303us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:55.110088 (+ 16139us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:55.110123 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:55.128529 (+ 18406us) log.cc:844] Serialized 52 byte log entry
0709 13:05:55.214427 (+ 85898us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:55.214497 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:55.214590 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:55.951554 (+736964us) raft_consensus.cc:1685] finished
0709 13:05:55.951576 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:55.951751 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9316,"raft.queue_time_us":11481,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":11752,"thread_start_us":60664,"threads_started":1,"wal-append.queue_time_us":802958}
I20260709 13:05:55.953833 15853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 513) took 1545 ms. Trace:
I20260709 13:05:55.954200 15853 rpcz_store.cc:276] 0709 13:05:54.408695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.414166 (+  5471us) service_pool.cc:224] Handling call
0709 13:05:54.464655 (+ 50489us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:54.464841 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:05:54.679845 (+215004us) log.cc:844] Serialized 14 byte log entry
0709 13:05:55.953433 (+1273588us) spinlock_profiling.cc:243] Waited 1.8 ms on lock 0x7b1800392b00. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e9f504bc 00007f73e729e48b 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72b8455 00007f73ea3a0060 00007f73ea3b2716 00007f73ea3c3287
0709 13:05:55.953517 (+    84us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:55.953558 (+    41us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:55.953597 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:55.953681 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:55.953782 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":167,"prepare.queue_time_us":132545,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":32,"raft.queue_time_us":74862,"raft.run_wall_time_us":17825,"spinlock_wait_cycles":24607232,"thread_start_us":326041,"threads_started":3,"wal-append.queue_time_us":124880}
I20260709 13:05:55.888051 18493 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:55.959383 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:55.962059 15935 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:05:55.964706 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:55.966109 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:55.969862 15853 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 4.
I20260709 13:05:55.957767 18669 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:55.969902 16007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60156 (request call id 477) took 1561 ms. Trace:
I20260709 13:05:55.971061 16007 rpcz_store.cc:276] 0709 13:05:54.409993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.469955 (+ 59962us) service_pool.cc:224] Handling call
0709 13:05:55.969747 (+1499792us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b540037b9d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:55.969849 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":259432704}
I20260709 13:05:55.968012 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:55.972466 15859 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
W20260709 13:05:55.956111 16562 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45246 (request call id 664) took 3640 ms (client timeout 3185 ms). Trace:
I20260709 13:05:55.944867 15854 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 4.
I20260709 13:05:55.942718 18577 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:55.959270 18490 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:55.973165 18640 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:55.983520 15935 raft_consensus.cc:1240] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 3. Current term is 4. Ops: []
I20260709 13:05:55.986397 15935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 1721) took 1008 ms. Trace:
I20260709 13:05:55.987020 15935 rpcz_store.cc:276] 0709 13:05:54.977764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:55.974078 (+996314us) service_pool.cc:224] Handling call
0709 13:05:55.982073 (+  7995us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:55.986132 (+  4059us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:55.986337 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:55.975409 16562 rpcz_store.cc:269] 0709 13:05:52.315642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:52.318537 (+  2895us) service_pool.cc:224] Handling call
0709 13:05:55.955862 (+3637325us) spinlock_profiling.cc:243] Waited 934 ms on lock 0x7b54002ff154. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:55.956015 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2147136256}
I20260709 13:05:55.989220 18490 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:55.989732 18490 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:55.990743 18667 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:55.992612 18605 raft_consensus.cc:3055] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:55.992947 18577 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:55.993024 18605 raft_consensus.cc:740] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:55.994825 18493 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:55.996137 18500 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:55.985199 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:56.002137 15935 raft_consensus.cc:1275] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
W20260709 13:05:55.984938 18372 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.022s	sys 0.032s
I20260709 13:05:56.004665 15853 raft_consensus.cc:2410] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:05:56.004994 18372 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.845s	user 0.022s	sys 0.033s
W20260709 13:05:56.013329 16569 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45232 (request call id 625) took 1346 ms (client timeout 1751 ms). Trace:
I20260709 13:05:56.013506 16565 raft_consensus.cc:1240] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 6. Current term is 7. Ops: []
W20260709 13:05:56.014053 16569 rpcz_store.cc:269] 0709 13:05:54.667378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.682071 (+ 14693us) service_pool.cc:224] Handling call
0709 13:05:56.013084 (+1331013us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b54000cdad4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:56.013250 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":308931328}
I20260709 13:05:56.015596 16004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 479) took 1605 ms. Trace:
I20260709 13:05:56.016175 16004 rpcz_store.cc:276] 0709 13:05:54.410588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:55.155531 (+744943us) service_pool.cc:224] Handling call
0709 13:05:55.166372 (+ 10841us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:55.208810 (+ 42438us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:55.208895 (+    85us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:55.211320 (+  2425us) log.cc:844] Serialized 26 byte log entry
0709 13:05:55.948780 (+737460us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:55.948826 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:55.948902 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:56.015309 (+ 66407us) raft_consensus.cc:1685] finished
0709 13:05:56.015338 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:56.015525 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4041,"raft.queue_time_us":48892,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":76995,"thread_start_us":546017,"threads_started":1,"wal-append.queue_time_us":663371}
I20260709 13:05:55.994074 18605 consensus_queue.cc:260] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:56.003136 18577 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.021726 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:56.022557 15853 raft_consensus.cc:2393] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 8: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:56.040905 16010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60156 (request call id 485) took 1345 ms. Trace:
I20260709 13:05:56.041498 16010 rpcz_store.cc:276] 0709 13:05:54.695608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.966778 (+271170us) service_pool.cc:224] Handling call
0709 13:05:56.040659 (+1073881us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b540010efd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:56.040820 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":418322688}
I20260709 13:05:56.043349 16010 raft_consensus.cc:1275] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:05:56.050973 18583 raft_consensus.cc:3055] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:56.052371 18627 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:56.061142 18460 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:56.086205 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:56.066954 15900 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:56.069550 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:56.090576 15935 raft_consensus.cc:2393] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 6: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:56.076232 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:56.044104 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:56.098583 15861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 1814) took 1597 ms. Trace:
I20260709 13:05:56.098961 15861 rpcz_store.cc:276] 0709 13:05:54.501033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.501277 (+   244us) service_pool.cc:224] Handling call
0709 13:05:56.076038 (+1574761us) spinlock_profiling.cc:243] Waited 1.99 ms on lock 0x7b5800005798. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:05:56.098531 (+ 22493us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4583680}
I20260709 13:05:56.080282 16010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1802) took 1339 ms. Trace:
I20260709 13:05:56.100812 16010 rpcz_store.cc:276] 0709 13:05:54.740822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.042489 (+1301667us) service_pool.cc:224] Handling call
0709 13:05:56.042914 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:56.079965 (+ 37051us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.080226 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:56.102102 16566 raft_consensus.cc:1240] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 7. Current term is 8. Ops: [5.8-7.9]
I20260709 13:05:56.106426 15827 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:56.010496 18680 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:56.098047 16006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60156 (request call id 478) took 1687 ms. Trace:
I20260709 13:05:56.116653 16006 rpcz_store.cc:276] 0709 13:05:54.410266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.460561 (+ 50295us) service_pool.cc:224] Handling call
0709 13:05:56.043929 (+1583368us) spinlock_profiling.cc:243] Waited 14.9 ms on lock 0x7b5800114398. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:05:56.097990 (+ 54061us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34215808}
I20260709 13:05:56.117957 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:56.113361 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:56.114163 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:56.119326 15935 raft_consensus.cc:2376] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6. Re-sending same reply.
I20260709 13:05:56.019348 18490 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:56.123739 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:56.124475 16006 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:56.127249 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:56.128001 16010 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:05:56.092432 18493 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.085183 18583 raft_consensus.cc:740] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:56.091740 18363 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.133816 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:56.136528 15935 raft_consensus.cc:2393] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 9: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:56.100216 18393 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.019187 18500 consensus_queue.cc:1048] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.104s
I20260709 13:05:56.171481 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:56.172619 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:05:56.139667 18693 log.cc:927] Time spent T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.153s	user 0.000s	sys 0.000s
W20260709 13:05:56.140429 18697 log.cc:927] Time spent T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
I20260709 13:05:56.190533 18163 raft_consensus.cc:1064] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
W20260709 13:05:56.142849 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.194869 16563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 679) took 1487 ms. Trace:
I20260709 13:05:56.126191 18490 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.109071 18549 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:56.162333 18493 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:56.164546 18583 consensus_queue.cc:260] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:56.166320 18667 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:56.080852 18577 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:56.182212 18234 consensus_queue.cc:1048] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:56.183081 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:56.185726 18363 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:56.186726 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:56.187399 18393 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:56.222234 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:56.220440 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:56.249816 18709 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:56.250036 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1715) took 1552 ms. Trace:
I20260709 13:05:56.250442 18709 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:56.249830 18677 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:05:56.249979 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.255511 18709 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:56.258306 18709 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.258913 18709 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:56.261461 18709 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:56.265084 15930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 1720) took 1292 ms. Trace:
I20260709 13:05:56.265730 15930 rpcz_store.cc:276] 0709 13:05:54.972756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:55.072532 (+ 99776us) service_pool.cc:224] Handling call
0709 13:05:55.072853 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:55.078113 (+  5260us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:55.078156 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:55.089440 (+ 11284us) log.cc:844] Serialized 26 byte log entry
0709 13:05:55.154700 (+ 65260us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:55.997661 (+842961us) spinlock_profiling.cc:243] Waited 2.79 ms on lock 0x7b18003a1b60. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72b8455 00007f73edda03c7 00007f73ea3b29a7 00007f73ea3c20bf 00007f73ea3c2038
0709 13:05:56.220719 (+223058us) log.cc:844] Serialized 14 byte log entry
0709 13:05:56.264602 (+ 43883us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.264714 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:56.264764 (+    50us) raft_consensus.cc:1685] finished
0709 13:05:56.264859 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:56.265014 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":36346,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":75,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":5921,"spinlock_wait_cycles":6407040,"thread_start_us":294710,"threads_started":3,"wal-append.queue_time_us":60527,"wal-append.run_cpu_time_us":1437,"wal-append.run_wall_time_us":1020572}
W20260709 13:05:56.272202 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 189389us
I20260709 13:05:56.250517 16002 rpcz_store.cc:276] 0709 13:05:54.697932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:55.566450 (+868518us) service_pool.cc:224] Handling call
0709 13:05:55.566837 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:55.567074 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:55.860911 (+293837us) log.cc:844] Serialized 14 byte log entry
0709 13:05:56.249559 (+388648us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:56.249619 (+    60us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:56.249666 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.249862 (+   196us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:56.249986 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":272195,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":2265,"raft.queue_time_us":189,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":87,"thread_start_us":370277,"threads_started":1}
W20260709 13:05:56.273080 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 206235us
I20260709 13:05:56.278087 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:56.283182 15933 raft_consensus.cc:2410] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:05:56.248279 15935 raft_consensus.cc:2376] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5. Re-sending same reply.
I20260709 13:05:56.279804 16007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1718) took 1560 ms. Trace:
W20260709 13:05:56.280810 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 173197us
I20260709 13:05:56.244299 18667 raft_consensus.cc:697] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:56.311498 16008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1720) took 1553 ms. Trace:
I20260709 13:05:56.316548 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:56.317291 16569 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:56.221451 18490 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:56.278939 18699 raft_consensus.cc:2749] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:05:56.277350 18649 raft_consensus.cc:697] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:56.279422 18500 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:56.313794 16008 rpcz_store.cc:276] 0709 13:05:54.757963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.045295 (+1287332us) service_pool.cc:224] Handling call
0709 13:05:56.045736 (+   441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:56.046221 (+   485us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:56.046260 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:56.046830 (+   570us) log.cc:844] Serialized 26 byte log entry
0709 13:05:56.289372 (+242542us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:56.289426 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.289523 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:56.311244 (+ 21721us) raft_consensus.cc:1685] finished
0709 13:05:56.311269 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:56.311438 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1587,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":109,"thread_start_us":242083,"threads_started":1,"wal-append.queue_time_us":215410}
I20260709 13:05:56.330415 15900 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:56.330685 16566 raft_consensus.cc:1275] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:05:56.331066 16009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 475) took 1922 ms. Trace:
I20260709 13:05:56.329051 16569 raft_consensus.cc:2468] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
I20260709 13:05:56.333856 15861 raft_consensus.cc:1240] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:05:56.333802 15827 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:05:56.418648 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 84899.7us
I20260709 13:05:56.351868 15855 raft_consensus.cc:1240] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260709 13:05:56.349779 18709 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:56.421092 18709 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:56.422101 16009 rpcz_store.cc:276] 0709 13:05:54.408479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.535522 (+127043us) service_pool.cc:224] Handling call
0709 13:05:54.536115 (+   593us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:54.699073 (+162958us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:54.699117 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:54.699651 (+   534us) log.cc:844] Serialized 26 byte log entry
0709 13:05:54.817263 (+117612us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:54.862917 (+ 45654us) log.cc:844] Serialized 14 byte log entry
0709 13:05:54.863148 (+   231us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.183234 (+1320086us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b1800455980. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb3f3 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd 000000000038f354 000000000038f199 00007f73e729e83a
0709 13:05:56.289319 (+106085us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b5400306454. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:56.289395 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:56.322100 (+ 32705us) raft_consensus.cc:1685] finished
0709 13:05:56.322127 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:56.331007 (+  8880us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":45617,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":7669,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":5077,"spinlock_wait_cycles":398759424,"thread_start_us":98275,"threads_started":1,"wal-append.queue_time_us":147914}
I20260709 13:05:56.288291 18583 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:56.389422 15931 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:05:56.350226 15934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1802) took 1931 ms. Trace:
I20260709 13:05:56.426007 15862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 1831) took 1253 ms. Trace:
I20260709 13:05:56.435500 15862 rpcz_store.cc:276] 0709 13:05:55.172667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.068362 (+895695us) service_pool.cc:224] Handling call
0709 13:05:56.068733 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:56.068980 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:05:56.252510 (+183530us) log.cc:844] Serialized 14 byte log entry
0709 13:05:56.425718 (+173208us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:56.425768 (+    50us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:56.425808 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.425865 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:56.425960 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16384,"prepare.queue_time_us":182579,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":423,"thread_start_us":303982,"threads_started":2}
I20260709 13:05:56.437640 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:56.386610 15853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 1830) took 1219 ms. Trace:
I20260709 13:05:56.433274 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:05:56.386045 18703 log.cc:927] Time spent T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:05:56.450181 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:56.450486 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:56.434473 15934 rpcz_store.cc:276] 0709 13:05:54.419161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.419333 (+   172us) service_pool.cc:224] Handling call
0709 13:05:54.466888 (+ 47555us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:54.920279 (+453391us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.349958 (+1429679us) spinlock_profiling.cc:243] Waited 453 ms on lock 0x7b5400328f54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:56.350157 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1041260416}
I20260709 13:05:56.413319 18177 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:56.457381 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 127006us
I20260709 13:05:56.458858 15900 leader_election.cc:400] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5.
W20260709 13:05:56.458942 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.459163 15900 leader_election.cc:403] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:05:56.284070 16007 rpcz_store.cc:276] 0709 13:05:54.719417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:55.975625 (+1256208us) service_pool.cc:224] Handling call
0709 13:05:55.975946 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:55.981595 (+  5649us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:55.981627 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:55.997596 (+ 15969us) log.cc:844] Serialized 26 byte log entry
0709 13:05:56.221454 (+223858us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:56.279047 (+ 57593us) log.cc:844] Serialized 14 byte log entry
0709 13:05:56.279345 (+   298us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.279440 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:56.279490 (+    50us) raft_consensus.cc:1685] finished
0709 13:05:56.279507 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:56.279655 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":57336,"raft.queue_time_us":5557,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":5664,"thread_start_us":210922,"threads_started":1,"wal-append.queue_time_us":249825}
I20260709 13:05:56.398918 18578 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:56.428449 18579 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:56.472970 16010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1722) took 1701 ms. Trace:
I20260709 13:05:56.474747 16008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1726) took 1684 ms. Trace:
I20260709 13:05:56.481976 16008 rpcz_store.cc:276] 0709 13:05:54.790167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.433754 (+1643587us) service_pool.cc:224] Handling call
0709 13:05:56.474126 (+ 40372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:56.474348 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:56.474386 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:56.474440 (+    54us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:56.474486 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.474553 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:56.474680 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:56.424532 18699 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:56.477331 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:56.478487 16007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1728) took 1661 ms. Trace:
I20260709 13:05:56.486949 16007 rpcz_store.cc:276] 0709 13:05:54.804095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.464716 (+1660621us) service_pool.cc:224] Handling call
0709 13:05:56.465085 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:56.465334 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:56.465370 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:56.465424 (+    54us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:56.465471 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.465539 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:56.465670 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:56.479254 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.482857 16010 rpcz_store.cc:276] 0709 13:05:54.771131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.312517 (+1541386us) service_pool.cc:224] Handling call
0709 13:05:56.331608 (+ 19091us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:56.332038 (+   430us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:56.332079 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:56.381660 (+ 49581us) log.cc:844] Serialized 26 byte log entry
0709 13:05:56.471640 (+ 89980us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:56.471714 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.471804 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:56.471869 (+    65us) raft_consensus.cc:1685] finished
0709 13:05:56.471885 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:56.472891 (+  1006us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17175,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":128,"thread_start_us":39639,"threads_started":1,"wal-append.queue_time_us":46367}
I20260709 13:05:56.195274 16563 rpcz_store.cc:276] 0709 13:05:54.707401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.880049 (+172648us) service_pool.cc:224] Handling call
0709 13:05:54.880372 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:54.961743 (+ 81371us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:05:55.758910 (+797167us) log.cc:844] Serialized 14 byte log entry
0709 13:05:55.950426 (+191516us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:55.985955 (+ 35529us) log.cc:844] Serialized 26 byte log entry
0709 13:05:56.000642 (+ 14687us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:56.008123 (+  7481us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.009966 (+  1843us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:56.194620 (+184654us) raft_consensus.cc:1685] finished
0709 13:05:56.194638 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:56.194809 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56329,"raft.queue_time_us":91350,"raft.run_cpu_time_us":300,"raft.run_wall_time_us":16792,"thread_start_us":161053,"threads_started":1,"wal-append.queue_time_us":214694}
W20260709 13:05:56.484547 18167 consensus_meta.cc:287] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.534s	user 0.009s	sys 0.001s
I20260709 13:05:56.432732 18500 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:56.503850 16006 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:05:56.481639 18579 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:05:56.512444 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.490962 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:56.488169 18724 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:05:56.526587 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 67917.8us
I20260709 13:05:56.480091 18177 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:56.509138 18722 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.536427 16533 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:56.506390 18727 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.519441 18579 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:56.492057 18699 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:56.564581 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:56.523593 16002 raft_consensus.cc:1275] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:05:56.513718 18711 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:05:56.579131 18700 log.cc:927] Time spent T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
W20260709 13:05:56.579892 18701 log.cc:927] Time spent T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
W20260709 13:05:56.611513 18698 log.cc:927] Time spent T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.291s	user 0.000s	sys 0.001s
I20260709 13:05:56.612236 16005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1716) took 1903 ms. Trace:
I20260709 13:05:56.612910 18737 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:05:56.532509 18725 log.cc:927] Time spent T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
W20260709 13:05:56.582803 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 46461.1us
I20260709 13:05:56.583925 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:56.589130 18376 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:56.506212 18167 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:56.627992 16008 raft_consensus.cc:1275] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:05:56.628152 16004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1719) took 1906 ms. Trace:
I20260709 13:05:56.528680 18724 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:56.518942 18627 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:56.628731 18177 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:56.628968 18724 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:56.629097 18627 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:56.631098 18627 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:56.591593 15902 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:56.593433 18727 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:56.596661 18500 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:56.560637 18393 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:56.539882 18577 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:56.598942 18258 raft_consensus.cc:3055] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:56.632735 18258 raft_consensus.cc:740] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:56.633517 18727 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:56.628486 16004 rpcz_store.cc:276] 0709 13:05:54.722015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.018406 (+1296391us) service_pool.cc:224] Handling call
0709 13:05:56.081441 (+ 63035us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:56.351102 (+269661us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:56.351136 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:56.384924 (+ 33788us) log.cc:844] Serialized 26 byte log entry
0709 13:05:56.469636 (+ 84712us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:56.528067 (+ 58431us) log.cc:844] Serialized 14 byte log entry
0709 13:05:56.528257 (+   190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.528326 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:56.627950 (+ 99624us) raft_consensus.cc:1685] finished
0709 13:05:56.627971 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:56.628088 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":122567,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":17556,"raft.run_cpu_time_us":332,"raft.run_wall_time_us":29927,"thread_start_us":333442,"threads_started":3,"wal-append.queue_time_us":45541}
I20260709 13:05:56.634472 18727 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:56.633826 18258 consensus_queue.cc:260] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:56.634827 18167 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:56.634934 18727 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:56.637029 18727 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:56.637217 18699 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:56.490110 18667 consensus_queue.cc:237] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:56.638238 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 46726.7us
I20260709 13:05:56.607234 18490 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:56.611446 18574 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:56.612638 16005 rpcz_store.cc:276] 0709 13:05:54.708824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:55.785959 (+1077135us) service_pool.cc:224] Handling call
0709 13:05:55.786582 (+   623us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:55.843017 (+ 56435us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:55.843056 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:55.873988 (+ 30932us) log.cc:844] Serialized 26 byte log entry
0709 13:05:56.243973 (+369985us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:56.244011 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.244080 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:56.612007 (+367927us) raft_consensus.cc:1685] finished
0709 13:05:56.612026 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:56.612169 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8355,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":131,"thread_start_us":306117,"threads_started":1,"wal-append.queue_time_us":488881}
I20260709 13:05:56.581950 16009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1725) took 1792 ms. Trace:
I20260709 13:05:56.656548 16009 rpcz_store.cc:276] 0709 13:05:54.789899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.430589 (+1640690us) service_pool.cc:224] Handling call
0709 13:05:56.432654 (+  2065us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:56.455109 (+ 22455us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:05:56.455130 (+    21us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:56.455453 (+   323us) log.cc:844] Serialized 26 byte log entry
0709 13:05:56.578130 (+122677us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:56.581625 (+  3495us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.581712 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:56.581755 (+    43us) raft_consensus.cc:1685] finished
0709 13:05:56.581771 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:56.581894 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1143,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":13721,"thread_start_us":81153,"threads_started":1,"wal-append.queue_time_us":91370}
I20260709 13:05:56.616755 16561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 749) took 1850 ms. Trace:
I20260709 13:05:56.616887 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1807) took 1840 ms. Trace:
I20260709 13:05:56.617712 18732 consensus_queue.cc:1059] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:56.493675 18649 consensus_queue.cc:237] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:56.584344 18722 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:56.625442 16011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1721) took 1862 ms. Trace:
I20260709 13:05:56.626417 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:56.603515 15862 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
I20260709 13:05:56.607321 18580 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.637496 18727 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:56.668370 15861 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
I20260709 13:05:56.650581 15902 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:56.528321 18163 consensus_queue.cc:237] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 4, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:56.686162 18722 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:56.664004 18490 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:05:56.635542 18258 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:56.619450 18738 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5.
I20260709 13:05:56.690433 18590 raft_consensus.cc:3055] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:56.686328 18727 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:56.690233 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:56.691039 15934 raft_consensus.cc:1275] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:05:56.700052 15932 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:56.619344 18711 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:56.671187 18363 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:56.666435 16011 rpcz_store.cc:276] 0709 13:05:54.762726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.192387 (+1429661us) service_pool.cc:224] Handling call
0709 13:05:56.192806 (+   419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:56.193178 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:56.193204 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:56.226873 (+ 33669us) log.cc:844] Serialized 26 byte log entry
0709 13:05:56.432274 (+205401us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:56.456286 (+ 24012us) log.cc:844] Serialized 14 byte log entry
0709 13:05:56.625095 (+168809us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.625197 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:56.625260 (+    63us) raft_consensus.cc:1685] finished
0709 13:05:56.625280 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:56.625390 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":18521,"prepare.run_wall_time_us":7225,"raft.queue_time_us":34089,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":7197,"spinlock_wait_cycles":19273600,"thread_start_us":304952,"threads_started":2,"wal-append.queue_time_us":261496}
I20260709 13:05:56.632759 18577 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:56.650904 18699 raft_consensus.cc:2804] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:56.689867 18732 raft_consensus.cc:3055] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:56.684973 15928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1792) took 3357 ms. Trace:
I20260709 13:05:56.690145 18709 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:56.686980 15756 catalog_manager.cc:5697] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:56.708942 18732 raft_consensus.cc:740] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:56.698740 18590 raft_consensus.cc:740] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:56.665166 18376 raft_consensus.cc:697] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:56.717705 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:56.620157 18737 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:56.718931 18643 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.931s	user 0.022s	sys 0.042s
I20260709 13:05:56.661918 16002 rpcz_store.cc:276] 0709 13:05:54.776092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.412386 (+1636294us) service_pool.cc:224] Handling call
0709 13:05:56.489275 (+ 76889us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:56.606574 (+117299us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.616809 (+ 10235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:56.687922 18743 consensus_queue.cc:1048] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:05:56.713677 18363 consensus_queue.cc:1048] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:05:56.713758 18699 raft_consensus.cc:697] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:56.684545 18580 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:05:56.726634 18643 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.946s	user 0.022s	sys 0.043s
I20260709 13:05:56.712380 18739 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:56.739985 15756 catalog_manager.cc:5697] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 3 to 4, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 4 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:56.700145 18724 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:56.742385 18579 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:56.735687 18709 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.712970 18577 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:05:56.752688 18699 consensus_queue.cc:237] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:56.753594 18577 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:56.750451 15756 catalog_manager.cc:5697] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 2 to 4, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 4 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
I20260709 13:05:56.758142 15934 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6.
I20260709 13:05:56.758899 15930 raft_consensus.cc:1240] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:56.744908 18643 raft_consensus.cc:697] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 4 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
W20260709 13:05:56.762444 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.767299 18724 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:56.767616 18579 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:56.749826 18732 consensus_queue.cc:260] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:56.768451 18732 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:56.711685 15928 rpcz_store.cc:276] 0709 13:05:53.327262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.329700 (+  2438us) service_pool.cc:224] Handling call
0709 13:05:53.382586 (+ 52886us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:54.207453 (+824867us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:54.214075 (+  6622us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:54.300445 (+ 86370us) log.cc:844] Serialized 52 byte log entry
0709 13:05:54.511856 (+211411us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:54.513338 (+  1482us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.684569 (+2171231us) spinlock_profiling.cc:243] Waited 403 ms on lock 0x7b54002cc754. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:56.684621 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:56.684678 (+    57us) raft_consensus.cc:1685] finished
0709 13:05:56.684691 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:56.684909 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":33545,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":118,"spinlock_wait_cycles":926916864,"thread_start_us":539109,"threads_started":2,"wal-append.queue_time_us":317738,"wal-append.run_cpu_time_us":2899,"wal-append.run_wall_time_us":1262839}
I20260709 13:05:56.661720 16561 rpcz_store.cc:276] 0709 13:05:54.765933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:55.184172 (+418239us) service_pool.cc:224] Handling call
0709 13:05:55.184536 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:55.837789 (+653253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:55.837834 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:55.838367 (+   533us) log.cc:844] Serialized 26 byte log entry
0709 13:05:56.134416 (+296049us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:56.136735 (+  2319us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.600300 (+463565us) spinlock_profiling.cc:243] Waited 248 ms on lock 0x7b5400500354. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:56.600357 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:56.613152 (+ 12795us) raft_consensus.cc:1685] finished
0709 13:05:56.613179 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:56.616657 (+  3478us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":364530,"raft.queue_time_us":366232,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":82,"spinlock_wait_cycles":570195328,"thread_start_us":273126,"threads_started":1,"wal-append.queue_time_us":403819}
I20260709 13:05:56.769672 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:56.769346 18643 consensus_queue.cc:237] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:56.770709 15928 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:56.757050 18709 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:56.760051 18590 consensus_queue.cc:260] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:56.779804 18376 consensus_queue.cc:237] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:56.780563 18590 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:56.781092 18709 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:56.787498 15756 catalog_manager.cc:5697] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:56.789088 15756 catalog_manager.cc:5697] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:56.792155 18727 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:56.790714 15756 catalog_manager.cc:5697] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 2 to 3, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 3 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:56.771651 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:56.795521 15900 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:56.800006 15756 catalog_manager.cc:5697] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 2 to 3, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 3 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:56.801291 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:56.801856 15756 catalog_manager.cc:5697] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 2 to 3, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 3 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:56.802685 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:56.803324 15858 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:05:56.803881 15756 catalog_manager.cc:5697] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 2 to 3, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 3 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:56.805636 15756 catalog_manager.cc:5697] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:56.807652 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:56.807049 15756 catalog_manager.cc:5697] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 2 to 3, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 3 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:56.810532 15900 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:56.811364 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:56.780617 18711 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:56.810528 15756 catalog_manager.cc:5697] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 5 to 6, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 6 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:56.810806 15928 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:56.812620 15756 catalog_manager.cc:5697] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: config changed from index 3 to 4, term changed from 3 to 4, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to <none>, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 4 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:05:56.816139 18727 raft_consensus.cc:2804] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:56.816625 18727 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:56.816974 18727 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:56.818956 18737 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:56.821969 18692 log.cc:927] Time spent T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.854s	user 0.004s	sys 0.001s
I20260709 13:05:56.824594 18493 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:56.826122 18580 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:56.828476 15928 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:05:56.833277 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:56.834438 16561 raft_consensus.cc:2393] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 6: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:56.846899 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:56.846148 15900 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:56.864420 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:56.865113 16566 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:05:56.866029 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:56.866910 18461 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:56.866997 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:56.835587 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:56.870986 18737 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:56.871397 18737 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:56.871747 18737 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:56.876156 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:05:56.876320 18749 log.cc:927] Time spent T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:05:56.877225 16566 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:56.876896 16567 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
I20260709 13:05:56.887765 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:56.889838 18597 consensus_queue.cc:1048] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:56.889935 15827 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:56.890753 18580 raft_consensus.cc:2804] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:56.891229 18580 raft_consensus.cc:697] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 10 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:56.892738 15827 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:56.876423 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:56.896169 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:56.861536 18711 raft_consensus.cc:2804] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:56.896104 18468 raft_consensus.cc:1027] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 6 LEADER]: attempt to promote peer eb1e0493af1e46dbb6dec7e25525bab5: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260709 13:05:56.902637 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:56.896138 18744 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:56.901417 18711 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:56.905288 18464 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:56.907490 18711 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:56.912755 18474 raft_consensus.cc:3055] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:56.913132 18474 raft_consensus.cc:740] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:56.913902 18474 consensus_queue.cc:260] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:56.914812 18474 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:56.916813 15827 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:56.919159 16562 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 LEARNER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:05:56.907472 18580 consensus_queue.cc:237] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:56.919463 18667 raft_consensus.cc:2749] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:56.930450 18605 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:56.933545 16533 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:05:56.934916 15827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.944649 16533 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:05:56.947300 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.947875 18461 raft_consensus.cc:2804] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:56.948009 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:56.948338 18461 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:56.948570 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:56.948688 15858 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:56.949733 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:56.950724 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:56.953254 16533 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:56.954139 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:56.958060 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:56.958920 15862 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
I20260709 13:05:56.948726 18461 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:56.962059 15855 raft_consensus.cc:1240] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:05:56.965476 15858 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
I20260709 13:05:56.967113 18363 consensus_queue.cc:1059] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:05:56.969012 18363 raft_consensus.cc:3055] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:56.969390 18363 raft_consensus.cc:740] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:56.970237 18363 consensus_queue.cc:260] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:56.971352 18363 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:56.940150 18738 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.971796 18493 raft_consensus.cc:2804] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:56.972326 18493 raft_consensus.cc:697] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:56.972208 18738 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:56.973222 18493 consensus_queue.cc:237] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:56.974273 18738 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:56.941525 18666 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:05:56.955744 18680 raft_consensus.cc:2804] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:56.977365 18680 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:56.977768 18680 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:56.981825 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:56.985320 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 241140us
I20260709 13:05:56.988698 15900 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:57.003157 15902 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:56.994788 15900 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:57.007859 18709 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:57.008493 18709 raft_consensus.cc:697] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 5 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:57.010018 18709 consensus_queue.cc:237] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:57.010445 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:56.990140 18616 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.000579 18738 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:57.018147 15902 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:57.004009 18732 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.024338 18699 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:57.024900 15902 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:57.030607 15857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 521) took 2566 ms. Trace:
I20260709 13:05:57.030735 18743 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:57.030630 18732 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:57.032917 18727 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:57.033756 18732 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:57.034584 15936 raft_consensus.cc:1275] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:05:57.034991 15928 raft_consensus.cc:1240] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:57.036402 18727 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:57.036480 18711 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:57.039057 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:57.039621 15858 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:57.028584 18461 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:57.040519 18737 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:05:57.041671 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.042444 18737 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:57.015977 18722 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:57.044148 18711 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:57.023054 18738 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:57.052088 18738 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:57.053141 16565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 745) took 2328 ms. Trace:
I20260709 13:05:57.053517 16565 rpcz_store.cc:276] 0709 13:05:54.724968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.963629 (+238661us) service_pool.cc:224] Handling call
0709 13:05:56.013264 (+1049635us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:56.014238 (+   974us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:57.052941 (+1038703us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54000cdad0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:57.053084 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2411431680}
I20260709 13:05:57.054807 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:57.062701 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:57.062819 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:57.063341 15860 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:57.063606 15855 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:05:57.064277 15854 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 1826) took 1919 ms (client timeout 1726 ms). Trace:
W20260709 13:05:57.064682 15854 rpcz_store.cc:269] 0709 13:05:55.144931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:55.631513 (+486582us) service_pool.cc:224] Handling call
0709 13:05:57.064098 (+1432585us) spinlock_profiling.cc:243] Waited 98.8 ms on lock 0x7b54003b6354. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:57.064208 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":227311360}
I20260709 13:05:57.065960 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:57.066707 16565 raft_consensus.cc:2393] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 10: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:57.068158 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:57.071097 18732 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.071700 18732 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:57.074124 15855 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:05:57.074574 18732 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:57.044171 18461 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:57.076009 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:57.076575 15861 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
W20260709 13:05:57.077209 15937 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 1712) took 2200 ms (client timeout 2625 ms). Trace:
I20260709 13:05:57.031065 15857 rpcz_store.cc:276] 0709 13:05:54.463958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.464093 (+   135us) service_pool.cc:224] Handling call
0709 13:05:54.490278 (+ 26185us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:54.629668 (+139390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:54.629697 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:54.652277 (+ 22580us) log.cc:844] Serialized 26 byte log entry
0709 13:05:54.947959 (+295682us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:54.948000 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:57.027290 (+2079290us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b5400218c54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:57.027323 (+    33us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:57.027350 (+    27us) raft_consensus.cc:1685] finished
0709 13:05:57.027356 (+     6us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:57.030535 (+  3179us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":429,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":12508,"spinlock_wait_cycles":319817600,"thread_start_us":233754,"threads_started":1,"wal-append.queue_time_us":276146,"wal-append.run_cpu_time_us":2091,"wal-append.run_wall_time_us":1086805}
W20260709 13:05:57.077517 15937 rpcz_store.cc:269] 0709 13:05:54.826801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.884121 (+ 57320us) service_pool.cc:224] Handling call
0709 13:05:57.026997 (+2142876us) spinlock_profiling.cc:243] Waited 519 ms on lock 0x7b54002c2754. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:57.027119 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1193944832}
W20260709 13:05:57.086540 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:57.087139 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:57.087549 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.088289 15902 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:57.089248 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:57.089958 15933 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:57.089306 15858 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:05:57.091876 15902 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:57.029769 18699 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:57.092866 18699 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:57.028278 18616 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:57.093514 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 1.01947e+06us
I20260709 13:05:57.094651 18616 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:57.094830 15974 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:57.096525 16008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1730) took 2267 ms. Trace:
I20260709 13:05:57.096917 16008 rpcz_store.cc:276] 0709 13:05:54.829279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.493302 (+1664023us) service_pool.cc:224] Handling call
0709 13:05:56.508458 (+ 15156us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:56.628365 (+119907us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:57.096321 (+467956us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b54000eddd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:57.096472 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":274061056}
I20260709 13:05:57.086410 18677 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:57.098551 16003 raft_consensus.cc:1240] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:57.098440 18677 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:57.100826 16003 raft_consensus.cc:1275] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:05:57.101819 15937 raft_consensus.cc:1275] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:05:57.107720 15902 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:57.107949 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:57.110404 15902 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:57.111850 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:57.114355 16007 raft_consensus.cc:1275] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:05:57.119473 15860 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:05:57.108629 18616 raft_consensus.cc:2749] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:57.106519 18638 consensus_queue.cc:1048] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 13:05:57.112363 18590 raft_consensus.cc:2749] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:57.112666 16008 raft_consensus.cc:2376] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6. Re-sending same reply.
I20260709 13:05:57.125458 15974 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:57.126442 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:57.126735 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:57.127823 16008 raft_consensus.cc:2410] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 9 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 8.
I20260709 13:05:57.128547 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:57.129153 16568 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:57.129228 15974 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:57.134179 15933 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:05:57.136842 15974 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:57.121868 18737 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:57.124287 18474 consensus_queue.cc:1048] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 13:05:57.121299 18743 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:57.134613 18722 raft_consensus.cc:2764] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:05:57.104099 18711 consensus_queue.cc:1059] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:05:57.145632 16568 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:05:57.134948 18728 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.134935 18590 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:57.148846 15902 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:57.148495 15974 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:57.151508 15974 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:57.153020 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:57.138410 18177 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:05:57.139935 18638 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:57.153728 16008 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:57.155239 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:57.156136 16568 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
I20260709 13:05:57.140863 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:57.160033 16563 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:05:57.163554 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:57.164153 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:57.154088 18605 raft_consensus.cc:2749] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:57.155297 18401 raft_consensus.cc:2749] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:57.162624 15902 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:57.155478 18177 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:57.172034 15974 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:57.172610 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:57.173276 16003 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:05:57.163514 18711 raft_consensus.cc:2804] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:57.155747 18583 consensus_queue.cc:1059] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:05:57.153458 18728 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:57.189649 15974 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:57.173276 18177 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:05:57.189674 16535 leader_election.cc:336] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 2.079s (SENT)
I20260709 13:05:57.174034 18732 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:57.176167 18605 raft_consensus.cc:2749] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:57.201853 18768 raft_consensus.cc:2749] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:57.205933 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:57.211320 15974 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:57.212538 16003 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:57.213999 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:57.204123 18728 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:57.195772 18732 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:57.219574 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:57.220638 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:05:57.221457 16564 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45246 (request call id 668) took 2555 ms (client timeout 2149 ms). Trace:
I20260709 13:05:57.210860 18474 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.221634 18732 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:57.222170 18474 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:56.438897 15853 rpcz_store.cc:276] 0709 13:05:55.167003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.023131 (+856128us) service_pool.cc:224] Handling call
0709 13:05:56.023444 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:56.023649 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:05:56.267386 (+243737us) log.cc:844] Serialized 14 byte log entry
0709 13:05:56.386212 (+118826us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:56.386278 (+    66us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:56.386331 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.386414 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:56.386545 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":682,"prepare.queue_time_us":227577,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":2678,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":27739,"spinlock_wait_cycles":8512768,"thread_start_us":360868,"threads_started":2,"wal-append.queue_time_us":157826}
W20260709 13:05:57.221948 16564 rpcz_store.cc:269] 0709 13:05:54.665617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.679692 (+ 14075us) service_pool.cc:224] Handling call
0709 13:05:57.220516 (+2540824us) spinlock_profiling.cc:243] Waited 3.15 ms on lock 0x7b5800044798. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283eee 0000000000375939 00007f73ef2f71e0 00007f73ef3c00c4 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:05:57.221373 (+   857us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7245184}
I20260709 13:05:57.226109 16564 raft_consensus.cc:1275] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:05:57.226586 18177 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:57.228067 18738 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:57.228581 18743 consensus_queue.cc:1048] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:05:57.229074 18177 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:57.214619 18680 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:57.230520 16565 raft_consensus.cc:1240] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:57.230849 18709 raft_consensus.cc:3055] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:57.231258 18709 raft_consensus.cc:740] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:57.234192 18680 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:05:57.237174 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.238778 15856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 1701) took 2220 ms. Trace:
I20260709 13:05:57.245211 15856 rpcz_store.cc:276] 0709 13:05:55.018332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:55.019176 (+   844us) service_pool.cc:224] Handling call
0709 13:05:57.238589 (+2219413us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b5400217854. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:57.238722 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3234776960}
W20260709 13:05:57.212663 18765 log.cc:927] Time spent T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.125s	user 0.002s	sys 0.000s
I20260709 13:05:57.194386 18711 raft_consensus.cc:697] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:57.248100 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:57.249238 15937 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:57.250607 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:57.251729 16007 raft_consensus.cc:2410] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 6 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:05:57.252535 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:57.202668 18677 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:57.254751 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:57.255072 15930 raft_consensus.cc:2410] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:05:57.255568 15932 raft_consensus.cc:2393] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 8: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:57.256006 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:57.256639 15930 raft_consensus.cc:2368] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 5. Current term is 6.
I20260709 13:05:57.257339 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:57.214931 18583 raft_consensus.cc:2764] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:05:57.239370 18699 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:57.264320 18711 consensus_queue.cc:237] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:57.270222 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.275985 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:57.276618 15929 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:57.279102 18679 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:57.279640 18679 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:57.281651 18679 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:57.282680 15900 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:05:57.283445 18743 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:57.283507 15900 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:57.284128 18743 raft_consensus.cc:697] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 7 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:57.284515 15900 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:57.285368 18724 raft_consensus.cc:2804] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:05:57.285569 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.285800 18724 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:57.286029 18616 raft_consensus.cc:2804] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:57.286222 18724 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:57.286641 18616 raft_consensus.cc:697] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:57.287621 18616 consensus_queue.cc:237] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:57.288021 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:05:57.290282 18766 log.cc:927] Time spent T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:05:57.291335 16533 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:57.291656 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:57.292933 18474 leader_election.cc:290] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:57.293493 18474 raft_consensus.cc:2804] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:57.293985 18474 raft_consensus.cc:697] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:57.285045 18743 consensus_queue.cc:237] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:57.307816 15937 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
W20260709 13:05:57.310431 15974 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.14.103.62:42985, user_credentials={real_user=slave}} blocked reactor thread for 46891us
I20260709 13:05:57.311753 15860 raft_consensus.cc:1240] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:57.313513 15862 raft_consensus.cc:1275] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:05:57.312060 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:57.313773 16533 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:57.314594 15929 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 4.
I20260709 13:05:57.314697 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:57.315958 15858 raft_consensus.cc:2410] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:05:57.300259 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:57.321556 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:57.321897 16007 raft_consensus.cc:2393] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 7: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:57.322369 16003 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:57.325457 15930 raft_consensus.cc:2393] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 5: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:57.326042 15974 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:57.326443 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:57.327646 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:57.327649 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:57.327394 15974 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:57.329957 15974 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:57.331961 15974 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:57.343689 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:57.343967 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:57.344571 15859 raft_consensus.cc:2393] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 5: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:57.344493 15974 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:57.344946 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:57.349988 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:57.350607 15856 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:57.354869 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:57.355146 15853 raft_consensus.cc:1240] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:57.355410 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:57.355531 15858 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
I20260709 13:05:57.357231 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:57.359330 15974 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:57.363725 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:57.364446 16003 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:57.367059 15827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.320963 18597 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:57.334611 18145 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:57.371158 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.343230 18167 raft_consensus.cc:2804] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:57.295382 18474 consensus_queue.cc:237] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:57.349519 18177 raft_consensus.cc:2749] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:57.357227 18583 raft_consensus.cc:2804] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:57.356065 18677 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:57.355139 18401 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:57.388994 16003 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:05:57.311159 18643 consensus_queue.cc:1059] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 13:05:57.392551 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:57.369648 18605 raft_consensus.cc:2749] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:57.398464 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:57.399142 15862 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:57.402133 16003 raft_consensus.cc:1275] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:05:57.403658 16008 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:05:57.405752 15862 raft_consensus.cc:1240] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:57.407428 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:57.408106 15857 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:57.411697 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:57.412482 15854 raft_consensus.cc:2393] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 6: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
W20260709 13:05:57.412838 16036 heartbeater.cc:646] Failed to heartbeat to 127.14.103.62:42985 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.14.103.62:42985 timed out after 15.000s (SENT)
I20260709 13:05:57.413252 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:57.312649 18709 consensus_queue.cc:260] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:57.384886 18145 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:57.381412 18597 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:57.414276 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:57.392073 18583 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:57.455004 15856 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:05:57.495023 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:57.414700 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:57.512508 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:57.434054 15857 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6.
W20260709 13:05:57.480002 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.580824 15859 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:57.587422 15902 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:57.393633 18401 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:57.393939 18167 raft_consensus.cc:697] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:57.603477 15862 raft_consensus.cc:2393] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 6: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:05:57.604269 15976 connection.cc:256] RPC call timeout handler was delayed by -1.4617057369998747s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1751s for a response
W20260709 13:05:57.605406 15976 connection.cc:256] RPC call timeout handler was delayed by -1.4230057369999258s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1794s for a response
I20260709 13:05:57.606427 15902 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:57.607810 15976 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:05:57.608939 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 248183us
I20260709 13:05:57.447496 18737 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:57.406822 18590 raft_consensus.cc:2764] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:05:57.508985 16564 raft_consensus.cc:1275] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:05:57.598533 18583 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:57.506927 18699 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:57.598261 18401 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:57.624297 18724 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:57.631804 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:57.632421 15930 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 6: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:57.633347 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:57.634213 16564 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:05:57.634562 18724 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:57.635325 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:57.635768 18167 consensus_queue.cc:237] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:57.637162 16564 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
I20260709 13:05:57.621249 15976 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:57.640156 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:57.612015 18746 raft_consensus.cc:2749] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:57.615182 18739 raft_consensus.cc:2749] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:57.445892 18732 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:57.622979 18709 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:57.619138 18677 consensus_queue.cc:1059] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:05:57.674419 15859 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:05:57.678330 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:57.684396 18145 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:57.684863 15934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53636 (request call id 549) took 1003 ms. Trace:
I20260709 13:05:57.685385 15934 rpcz_store.cc:276] 0709 13:05:56.682380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.706951 (+ 24571us) service_pool.cc:224] Handling call
0709 13:05:57.684640 (+977689us) spinlock_profiling.cc:243] Waited 38.1 ms on lock 0x7b540034ba54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:57.684792 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":87538560}
I20260709 13:05:57.686859 16533 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:57.644129 18780 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:57.694236 18688 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:57.694718 18688 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:57.695091 18688 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:57.696048 15935 raft_consensus.cc:1275] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:05:57.697317 18597 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:57.701097 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 94751.4us
I20260709 13:05:57.702327 16562 raft_consensus.cc:1240] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:57.707487 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:57.707733 15935 raft_consensus.cc:1275] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:05:57.708060 16562 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:57.709100 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 110541us
I20260709 13:05:57.644147 18768 consensus_queue.cc:1048] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.058s
W20260709 13:05:57.721766 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 100586us
I20260709 13:05:57.713104 18739 consensus_queue.cc:1048] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:05:57.726987 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:57.727710 16003 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
W20260709 13:05:57.721750 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 109492us
I20260709 13:05:57.732580 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:57.733428 16011 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:05:57.734464 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:57.709035 18743 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:05:57.737556 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:57.738173 16009 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:57.712566 18649 consensus_queue.cc:1048] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:05:57.744483 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:57.745837 16009 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:57.744784 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:57.746759 16006 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:57.727767 18786 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:57.753345 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:57.753353 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:57.754145 16003 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:57.754133 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:57.754130 16010 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
W20260709 13:05:57.756435 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 45469.3us
I20260709 13:05:57.757719 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:57.758181 16535 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:05:57.798178 16533 leader_election.cc:336] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.516s (SENT)
I20260709 13:05:57.826462 15974 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:57.827464 16562 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:05:57.830914 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:57.848909 16562 raft_consensus.cc:1275] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:05:57.885347 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:57.886257 16009 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
W20260709 13:05:57.887434 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 61060.5us
I20260709 13:05:57.888698 15974 leader_election.cc:400] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Vote denied by peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) with higher term. Message: Invalid argument: T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 5. Current term is 6.
I20260709 13:05:57.889060 15974 leader_election.cc:403] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:05:57.849476 18771 log.cc:927] Time spent T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
W20260709 13:05:57.952045 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 63530.3us
I20260709 13:05:57.953510 15974 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:57.955420 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:57.956079 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.960875 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:57.976086 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:57.976899 16009 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
I20260709 13:05:57.977909 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:57.979957 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:57.980743 16006 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:57.981727 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:57.982921 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:57.983534 16006 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:57.985076 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:57.986186 16009 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:57.987464 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:57.986363 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:57.988135 16009 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:57.988897 16003 raft_consensus.cc:2368] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5.
I20260709 13:05:58.009266 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:57.974474 18785 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:57.895783 18677 consensus_queue.cc:1048] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.114s
W20260709 13:05:58.017930 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 64473.3us
I20260709 13:05:57.997790 18732 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:58.018784 15974 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:58.024986 16009 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:05:58.025911 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:58.027338 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:58.027750 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 269691us
I20260709 13:05:58.026605 16009 raft_consensus.cc:2393] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 6: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:58.030392 16006 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:05:58.032200 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:58.032750 16006 raft_consensus.cc:2376] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6. Re-sending same reply.
I20260709 13:05:58.033514 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:58.033594 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:58.034348 16006 raft_consensus.cc:2410] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 7 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:05:58.036968 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:58.037835 16006 raft_consensus.cc:1275] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:05:58.039301 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:58.040248 16011 raft_consensus.cc:2393] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 5: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:58.035650 18793 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:05:58.033041 18790 raft_consensus.cc:2749] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:58.037580 18768 raft_consensus.cc:2749] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:58.056456 16535 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:58.058737 18500 consensus_queue.cc:1048] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:58.062721 18363 consensus_queue.cc:1048] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:58.063263 18401 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:58.061709 18688 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:58.065358 18401 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:58.070479 18688 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:58.076726 18638 raft_consensus.cc:3055] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:58.077282 18638 raft_consensus.cc:740] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:58.081168 16535 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:58.084302 18796 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:58.117334 18322 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:58.079509 18677 raft_consensus.cc:2749] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:58.117915 18322 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:05:58.119930 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 101209us
I20260709 13:05:58.120355 16535 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:58.120714 15974 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
W20260709 13:05:58.123637 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 197219us
I20260709 13:05:58.122879 16535 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:58.129825 15974 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:05:58.135149 15974 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:58.136454 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:58.137603 16562 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:58.139890 16535 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:58.145269 15902 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:58.148650 16562 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:05:58.148985 15902 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:58.121851 18322 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:58.124392 18688 raft_consensus.cc:2749] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:05:58.151786 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:58.130812 18798 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:58.154367 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:58.078300 18638 consensus_queue.cc:260] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:58.157503 15902 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:58.133626 18401 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:58.160429 15862 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:05:58.165244 16006 raft_consensus.cc:1275] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:05:58.149598 18739 raft_consensus.cc:2804] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:58.170974 15857 raft_consensus.cc:1240] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 6. Current term is 7. Ops: [3.3-6.4]
I20260709 13:05:58.149353 18624 raft_consensus.cc:2804] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:58.168500 15902 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:58.168992 16562 raft_consensus.cc:1240] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 7. Current term is 8. Ops: [6.8-7.9]
I20260709 13:05:58.155874 18363 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:58.159200 18768 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:58.151491 18786 raft_consensus.cc:697] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:58.157474 18677 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:58.157713 18799 raft_consensus.cc:2749] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:58.159603 18722 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:58.151355 18500 consensus_queue.cc:1048] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 13:05:58.161690 18790 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:58.170258 18739 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:05:58.259815 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 70775.7us
I20260709 13:05:58.244444 18363 raft_consensus.cc:697] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:58.170324 18711 raft_consensus.cc:2749] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:58.173022 18798 raft_consensus.cc:697] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 4 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:58.176134 18638 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:58.175911 18624 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:58.250366 18743 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:58.251511 18768 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:58.283592 15862 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:05:58.290860 18739 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:58.260416 18677 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:58.300690 18709 consensus_meta.cc:287] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.629s	user 0.013s	sys 0.000s
I20260709 13:05:58.313953 18624 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:58.293428 18790 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:58.289314 18722 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:58.345956 15932 raft_consensus.cc:2410] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:05:58.336800 18743 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:05:58.353994 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 185555us
I20260709 13:05:58.357697 15902 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:58.349849 18722 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:58.359194 15902 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:58.362079 15902 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:58.360301 15976 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:58.364744 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:58.314565 18786 consensus_queue.cc:237] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:58.359517 18743 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:58.368373 15935 raft_consensus.cc:1240] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:58.373728 15976 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:58.359764 18709 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:58.376276 15935 raft_consensus.cc:1240] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 7 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:58.379329 18401 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:58.384886 18401 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:58.385315 18401 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:58.388284 18583 consensus_meta.cc:287] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.733s	user 0.008s	sys 0.004s
I20260709 13:05:58.388720 18583 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:58.389734 16008 raft_consensus.cc:2393] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 5: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:58.390678 18583 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:58.376222 18799 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:58.374051 18805 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:58.404299 18805 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:58.406400 18805 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:58.407157 18738 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:58.408541 18768 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:58.364071 18711 raft_consensus.cc:2804] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:58.420467 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:58.421125 15902 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:58.425580 15932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 1733) took 1172 ms. Trace:
I20260709 13:05:58.425974 15932 rpcz_store.cc:276] 0709 13:05:57.253641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.291392 (+ 37751us) service_pool.cc:224] Handling call
0709 13:05:58.425395 (+1134003us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540008a7d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:58.425522 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2422603520}
I20260709 13:05:58.419059 18711 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:58.428277 18711 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:58.429116 18783 consensus_queue.cc:1048] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:58.430480 18401 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:58.433667 18401 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:58.434397 18401 consensus_queue.cc:1059] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:58.435328 18401 consensus_queue.cc:1243] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer 8430db80a39d4868bcf7641c16de35ea log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:05:58.355320 18363 consensus_queue.cc:237] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:58.436771 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:58.438531 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:58.421126 15853 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:58.442359 16564 raft_consensus.cc:1240] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 3. Current term is 5. Ops: []
I20260709 13:05:58.443233 16011 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:05:58.375335 18500 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:58.445885 16011 raft_consensus.cc:1275] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:05:58.430529 18783 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:58.450902 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:58.453204 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:58.453965 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:58.454449 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:58.454701 16562 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:58.455163 16569 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:58.458240 18500 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:58.447778 18768 consensus_queue.cc:1059] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 13:05:58.460405 18500 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:58.460772 18460 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:58.449210 18783 raft_consensus.cc:697] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:58.461381 18460 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:58.462145 18783 consensus_queue.cc:237] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:58.460930 18768 raft_consensus.cc:3055] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:58.465538 16569 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:05:58.467829 18768 raft_consensus.cc:740] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:58.468931 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:58.468813 18768 consensus_queue.cc:260] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:58.469614 16569 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:58.470088 18768 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:58.466085 18322 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:58.463493 18460 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:58.476972 18793 raft_consensus.cc:2749] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) with higher term. Message: Invalid argument: T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 5. Current term is 6.
I20260709 13:05:58.477293 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:05:58.494642 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 73564.1us
I20260709 13:05:58.495826 18401 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.496060 18817 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.496383 18401 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:58.496779 18817 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:58.497684 16567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 766) took 1508 ms. Trace:
I20260709 13:05:58.497908 18796 raft_consensus.cc:2749] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:58.498790 18401 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:58.498091 16567 rpcz_store.cc:276] 0709 13:05:56.989327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.989450 (+   123us) service_pool.cc:224] Handling call
0709 13:05:56.989769 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:56.990101 (+   332us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:56.990126 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:56.995486 (+  5360us) log.cc:844] Serialized 26 byte log entry
0709 13:05:57.236997 (+241511us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b18003474a0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e9f504bc 00007f73e729e48b 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f480d1
0709 13:05:57.237135 (+   138us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:57.237168 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:57.237229 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:58.455323 (+1218094us) spinlock_profiling.cc:243] Waited 5.85 ms on lock 0x7b18003474a0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb3f3 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd 000000000038f354 000000000038f199 00007f73e729e83a
0709 13:05:58.497486 (+ 42163us) raft_consensus.cc:1685] finished
0709 13:05:58.497507 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:58.497632 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":527,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":6767,"spinlock_wait_cycles":39486464,"thread_start_us":16453,"threads_started":1,"wal-append.queue_time_us":16414}
I20260709 13:05:58.499233 18817 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:58.502571 16562 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
I20260709 13:05:58.503545 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:05:58.504634 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:58.505484 16567 raft_consensus.cc:3055] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:58.508430 16567 raft_consensus.cc:740] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:05:58.508440 16003 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 3.
I20260709 13:05:58.509608 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:58.510221 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:58.510248 16003 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:05:58.510848 16011 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
I20260709 13:05:58.512769 16566 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
W20260709 13:05:58.517270 16566 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45244 (request call id 767) took 1526 ms (client timeout 1903 ms). Trace:
I20260709 13:05:58.506863 15902 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:58.514003 16535 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:05:58.518182 16566 rpcz_store.cc:269] 0709 13:05:56.991271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.991388 (+   117us) service_pool.cc:224] Handling call
0709 13:05:58.477139 (+1485751us) spinlock_profiling.cc:243] Waited 7.74 ms on lock 0x7b5800044798. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:05:58.517219 (+ 40080us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17812736}
I20260709 13:05:58.509215 16567 consensus_queue.cc:260] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:58.516119 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:58.418016 18709 consensus_queue.cc:1048] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.084s
I20260709 13:05:58.520257 16567 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:58.505900 16569 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:05:58.514261 15976 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:58.524067 15976 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:58.524121 15902 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:58.525993 16009 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:05:58.525701 15976 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:58.526837 18813 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:58.528061 18813 raft_consensus.cc:697] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:58.528388 18677 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:58.528941 18813 consensus_queue.cc:237] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:58.530719 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:58.527324 18638 raft_consensus.cc:2804] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:58.532039 18638 raft_consensus.cc:697] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 5 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:58.533170 18638 consensus_queue.cc:237] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:58.538594 18793 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:58.361578 18798 consensus_queue.cc:237] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:58.539397 18793 raft_consensus.cc:697] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:58.539494 18743 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
W20260709 13:05:58.543074 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:58.543665 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:58.535656 16567 raft_consensus.cc:1275] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:05:58.544507 15902 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:58.448649 18474 consensus_queue.cc:1048] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.072s
W20260709 13:05:58.552454 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:58.449435 18797 consensus_queue.cc:1059] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 13:05:58.554000 15902 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:05:58.556878 15931 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53636 (request call id 537) took 2491 ms (client timeout 2364 ms). Trace:
I20260709 13:05:58.548666 15853 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
W20260709 13:05:58.557416 15931 rpcz_store.cc:269] 0709 13:05:56.065095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.171145 (+106050us) service_pool.cc:224] Handling call
0709 13:05:58.556635 (+2385490us) spinlock_profiling.cc:243] Waited 217 ms on lock 0x7b54000992d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:58.556773 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":499173120}
I20260709 13:05:58.541097 18793 consensus_queue.cc:237] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:58.542407 18500 raft_consensus.cc:2804] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:58.566915 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:58.567934 18790 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:58.586221 18640 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.586776 18640 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:58.581624 18797 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.589615 18797 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:58.537948 18711 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:58.592967 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:58.569211 18605 raft_consensus.cc:3055] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:58.556381 18738 raft_consensus.cc:2749] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:58.551374 18649 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:58.632552 16570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 683) took 2511 ms. Trace:
I20260709 13:05:58.633029 18649 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:58.633076 16570 rpcz_store.cc:276] 0709 13:05:56.121023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.121171 (+   148us) service_pool.cc:224] Handling call
0709 13:05:56.844944 (+723773us) spinlock_profiling.cc:243] Waited 34.3 ms on lock 0x7b5800044798. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef381fc0 00007f73ef36c929 00007f73ef36c3d2 00007f73e88e3bd4
0709 13:05:56.845187 (+   243us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:56.861345 (+ 16158us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:56.861373 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:58.474151 (+1612778us) spinlock_profiling.cc:243] Waited 15.5 ms on lock 0x7b2c00047e58. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e8ca1a11 00007f73e8cb1372 00007f73ea3e3a01 00007f73ea3a843a 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2
0709 13:05:58.474584 (+   433us) log.cc:844] Serialized 26 byte log entry
0709 13:05:58.526259 (+ 51675us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:58.526298 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:58.526367 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:58.632279 (+105912us) raft_consensus.cc:1685] finished
0709 13:05:58.632312 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:58.632484 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":82,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":82,"spinlock_wait_cycles":114566272,"thread_start_us":51318,"threads_started":1,"wal-append.queue_time_us":85395}
I20260709 13:05:58.572474 18709 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:05:58.635413 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:58.551856 18818 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:05:58.553114 18817 raft_consensus.cc:2804] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:58.583207 16011 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:58.636286 18817 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:58.636732 18817 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:58.581568 18500 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:58.584812 18743 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:58.581347 18722 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:58.572444 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:58.650520 16568 raft_consensus.cc:2376] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5. Re-sending same reply.
I20260709 13:05:58.651444 16568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45232 (request call id 630) took 1070 ms. Trace:
I20260709 13:05:58.651901 16568 rpcz_store.cc:276] 0709 13:05:57.580654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.591433 (+ 10779us) service_pool.cc:224] Handling call
0709 13:05:58.572286 (+980853us) spinlock_profiling.cc:243] Waited 4.23 ms on lock 0x7b5800044798. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:05:58.651350 (+ 79064us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9731968}
I20260709 13:05:58.593605 15859 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
I20260709 13:05:58.556149 18702 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:58.605075 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:58.655018 16562 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
I20260709 13:05:58.504331 18739 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:58.611171 18711 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:58.635294 18709 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:58.579351 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:58.563467 18322 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:58.670529 18822 log.cc:927] Time spent T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.104s	user 0.002s	sys 0.000s
I20260709 13:05:58.622241 18799 raft_consensus.cc:2749] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:58.622521 18797 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:58.639688 18500 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:58.620424 18605 raft_consensus.cc:740] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:58.639791 18460 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.674733 18709 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:58.722261 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 168321us
I20260709 13:05:58.675020 18743 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:58.724331 15902 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:58.725751 15902 leader_election.cc:400] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5.
I20260709 13:05:58.727391 15902 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:58.729379 15902 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:58.673669 18702 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:58.720750 18460 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:58.409013 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:58.791400 18709 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:58.726851 18743 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:58.776157 18739 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:58.781405 18722 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:58.726976 18783 raft_consensus.cc:2804] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:58.838629 18817 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:58.767789 18809 raft_consensus.cc:2749] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:58.764326 18825 raft_consensus.cc:2749] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:58.850164 18817 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:58.843200 18783 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:58.854326 18783 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:58.873051 16007 raft_consensus.cc:1240] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:05:58.874521 16007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1812) took 1151 ms. Trace:
I20260709 13:05:58.874862 16007 rpcz_store.cc:276] 0709 13:05:57.722757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.722995 (+   238us) service_pool.cc:224] Handling call
0709 13:05:57.723345 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:58.873525 (+1150180us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:58.874314 (+   789us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54000ca654. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:58.874471 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2643487616}
I20260709 13:05:58.875079 18783 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:58.878844 18783 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:58.939956 18761 log.cc:927] Time spent T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 1.325s	user 0.005s	sys 0.008s
I20260709 13:05:58.953765 18743 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:59.037894 18787 log.cc:927] Time spent T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 1.026s	user 0.002s	sys 0.002s
I20260709 13:05:58.730509 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:05:59.075644 16563 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45232 (request call id 629) took 1496 ms (client timeout 1726 ms). Trace:
W20260709 13:05:59.076123 16563 rpcz_store.cc:269] 0709 13:05:57.579605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.598897 (+ 19292us) service_pool.cc:224] Handling call
0709 13:05:58.730366 (+1131469us) spinlock_profiling.cc:243] Waited 2.12 ms on lock 0x7b5800044798. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283eee 0000000000375939 00007f73ef2f71e0 00007f73ef3c00c4 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:05:59.075555 (+345189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4873600}
W20260709 13:05:59.091653 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.094568 15853 raft_consensus.cc:1275] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 13:05:59.107237 16533 leader_election.cc:336] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 2.050s (SENT)
I20260709 13:05:59.107870 16533 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:59.108809 18363 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:58.750195 15902 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:58.790009 18624 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:59.125444 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.126811 18789 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.127286 18624 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:59.129733 15853 raft_consensus.cc:3055] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:05:59.127494 18789 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:59.130122 15853 raft_consensus.cc:740] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:59.130825 15853 consensus_queue.cc:260] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:59.131834 15853 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:05:59.132901 18789 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:59.137876 15853 raft_consensus.cc:1275] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:05:59.138892 18789 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.139387 18789 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:59.141441 18789 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:59.142867 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:59.143721 15853 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
I20260709 13:05:59.146832 15853 raft_consensus.cc:1275] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:05:58.785085 16011 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6.
I20260709 13:05:59.155099 16007 raft_consensus.cc:1240] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 5. Current term is 6. Ops: [4.6-5.7]
I20260709 13:05:58.767449 15929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 1740) took 1507 ms. Trace:
I20260709 13:05:59.157006 15929 rpcz_store.cc:276] 0709 13:05:57.260006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.390412 (+130406us) service_pool.cc:224] Handling call
0709 13:05:57.402554 (+ 12142us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:57.402746 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:05:57.440375 (+ 37629us) log.cc:844] Serialized 14 byte log entry
0709 13:05:58.732154 (+1291779us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b180043af80. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f4823a 00007f73ea3b2a2a
0709 13:05:58.767044 (+ 34890us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:58.767128 (+    84us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:58.767199 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:58.767267 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:58.767392 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":31624,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":87,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":81,"spinlock_wait_cycles":38489728,"thread_start_us":34367,"threads_started":1,"wal-append.queue_time_us":1326078}
I20260709 13:05:59.164505 16011 raft_consensus.cc:1240] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:58.736079 18605 consensus_queue.cc:260] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:59.175889 18802 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:59.176581 18802 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:59.179581 18802 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:59.192413 15853 raft_consensus.cc:3055] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:59.201537 15853 raft_consensus.cc:740] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:59.202400 15853 consensus_queue.cc:260] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:59.203431 15853 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:59.207666 18586 raft_consensus.cc:3055] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:59.211566 18586 raft_consensus.cc:740] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:05:59.212445 18586 consensus_queue.cc:260] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:59.213521 18586 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:59.210144 16011 raft_consensus.cc:1275] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:05:59.216264 15827 connection.cc:256] RPC call timeout handler was delayed by -1.5347730490000231s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1785s for a response
W20260709 13:05:59.218130 15827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.223563 18840 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.218779 16006 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 3.
I20260709 13:05:59.224496 18163 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.225629 18163 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:59.228384 18163 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:59.224290 18840 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:59.232900 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.234714 18728 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:59.235592 15853 raft_consensus.cc:1275] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:05:59.299201 16009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41662 (request call id 1842) took 1332 ms. Trace:
I20260709 13:05:59.351958 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:59.536675 16009 rpcz_store.cc:276] 0709 13:05:57.966944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:58.036749 (+ 69805us) service_pool.cc:224] Handling call
0709 13:05:59.298951 (+1262202us) spinlock_profiling.cc:243] Waited 440 ms on lock 0x7b54001f72d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:59.299122 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1012467584}
W20260709 13:05:59.307603 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.409590 16561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 783) took 1700 ms. Trace:
I20260709 13:05:59.294108 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:59.538043 16562 raft_consensus.cc:1240] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:59.382828 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:58.786751 15928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 555) took 1892 ms. Trace:
I20260709 13:05:59.540452 16562 raft_consensus.cc:1275] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:05:59.249321 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:59.542529 16562 raft_consensus.cc:1275] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:05:59.290684 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:05:59.236208 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.352111 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:05:59.464965 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 166832us
W20260709 13:05:58.750911 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:59.544530 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.352679 15861 raft_consensus.cc:1240] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 4. Current term is 5. Ops: []
W20260709 13:05:59.287871 16008 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60156 (request call id 506) took 1839 ms (client timeout 1705 ms). Trace:
W20260709 13:05:59.545183 16008 rpcz_store.cc:269] 0709 13:05:57.448156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.494693 (+ 46537us) service_pool.cc:224] Handling call
0709 13:05:59.274197 (+1779504us) spinlock_profiling.cc:243] Waited 811 ms on lock 0x7b54001f86d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:59.287785 (+ 13588us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1864274816}
I20260709 13:05:59.545758 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:59.285403 15974 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:59.347522 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:05:59.445776 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.357407 15862 raft_consensus.cc:1240] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 5. Current term is 6. Ops: []
W20260709 13:05:59.423552 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 70470.4us
I20260709 13:05:59.285109 15930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 1738) took 2026 ms. Trace:
I20260709 13:05:59.263412 15936 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 1719) took 4294 ms. Trace:
I20260709 13:05:59.347468 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:05:59.382534 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 146069us
I20260709 13:05:59.558342 15930 rpcz_store.cc:276] 0709 13:05:57.258209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.327418 (+ 69209us) service_pool.cc:224] Handling call
0709 13:05:59.284803 (+1957385us) spinlock_profiling.cc:243] Waited 303 ms on lock 0x7b54003b8154. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:59.284990 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":697579776}
I20260709 13:05:59.558421 15936 rpcz_store.cc:276] 0709 13:05:54.969319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.997420 (+ 28101us) service_pool.cc:224] Handling call
0709 13:05:57.027130 (+2029710us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:57.034950 (+  7820us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:58.553702 (+1518752us) spinlock_profiling.cc:243] Waited 7.11 ms on lock 0x7b54002c2754. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:59.263065 (+709363us) spinlock_profiling.cc:243] Waited 2.03 s on lock 0x7b54002c2750. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:59.263288 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4683233920}
I20260709 13:05:59.528417 18732 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:59.322636 18783 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:59.262593 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:59.361374 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:05:59.320462 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 570334us
I20260709 13:05:59.257635 18709 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:05:59.548556 16005 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41662 (request call id 1820) took 1811 ms (client timeout 1880 ms). Trace:
W20260709 13:05:59.563664 16005 rpcz_store.cc:269] 0709 13:05:57.737487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.737645 (+   158us) service_pool.cc:224] Handling call
0709 13:05:59.293935 (+1556290us) spinlock_profiling.cc:243] Waited 3.3 ms on lock 0x7b5800114398. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:05:59.548484 (+254549us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7593344}
I20260709 13:05:59.549047 15860 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:59.335621 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:59.555809 18792 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:59.536357 15853 pending_rounds.cc:85] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8: Aborting all ops after (but not including) 6
I20260709 13:05:59.540982 15928 rpcz_store.cc:276] 0709 13:05:56.894179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.906676 (+ 12497us) service_pool.cc:224] Handling call
0709 13:05:56.911011 (+  4335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:57.035460 (+124449us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:58.786455 (+1750995us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b54002c1d54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:58.786678 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":284602240}
I20260709 13:05:59.551506 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:59.565042 18840 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:59.565210 15853 pending_rounds.cc:107] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:05:59.565184 18792 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:59.565229 15854 raft_consensus.cc:2393] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 5: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:59.552647 18728 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:59.566514 16570 raft_consensus.cc:2410] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 9 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:05:59.567313 18792 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:59.551800 18768 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:05:59.567840 16570 raft_consensus.cc:1275] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:05:59.568049 18768 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:59.569068 16570 raft_consensus.cc:1275] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:05:59.569977 18768 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:59.570271 16570 raft_consensus.cc:1275] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:05:59.552830 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:59.551707 18177 raft_consensus.cc:2804] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:59.571177 16567 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:05:59.571322 18177 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:05:59.543794 16565 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45246 (request call id 700) took 2195 ms (client timeout 1647 ms). Trace:
W20260709 13:05:59.572185 16565 rpcz_store.cc:269] 0709 13:05:57.348085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.348202 (+   117us) service_pool.cc:224] Handling call
0709 13:05:59.290465 (+1942263us) spinlock_profiling.cc:243] Waited 2.43 ms on lock 0x7b5800044798. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:05:59.543734 (+253269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5595520}
I20260709 13:05:59.565933 15853 raft_consensus.cc:2889] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:59.317276 18500 consensus_meta.cc:287] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.598s	user 0.005s	sys 0.006s
I20260709 13:05:59.574317 18500 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:59.575958 15937 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 1739) took 2317 ms. Trace:
W20260709 13:05:59.576380 16006 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41674 (request call id 1743) took 1405 ms (client timeout 1647 ms). Trace:
I20260709 13:05:59.576726 18500 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:59.567554 18728 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:05:59.576833 16006 rpcz_store.cc:269] 0709 13:05:58.170683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:58.364499 (+193816us) service_pool.cc:224] Handling call
0709 13:05:59.576212 (+1211713us) spinlock_profiling.cc:243] Waited 853 ms on lock 0x7b5400306454. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:59.576325 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1962552576}
I20260709 13:05:59.578223 16569 raft_consensus.cc:2393] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 6: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:05:59.578383 15860 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:05:58.730031 18711 raft_consensus.cc:2804] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:05:59.556054 16568 raft_consensus.cc:1240] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:59.322855 18738 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:05:59.580530 18738 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:59.583317 18738 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:59.444461 18782 consensus_queue.cc:1048] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.117s
I20260709 13:05:59.294194 18780 consensus_queue.cc:1059] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.268s
I20260709 13:05:59.556900 16566 raft_consensus.cc:1240] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:59.590246 16010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 514) took 1009 ms. Trace:
I20260709 13:05:59.590735 16010 rpcz_store.cc:276] 0709 13:05:58.580415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:59.050878 (+470463us) service_pool.cc:224] Handling call
0709 13:05:59.577343 (+526465us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:59.577516 (+   173us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:59.577540 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:59.577576 (+    36us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:59.577606 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:59.577657 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:59.590038 (+ 12381us) spinlock_profiling.cc:243] Waited 526 ms on lock 0x7b5400306450. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:59.590188 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1210004736}
W20260709 13:05:59.591542 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.584477 18711 raft_consensus.cc:697] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:59.604507 18711 consensus_queue.cc:237] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:59.272137 18640 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:05:59.559716 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:59.537797 16561 rpcz_store.cc:276] 0709 13:05:57.709055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.710093 (+  1038us) service_pool.cc:224] Handling call
0709 13:05:57.718639 (+  8546us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:57.718903 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:05:57.731855 (+ 12952us) log.cc:844] Serialized 14 byte log entry
0709 13:05:59.209763 (+1477908us) spinlock_profiling.cc:243] Waited 21.6 ms on lock 0x7b18003f0460. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e9f504bc 00007f73e729e48b 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72b8455 00007f73ea3a0060 00007f73ea3b2716 00007f73ea3c3287
0709 13:05:59.210094 (+   331us) log.cc:844] Serialized 14 byte log entry
0709 13:05:59.319986 (+109892us) log.cc:844] Serialized 14 byte log entry
0709 13:05:59.336828 (+ 16842us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:59.336899 (+    71us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:59.336960 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:59.337048 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:59.409473 (+ 72425us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":10543,"prepare.run_cpu_time_us":107,"prepare.run_wall_time_us":109,"raft.queue_time_us":216446,"raft.run_cpu_time_us":296,"raft.run_wall_time_us":5375,"spinlock_wait_cycles":49784192,"thread_start_us":345388,"threads_started":2,"wal-append.queue_time_us":107880}
I20260709 13:05:59.635035 15857 raft_consensus.cc:2376] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5. Re-sending same reply.
I20260709 13:05:59.636941 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:59.638056 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:59.638533 15929 raft_consensus.cc:1240] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:59.639125 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:59.560168 15859 raft_consensus.cc:2393] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 9: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:59.640354 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:59.640854 16569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45232 (request call id 644) took 1097 ms. Trace:
I20260709 13:05:59.560216 15855 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:59.641336 16569 rpcz_store.cc:276] 0709 13:05:58.543245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:58.543402 (+   157us) service_pool.cc:224] Handling call
0709 13:05:59.640636 (+1097234us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400545e54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:05:59.640794 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2359827328}
I20260709 13:05:59.642401 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:59.643203 15929 raft_consensus.cc:2410] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:05:59.254132 18460 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:59.423131 18401 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.644119 18401 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:59.645071 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:59.645612 15929 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:59.646080 18401 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:59.562536 15856 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:05:59.646337 15857 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6.
I20260709 13:05:59.563024 18783 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:59.320801 18727 raft_consensus.cc:2804] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:05:59.648177 18846 log.cc:927] Time spent T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:05:59.648545 18727 raft_consensus.cc:697] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:59.649317 18783 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:59.563227 15902 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:05:59.649505 18727 consensus_queue.cc:237] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:59.650440 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 87279.6us
I20260709 13:05:59.651185 15902 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:05:59.529834 15900 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:05:59.652021 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 122302us
I20260709 13:05:59.652441 16562 raft_consensus.cc:1240] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:59.527791 18737 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.564193 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:05:59.566422 15854 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58214 (request call id 529) took 2158 ms (client timeout 2367 ms). Trace:
I20260709 13:05:59.657433 18709 raft_consensus.cc:2749] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:05:59.657785 15854 rpcz_store.cc:269] 0709 13:05:57.408383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:59.335262 (+1926879us) service_pool.cc:224] Handling call
0709 13:05:59.566366 (+231104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:59.658269 18709 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:59.658994 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:59.659623 15929 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:05:59.659595 18737 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:59.660950 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:59.661963 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:59.662474 18737 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:05:59.662647 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:05:59.546958 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 261631us
I20260709 13:05:59.585253 18732 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:59.576335 15937 rpcz_store.cc:276] 0709 13:05:57.258548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.367491 (+108943us) service_pool.cc:224] Handling call
0709 13:05:57.380426 (+ 12935us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:57.380805 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:57.380835 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:57.381224 (+   389us) log.cc:844] Serialized 26 byte log entry
0709 13:05:57.592047 (+210823us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:57.616030 (+ 23983us) log.cc:844] Serialized 14 byte log entry
0709 13:05:57.616282 (+   252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:57.616380 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:58.904989 (+1288609us) spinlock_profiling.cc:243] Waited 3.41 ms on lock 0x7b18003b5480. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72892a6 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb3f3 00007f73e72bd7c1
0709 13:05:59.264883 (+359894us) spinlock_profiling.cc:243] Waited 2.81 ms on lock 0x7b18003b5480. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb3f3 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd 000000000038f354 000000000038f199 00007f73e729e83a
0709 13:05:59.574058 (+309175us) spinlock_profiling.cc:243] Waited 4.24 ms on lock 0x7b18003b5480. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb3f3 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd 000000000038f354 000000000038f199 00007f73e729e83a
0709 13:05:59.575749 (+  1691us) raft_consensus.cc:1685] finished
0709 13:05:59.575769 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:59.575902 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":80936,"prepare.queue_time_us":23392,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":3277,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":8496,"spinlock_wait_cycles":24068096,"thread_start_us":186142,"threads_started":1,"wal-append.queue_time_us":218872}
I20260709 13:05:59.679771 18780 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:59.658944 18709 raft_consensus.cc:697] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:05:59.680452 18780 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:59.267530 18702 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:05:59.552194 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.689860 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:05:59.690513 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:59.690624 16006 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:59.690776 15976 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:59.561847 16535 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:05:59.619536 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:59.691695 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:59.692276 16008 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:05:59.692538 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:05:59.691704 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 129900us
I20260709 13:05:59.693198 15860 raft_consensus.cc:2393] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 6: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:59.621281 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:59.694694 15862 raft_consensus.cc:2410] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:05:59.522826 18590 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.622026 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:59.695634 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:59.699921 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:59.701427 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:59.629038 18649 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:05:59.713002 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:05:59.637036 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:59.718312 15930 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:05:59.719271 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:59.720278 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:59.720314 15854 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:05:59.641026 15859 raft_consensus.cc:2393] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 5: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:59.720697 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:59.721247 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:59.721675 15862 raft_consensus.cc:2393] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 6: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:59.721931 15854 raft_consensus.cc:2393] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 5: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:05:59.544718 15858 raft_consensus.cc:2376] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6. Re-sending same reply.
I20260709 13:05:59.571765 18177 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:59.349124 18605 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:59.667656 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:59.674299 15855 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:05:59.679694 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:59.680285 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:05:59.727119 15932 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:59.680804 18790 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:59.559248 15827 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:05:59.665944 18401 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.298466 18677 raft_consensus.cc:3055] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:59.681684 16564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 635) took 2091 ms. Trace:
I20260709 13:05:59.681951 15929 raft_consensus.cc:1240] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:59.682379 15934 raft_consensus.cc:1240] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 4. Current term is 5. Ops: []
W20260709 13:05:59.684033 15825 connection.cc:256] RPC call timeout handler was delayed by -2.494160609000005s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3034s for a response
W20260709 13:05:59.764505 15825 connection.cc:256] RPC call timeout handler was delayed by -1.2208303849999993s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:05:59.765463 15934 raft_consensus.cc:1275] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:05:59.685534 15902 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
W20260709 13:05:59.653545 16533 leader_election.cc:336] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 2.463s (SENT)
I20260709 13:05:59.693543 18460 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.708076 15935 raft_consensus.cc:1240] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:59.709349 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:59.710136 15931 raft_consensus.cc:1240] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:59.771817 15931 raft_consensus.cc:1240] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 3. Current term is 5. Ops: []
I20260709 13:05:59.711491 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:59.632560 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:59.647125 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:59.774362 15856 raft_consensus.cc:2393] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 5: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:59.774472 15928 raft_consensus.cc:2410] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:05:59.723615 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:59.724941 15853 raft_consensus.cc:1275] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:05:59.724747 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:05:59.729101 15854 raft_consensus.cc:1240] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:59.742664 16006 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:05:59.776142 16005 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6.
W20260709 13:05:59.747738 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 188666us
W20260709 13:05:59.354231 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:59.750530 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:59.778102 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:59.778499 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.760104 15862 raft_consensus.cc:1275] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:05:59.766551 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 113026us
I20260709 13:05:59.722209 18620 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:59.785310 15930 raft_consensus.cc:2393] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 7: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
W20260709 13:05:59.766747 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 81287.5us
I20260709 13:05:59.788298 15932 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 7.
I20260709 13:05:59.768232 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:05:59.789403 15859 raft_consensus.cc:2393] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 9: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:05:59.769395 15934 raft_consensus.cc:1240] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:59.691120 16010 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:05:59.721869 18838 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:05:59.767124 16564 rpcz_store.cc:276] 0709 13:05:57.590550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.681424 (+ 90874us) service_pool.cc:224] Handling call
0709 13:05:57.681768 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:58.447596 (+765828us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:59.681467 (+1233871us) spinlock_profiling.cc:243] Waited 760 ms on lock 0x7b5400724554. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:05:59.681626 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1748716800}
I20260709 13:05:59.791150 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:59.810316 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:59.810830 15932 raft_consensus.cc:2393] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 7: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:05:59.811632 15934 raft_consensus.cc:1240] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:59.812206 15937 raft_consensus.cc:2468] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
I20260709 13:05:59.812740 15934 raft_consensus.cc:1240] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:59.787204 15902 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:05:59.815927 15932 raft_consensus.cc:1275] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:05:59.815968 15934 raft_consensus.cc:1240] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 4. Current term is 5. Ops: []
W20260709 13:05:59.817435 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 126734us
I20260709 13:05:59.809196 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:59.810611 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:05:59.822944 15756 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58426 (request call id 342) took 14841 ms (client timeout 15000 ms). Trace:
I20260709 13:05:59.833297 15928 raft_consensus.cc:1275] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:05:59.765090 18841 raft_consensus.cc:2804] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:59.838846 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:59.763080 18677 raft_consensus.cc:740] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:05:59.774896 18616 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:05:59.838215 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 51120.6us
I20260709 13:05:59.769496 18500 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:59.834347 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 112547us
I20260709 13:05:59.869788 15932 raft_consensus.cc:1275] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:05:59.722615 18590 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:59.866400 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:59.866721 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:59.876175 15934 raft_consensus.cc:2410] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:05:59.814968 18711 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.872720 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:59.873692 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:59.809985 18802 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:59.823642 15756 rpcz_store.cc:269] 0709 13:05:44.981878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.990268 (+  8390us) service_pool.cc:224] Handling call
0709 13:05:45.428221 (+437953us) spinlock_profiling.cc:243] Waited 7.83 ms on lock 0x7b5400030890. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e72840b2 00007f73f01fda89 00007f73f0195da6 00007f73f0167088 00007f73f026537e 00007f73ee7262a4 00007f73ee726232 00007f73ee726161 00007f73ee7260dc
0709 13:05:59.822808 (+14394587us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:56.898674 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:56.899300 (+   626us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:56.899344 (+    44us) write_op.cc:435] Acquired schema lock
0709 13:05:56.899361 (+    17us) tablet.cc:662] Decoding operations
0709 13:05:56.968941 (+ 69580us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:56.969021 (+    80us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:56.969053 (+    32us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:05:56.970243 (+  1190us) tablet.cc:701] Row locks acquired
0709 13:05:56.970295 (+    52us) write_op.cc:260] PREPARE: finished
0709 13:05:56.970475 (+   180us) write_op.cc:270] Start()
0709 13:05:56.970612 (+   137us) write_op.cc:276] Timestamp: P: 1783602356970409 usec, L: 0
0709 13:05:56.970636 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:05:56.971308 (+   672us) log.cc:844] Serialized 5147 byte log entry
0709 13:05:57.374911 (+403603us) op_driver.cc:464] REPLICATION: finished
0709 13:05:57.664356 (+289445us) write_op.cc:301] APPLY: starting
0709 13:05:57.676832 (+ 12476us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:57.678592 (+  1760us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:57.678764 (+   172us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:57.680755 (+  1991us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:57.681203 (+   448us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=13
0709 13:05:57.681267 (+    64us) write_op.cc:312] APPLY: finished
0709 13:05:57.681701 (+   434us) log.cc:844] Serialized 95 byte log entry
0709 13:05:57.715352 (+ 33651us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:57.721448 (+  6096us) write_op.cc:454] Released schema lock
0709 13:05:57.726703 (+  5255us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:57.727098 (+   395us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:57.727127 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":650122,"reported_tablets":29,"spinlock_wait_cycles":18024704,"child_traces":[["op",{"apply.queue_time_us":289069,"apply.run_cpu_time_us":4779,"apply.run_wall_time_us":62763,"num_ops":13,"prepare.queue_time_us":78315,"prepare.run_cpu_time_us":41515,"prepare.run_wall_time_us":184177,"raft.queue_time_us":184837,"raft.run_cpu_time_us":27107,"raft.run_wall_time_us":234060,"replication_time_us":404080,"spinlock_wait_cycles":90368,"thread_start_us":501558,"threads_started":4,"wal-append.queue_time_us":112085,"wal-append.run_cpu_time_us":33837,"wal-append.run_wall_time_us":1628659}]]}
I20260709 13:05:59.769744 18782 consensus_queue.cc:1059] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.092s
I20260709 13:05:59.764923 18790 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:59.902690 15858 raft_consensus.cc:1275] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:05:59.907095 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:59.887356 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:59.911509 15934 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
I20260709 13:05:59.885437 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:05:59.892915 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:59.770308 18401 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:59.725373 18709 consensus_queue.cc:237] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:05:59.865335 18616 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:05:59.860996 18737 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:05:59.913415 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.780894 18460 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:05:59.913365 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 126911us
I20260709 13:05:59.889796 15928 raft_consensus.cc:1275] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:05:59.858033 18805 consensus_queue.cc:1059] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:05:59.895812 18624 consensus_queue.cc:1059] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:05:59.832616 18838 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:00.341012 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:00.406796 15929 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
I20260709 13:05:59.917177 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:06:00.409036 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:00.410782 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:00.411971 16565 raft_consensus.cc:1240] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:00.091500 18677 consensus_queue.cc:260] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:59.905133 18783 consensus_queue.cc:1059] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.508s
I20260709 13:06:00.412654 18853 consensus_queue.cc:1059] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:00.413122 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:05:59.967285 15930 raft_consensus.cc:1240] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:00.414359 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:00.415082 16570 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
I20260709 13:06:00.415833 18460 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:00.416450 16569 raft_consensus.cc:1275] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:05:59.914876 18738 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:59.895638 18376 consensus_queue.cc:1048] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.523s
I20260709 13:05:59.900238 18711 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:00.258782 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 388113us
I20260709 13:06:00.019095 18620 raft_consensus.cc:697] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:00.420913 18711 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:00.421491 18711 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:00.421653 18620 consensus_queue.cc:237] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:00.422103 18711 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:00.011997 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 193816us
I20260709 13:05:59.881188 18500 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:00.003269 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 182019us
I20260709 13:06:00.423549 18883 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:00.084822 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:00.428215 18883 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:00.429436 18500 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:00.430747 18883 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:06:00.431914 16536 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.010s	sys 0.031s
W20260709 13:06:00.432274 16536 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.010s	sys 0.031s
I20260709 13:05:59.909646 18802 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:05:59.871510 18768 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:06:00.229254 15791 debug-util.cc:398] Leaking SignalData structure 0x7b0800278c00 after lost signal to thread 14751
I20260709 13:06:00.427753 16564 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:00.432940 18768 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:00.433542 18768 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:00.433851 15791 debug-util.cc:398] Leaking SignalData structure 0x7b080026e6a0 after lost signal to thread 15794
I20260709 13:06:00.433959 18802 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:00.425765 15928 pending_rounds.cc:85] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 6
I20260709 13:06:00.436218 15928 pending_rounds.cc:107] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:06:00.436631 15928 raft_consensus.cc:2889] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:06:00.437973 18649 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.012s	sys 0.014s
W20260709 13:06:00.438465 18649 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.012s	sys 0.014s
I20260709 13:06:00.438701 18880 consensus_queue.cc:1048] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:00.441046 18887 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:00.441720 18887 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:00.442346 18879 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:00.445075 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:00.445338 18879 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:00.469679 18861 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:00.470952 18852 log.cc:927] Time spent T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:06:00.471362 18861 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:00.473472 18728 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.473754 18861 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:00.473984 18728 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:00.443951 15974 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:00.440337 18865 consensus_queue.cc:1048] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:06:00.447363 15902 connection.cc:256] RPC call timeout handler was delayed by -1.346653172999936s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20520000000000002s for a response
I20260709 13:06:00.034641 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:06:00.448961 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 492299us
W20260709 13:06:00.476446 15902 connection.cc:256] RPC call timeout handler was delayed by -1.1889688239999714s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19030000000000002s for a response
I20260709 13:06:00.044461 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:00.140136 18732 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:00.449771 15976 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:00.469901 18805 raft_consensus.cc:3055] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:06:00.470126 18850 log.cc:927] Time spent T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.331s	user 0.001s	sys 0.000s
I20260709 13:06:00.470790 18792 raft_consensus.cc:3055] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:00.471807 18790 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:00.150945 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:00.474378 18861 consensus_queue.cc:1048] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:00.450810 18616 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:00.426370 18859 consensus_queue.cc:1048] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:00.431396 18854 consensus_queue.cc:1048] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:06:00.467660 18847 log.cc:927] Time spent T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.506s	user 0.001s	sys 0.001s
I20260709 13:05:59.961086 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:05:59.937765 18851 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.386309 18590 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:00.450655 18737 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:00.476336 18401 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:00.489589 18805 raft_consensus.cc:740] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:00.487625 18792 raft_consensus.cc:740] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:00.440372 18880 consensus_queue.cc:1048] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 13:06:00.474751 18768 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:00.505923 18882 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.560804 18867 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.561453 18867 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:00.515798 18881 consensus_queue.cc:1048] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:06:00.565713 18879 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:00.566962 18887 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:00.511435 18862 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:00.567656 18862 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:00.568068 18862 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:00.568928 18711 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:00.482024 18889 consensus_queue.cc:1048] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.089s
I20260709 13:06:00.523181 18737 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:00.569628 18882 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:00.572469 18882 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:00.569047 18838 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:00.572844 18805 consensus_queue.cc:260] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:00.574069 18805 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:00.559245 18728 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:00.575248 18792 consensus_queue.cc:260] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:00.576530 18792 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:06:00.576756 18605 consensus_meta.cc:287] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.814s	user 0.011s	sys 0.000s
I20260709 13:06:00.519945 18780 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:00.556885 18896 consensus_queue.cc:1048] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
W20260709 13:06:00.577951 18847 log.cc:927] Time spent T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:06:00.564086 18790 consensus_queue.cc:1048] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 13:06:00.578207 18883 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.557890 18891 consensus_queue.cc:1048] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:06:00.582728 18891 consensus_queue.cc:1243] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer eb1e0493af1e46dbb6dec7e25525bab5 log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:06:00.582718 18883 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:00.587427 18862 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:00.526415 18851 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:00.590571 18851 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:00.592761 18867 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:00.207126 15759 catalog_manager.cc:5697] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 3 to 5, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 5 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
W20260709 13:06:00.435485 15791 debug-util.cc:398] Leaking SignalData structure 0x7b0800224320 after lost signal to thread 15886
W20260709 13:06:00.624538 15791 debug-util.cc:398] Leaking SignalData structure 0x7b0800252840 after lost signal to thread 15887
W20260709 13:06:00.626940 15791 debug-util.cc:398] Leaking SignalData structure 0x7b0800268c20 after lost signal to thread 15961
W20260709 13:06:00.627321 15791 debug-util.cc:398] Leaking SignalData structure 0x7b080030b1e0 after lost signal to thread 15962
I20260709 13:06:00.470549 15825 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:00.632249 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 147605us
W20260709 13:06:00.634317 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:00.635751 15977 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.781s	user 0.008s	sys 0.004s
W20260709 13:06:00.638407 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 188525us
W20260709 13:06:00.510706 16533 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.000s	sys 0.012s
W20260709 13:06:00.640323 16533 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.800s	user 0.001s	sys 0.012s
W20260709 13:06:00.640893 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 801225us
W20260709 13:06:00.636286 15977 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.008s	sys 0.005s
W20260709 13:06:00.648888 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:00.649664 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 180869us
I20260709 13:06:00.638708 16004 raft_consensus.cc:2410] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:06:00.528564 18590 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:00.654227 18709 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:00.657065 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:00.660348 18728 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.661641 18586 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:06:00.662065 15791 debug-util.cc:398] Leaking SignalData structure 0x7b080037e7a0 after lost signal to thread 16035
I20260709 13:06:00.661846 18728 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:00.662755 18586 raft_consensus.cc:697] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
W20260709 13:06:00.664956 15791 debug-util.cc:398] Leaking SignalData structure 0x7b080037cf00 after lost signal to thread 16594
I20260709 13:06:00.665612 18792 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.665349 18586 consensus_queue.cc:237] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:00.667532 18851 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.668566 18851 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:06:00.670990 15853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 542) took 1050 ms. Trace:
I20260709 13:06:00.581737 18768 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:00.593705 18862 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:00.690840 18869 consensus_queue.cc:1059] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:00.692591 18869 raft_consensus.cc:3055] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:00.693051 18869 raft_consensus.cc:740] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:00.694756 18869 consensus_queue.cc:260] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:00.660631 15827 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:00.665351 15791 debug-util.cc:398] Leaking SignalData structure 0x7b080031d260 after lost signal to thread 16595
W20260709 13:06:00.776822 15791 debug-util.cc:398] Leaking SignalData structure 0x7b08003b0240 after lost signal to thread 18372
W20260709 13:06:00.777549 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 117371us
W20260709 13:06:00.778228 15791 debug-util.cc:398] Leaking SignalData structure 0x7b08002d40e0 after lost signal to thread 18718
W20260709 13:06:00.779011 15791 debug-util.cc:398] Leaking SignalData structure 0x7b0800353d40 after lost signal to thread 18749
W20260709 13:06:00.779452 15791 debug-util.cc:398] Leaking SignalData structure 0x7b0800314560 after lost signal to thread 18762
I20260709 13:06:00.626515 15759 catalog_manager.cc:5697] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 2 to 4, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 4 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:06:00.679051 18867 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:00.779685 18876 raft_consensus.cc:2749] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:00.780272 18867 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:00.780782 18876 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:00.780628 15759 catalog_manager.cc:5697] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 3 to 4, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 4 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:00.783025 18867 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:00.782650 15759 catalog_manager.cc:5697] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 2 to 4, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 4 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
I20260709 13:06:00.655380 18709 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:00.784083 18620 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.784968 18620 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:00.785740 18709 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:00.779132 18848 log.cc:927] Time spent T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.772s	user 0.003s	sys 0.007s
I20260709 13:06:00.783437 18876 consensus_queue.cc:1048] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:06:00.788116 18620 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:00.787529 15759 catalog_manager.cc:5697] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 3 to 4, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 4 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:06:00.789448 15759 catalog_manager.cc:5697] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:00.688354 18868 consensus_queue.cc:1059] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:06:00.634176 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 163569us
I20260709 13:06:00.792240 18868 raft_consensus.cc:3055] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:00.792663 18868 raft_consensus.cc:740] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:00.793870 18868 consensus_queue.cc:260] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:00.794691 15825 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:00.675148 18854 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.795423 18868 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:00.795917 18854 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:00.798580 18854 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:00.799645 18860 consensus_queue.cc:1059] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:00.800901 18869 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:00.802953 16535 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:00.804050 18861 raft_consensus.cc:3055] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:00.804605 18861 raft_consensus.cc:740] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:00.806098 16535 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:00.703696 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:00.727416 18728 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:00.807804 18728 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:00.803192 18860 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:00.709276 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:00.724146 16533 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:00.733021 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:00.735396 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:00.737304 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:00.737907 18883 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:00.740675 18884 raft_consensus.cc:3055] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:00.746395 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:00.748026 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:00.706954 18677 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:00.818024 15759 catalog_manager.cc:5697] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 3 to 4, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 4 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:06:00.825886 15759 catalog_manager.cc:5697] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 3 to 6, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 6 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:00.768749 15936 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1808) took 1135 ms. Trace:
I20260709 13:06:00.773051 18796 raft_consensus.cc:3055] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:00.672385 18851 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:00.805989 18861 consensus_queue.cc:260] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:00.811276 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:00.835199 16570 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:00.829977 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 186491us
W20260709 13:06:00.712524 16004 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41662 (request call id 1821) took 2974 ms (client timeout 2052 ms). Trace:
I20260709 13:06:00.713438 18727 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.827754 18883 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:00.828790 16563 raft_consensus.cc:2393] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 5: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:00.829501 18782 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:00.671481 15853 rpcz_store.cc:276] 0709 13:05:59.620405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:59.623762 (+  3357us) service_pool.cc:224] Handling call
0709 13:05:59.724649 (+100887us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:59.775507 (+ 50858us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:00.670520 (+895013us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b5400427cd0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:00.670746 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":231356288}
I20260709 13:06:00.697286 18768 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:00.818078 18728 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:00.835949 16004 rpcz_store.cc:269] 0709 13:05:57.737919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.738022 (+   103us) service_pool.cc:224] Handling call
0709 13:05:59.262381 (+1524359us) spinlock_profiling.cc:243] Waited 3.14 ms on lock 0x7b5800114398. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:06:00.712050 (+1449669us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54002e1b54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:00.712331 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2479616512}
I20260709 13:06:00.822074 18884 raft_consensus.cc:740] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:00.859881 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:00.831626 18796 raft_consensus.cc:740] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:00.856762 18728 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:00.838740 18854 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.849292 18851 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.882500 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:00.863584 15759 catalog_manager.cc:5697] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:06:00.838346 18792 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:00.885881 15859 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
I20260709 13:06:00.868337 18782 raft_consensus.cc:697] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:00.862177 18727 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:00.896071 15827 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:00.899716 18640 raft_consensus.cc:2804] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:00.900309 18640 raft_consensus.cc:697] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:00.901535 18640 consensus_queue.cc:237] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:00.902769 15974 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:00.903040 18884 consensus_queue.cc:260] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:00.903735 18790 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:00.904685 18884 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:00.904785 15860 raft_consensus.cc:1275] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:06:00.905880 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 409664us
I20260709 13:06:00.906211 18780 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:06:00.908006 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 113511us
W20260709 13:06:00.908612 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:00.909368 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:00.909627 18840 consensus_queue.cc:1048] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 13:06:00.910079 18854 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:00.910488 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:00.911360 18840 raft_consensus.cc:2804] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:06:00.911779 18840 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:00.912186 18840 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:00.912485 18854 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:00.912715 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:00.913431 16007 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:00.913856 18782 consensus_queue.cc:237] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:00.916209 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:00.916916 16010 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:00.920023 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 114230us
I20260709 13:06:00.920253 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:00.921567 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:00.923170 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:00.924479 18796 consensus_queue.cc:260] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:00.925886 18796 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:00.929617 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:00.930413 16005 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:00.932525 18883 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:00.935977 18861 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:00.909256 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:00.925678 18620 consensus_queue.cc:1048] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:00.904487 18790 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:00.909148 18851 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:00.950093 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:00.950686 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:00.951363 15860 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:00.951822 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:00.953085 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:00.942193 16003 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
W20260709 13:06:00.954010 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:00.945477 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:00.949726 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:06:00.957899 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 51125.1us
I20260709 13:06:00.957508 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:00.960296 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:00.966369 15856 raft_consensus.cc:1275] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:06:00.967785 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:06:00.968379 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 59647.2us
I20260709 13:06:00.969028 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:00.969261 15825 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:00.969681 15857 raft_consensus.cc:1275] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:06:00.970212 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:00.970711 15825 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:00.971161 18860 raft_consensus.cc:2804] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:00.951519 18898 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.971622 18860 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:00.972160 18860 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:00.976164 18163 raft_consensus.cc:2749] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:00.977293 18790 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:00.978226 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:00.978561 15853 raft_consensus.cc:1275] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:06:00.977308 15858 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:00.978899 16011 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:06:00.979586 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:00.980333 15853 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
I20260709 13:06:00.981263 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:00.981981 15853 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:06:00.982384 18899 consensus_queue.cc:1059] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:00.982923 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:00.983386 18899 raft_consensus.cc:3055] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:00.830219 15936 rpcz_store.cc:276] 0709 13:05:59.632851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:59.632976 (+   125us) service_pool.cc:224] Handling call
0709 13:05:59.633257 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:59.666035 (+ 32778us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:05:59.719339 (+ 53304us) log.cc:844] Serialized 14 byte log entry
0709 13:05:59.784068 (+ 64729us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:59.784121 (+    53us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:59.784165 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:00.766960 (+982795us) spinlock_profiling.cc:243] Waited 31.3 ms on lock 0x7b1800186a00. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72892a6 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419
0709 13:06:00.768175 (+  1215us) spinlock_profiling.cc:243] Waited 32.5 ms on lock 0x7b540034a154. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:00.768238 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:00.768522 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57414,"prepare.queue_time_us":56592,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":105,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":23008,"spinlock_wait_cycles":146708480,"thread_start_us":60767,"threads_started":1,"wal-append.queue_time_us":517498}
I20260709 13:06:00.970891 15859 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
I20260709 13:06:00.983633 15853 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
I20260709 13:06:00.975585 16533 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:00.982419 15902 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:00.970736 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:00.967584 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:00.958030 18851 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:01.017048 18790 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:01.030519 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:01.031091 18841 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.001s	user 0.010s	sys 0.029s
I20260709 13:06:00.972173 18898 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:01.031333 18709 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:01.033219 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:01.002890 18899 raft_consensus.cc:740] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:06:01.031140 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 48841.7us
I20260709 13:06:01.034301 18702 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:06:01.035174 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 59778us
I20260709 13:06:01.036712 16567 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:06:01.036918 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:01.037241 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:06:01.038286 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:00.899006 18768 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:01.043177 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 140317us
W20260709 13:06:01.042899 18841 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.162s	user 0.010s	sys 0.029s
I20260709 13:06:01.054788 16005 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
I20260709 13:06:01.049237 16004 raft_consensus.cc:1275] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 9. (term mismatch)
I20260709 13:06:01.057286 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:00.962419 18883 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:01.057608 16004 pending_rounds.cc:85] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting all ops after (but not including) 8
W20260709 13:06:01.041157 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 70566.5us
I20260709 13:06:00.956300 15759 catalog_manager.cc:5697] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 1 to 3, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 3 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
I20260709 13:06:01.057052 18841 raft_consensus.cc:697] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 8 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:01.061650 18790 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:01.062453 16003 raft_consensus.cc:2393] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 5: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:01.062282 18841 consensus_queue.cc:237] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:06:01.063947 18790 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:01.063982 18891 consensus_queue.cc:1048] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:01.058979 16004 pending_rounds.cc:107] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 7.9
I20260709 13:06:01.070004 16004 raft_consensus.cc:2889] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:01.072297 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:01.044893 18709 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:01.089471 18709 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:01.109483 16006 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:01.112876 15856 raft_consensus.cc:1275] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:06:01.120709 15860 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:06:01.121812 16010 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:01.122601 16570 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:06:01.123551 15858 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
W20260709 13:06:01.128702 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 69331.8us
I20260709 13:06:01.132820 15902 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:01.135576 18732 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:01.136001 18732 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:01.136345 18732 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:01.137910 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:01.138739 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:01.139137 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:01.139530 16004 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 6: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:01.139950 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:01.140671 16567 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:06:01.140740 15825 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:01.142865 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:01.144112 16566 raft_consensus.cc:1275] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:06:01.144212 18460 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:01.144888 18460 consensus_queue.cc:1059] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:01.146080 18163 raft_consensus.cc:3055] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:01.146481 18163 raft_consensus.cc:740] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:01.147310 18163 consensus_queue.cc:260] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:01.148166 16562 raft_consensus.cc:1240] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:01.149689 18898 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:01.149823 18732 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:01.150717 18768 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:01.153507 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:01.153815 18899 consensus_queue.cc:260] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:01.154838 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:01.158586 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:01.159277 16563 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:06:01.160429 16005 raft_consensus.cc:1275] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:06:01.164073 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:01.165429 16562 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 6: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:01.155534 18899 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:01.160248 15759 catalog_manager.cc:5697] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:01.172065 18177 consensus_meta.cc:287] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 1.404s	user 0.010s	sys 0.014s
I20260709 13:06:01.172196 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:01.172868 18727 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:01.173015 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:01.177429 18586 consensus_queue.cc:1059] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:06:01.207201 16006 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
I20260709 13:06:01.209645 15759 catalog_manager.cc:5697] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 2 to 4, leader changed from <none> to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 4 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:06:01.226189 16561 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
W20260709 13:06:01.279541 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 113106us
I20260709 13:06:01.225760 18792 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:01.204869 18177 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:01.308809 16004 raft_consensus.cc:1240] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:01.287994 15759 catalog_manager.cc:5697] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 3 to 4, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 4 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:01.227483 18867 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:06:01.317749 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 141027us
I20260709 13:06:01.299690 18884 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:01.319386 15900 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.792s (SENT) [suppressed 43 similar messages]
I20260709 13:06:01.309543 18883 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:06:01.333204 18906 log.cc:927] Time spent T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
W20260709 13:06:01.333714 18908 log.cc:927] Time spent T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:06:01.335810 18867 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:01.338122 18867 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:01.338632 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:01.280792 18728 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:01.389889 16006 raft_consensus.cc:1240] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 13:06:01.391038 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:01.391398 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 183359us
W20260709 13:06:01.393890 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 63741.2us
W20260709 13:06:01.381996 18909 log.cc:927] Time spent T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
W20260709 13:06:01.398746 15827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:01.331223 18732 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:01.410372 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:01.410796 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:01.411484 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:01.412492 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:01.413486 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:01.414583 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:01.418846 18728 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:01.424862 16006 raft_consensus.cc:1275] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:06:01.452385 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 63387.5us
I20260709 13:06:01.430513 18860 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:01.443794 18590 consensus_meta.cc:287] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.687s	user 0.014s	sys 0.001s
I20260709 13:06:01.445243 15933 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
W20260709 13:06:01.451426 16007 consensus_meta.cc:287] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.537s	user 0.007s	sys 0.008s
I20260709 13:06:01.456804 16007 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
W20260709 13:06:01.394356 16533 connection.cc:256] RPC call timeout handler was delayed by -1.0603792530000646s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17010000000000003s for a response
I20260709 13:06:01.403129 15974 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:06:01.462185 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 58422.3us
I20260709 13:06:01.475505 18586 consensus_queue.cc:1059] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 13:06:01.492537 15825 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:01.494490 15929 raft_consensus.cc:2393] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 5: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:01.495595 15928 raft_consensus.cc:2393] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 7: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:01.461176 18460 consensus_queue.cc:1048] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
W20260709 13:06:01.498581 18616 consensus_meta.cc:287] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.951s	user 0.007s	sys 0.011s
I20260709 13:06:01.499046 18616 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:01.499713 18790 raft_consensus.cc:2749] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:01.501363 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:01.501634 18616 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:01.502393 15931 raft_consensus.cc:2393] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 4: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:01.502978 15854 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "8430db80a39d4868bcf7641c16de35ea"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:06:01.503154 18590 raft_consensus.cc:2749] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:01.503758 15854 raft_consensus.cc:606] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Received request to transfer leadership to TS 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:01.504122 15854 raft_consensus.cc:614] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:06:01.504928 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:01.507270 15934 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:06:01.508267 16009 raft_consensus.cc:1240] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
W20260709 13:06:01.509711 18914 log.cc:927] Time spent T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
W20260709 13:06:01.510933 16533 leader_election.cc:336] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.705s (SENT)
I20260709 13:06:01.511425 16533 leader_election.cc:304] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:01.512770 16533 leader_election.cc:336] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.751s (SENT)
W20260709 13:06:01.514029 16533 leader_election.cc:336] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.992s (SENT)
I20260709 13:06:01.514394 16533 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:01.515312 18640 raft_consensus.cc:2749] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:01.516043 18868 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:01.479763 15937 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
W20260709 13:06:01.517114 16533 leader_election.cc:336] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 2.002s (SENT)
I20260709 13:06:01.517285 18860 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:01.518270 18854 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:01.518465 16533 leader_election.cc:336] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 2.147s (SENT)
I20260709 13:06:01.518937 16533 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:01.518976 18854 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:01.519892 18802 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:06:01.520618 16533 leader_election.cc:336] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 2.364s (SENT)
W20260709 13:06:01.520901 18861 consensus_meta.cc:287] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.584s	user 0.006s	sys 0.003s
I20260709 13:06:01.521067 16533 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:01.521623 15855 raft_consensus.cc:1275] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:01.521831 18802 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:06:01.522251 16533 leader_election.cc:336] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 2.367s (SENT)
I20260709 13:06:01.522687 16533 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:01.523377 18802 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:06:01.523871 18840 consensus_meta.cc:287] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.611s	user 0.004s	sys 0.010s
I20260709 13:06:01.524259 18840 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:01.456898 18884 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:01.527181 18840 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:01.527705 18884 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:01.528784 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 364258us
W20260709 13:06:01.544309 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:01.546021 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:01.521662 18854 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:01.451952 18883 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:01.559002 16006 raft_consensus.cc:1275] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:06:01.562496 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:01.552714 18677 consensus_meta.cc:287] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.729s	user 0.013s	sys 0.001s
I20260709 13:06:01.547134 18861 raft_consensus.cc:2749] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:01.569511 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:01.549957 18884 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:01.479224 18841 consensus_queue.cc:1059] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.085s
I20260709 13:06:01.554235 18616 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:06:01.578286 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 185186us
I20260709 13:06:01.585280 16011 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:06:01.581732 16002 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6.
I20260709 13:06:01.583097 15937 raft_consensus.cc:2393] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 8: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:01.587658 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:01.583770 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:01.588338 15931 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
W20260709 13:06:01.605366 18905 log.cc:927] Time spent T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.342s	user 0.000s	sys 0.001s
I20260709 13:06:01.587958 18851 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:01.590969 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:01.606942 15928 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6.
I20260709 13:06:01.571847 18177 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:01.585419 18869 consensus_meta.cc:287] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.784s	user 0.012s	sys 0.000s
I20260709 13:06:01.609220 15936 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
I20260709 13:06:01.612280 15759 catalog_manager.cc:5697] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 1 to 3, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 3 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
I20260709 13:06:01.617779 16006 raft_consensus.cc:1240] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:01.628469 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:01.626320 15759 catalog_manager.cc:5697] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:06:01.634394 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 160115us
I20260709 13:06:01.634824 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:06:01.627840 18737 consensus_meta.cc:287] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 1.008s	user 0.014s	sys 0.009s
I20260709 13:06:01.638240 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:06:01.633332 18796 consensus_meta.cc:287] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.707s	user 0.014s	sys 0.000s
I20260709 13:06:01.651538 15937 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:01.650904 15935 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
I20260709 13:06:01.590487 18853 consensus_queue.cc:1048] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.254s
I20260709 13:06:01.618361 18851 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:01.661991 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:01.656031 15974 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:01.628090 18884 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:01.630405 18616 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:01.667515 15827 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:01.669646 18738 consensus_queue.cc:1048] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:01.670203 18898 consensus_queue.cc:1048] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:01.672251 18899 consensus_queue.cc:1048] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:06:01.672935 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 180525us
I20260709 13:06:01.678503 18898 consensus_queue.cc:1048] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:06:01.679724 18915 log.cc:927] Time spent T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:06:01.679955 16006 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6.
I20260709 13:06:01.680586 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:01.680807 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:01.680907 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:01.681562 16006 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6.
W20260709 13:06:01.681946 18918 log.cc:927] Time spent T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.200s	user 0.000s	sys 0.000s
I20260709 13:06:01.682205 16562 pending_rounds.cc:85] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting all ops after (but not including) 5
I20260709 13:06:01.682354 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:01.682626 16562 pending_rounds.cc:107] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:06:01.682917 16562 raft_consensus.cc:2889] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:01.683236 16006 raft_consensus.cc:2393] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 6: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:01.684506 16564 raft_consensus.cc:1240] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:01.682291 18869 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:01.685530 18869 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:01.685870 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:01.688048 18869 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:01.687497 16566 raft_consensus.cc:1275] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:06:01.694311 15936 raft_consensus.cc:1275] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:06:01.654629 18737 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:01.704830 15902 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:01.692742 18177 raft_consensus.cc:2749] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:06:01.778188 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 110818us
I20260709 13:06:01.780965 16007 raft_consensus.cc:1240] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:01.696075 15759 catalog_manager.cc:5697] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 2 to 4, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 4 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:06:01.794301 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 119460us
I20260709 13:06:01.803624 18860 raft_consensus.cc:3055] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:01.804355 18860 raft_consensus.cc:740] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:01.805860 18884 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:01.805517 18860 consensus_queue.cc:260] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:01.807106 18860 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:01.776077 18929 raft_consensus.cc:2749] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:06:01.808910 16533 leader_election.cc:336] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 2.176s (SENT)
W20260709 13:06:01.810004 16533 leader_election.cc:336] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 2.659s (SENT)
I20260709 13:06:01.812829 16533 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:01.816453 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:01.817382 16011 raft_consensus.cc:2410] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 9 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:06:01.818223 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:01.818887 18677 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:01.819716 16011 raft_consensus.cc:1275] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:06:01.819653 18677 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:01.821055 18869 raft_consensus.cc:2749] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:01.822481 18677 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:01.824354 18858 raft_consensus.cc:3055] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:01.824877 18858 raft_consensus.cc:740] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:01.826021 18858 consensus_queue.cc:260] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:01.828305 18883 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:01.830070 18616 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:01.830873 18709 consensus_meta.cc:287] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.741s	user 0.010s	sys 0.001s
I20260709 13:06:01.831249 18709 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:01.810616 15932 raft_consensus.cc:1275] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
W20260709 13:06:01.833642 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:01.847168 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:01.853420 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 181569us
W20260709 13:06:01.874243 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 242752us
I20260709 13:06:01.853638 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:01.875522 15827 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:01.876297 15759 catalog_manager.cc:5697] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 3 to 6, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 6 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
I20260709 13:06:01.880672 16007 raft_consensus.cc:1275] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:06:01.851275 18929 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:01.912313 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:01.938601 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 85673.8us
I20260709 13:06:01.939769 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:01.947865 18917 log.cc:927] Time spent T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.332s	user 0.000s	sys 0.006s
I20260709 13:06:01.949678 18851 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:01.950659 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:01.955853 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:01.958030 18792 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:01.958082 18163 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:01.958829 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:01.959028 18782 consensus_queue.cc:1048] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:01.882561 18732 raft_consensus.cc:2804] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:01.880316 18858 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:01.985867 15902 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:01.960496 18802 consensus_queue.cc:1048] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:01.961901 18929 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:01.964103 15976 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:01.965871 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:02.026939 15937 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:06:01.966977 18868 consensus_queue.cc:1059] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:06:01.938898 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 84218.1us
I20260709 13:06:02.028399 18868 raft_consensus.cc:3055] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:02.028769 18868 raft_consensus.cc:740] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:01.959483 16535 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:01.984617 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 86157.6us
I20260709 13:06:01.943370 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:02.000196 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:01.998983 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 159902us
I20260709 13:06:02.030534 16570 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
I20260709 13:06:02.029232 15900 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:02.036159 15974 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:01.996129 18732 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.037320 15855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1752) took 1099 ms. Trace:
I20260709 13:06:02.037452 18732 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:02.037693 15855 rpcz_store.cc:276] 0709 13:06:00.938139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.064010 (+125871us) service_pool.cc:224] Handling call
0709 13:06:01.064350 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:01.522124 (+457774us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:02.037088 (+514964us) spinlock_profiling.cc:243] Waited 457 ms on lock 0x7b54004272d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:02.037265 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1050805504}
W20260709 13:06:02.038790 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 79377.8us
W20260709 13:06:02.060572 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.045390 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:06:02.048067 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 84021.5us
I20260709 13:06:02.042757 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:02.064556 15974 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:01.996105 18586 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.057921 15900 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:02.121068 15976 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:02.120271 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 55783.3us
W20260709 13:06:02.132872 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 97134.8us
W20260709 13:06:02.133747 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:02.135612 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 149841us
W20260709 13:06:02.046283 18925 log.cc:927] Time spent T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.410s	user 0.000s	sys 0.003s
I20260709 13:06:02.040419 18902 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:02.138643 15974 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:02.049396 18177 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:02.046573 18869 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:02.152014 18177 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.152145 18869 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.152530 18177 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:02.152667 18869 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:02.156329 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:02.163775 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 105958us
I20260709 13:06:02.163937 15902 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:02.165496 15902 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:02.166318 18898 raft_consensus.cc:2749] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:02.166862 15902 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:02.166950 16533 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:02.167229 18802 consensus_queue.cc:1048] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:02.142664 16010 raft_consensus.cc:1240] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:02.167732 18616 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:02.138159 18732 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:02.168347 18616 raft_consensus.cc:697] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 5 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:02.168855 16010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1751) took 1031 ms. Trace:
I20260709 13:06:02.072278 18709 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:02.142558 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60156 (request call id 525) took 1017 ms. Trace:
I20260709 13:06:02.059567 18737 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:02.169267 18616 consensus_queue.cc:237] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:02.166990 18898 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:02.170876 18898 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:02.171617 18782 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:02.169726 16002 rpcz_store.cc:276] 0709 13:06:01.124988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.125100 (+   112us) service_pool.cc:224] Handling call
0709 13:06:02.142359 (+1017259us) spinlock_profiling.cc:243] Waited 361 ms on lock 0x7b5400305054. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:02.142498 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":829685632}
I20260709 13:06:02.171263 18727 consensus_queue.cc:1048] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 13:06:02.064810 18853 consensus_queue.cc:1048] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.078s
I20260709 13:06:01.987440 18792 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:02.146914 18854 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:02.175879 18854 raft_consensus.cc:697] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:02.176339 18792 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:01.987646 16563 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
I20260709 13:06:02.169806 18732 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:02.176942 18854 consensus_queue.cc:237] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:02.149729 18861 raft_consensus.cc:3055] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:02.178691 18861 raft_consensus.cc:740] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:02.138288 18586 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:02.179646 18861 consensus_queue.cc:260] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:02.180352 18869 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:02.180907 18861 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:02.181213 18586 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:02.181567 16568 raft_consensus.cc:2393] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 7: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:02.181715 15929 raft_consensus.cc:1275] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:06:02.181789 18586 consensus_queue.cc:1059] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:02.182942 18841 raft_consensus.cc:3055] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:02.183283 18841 raft_consensus.cc:740] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
W20260709 13:06:02.150934 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.184132 18841 consensus_queue.cc:260] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:02.057531 18932 raft_consensus.cc:2749] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:02.184691 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:02.185154 18841 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:02.039518 18868 consensus_queue.cc:260] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:02.171273 18898 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:02.171617 18709 consensus_queue.cc:1048] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 13:06:02.164213 15827 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:02.188006 18709 consensus_queue.cc:1243] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer 85e997ba5003439781ec4f1f8db0c3ee log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:06:02.058223 18867 raft_consensus.cc:2804] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:02.190690 15827 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:02.191524 18460 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:02.191923 18460 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.192246 18460 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:02.192621 18926 consensus_queue.cc:1048] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:02.192735 18867 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:02.193524 18867 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:02.193418 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:02.194144 18926 raft_consensus.cc:2749] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:02.081664 18929 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:02.169232 16010 rpcz_store.cc:276] 0709 13:06:01.137117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.166287 (+ 29170us) service_pool.cc:224] Handling call
0709 13:06:02.142489 (+976202us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:02.168032 (+ 25543us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:02.168678 (+   646us) spinlock_profiling.cc:243] Waited 976 ms on lock 0x7b5400305050. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:02.168798 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2244365696}
I20260709 13:06:02.200101 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:02.171284 15902 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:02.201923 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:02.185273 18869 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:02.207870 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:02.207870 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:02.224023 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 52815.9us
I20260709 13:06:02.145493 18902 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:02.225025 15902 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:02.040701 18927 consensus_queue.cc:1059] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.210s
I20260709 13:06:02.265170 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:02.207878 18709 raft_consensus.cc:2804] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:02.264181 15900 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:02.269109 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:02.270340 16007 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:06:02.259145 18899 raft_consensus.cc:2804] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:02.274060 16004 raft_consensus.cc:1240] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:02.261826 18902 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:02.215517 18177 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:02.284372 16005 raft_consensus.cc:1275] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:06:02.289036 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 156045us
I20260709 13:06:02.294268 16003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1748) took 1226 ms. Trace:
W20260709 13:06:02.292852 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 155552us
I20260709 13:06:02.271051 18728 consensus_queue.cc:1048] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:06:02.268590 18869 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:06:02.279114 18929 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.286834 18732 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:02.280800 18932 raft_consensus.cc:2749] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:02.377110 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:02.291252 18709 raft_consensus.cc:697] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 5 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:02.294274 18851 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:02.393275 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:02.380323 15934 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.025s	sys 0.052s
W20260709 13:06:02.399667 15934 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.026s	sys 0.052s
I20260709 13:06:02.373790 16003 rpcz_store.cc:276] 0709 13:06:01.063771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.064224 (+   453us) service_pool.cc:224] Handling call
0709 13:06:01.064584 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:01.064926 (+   342us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:06:01.195600 (+130674us) log.cc:844] Serialized 14 byte log entry
0709 13:06:02.151714 (+956114us) spinlock_profiling.cc:243] Waited 880 us on lock 0x7b18002640a0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e9f504bc 00007f73e729e48b 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f4823a
0709 13:06:02.288613 (+136899us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:02.289826 (+  1213us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:02.289894 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:02.289982 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:02.290126 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17244,"prepare.queue_time_us":213730,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":4954,"spinlock_wait_cycles":2023808,"thread_start_us":416339,"threads_started":3,"wal-append.queue_time_us":291843}
I20260709 13:06:02.376833 18860 consensus_queue.cc:1059] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:06:02.415376 15937 raft_consensus.cc:1240] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:02.416424 18899 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:02.417095 18899 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:02.420722 18938 log.cc:927] Time spent T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:06:02.420408 18460 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:02.422478 18709 consensus_queue.cc:237] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:02.370951 18727 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:02.424561 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 208554us
I20260709 13:06:02.429008 16008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1755) took 1290 ms. Trace:
I20260709 13:06:02.392798 18891 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.434480 15974 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:06:02.432674 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 159308us
I20260709 13:06:02.422827 18460 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:02.427578 18732 raft_consensus.cc:697] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 7 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:02.409725 18737 consensus_queue.cc:1048] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:06:02.433024 18616 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.431044 18950 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:02.465621 18177 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:02.466287 18941 log.cc:927] Time spent T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:06:02.453697 18727 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
W20260709 13:06:02.476886 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 51948.4us
W20260709 13:06:02.465711 18939 log.cc:927] Time spent T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:06:02.466390 18868 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:02.477653 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.426663 18929 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:02.516690 15974 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:02.475204 18899 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:02.519860 18958 consensus_queue.cc:1048] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:02.520759 18586 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.521337 18586 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:02.522092 18899 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:02.522704 18899 consensus_queue.cc:1048] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:02.523343 18586 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:02.527614 18929 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:02.491550 18950 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:02.511078 18616 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:02.551429 18732 consensus_queue.cc:237] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:06:02.556068 18953 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:02.443511 18891 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:02.566952 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 55672.6us
I20260709 13:06:02.569762 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:02.523502 18958 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.571844 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:02.580462 16565 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:06:02.571971 15974 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:02.572557 18883 raft_consensus.cc:2804] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:02.577108 18727 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
I20260709 13:06:02.581712 15974 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:02.586776 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 194231us
I20260709 13:06:02.594849 15976 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:02.596462 15976 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:02.589354 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:02.587409 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 120274us
I20260709 13:06:02.569082 18782 raft_consensus.cc:2749] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:02.565044 18950 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:02.429440 16008 rpcz_store.cc:276] 0709 13:06:01.138455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.232632 (+ 94177us) service_pool.cc:224] Handling call
0709 13:06:01.292795 (+ 60163us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:01.402527 (+109732us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:01.402568 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:01.466412 (+ 63844us) log.cc:844] Serialized 26 byte log entry
0709 13:06:02.139858 (+673446us) spinlock_profiling.cc:243] Waited 19.1 ms on lock 0x7b18002d3f40. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f480d1 00007f73ea363a0f
0709 13:06:02.295388 (+155530us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:02.295446 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:02.374605 (+ 79159us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:02.424144 (+ 49539us) raft_consensus.cc:1685] finished
0709 13:06:02.424164 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:02.428955 (+  4791us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49795,"raft.queue_time_us":59402,"raft.run_cpu_time_us":286,"raft.run_wall_time_us":79706,"spinlock_wait_cycles":43871104,"thread_start_us":155325,"threads_started":1,"wal-append.queue_time_us":922603}
I20260709 13:06:02.568265 18953 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:06:02.648862 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 254878us
I20260709 13:06:02.593127 18883 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.613740 18952 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:02.673715 18927 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.644371 18953 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:02.690689 18952 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:02.699187 18952 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:02.700062 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:02.700781 15853 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
I20260709 13:06:02.680078 18883 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:02.709508 15859 raft_consensus.cc:1275] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:06:02.741350 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:02.742868 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:02.746302 15827 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:02.690322 18854 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:02.747895 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:02.748792 18956 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.751266 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:02.751780 18927 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:02.753530 18957 consensus_queue.cc:1048] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:02.754305 18796 consensus_queue.cc:1048] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:02.755219 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:02.691363 18727 consensus_queue.cc:1048] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.067s
I20260709 13:06:02.691915 18929 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.705273 18964 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.761361 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:02.705186 18948 raft_consensus.cc:2764] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:06:02.707868 18586 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:02.780237 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.612887 18958 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:02.775271 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:02.782255 15855 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:02.791502 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.801653 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:02.802501 15856 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:02.764534 18854 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.805941 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:06:02.806391 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 60304.1us
I20260709 13:06:02.806581 16006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 533) took 1260 ms. Trace:
I20260709 13:06:02.812529 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:02.808941 16006 rpcz_store.cc:276] 0709 13:06:01.546517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.703594 (+157077us) service_pool.cc:224] Handling call
0709 13:06:01.795116 (+ 91522us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:01.882341 (+ 87225us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:01.945752 (+ 63411us) log.cc:844] Serialized 14 byte log entry
0709 13:06:02.805953 (+860201us) spinlock_profiling.cc:243] Waited 5.68 ms on lock 0x7b180014fb60. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e9f504bc 00007f73e729e48b 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72b8455 00007f73ea3a0060 00007f73ea3b2716 00007f73ea3c3287
0709 13:06:02.806071 (+   118us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:02.806145 (+    74us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:02.806214 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:02.806319 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:02.806495 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43654,"prepare.queue_time_us":83936,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":256073,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":21085,"spinlock_wait_cycles":13070080,"thread_start_us":538170,"threads_started":2,"wal-append.queue_time_us":348689}
I20260709 13:06:02.826767 15856 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:06:02.829191 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:02.832571 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 193103us
I20260709 13:06:02.803457 18460 raft_consensus.cc:2749] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:02.811208 18854 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:02.960094 16565 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:06:02.868009 15827 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:02.962390 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 269094us
I20260709 13:06:02.809134 18677 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:02.876575 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:02.877153 18960 log.cc:927] Time spent T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:06:02.877347 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:02.879755 18782 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:02.876596 18940 log.cc:927] Time spent T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.175s	user 0.000s	sys 0.000s
W20260709 13:06:02.919397 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.932164 15855 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:06:02.867519 15860 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
W20260709 13:06:02.815433 18841 consensus_meta.cc:287] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.630s	user 0.003s	sys 0.009s
I20260709 13:06:02.964673 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:06:02.971786 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.978180 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:02.978276 15937 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:02.978891 16007 raft_consensus.cc:1275] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:06:02.807134 18586 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:02.986913 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 405248us
I20260709 13:06:02.989223 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:02.992122 16006 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:06:02.989711 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:02.989825 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:02.994910 15934 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:06:02.997011 16011 raft_consensus.cc:1275] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:06:02.813596 18929 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:02.792476 18956 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:03.003319 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:03.008101 16004 raft_consensus.cc:1275] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:03.001735 15974 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:02.803822 18964 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:03.010581 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:02.974058 18971 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:03.020505 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:06:02.994841 18858 consensus_meta.cc:287] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.936s	user 0.010s	sys 0.002s
I20260709 13:06:03.031036 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:03.044818 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:03.044979 16007 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:03.046329 15859 raft_consensus.cc:2410] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:06:03.047144 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:02.998646 18616 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:03.030828 15854 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
I20260709 13:06:03.047921 16011 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
W20260709 13:06:03.061359 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:03.055054 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:03.063395 16004 raft_consensus.cc:2410] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 13:06:03.072870 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 204957us
I20260709 13:06:03.073871 15827 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:03.076165 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:03.024416 18841 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:03.081058 15937 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 7.
W20260709 13:06:03.085230 18867 consensus_meta.cc:287] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.890s	user 0.001s	sys 0.010s
I20260709 13:06:02.995932 18782 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:03.086472 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:03.010070 18968 raft_consensus.cc:2764] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:06:03.041460 18163 raft_consensus.cc:3055] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:03.091485 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:03.099474 16010 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:06:03.092407 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:03.100895 16011 raft_consensus.cc:2393] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 7: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:03.098038 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:03.101536 16004 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:03.098261 15931 raft_consensus.cc:1240] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:03.048893 18984 raft_consensus.cc:2749] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:03.062839 18851 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:03.111563 15928 raft_consensus.cc:1240] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 5. Current term is 6. Ops: []
W20260709 13:06:03.137792 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 205613us
I20260709 13:06:03.138888 18853 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:03.151909 16008 raft_consensus.cc:1240] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:03.104452 18163 raft_consensus.cc:740] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:03.131279 18851 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:03.098780 18867 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:03.171619 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:03.172223 16010 raft_consensus.cc:2376] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7. Re-sending same reply.
I20260709 13:06:03.173105 18867 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:03.173813 15933 raft_consensus.cc:1240] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:03.024322 18891 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:03.056543 18971 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:03.085461 18964 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:03.179253 16007 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:06:03.180402 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:03.181811 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:03.182428 15928 raft_consensus.cc:1275] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:06:03.110781 18841 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:03.206842 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 101627us
I20260709 13:06:03.202570 18802 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:03.210652 18802 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:03.211794 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:03.211895 18460 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:06:03.207067 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 213059us
I20260709 13:06:03.213110 18460 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:03.216150 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:03.219116 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:03.220429 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:03.221436 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:03.223760 18460 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:03.213109 18802 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:06:03.209910 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 207247us
I20260709 13:06:03.226455 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:03.226848 15902 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:03.227138 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:03.227952 15974 connection.cc:256] RPC call timeout handler was delayed by -1.1484460659999058s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15910000000000002s for a response
W20260709 13:06:03.228320 15974 connection.cc:256] RPC call timeout handler was delayed by -1.1250460659999817s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1617s for a response
I20260709 13:06:03.229714 18964 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:03.229841 15827 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:03.230338 18964 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:03.230659 18973 consensus_queue.cc:1059] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:03.233045 15937 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:06:03.231995 16533 leader_election.cc:336] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 3.337s (SENT)
W20260709 13:06:03.203138 18861 consensus_meta.cc:287] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 1.022s	user 0.011s	sys 0.012s
I20260709 13:06:03.231042 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:03.228617 15974 connection.cc:256] RPC call timeout handler was delayed by -1.0890460659999235s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16570000000000001s for a response
W20260709 13:06:03.238600 15974 connection.cc:256] RPC call timeout handler was delayed by -1.0791460659999075s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1668s for a response
W20260709 13:06:03.239331 15974 connection.cc:256] RPC call timeout handler was delayed by -1.0440460659999644s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17070000000000002s for a response
I20260709 13:06:03.243000 15855 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6.
I20260709 13:06:03.243424 15937 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5.
I20260709 13:06:03.141202 18853 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:03.206528 18177 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:03.272123 15974 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:03.275794 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:03.205281 18926 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:03.244378 18970 raft_consensus.cc:2804] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:03.284730 15859 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 7.
I20260709 13:06:03.312718 16007 raft_consensus.cc:1240] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:03.284847 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:03.313057 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:03.311513 16564 consensus_meta.cc:287] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.556s	user 0.006s	sys 0.002s
I20260709 13:06:03.317762 16004 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:06:03.205602 18796 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.320178 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:03.235618 18958 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:03.236860 18891 raft_consensus.cc:2804] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:03.235457 18927 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:03.244904 18858 consensus_queue.cc:1048] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.258s
I20260709 13:06:03.541397 15937 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5.
I20260709 13:06:03.547605 16564 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:06:03.539839 16569 raft_consensus.cc:1240] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:03.369004 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:03.345101 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:03.546555 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:06:03.475047 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:03.344971 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 73014.5us
W20260709 13:06:03.372839 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 140859us
I20260709 13:06:03.545735 16008 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:03.437309 18982 log.cc:927] Time spent T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
I20260709 13:06:03.554325 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:03.574608 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:03.574723 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:03.540349 18988 log.cc:927] Time spent T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:06:03.540648 18802 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.531666 18860 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.584720 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:03.326170 18984 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.346134 18792 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:03.475430 18460 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.588899 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:03.439694 18978 raft_consensus.cc:2749] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:03.556317 15827 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:03.644227 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:03.644346 18990 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:06:03.644742 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 88505us
I20260709 13:06:03.645095 16565 raft_consensus.cc:2410] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:06:03.644943 18990 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:03.647185 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:03.647324 18990 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:03.648000 18990 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:06:03.648648 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:03.649434 16569 raft_consensus.cc:2410] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 6 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:06:03.548091 18586 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:03.609087 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:03.650838 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:03.499001 18177 raft_consensus.cc:697] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:03.503490 18782 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:03.580418 18891 raft_consensus.cc:697] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:03.478503 18163 consensus_queue.cc:260] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:03.649930 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:03.650962 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:03.655632 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:03.657238 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:03.548360 18970 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:03.371309 18985 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.591536 15974 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:06:03.710930 16564 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45232 (request call id 670) took 1697 ms (client timeout 1833 ms). Trace:
I20260709 13:06:03.686820 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:03.721971 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:03.723127 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:03.723308 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 133349us
I20260709 13:06:03.725790 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:03.688149 18969 log.cc:927] Time spent T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.521s	user 0.001s	sys 0.000s
W20260709 13:06:03.722230 16564 rpcz_store.cc:269] 0709 13:06:01.944562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.042488 (+ 97926us) service_pool.cc:224] Handling call
0709 13:06:03.610712 (+1568224us) spinlock_profiling.cc:243] Waited 654 ms on lock 0x7b5400573ad4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:03.641399 (+ 30687us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1504485760}
I20260709 13:06:03.717203 18927 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:03.656095 18983 log.cc:927] Time spent T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.481s	user 0.001s	sys 0.000s
W20260709 13:06:03.801890 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 75682.4us
W20260709 13:06:03.694370 18883 consensus_meta.cc:287] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.920s	user 0.009s	sys 0.000s
I20260709 13:06:03.727579 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:03.801817 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:03.807708 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:03.810990 15937 raft_consensus.cc:1240] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:03.806699 15854 raft_consensus.cc:2376] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6. Re-sending same reply.
W20260709 13:06:03.813283 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 77432.6us
I20260709 13:06:03.813804 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:03.814167 19009 consensus_queue.cc:1059] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:03.814875 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:03.814976 19009 consensus_queue.cc:1059] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:03.815580 19007 raft_consensus.cc:3055] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:03.815972 19007 raft_consensus.cc:740] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:03.816208 18990 raft_consensus.cc:3055] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:03.816583 18990 raft_consensus.cc:740] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:03.816771 19007 consensus_queue.cc:260] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:03.817240 18945 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.817404 18990 consensus_queue.cc:260] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:03.817960 19007 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:03.817761 18945 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:03.818657 18990 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:03.814080 15825 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:03.820278 18985 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:03.822340 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:03.823089 16570 raft_consensus.cc:2393] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 6: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:03.823170 18985 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:03.824450 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 232982us
I20260709 13:06:03.726626 18970 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:03.803337 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:03.692826 18898 consensus_meta.cc:287] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 1.489s	user 0.010s	sys 0.002s
W20260709 13:06:03.655637 18991 log.cc:927] Time spent T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 13:06:03.670482 18964 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:03.804836 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:03.858203 16565 raft_consensus.cc:2410] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:06:03.588814 18796 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:03.656078 18460 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:03.665143 18987 log.cc:927] Time spent T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.344s	user 0.000s	sys 0.000s
W20260709 13:06:03.801436 18953 consensus_meta.cc:287] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 1.043s	user 0.005s	sys 0.005s
I20260709 13:06:03.870113 18953 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:03.872268 18953 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:03.872969 16009 raft_consensus.cc:2393] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 7: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
W20260709 13:06:03.873263 18981 log.cc:927] Time spent T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.702s	user 0.003s	sys 0.000s
I20260709 13:06:03.874490 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:03.875089 16009 raft_consensus.cc:2376] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6. Re-sending same reply.
I20260709 13:06:03.875901 16004 raft_consensus.cc:1240] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:03.876020 15858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1763) took 1181 ms. Trace:
I20260709 13:06:03.876976 18962 raft_consensus.cc:3055] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:03.877487 18962 raft_consensus.cc:740] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:03.878458 18962 consensus_queue.cc:260] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:03.876398 15858 rpcz_store.cc:276] 0709 13:06:02.694501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.694615 (+   114us) service_pool.cc:224] Handling call
0709 13:06:02.694935 (+   320us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:02.713832 (+ 18897us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:02.743876 (+ 30044us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:02.760816 (+ 16940us) log.cc:844] Serialized 52 byte log entry
0709 13:06:03.016282 (+255466us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:03.542041 (+525759us) log.cc:844] Serialized 14 byte log entry
0709 13:06:03.553036 (+ 10995us) log.cc:844] Serialized 14 byte log entry
0709 13:06:03.557752 (+  4716us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.557858 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:03.872626 (+314768us) spinlock_profiling.cc:243] Waited 2.66 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:06:03.875805 (+  3179us) raft_consensus.cc:1685] finished
0709 13:06:03.875825 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:03.875943 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15928,"prepare.queue_time_us":215603,"raft.queue_time_us":131481,"raft.run_cpu_time_us":416,"raft.run_wall_time_us":71932,"spinlock_wait_cycles":12787328,"thread_start_us":456216,"threads_started":2,"wal-append.queue_time_us":316857}
I20260709 13:06:03.885025 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:03.885638 15860 raft_consensus.cc:2376] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6. Re-sending same reply.
I20260709 13:06:03.885762 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:03.885958 18902 consensus_meta.cc:287] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 1.597s	user 0.010s	sys 0.001s
I20260709 13:06:03.886377 15858 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:06:03.886523 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:03.887074 15860 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:03.639766 18860 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:03.846010 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:03.610932 18802 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:03.894328 15861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1776) took 1189 ms. Trace:
I20260709 13:06:03.680935 18853 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:03.855836 15937 raft_consensus.cc:1275] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 6 index: 9. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:06:03.701328 18792 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:03.859175 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:03.861560 16562 raft_consensus.cc:1275] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:06:03.907169 15931 raft_consensus.cc:2393] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 7: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:03.668838 18841 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:03.909504 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:03.869374 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:03.805387 18948 raft_consensus.cc:2749] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:03.843999 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:03.890699 15902 leader_election.cc:336] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 1.880s (SENT)
I20260709 13:06:03.851521 15974 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:06:03.898777 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:03.902570 18177 consensus_queue.cc:237] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:03.886955 18891 consensus_queue.cc:237] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:03.919302 15862 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:06:03.913125 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:03.913131 15902 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:03.912130 15928 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:03.919656 16561 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:03.697752 18984 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:03.923739 15933 raft_consensus.cc:2393] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 5: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:03.925415 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:03.903795 18983 log.cc:927] Time spent T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:06:03.931869 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:03.903752 18962 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:03.927256 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:03.931095 16568 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5.
W20260709 13:06:03.908020 18950 consensus_meta.cc:287] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 1.226s	user 0.010s	sys 0.002s
I20260709 13:06:03.933234 16567 raft_consensus.cc:2410] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:06:03.896639 15861 rpcz_store.cc:276] 0709 13:06:02.704498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.833491 (+128993us) service_pool.cc:224] Handling call
0709 13:06:02.884975 (+ 51484us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:06:02.885451 (+   476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:02.885490 (+    39us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:06:02.886263 (+   773us) log.cc:844] Serialized 78 byte log entry
0709 13:06:03.088139 (+201876us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:03.237043 (+148904us) log.cc:844] Serialized 14 byte log entry
0709 13:06:03.893932 (+656889us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.894030 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:03.894089 (+    59us) raft_consensus.cc:1685] finished
0709 13:06:03.894107 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:03.894277 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":254042,"prepare.queue_time_us":565294,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":401906,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":32128,"thread_start_us":781910,"threads_started":3,"wal-append.queue_time_us":591039}
I20260709 13:06:03.929837 15860 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
W20260709 13:06:03.933679 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 82249us
I20260709 13:06:03.895462 18858 raft_consensus.cc:2749] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:06:03.933480 15857 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.760s	user 0.000s	sys 0.015s
I20260709 13:06:03.847358 18883 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:03.905856 18971 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:03.937330 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:03.938553 15931 raft_consensus.cc:2393] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 5: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
W20260709 13:06:03.940073 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 358298us
I20260709 13:06:03.847206 18927 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:03.932605 18998 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:06:03.959022 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 138369us
I20260709 13:06:03.928602 18945 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:03.932780 18853 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:03.862393 18898 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:03.979434 16005 raft_consensus.cc:2393] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 6: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:03.962558 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:03.862222 19009 consensus_queue.cc:1059] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
W20260709 13:06:03.927908 15900 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.877s	user 0.006s	sys 0.007s
W20260709 13:06:03.985496 15900 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.934s	user 0.006s	sys 0.007s
W20260709 13:06:03.985942 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 935066us
W20260709 13:06:03.943837 15857 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.770s	user 0.001s	sys 0.015s
I20260709 13:06:03.886440 18902 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:03.960454 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:03.966841 15931 raft_consensus.cc:1240] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:03.973465 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:03.960193 15853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1769) took 1257 ms. Trace:
I20260709 13:06:04.007195 15861 raft_consensus.cc:2410] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
W20260709 13:06:03.945204 16563 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.961s	user 0.017s	sys 0.057s
I20260709 13:06:03.980151 15860 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6.
I20260709 13:06:04.006836 15853 rpcz_store.cc:276] 0709 13:06:02.702646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.702752 (+   106us) service_pool.cc:224] Handling call
0709 13:06:02.757042 (+ 54290us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:02.779822 (+ 22780us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:02.779873 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:02.879644 (+ 99771us) log.cc:844] Serialized 52 byte log entry
0709 13:06:03.027657 (+148013us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:03.027730 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.027805 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:03.959975 (+932170us) raft_consensus.cc:1685] finished
0709 13:06:03.959994 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:03.960132 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16039,"raft.queue_time_us":21517,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":25766,"thread_start_us":147614,"threads_started":1,"wal-append.queue_time_us":323059}
I20260709 13:06:03.981954 15974 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:04.042279 15858 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 7.
I20260709 13:06:04.102123 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:04.111057 15853 raft_consensus.cc:2393] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 6: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
W20260709 13:06:04.150925 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 184408us
I20260709 13:06:04.153505 15825 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:04.156324 15825 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:04.171088 16533 leader_election.cc:336] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.576s (SENT)
I20260709 13:06:04.170888 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:04.171813 16533 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:06:04.037667 16563 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.053s	user 0.017s	sys 0.057s
W20260709 13:06:04.201213 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 219254us
I20260709 13:06:04.170897 15929 raft_consensus.cc:1240] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:04.172812 15856 raft_consensus.cc:1240] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:04.201197 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:04.207265 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 51025.1us
I20260709 13:06:04.214726 15825 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:03.947240 18950 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:04.217308 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:04.220396 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 382586us
I20260709 13:06:04.222867 15931 raft_consensus.cc:1240] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:06:04.212101 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:04.218174 15861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 1872) took 1001 ms. Trace:
I20260709 13:06:04.226913 15861 rpcz_store.cc:276] 0709 13:06:03.216366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.093829 (+877463us) service_pool.cc:224] Handling call
0709 13:06:04.218084 (+124255us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:04.221674 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:04.229581 15854 raft_consensus.cc:2393] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 7: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:04.234846 15853 raft_consensus.cc:2368] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 5. Current term is 6.
I20260709 13:06:04.239517 15853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58214 (request call id 578) took 1019 ms. Trace:
I20260709 13:06:04.239944 15853 rpcz_store.cc:276] 0709 13:06:03.220025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.200227 (+980202us) service_pool.cc:224] Handling call
0709 13:06:04.239448 (+ 39221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:04.237062 15902 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
W20260709 13:06:04.247853 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 125780us
W20260709 13:06:04.249327 16533 leader_election.cc:336] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.647s (SENT)
I20260709 13:06:03.987784 18853 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:04.252200 15825 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:04.251664 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:04.267933 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:04.253731 16006 raft_consensus.cc:2393] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 9: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
W20260709 13:06:04.254693 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 388801us
I20260709 13:06:04.170857 18586 consensus_queue.cc:1048] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.074s
W20260709 13:06:04.240897 18952 consensus_meta.cc:287] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 1.541s	user 0.008s	sys 0.006s
W20260709 13:06:04.263583 16533 leader_election.cc:336] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.650s (SENT)
I20260709 13:06:04.266185 15974 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:04.280803 15902 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:04.235234 18945 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:04.280467 16533 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:04.271615 15825 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:04.259007 18899 raft_consensus.cc:2804] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:04.279312 18840 raft_consensus.cc:2749] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:06:04.326436 16008 consensus_meta.cc:287] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.746s	user 0.010s	sys 0.000s
I20260709 13:06:04.326848 16008 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:06:04.250988 18985 consensus_queue.cc:1048] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.068s
I20260709 13:06:04.282058 18999 raft_consensus.cc:2749] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:06:04.321797 18851 consensus_meta.cc:287] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 1.090s	user 0.008s	sys 0.000s
I20260709 13:06:04.332309 16008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41674 (request call id 1765) took 1080 ms. Trace:
I20260709 13:06:04.336398 16008 rpcz_store.cc:276] 0709 13:06:03.247503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.247655 (+   152us) service_pool.cc:224] Handling call
0709 13:06:04.327420 (+1079765us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:04.307577 19016 raft_consensus.cc:2804] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:04.313843 18899 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:04.340170 18902 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:04.318140 18860 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:04.287807 18952 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:04.416157 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:04.516065 16003 raft_consensus.cc:2393] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 6: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:04.517179 18952 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:06:04.517670 16003 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41662 (request call id 1866) took 1553 ms (client timeout 1917 ms). Trace:
W20260709 13:06:04.518189 16003 rpcz_store.cc:269] 0709 13:06:02.963880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.964030 (+   150us) service_pool.cc:224] Handling call
0709 13:06:04.517440 (+1553410us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b54001dd154. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:04.517598 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3500948736}
I20260709 13:06:04.519702 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:04.520406 16570 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
I20260709 13:06:04.521486 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:04.522053 16570 raft_consensus.cc:2376] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6. Re-sending same reply.
I20260709 13:06:04.522830 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:04.524806 16570 raft_consensus.cc:1275] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:06:04.339627 18949 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:04.463303 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:04.467885 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:04.331979 18586 raft_consensus.cc:2749] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:04.329363 18460 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:04.473474 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:04.479280 18999 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:06:04.481441 16006 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41662 (request call id 1871) took 1514 ms (client timeout 1645 ms). Trace:
W20260709 13:06:04.539338 16006 rpcz_store.cc:269] 0709 13:06:02.966692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.993005 (+ 26313us) service_pool.cc:224] Handling call
0709 13:06:04.481193 (+1488188us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b540025a3d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:04.481368 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2823328640}
I20260709 13:06:04.482859 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:04.343416 18851 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:04.483829 19000 raft_consensus.cc:2804] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:06:04.484797 19018 log.cc:927] Time spent T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.254s	user 0.000s	sys 0.008s
W20260709 13:06:04.485147 19030 log.cc:927] Time spent T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260709 13:06:04.488376 18950 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:04.489379 18985 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:04.491236 15936 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 614) took 1620 ms. Trace:
I20260709 13:06:04.491724 18927 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:04.556941 15936 rpcz_store.cc:276] 0709 13:06:02.871043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.871158 (+   115us) service_pool.cc:224] Handling call
0709 13:06:02.879647 (+  8489us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:03.358236 (+478589us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:03.358278 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:03.371605 (+ 13327us) log.cc:844] Serialized 26 byte log entry
0709 13:06:03.931024 (+559419us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:03.931100 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.931190 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:04.490998 (+559808us) raft_consensus.cc:1685] finished
0709 13:06:04.491021 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:04.491179 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":99953,"raft.queue_time_us":209573,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":15427,"thread_start_us":860936,"threads_started":2,"wal-append.queue_time_us":845478}
I20260709 13:06:04.491866 15930 raft_consensus.cc:1240] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:04.338630 19016 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:06:04.492312 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 228762us
I20260709 13:06:04.492826 18728 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:04.494607 18898 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:04.495030 18984 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:04.496037 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:04.496874 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:04.497936 16565 raft_consensus.cc:1240] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 7. Current term is 8. Ops: []
W20260709 13:06:04.559430 18854 consensus_meta.cc:287] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 1.557s	user 0.005s	sys 0.007s
I20260709 13:06:04.501164 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:04.503701 15937 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1843) took 1277 ms. Trace:
W20260709 13:06:04.505182 16005 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41662 (request call id 1867) took 1541 ms (client timeout 1994 ms). Trace:
I20260709 13:06:04.330565 19010 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:04.379163 18993 consensus_queue.cc:1048] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.188s
W20260709 13:06:04.379122 19012 log.cc:927] Time spent T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.287s	user 0.001s	sys 0.000s
W20260709 13:06:04.512076 15862 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 1880) took 1192 ms (client timeout 1539 ms). Trace:
I20260709 13:06:04.514321 18853 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:04.476796 19021 log.cc:927] Time spent T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.215s	user 0.000s	sys 0.001s
I20260709 13:06:04.526611 15976 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:04.519542 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:06:04.582325 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 55822.5us
I20260709 13:06:04.582605 16003 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:06:04.339877 19024 raft_consensus.cc:2804] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:04.533440 15860 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
W20260709 13:06:04.533527 16561 consensus_meta.cc:287] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.607s	user 0.010s	sys 0.000s
I20260709 13:06:04.538434 15861 raft_consensus.cc:2376] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5. Re-sending same reply.
I20260709 13:06:04.540401 15854 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:04.588169 15860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 1878) took 1304 ms. Trace:
I20260709 13:06:04.475970 18899 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:04.534199 18949 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:04.562251 16011 raft_consensus.cc:2393] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 6: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:04.563015 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:06:04.560726 16533 leader_election.cc:336] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.650s (SENT)
I20260709 13:06:04.565160 15900 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:04.498554 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:04.595151 16564 raft_consensus.cc:2393] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 7: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:04.595666 15900 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:04.596038 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:04.596873 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:04.597250 16564 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:06:04.554802 19000 raft_consensus.cc:697] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:04.543524 18860 consensus_queue.cc:1059] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 13:06:04.568295 16562 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
I20260709 13:06:04.561200 18883 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:04.570716 15937 rpcz_store.cc:276] 0709 13:06:03.226451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.812337 (+585886us) service_pool.cc:224] Handling call
0709 13:06:03.812606 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:03.896320 (+ 83714us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.503388 (+607068us) spinlock_profiling.cc:243] Waited 26.5 ms on lock 0x7b5400153e54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:04.503617 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":60999424}
W20260709 13:06:04.571127 16005 rpcz_store.cc:269] 0709 13:06:02.964215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.964380 (+   165us) service_pool.cc:224] Handling call
0709 13:06:04.505005 (+1540625us) spinlock_profiling.cc:243] Waited 997 ms on lock 0x7b5400304b54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:04.505126 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2293560576}
I20260709 13:06:04.601300 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:06:04.574681 15862 rpcz_store.cc:269] 0709 13:06:03.322682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.641770 (+319088us) service_pool.cc:224] Handling call
0709 13:06:04.511883 (+870113us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b54004263d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:04.512012 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":449634048}
W20260709 13:06:04.601727 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:04.570847 19033 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.548445 15827 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:04.603639 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 332094us
W20260709 13:06:04.603955 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 55576.8us
I20260709 13:06:04.604272 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1884) took 1328 ms. Trace:
I20260709 13:06:04.588544 15861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58214 (request call id 569) took 1371 ms. Trace:
I20260709 13:06:04.606483 15861 rpcz_store.cc:276] 0709 13:06:03.216930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.268019 (+1051089us) service_pool.cc:224] Handling call
0709 13:06:04.588467 (+320448us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:04.607664 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:04.588200 16561 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:06:04.589926 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:04.588611 15860 rpcz_store.cc:276] 0709 13:06:03.284259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.029042 (+744783us) service_pool.cc:224] Handling call
0709 13:06:04.588090 (+559048us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:04.593969 15932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 612) took 1725 ms. Trace:
I20260709 13:06:04.594391 16533 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:04.580883 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:04.562170 19016 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:04.604663 16002 rpcz_store.cc:276] 0709 13:06:03.285601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.550636 (+265035us) service_pool.cc:224] Handling call
0709 13:06:03.586194 (+ 35558us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:03.590373 (+  4179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:03.590402 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:03.594415 (+  4013us) log.cc:844] Serialized 26 byte log entry
0709 13:06:03.904186 (+309771us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:03.904234 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.904322 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:04.604088 (+699766us) raft_consensus.cc:1685] finished
0709 13:06:04.604106 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:04.604220 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3827,"raft.queue_time_us":4416,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":60230,"thread_start_us":224251,"threads_started":1,"wal-append.queue_time_us":379719}
I20260709 13:06:04.591620 18950 raft_consensus.cc:2804] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:04.612166 16565 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
W20260709 13:06:04.599802 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 333675us
I20260709 13:06:04.568282 18985 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:04.598892 18949 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:04.617440 15861 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:06:04.624173 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:04.624707 15861 raft_consensus.cc:606] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Received request to transfer leadership to TS 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:04.573836 18854 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:04.633976 18854 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:04.634809 16010 raft_consensus.cc:2393] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 6: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:04.621218 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:04.602766 18993 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:04.615886 15974 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:04.625108 15861 raft_consensus.cc:614] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:04.624173 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:04.640450 16007 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5.
I20260709 13:06:04.641181 15861 raft_consensus.cc:1275] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:06:04.641351 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:04.641779 15861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 568) took 1425 ms. Trace:
I20260709 13:06:04.642148 15861 rpcz_store.cc:276] 0709 13:06:03.216599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.640629 (+1424030us) service_pool.cc:224] Handling call
0709 13:06:04.640886 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:04.641544 (+   658us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.641729 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:04.643021 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:04.625953 18891 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:04.627017 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:04.612715 18984 raft_consensus.cc:2749] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:04.629340 19007 consensus_queue.cc:1048] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:04.620608 18927 consensus_queue.cc:1059] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:06:04.652935 19007 raft_consensus.cc:3055] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:04.653339 19007 raft_consensus.cc:740] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:04.653729 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:04.654366 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:04.654244 19007 consensus_queue.cc:260] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:04.630035 19022 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:04.654933 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:04.610143 15932 rpcz_store.cc:276] 0709 13:06:02.868627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.868755 (+   128us) service_pool.cc:224] Handling call
0709 13:06:02.878707 (+  9952us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:03.203025 (+324318us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:06:03.203067 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:03.244170 (+ 41103us) log.cc:844] Serialized 26 byte log entry
0709 13:06:03.530041 (+285871us) spinlock_profiling.cc:243] Waited 5.47 ms on lock 0x7b4800002dd8. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ef2d1aa1 00007f73ef2c8b37 00007f73ef2c89ca 00007f73ef3c026f 00007f73ef3bff37 00007f73ef3d455c 00007f73ef3d4508 00007f73ef3d4488 00007f73ef3d4443 00007f73ef3d36ec 00007f73ee2fda96
0709 13:06:03.804172 (+274131us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:03.804253 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.804342 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:04.581598 (+777256us) raft_consensus.cc:1685] finished
0709 13:06:04.581617 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:04.593906 (+ 12289us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":100621,"raft.queue_time_us":167107,"raft.run_cpu_time_us":3682,"raft.run_wall_time_us":484694,"spinlock_wait_cycles":12585344,"thread_start_us":555999,"threads_started":2,"wal-append.queue_time_us":573768}
I20260709 13:06:04.631990 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:04.636029 16003 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:06:04.617264 15900 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:04.598416 19024 raft_consensus.cc:697] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:04.639305 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:04.636543 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 355804us
I20260709 13:06:04.644531 15934 raft_consensus.cc:2393] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 6: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:04.627068 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:04.642023 16007 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:06:04.616166 18950 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:06:04.647707 18962 consensus_meta.cc:287] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.705s	user 0.004s	sys 0.005s
I20260709 13:06:04.648176 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:04.648694 18851 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:04.649959 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:04.650658 18985 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:04.651245 16563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 720) took 2390 ms. Trace:
I20260709 13:06:04.593895 18853 consensus_queue.cc:1048] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:06:04.599473 19003 consensus_queue.cc:1059] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:06:04.658847 16005 raft_consensus.cc:2393] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 7: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:04.711526 16007 pending_rounds.cc:85] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting all ops after (but not including) 3
I20260709 13:06:04.751649 16007 pending_rounds.cc:107] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:06:04.752039 16007 raft_consensus.cc:2889] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:04.753151 16005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60156 (request call id 548) took 1027 ms. Trace:
I20260709 13:06:04.753548 15854 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:06:04.753636 16005 rpcz_store.cc:276] 0709 13:06:03.725311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.636480 (+911169us) service_pool.cc:224] Handling call
0709 13:06:04.753079 (+116599us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:04.662420 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:04.717612 15937 raft_consensus.cc:2410] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:06:04.717931 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 100690us
I20260709 13:06:04.718050 15902 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:04.754935 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:04.756178 15854 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6.
I20260709 13:06:04.756596 15902 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:04.691216 16004 raft_consensus.cc:2410] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 9 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:06:04.757789 16004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41662 (request call id 1886) took 1103 ms. Trace:
I20260709 13:06:04.758112 16004 rpcz_store.cc:276] 0709 13:06:03.681282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.654727 (+973445us) service_pool.cc:224] Handling call
0709 13:06:04.757734 (+103007us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:04.722149 15825 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:04.719301 18950 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:04.638893 19035 consensus_queue.cc:1048] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.098s
I20260709 13:06:04.723702 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:04.724187 16003 raft_consensus.cc:1275] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:06:04.717315 18891 raft_consensus.cc:2804] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:04.737282 15974 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:04.737869 18177 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.739166 19039 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.739938 18948 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:04.635878 18945 raft_consensus.cc:2749] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:04.742133 18929 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:04.725373 18999 consensus_queue.cc:1048] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
W20260709 13:06:04.749029 18990 consensus_meta.cc:287] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.930s	user 0.011s	sys 0.003s
I20260709 13:06:04.640005 18993 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:06:04.657008 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 96324.6us
I20260709 13:06:04.761912 18985 raft_consensus.cc:2749] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:04.736333 18883 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:06:04.724541 18970 consensus_meta.cc:287] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.868s	user 0.009s	sys 0.000s
I20260709 13:06:04.762472 15900 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:04.762840 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:04.811096 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:04.811889 15854 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:04.763449 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:04.719468 18854 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:04.719640 18927 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.801065 16005 raft_consensus.cc:1240] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:06:04.823743 16005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 544) took 1116 ms. Trace:
I20260709 13:06:04.824215 16005 rpcz_store.cc:276] 0709 13:06:03.724009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.772505 (+1048496us) service_pool.cc:224] Handling call
0709 13:06:04.800829 (+ 28324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:04.823351 (+ 22522us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.823649 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:04.615780 19033 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:04.765146 18616 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:06:04.804245 16533 leader_election.cc:336] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.701s (SENT)
I20260709 13:06:04.761125 16003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1887) took 1079 ms. Trace:
I20260709 13:06:04.766324 18984 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:04.827210 16003 rpcz_store.cc:276] 0709 13:06:03.681738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.718939 (+1037201us) service_pool.cc:224] Handling call
0709 13:06:04.723875 (+  4936us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:04.760885 (+ 37010us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.761073 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:04.807792 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 45599.9us
W20260709 13:06:04.828737 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 91543.4us
I20260709 13:06:04.829413 15902 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:04.821884 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 59478.5us
I20260709 13:06:04.792632 18891 raft_consensus.cc:697] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:04.806210 19007 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:04.836117 15900 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:04.807400 18993 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:04.800292 18929 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:04.804488 18851 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:04.830442 15974 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:04.846395 15854 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:06:04.847440 16533 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:04.886431 15861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 1874) took 1626 ms. Trace:
I20260709 13:06:04.791651 18177 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:04.808454 19003 raft_consensus.cc:3055] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:06:04.897272 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 61243.9us
I20260709 13:06:04.886813 15861 rpcz_store.cc:276] 0709 13:06:03.216926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.642835 (+1425909us) service_pool.cc:224] Handling call
0709 13:06:04.643135 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:04.690127 (+ 46992us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:04.690166 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:04.725085 (+ 34919us) log.cc:844] Serialized 26 byte log entry
0709 13:06:04.841166 (+116081us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:04.841211 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.841279 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:04.842614 (+  1335us) raft_consensus.cc:1685] finished
0709 13:06:04.842636 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:04.843573 (+   937us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12077,"raft.queue_time_us":46566,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":97,"thread_start_us":91410,"threads_started":1,"wal-append.queue_time_us":91840}
I20260709 13:06:04.889899 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:06:04.891064 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 86861.2us
W20260709 13:06:04.899243 16533 leader_election.cc:336] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.910s (SENT)
W20260709 13:06:04.892328 19042 log.cc:927] Time spent T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:06:04.892107 18945 raft_consensus.cc:2749] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:06:04.893817 19037 log.cc:927] Time spent T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:06:04.651748 19000 consensus_queue.cc:237] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:04.900506 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 70195.8us
W20260709 13:06:04.901486 19048 log.cc:927] Time spent T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
W20260709 13:06:04.902559 19041 log.cc:927] Time spent T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:06:04.903038 18728 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:04.826649 18616 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:04.904038 18891 consensus_queue.cc:237] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:04.905594 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:04.836402 19045 raft_consensus.cc:2804] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:06:04.925419 16533 leader_election.cc:336] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 2.064s (SENT)
I20260709 13:06:04.800279 18948 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:04.915728 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 86397us
I20260709 13:06:04.821602 18970 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:04.791989 19039 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:04.811650 18883 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:04.810189 18899 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:04.928170 18616 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:04.942203 18899 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:04.944444 18883 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:04.944916 18949 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:04.946859 19033 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:04.948833 18177 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:04.953306 19039 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:04.725111 16563 rpcz_store.cc:276] 0709 13:06:02.261090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.261273 (+   183us) service_pool.cc:224] Handling call
0709 13:06:02.268512 (+  7239us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:02.684487 (+415975us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:02.686875 (+  2388us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:02.780048 (+ 93173us) log.cc:844] Serialized 26 byte log entry
0709 13:06:04.223896 (+1443848us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:04.230721 (+  6825us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.244231 (+ 13510us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:04.650979 (+406748us) raft_consensus.cc:1685] finished
0709 13:06:04.651013 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:04.651166 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":139550,"raft.run_cpu_time_us":53,"raft.run_wall_time_us":52,"thread_start_us":1584173,"threads_started":2,"wal-append.queue_time_us":1300069}
I20260709 13:06:04.931102 18728 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:04.956992 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 546) took 1232 ms. Trace:
I20260709 13:06:04.957559 16002 rpcz_store.cc:276] 0709 13:06:03.724724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.955990 (+1231266us) service_pool.cc:224] Handling call
0709 13:06:04.956374 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:04.956591 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:04.956628 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:04.956679 (+    51us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:04.956724 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.956790 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:04.956914 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:04.885442 19024 consensus_queue.cc:237] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:04.905649 18985 raft_consensus.cc:2749] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:04.960469 15930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1847) took 1731 ms. Trace:
I20260709 13:06:04.973685 16005 raft_consensus.cc:1240] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:06:04.888090 18929 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:04.912147 15861 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7c0fef632f9a4485838855b553a79163"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "8430db80a39d4868bcf7641c16de35ea"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:06:04.931942 15902 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:04.915719 15974 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:04.932739 16533 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:04.897771 19003 raft_consensus.cc:740] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:04.914252 18898 raft_consensus.cc:2804] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:04.836639 18927 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:05.069880 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:04.969655 15857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1762) took 2275 ms. Trace:
I20260709 13:06:04.974164 15929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 606) took 2791 ms. Trace:
I20260709 13:06:05.197039 15929 rpcz_store.cc:276] 0709 13:06:02.183035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.183141 (+   106us) service_pool.cc:224] Handling call
0709 13:06:02.183400 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:04.203379 (+2019979us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.973931 (+770552us) spinlock_profiling.cc:243] Waited 1.97 s on lock 0x7b54001a81d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:04.974087 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4532433792}
I20260709 13:06:04.973377 18970 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:05.153440 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 392205us
W20260709 13:06:05.200261 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 268397us
I20260709 13:06:05.201190 15902 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:05.202049 18899 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:05.202190 18927 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:05.202495 18899 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:05.202904 18899 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:05.203269 15827 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:05.204447 18966 raft_consensus.cc:2749] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:05.204924 15856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1770) took 2502 ms. Trace:
I20260709 13:06:05.205015 18926 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:05.040657 18949 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:05.205389 15856 rpcz_store.cc:276] 0709 13:06:02.702933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.827689 (+124756us) service_pool.cc:224] Handling call
0709 13:06:02.828007 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:04.203560 (+1375553us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.204723 (+1001163us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b5400218254. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:05.204872 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3092207360}
I20260709 13:06:05.014039 18984 raft_consensus.cc:3055] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:05.206511 18984 raft_consensus.cc:740] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:05.207005 15856 raft_consensus.cc:1240] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:04.972473 15853 raft_consensus.cc:2393] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 5: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:04.967293 15933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 1772) took 1235 ms. Trace:
I20260709 13:06:04.971761 16008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 550) took 1246 ms. Trace:
W20260709 13:06:05.208580 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:05.138221 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 232164us
I20260709 13:06:05.211182 16008 rpcz_store.cc:276] 0709 13:06:03.725795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.971012 (+1245217us) service_pool.cc:224] Handling call
0709 13:06:04.971281 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:04.971457 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:04.971481 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:04.971519 (+    38us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:04.971547 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.971599 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:04.971711 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:04.960909 15858 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58214 (request call id 589) took 1737 ms (client timeout 2223 ms). Trace:
I20260709 13:06:05.212512 15900 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:06:05.212638 15858 rpcz_store.cc:269] 0709 13:06:03.223687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.910643 (+686956us) service_pool.cc:224] Handling call
0709 13:06:04.960719 (+1050076us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b54003b5e54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:04.960822 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":252451584}
I20260709 13:06:04.973991 15930 rpcz_store.cc:276] 0709 13:06:03.229175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.930517 (+701342us) service_pool.cc:224] Handling call
0709 13:06:04.491688 (+561171us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:04.559065 (+ 67377us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.960253 (+401188us) spinlock_profiling.cc:243] Waited 561 ms on lock 0x7b540034ba50. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:04.960407 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1289379200}
I20260709 13:06:04.959658 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 549) took 1234 ms. Trace:
W20260709 13:06:05.210096 15859 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58214 (request call id 566) took 1998 ms (client timeout 1576 ms). Trace:
I20260709 13:06:05.198917 18460 raft_consensus.cc:3055] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:05.208978 15861 raft_consensus.cc:606] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Received request to transfer leadership to TS 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:05.216059 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:05.216454 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:05.214903 16002 rpcz_store.cc:276] 0709 13:06:03.725569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.958717 (+1233148us) service_pool.cc:224] Handling call
0709 13:06:04.959095 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:04.959292 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:04.959326 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:04.959374 (+    48us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:04.959414 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.959476 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:04.959597 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:05.210713 15933 rpcz_store.cc:276] 0709 13:06:03.731617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.937905 (+206288us) service_pool.cc:224] Handling call
0709 13:06:03.961965 (+ 24060us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:03.978514 (+ 16549us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:03.978560 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:04.205121 (+226561us) log.cc:844] Serialized 26 byte log entry
0709 13:06:04.587538 (+382417us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:04.587567 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.587616 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:04.967031 (+379415us) raft_consensus.cc:1685] finished
0709 13:06:04.967059 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:04.967228 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14677,"raft.queue_time_us":12594,"raft.run_cpu_time_us":282,"raft.run_wall_time_us":51512,"thread_start_us":339380,"threads_started":1,"wal-append.queue_time_us":487173}
I20260709 13:06:05.266718 15930 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
W20260709 13:06:05.266722 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 351088us
I20260709 13:06:05.285683 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:05.218602 15856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 573) took 1992 ms. Trace:
W20260709 13:06:05.268827 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 343391us
W20260709 13:06:05.213954 15859 rpcz_store.cc:269] 0709 13:06:03.215758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.215911 (+   153us) service_pool.cc:224] Handling call
0709 13:06:05.209938 (+1994027us) spinlock_profiling.cc:243] Waited 67.9 ms on lock 0x7b540032e454. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:05.210034 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":156174208}
W20260709 13:06:05.288789 16533 leader_election.cc:336] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 2.114s (SENT)
I20260709 13:06:05.289011 15861 raft_consensus.cc:614] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:06:05.253229 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 52268us
I20260709 13:06:05.289371 15856 rpcz_store.cc:276] 0709 13:06:03.218348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.206553 (+1988205us) service_pool.cc:224] Handling call
0709 13:06:05.206859 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:05.210509 (+  3650us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.210682 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:04.957058 18728 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:05.198328 15857 rpcz_store.cc:276] 0709 13:06:02.694238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.694332 (+    94us) service_pool.cc:224] Handling call
0709 13:06:02.709897 (+ 15565us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:03.104595 (+394698us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:03.104628 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:03.173524 (+ 68896us) log.cc:844] Serialized 26 byte log entry
0709 13:06:04.005438 (+831914us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:04.143957 (+138519us) log.cc:844] Serialized 14 byte log entry
0709 13:06:04.175906 (+ 31949us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.968605 (+792699us) spinlock_profiling.cc:243] Waited 29.9 ms on lock 0x7b18003dfd20. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72892a6 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419
0709 13:06:04.969351 (+   746us) spinlock_profiling.cc:243] Waited 354 ms on lock 0x7b54002ebb54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:04.969397 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:04.969438 (+    41us) raft_consensus.cc:1685] finished
0709 13:06:04.969451 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:04.969582 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":126174,"prepare.queue_time_us":138314,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":46262,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":37040,"spinlock_wait_cycles":882185856,"thread_start_us":831547,"threads_started":1,"wal-append.queue_time_us":1072963}
I20260709 13:06:05.268250 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:05.205571 18926 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:05.196846 18898 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:05.101593 18861 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:05.425289 15931 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6.
W20260709 13:06:05.432165 15862 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.000s	sys 0.016s
I20260709 13:06:04.973029 18802 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:05.517015 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 551) took 1791 ms. Trace:
I20260709 13:06:05.517903 16002 rpcz_store.cc:276] 0709 13:06:03.726023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.409585 (+1683562us) service_pool.cc:224] Handling call
0709 13:06:05.516383 (+106798us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:05.516615 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:05.516642 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:05.516694 (+    52us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:05.516725 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.516791 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:05.516934 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:05.102444 18990 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:05.424506 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:04.971424 18948 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:05.428601 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 138254us
I20260709 13:06:05.526446 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:05.007030 19045 raft_consensus.cc:697] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 7 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:06:05.432953 15862 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.000s	sys 0.016s
I20260709 13:06:05.218451 18999 raft_consensus.cc:2804] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:06:05.432476 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 143706us
W20260709 13:06:05.544693 16533 leader_election.cc:336] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 2.223s (SENT)
I20260709 13:06:05.545244 16533 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:05.425676 16008 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:04.968070 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:05.268020 19022 raft_consensus.cc:2749] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:05.287500 15974 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:05.427191 19049 log.cc:927] Time spent T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.533s	user 0.001s	sys 0.000s
I20260709 13:06:05.214366 19010 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.285569 18998 raft_consensus.cc:2749] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:05.291795 18949 raft_consensus.cc:2749] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:04.966063 19016 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:05.218452 18460 raft_consensus.cc:740] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:05.266252 18966 consensus_queue.cc:1059] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.255s
I20260709 13:06:05.432770 18926 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:05.566009 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:05.675460 15857 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:05.516314 18898 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:05.575435 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 288101us
I20260709 13:06:05.677816 15974 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:05.529029 18993 consensus_meta.cc:287] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.636s	user 0.011s	sys 0.000s
I20260709 13:06:05.579749 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:05.581724 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1774) took 1777 ms. Trace:
I20260709 13:06:05.679931 16002 rpcz_store.cc:276] 0709 13:06:03.806175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.527661 (+1721486us) service_pool.cc:224] Handling call
0709 13:06:05.572488 (+ 44827us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:05.572704 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:05.572730 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:05.572778 (+    48us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:05.572818 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.572878 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:05.581667 (+  8789us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:05.601426 18782 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:05.602841 19009 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.608009 16010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41674 (request call id 1767) took 2301 ms. Trace:
I20260709 13:06:05.684110 16010 rpcz_store.cc:276] 0709 13:06:03.306162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.541812 (+235650us) service_pool.cc:224] Handling call
0709 13:06:05.607695 (+2065883us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54001dc754. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:05.607924 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2420453248}
I20260709 13:06:05.609367 18956 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:05.616516 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 71856.5us
W20260709 13:06:05.687934 16533 leader_election.cc:336] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 2.237s (SENT)
I20260709 13:06:05.617389 15854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 1876) took 2399 ms. Trace:
I20260709 13:06:05.619186 18792 leader_election.cc:290] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:05.624397 18854 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:05.696238 18854 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:05.697669 18899 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:05.624727 18948 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:05.628110 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:05.207352 18984 consensus_queue.cc:260] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:05.636569 18973 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.637316 16003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1889) took 1806 ms. Trace:
I20260709 13:06:05.644845 16004 raft_consensus.cc:1240] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 8. Current term is 9. Ops: []
W20260709 13:06:05.649950 19021 log.cc:927] Time spent T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:06:05.577423 18999 raft_consensus.cc:697] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:05.526715 18861 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:05.700824 18984 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:05.702723 18861 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:05.656302 18163 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:05.650027 15861 raft_consensus.cc:1240] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 3. Current term is 4. Ops: []
I20260709 13:06:05.704015 15861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 579) took 2483 ms. Trace:
I20260709 13:06:05.703879 18973 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:05.704432 15861 rpcz_store.cc:276] 0709 13:06:03.220328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.649476 (+2429148us) service_pool.cc:224] Handling call
0709 13:06:05.649827 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:05.703721 (+ 53894us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.703943 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:05.705523 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:05.706097 18973 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:05.658504 15860 raft_consensus.cc:1240] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:05.706940 15860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 576) took 2487 ms. Trace:
I20260709 13:06:05.707191 16004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1776) took 1846 ms. Trace:
I20260709 13:06:05.707345 15860 rpcz_store.cc:276] 0709 13:06:03.219263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.579839 (+2360576us) service_pool.cc:224] Handling call
0709 13:06:05.658330 (+ 78491us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:05.706687 (+ 48357us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.706883 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:05.707703 16004 rpcz_store.cc:276] 0709 13:06:03.860755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.644271 (+1783516us) service_pool.cc:224] Handling call
0709 13:06:05.644638 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:05.706930 (+ 62292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.707134 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:05.708814 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:06:05.710469 16533 leader_election.cc:336] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 2.541s (SENT)
I20260709 13:06:05.710952 16533 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:05.537477 18990 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:05.716840 16002 raft_consensus.cc:1240] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:05.717501 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1778) took 1789 ms. Trace:
I20260709 13:06:05.717808 16002 rpcz_store.cc:276] 0709 13:06:03.928161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.681541 (+1753380us) service_pool.cc:224] Handling call
0709 13:06:05.716660 (+ 35119us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:05.717255 (+   595us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.717437 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:05.718386 15860 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
W20260709 13:06:05.672160 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 144823us
W20260709 13:06:05.674486 18950 consensus_meta.cc:287] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.899s	user 0.010s	sys 0.000s
I20260709 13:06:05.688819 15854 rpcz_store.cc:276] 0709 13:06:03.217641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.933228 (+1715587us) service_pool.cc:224] Handling call
0709 13:06:04.935366 (+  2138us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:04.935720 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:04.935740 (+    20us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:04.936095 (+   355us) log.cc:844] Serialized 26 byte log entry
0709 13:06:05.273847 (+337752us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:05.273897 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.273978 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:05.605654 (+331676us) raft_consensus.cc:1685] finished
0709 13:06:05.605716 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:05.617303 (+ 11587us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":906,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":7469,"thread_start_us":199459,"threads_started":1,"wal-append.queue_time_us":620662}
I20260709 13:06:05.671356 19010 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:05.727885 18929 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:05.728963 19010 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:05.706256 15861 raft_consensus.cc:2393] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 4: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:05.730201 16007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1885) took 2144 ms. Trace:
I20260709 13:06:05.730340 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:05.713053 16003 rpcz_store.cc:276] 0709 13:06:03.830492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.635947 (+1805455us) service_pool.cc:224] Handling call
0709 13:06:05.636403 (+   456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:05.636843 (+   440us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:05.636869 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:05.636911 (+    42us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:05.636957 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.637035 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:05.637233 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":142,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":115}
I20260709 13:06:05.730681 16007 rpcz_store.cc:276] 0709 13:06:03.585369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.710903 (+1125534us) service_pool.cc:224] Handling call
0709 13:06:04.711323 (+   420us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:04.907304 (+195981us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:04.907348 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:04.908072 (+   724us) log.cc:844] Serialized 52 byte log entry
0709 13:06:05.273331 (+365259us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:05.273382 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.273529 (+   147us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:05.729882 (+456353us) raft_consensus.cc:1685] finished
0709 13:06:05.729907 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:05.730125 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":20807,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":5646,"thread_start_us":448396,"threads_started":2,"wal-append.queue_time_us":796557}
I20260709 13:06:05.733479 18950 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:05.734110 18999 consensus_queue.cc:237] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:05.735853 16006 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:06:05.736007 18796 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:05.737802 16006 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60156 (request call id 547) took 2012 ms (client timeout 2093 ms). Trace:
W20260709 13:06:05.738219 16006 rpcz_store.cc:269] 0709 13:06:03.724957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.552987 (+828030us) service_pool.cc:224] Handling call
0709 13:06:05.737614 (+1184627us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54003edd54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:05.737741 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2625325824}
W20260709 13:06:05.739482 16561 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45246 (request call id 721) took 2456 ms (client timeout 1513 ms). Trace:
W20260709 13:06:05.739919 16561 rpcz_store.cc:269] 0709 13:06:03.283529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.283739 (+   210us) service_pool.cc:224] Handling call
0709 13:06:05.739287 (+2455548us) spinlock_profiling.cc:243] Waited 324 ms on lock 0x7b5400695ad4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:05.739390 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":745672576}
I20260709 13:06:05.740118 18854 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:05.742796 18854 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:06:05.670507 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:05.724808 15902 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:05.744103 16535 connection.cc:256] RPC call timeout handler was delayed by -2.1150173019999556s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1647s for a response
I20260709 13:06:05.744730 18732 raft_consensus.cc:2804] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:06:05.745021 19020 log.cc:927] Time spent T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 1.434s	user 0.010s	sys 0.010s
I20260709 13:06:05.745419 15861 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:06:05.746387 15935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 613) took 2877 ms. Trace:
I20260709 13:06:05.747733 18950 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:05.746873 15935 rpcz_store.cc:276] 0709 13:06:02.869088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.869212 (+   124us) service_pool.cc:224] Handling call
0709 13:06:02.879184 (+  9972us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:03.354946 (+475762us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:03.354991 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:03.518089 (+163098us) log.cc:844] Serialized 26 byte log entry
0709 13:06:03.936294 (+418205us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:03.936328 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.936392 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:05.744399 (+1808007us) spinlock_profiling.cc:243] Waited 4.54 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:06:05.746085 (+  1686us) raft_consensus.cc:1685] finished
0709 13:06:05.746107 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:05.746323 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":112282,"raft.queue_time_us":189119,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":21222,"spinlock_wait_cycles":10431232,"thread_start_us":717911,"threads_started":2,"wal-append.queue_time_us":630157}
I20260709 13:06:05.745414 18732 raft_consensus.cc:697] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:05.750286 19016 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:05.751017 18732 consensus_queue.cc:237] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:05.711849 18792 raft_consensus.cc:2749] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:05.749379 18737 raft_consensus.cc:3055] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:05.726215 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:05.754463 15854 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:05.653225 19045 consensus_queue.cc:237] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:05.746295 15902 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:06:05.722293 19007 consensus_meta.cc:287] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.871s	user 0.011s	sys 0.000s
I20260709 13:06:05.522089 19003 consensus_queue.cc:260] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:05.752808 16004 raft_consensus.cc:1240] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 5. Current term is 6. Ops: []
W20260709 13:06:05.744796 16535 connection.cc:256] RPC call timeout handler was delayed by -1.4934319550000055s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2093s for a response
I20260709 13:06:05.242607 18970 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.712527 16533 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:05.769512 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 57096.3us
W20260709 13:06:05.751513 16011 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41662 (request call id 1883) took 2542 ms (client timeout 1530 ms). Trace:
I20260709 13:06:05.756490 15902 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:05.771988 15902 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:05.749362 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:05.754099 18737 raft_consensus.cc:740] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:05.762269 18932 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:05.776937 16535 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:05.712620 18949 raft_consensus.cc:2749] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:06:05.770355 16011 rpcz_store.cc:269] 0709 13:06:03.208877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.234683 (+ 25806us) service_pool.cc:224] Handling call
0709 13:06:05.751296 (+2516613us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b54003ee254. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:05.751434 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2928876416}
I20260709 13:06:05.771390 16533 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:05.780614 18792 raft_consensus.cc:2804] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:05.687650 18993 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:05.794332 18792 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:05.811265 19022 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:05.811547 18792 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:05.812150 19022 consensus_queue.cc:1048] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:05.816095 18728 consensus_queue.cc:1059] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:05.816305 18737 consensus_queue.cc:260] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:05.817298 18616 consensus_meta.cc:287] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.876s	user 0.013s	sys 0.000s
I20260709 13:06:05.667357 16008 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5.
I20260709 13:06:05.794648 15854 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:06:05.712066 19009 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:05.785322 19003 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:05.781706 19035 raft_consensus.cc:2749] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:05.781520 18932 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:05.855370 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:05.856180 16565 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
W20260709 13:06:05.858103 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:05.825848 18616 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:05.840768 18796 raft_consensus.cc:3055] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:05.844106 18993 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:05.846868 15862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 1875) took 2629 ms. Trace:
I20260709 13:06:05.847658 18949 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:06:05.848345 15853 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 1879) took 2542 ms (client timeout 3080 ms). Trace:
W20260709 13:06:05.874145 15853 rpcz_store.cc:269] 0709 13:06:03.305901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.240757 (+934856us) service_pool.cc:224] Handling call
0709 13:06:05.848149 (+1607392us) spinlock_profiling.cc:243] Waited 701 ms on lock 0x7b54004272d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:05.848287 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1611843328}
I20260709 13:06:05.828006 15855 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 7.
I20260709 13:06:05.784775 15859 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:06:05.854339 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:05.870618 15862 rpcz_store.cc:276] 0709 13:06:03.217257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.913311 (+1696054us) service_pool.cc:224] Handling call
0709 13:06:04.919943 (+  6632us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:04.920356 (+   413us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:06:04.920387 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:04.920907 (+   520us) log.cc:844] Serialized 26 byte log entry
0709 13:06:05.536010 (+615103us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:05.536068 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.536149 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:05.845065 (+308916us) spinlock_profiling.cc:243] Waited 32.6 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:06:05.846583 (+  1518us) raft_consensus.cc:1685] finished
0709 13:06:05.846604 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:05.846807 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3664,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":2383,"spinlock_wait_cycles":74891648,"thread_start_us":614684,"threads_started":1,"wal-append.queue_time_us":847717}
I20260709 13:06:05.860240 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:05.861955 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:05.876036 16567 raft_consensus.cc:2393] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 6: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:05.848763 18792 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:05.807976 18971 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:05.799168 18899 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:05.877846 16008 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 4.
I20260709 13:06:05.878414 15859 raft_consensus.cc:1275] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:06:05.878829 15859 pending_rounds.cc:85] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8: Aborting all ops after (but not including) 2
I20260709 13:06:05.864310 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:05.879153 15859 pending_rounds.cc:107] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:06:05.867610 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:05.869788 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:05.865739 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:05.874058 15976 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:05.871101 15860 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:06:05.858706 18932 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:05.875201 15825 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.025s	sys 0.035s
I20260709 13:06:05.784235 18966 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:05.862490 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:05.859095 18854 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.867491 18796 raft_consensus.cc:740] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:05.876757 18840 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.877365 15853 raft_consensus.cc:1240] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:06:05.890373 15853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1779) took 2243 ms. Trace:
I20260709 13:06:05.781178 18970 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:05.890842 15853 rpcz_store.cc:276] 0709 13:06:03.647075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.876777 (+2229702us) service_pool.cc:224] Handling call
0709 13:06:05.877150 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:05.889918 (+ 12768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.890291 (+   373us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:05.892684 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:05.893014 18970 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:05.880393 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:05.893874 16007 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:06:05.881165 15859 raft_consensus.cc:2889] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:05.882941 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:05.894830 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:05.883260 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:05.895593 15859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1781) took 1421 ms. Trace:
I20260709 13:06:05.891319 18792 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:05.894830 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:05.896533 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:05.897702 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:05.898062 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:05.899348 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:05.895620 16006 raft_consensus.cc:2393] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 6: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:05.883396 16565 raft_consensus.cc:2368] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 5. Current term is 6.
W20260709 13:06:05.884742 15825 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.025s	sys 0.035s
I20260709 13:06:05.885437 16570 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:06:05.896075 15859 rpcz_store.cc:276] 0709 13:06:04.474350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.877867 (+1403517us) service_pool.cc:224] Handling call
0709 13:06:05.878130 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:05.895349 (+ 17219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.895520 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:05.906044 15976 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:05.907531 18948 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:05.907784 18840 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:05.908222 18948 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:05.908819 18948 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:05.909269 18796 consensus_queue.cc:260] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:05.808739 18460 consensus_queue.cc:260] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:05.921417 15827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:05.929530 16008 raft_consensus.cc:1240] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 6. Current term is 7. Ops: []
W20260709 13:06:05.934545 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:05.888319 18971 raft_consensus.cc:697] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:05.903842 18950 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:05.928448 18163 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.884522 18949 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:05.951742 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:05.929093 16563 raft_consensus.cc:1240] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:05.956110 15934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1858) took 1313 ms. Trace:
I20260709 13:06:05.946611 18984 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.903126 18854 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:06.023514 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 310116us
I20260709 13:06:05.953665 19016 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.013707 15934 rpcz_store.cc:276] 0709 13:06:04.642934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.729867 (+ 86933us) service_pool.cc:224] Handling call
0709 13:06:04.767041 (+ 37174us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:04.767289 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:06:04.796168 (+ 28879us) log.cc:844] Serialized 14 byte log entry
0709 13:06:05.639095 (+842927us) spinlock_profiling.cc:243] Waited 5.42 ms on lock 0x7b18003afd80. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f4823a 00007f73ea3b2a2a
0709 13:06:05.955725 (+316630us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:05.955783 (+    58us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:05.955835 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.955902 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:05.956044 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":28582,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":2189,"raft.queue_time_us":1792,"spinlock_wait_cycles":12465408,"thread_start_us":315251,"threads_started":1}
I20260709 13:06:06.026867 15974 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:06.028426 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:06.031425 16570 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:06:06.012399 18460 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:06.035552 15827 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:06.038543 16011 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6.
W20260709 13:06:06.041278 16568 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45246 (request call id 723) took 2752 ms (client timeout 3296 ms). Trace:
W20260709 13:06:06.043489 16568 rpcz_store.cc:269] 0709 13:06:03.288475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.288650 (+   175us) service_pool.cc:224] Handling call
0709 13:06:06.041083 (+2752433us) spinlock_profiling.cc:243] Waited 355 ms on lock 0x7b54007017d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:06.041203 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":817438592}
I20260709 13:06:06.046106 15861 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "8430db80a39d4868bcf7641c16de35ea"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:06:06.046715 15861 raft_consensus.cc:606] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Received request to transfer leadership to TS 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:06.047055 15861 raft_consensus.cc:614] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:06:06.044585 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 251270us
I20260709 13:06:05.958657 18163 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:06.058281 15861 raft_consensus.cc:1240] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:06.044986 15974 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:06.058970 15861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 1886) took 1367 ms. Trace:
I20260709 13:06:06.059329 15861 rpcz_store.cc:276] 0709 13:06:04.691438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.057848 (+1366410us) service_pool.cc:224] Handling call
0709 13:06:06.058124 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:06.058737 (+   613us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:06.058918 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:06.034674 19000 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:06.062296 15827 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:06.060551 15861 raft_consensus.cc:1240] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 13:06:06.068727 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 274556us
I20260709 13:06:06.072463 15861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 596) took 1380 ms. Trace:
I20260709 13:06:06.073284 15861 rpcz_store.cc:276] 0709 13:06:04.692302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.060058 (+1367756us) service_pool.cc:224] Handling call
0709 13:06:06.060327 (+   269us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:06.071849 (+ 11522us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:06.072369 (+   520us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:06.070892 15974 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:06.075557 15827 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:06.075222 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 271431us
I20260709 13:06:06.029639 18984 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:06.048676 19024 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.079s	user 0.011s	sys 0.015s
I20260709 13:06:06.085547 15827 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:06.086722 15827 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:06.081130 18616 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:06.057694 18970 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:06.082127 16010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1779) took 2122 ms. Trace:
I20260709 13:06:06.083688 16005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 545) took 2359 ms. Trace:
I20260709 13:06:06.078131 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:06.081741 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:06.034986 19016 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:06.089406 15827 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:06.092978 16005 rpcz_store.cc:276] 0709 13:06:03.724324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.843488 (+1119164us) service_pool.cc:224] Handling call
0709 13:06:04.896091 (+ 52603us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:04.974117 (+ 78026us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:06.083432 (+1109315us) spinlock_profiling.cc:243] Waited 77.3 ms on lock 0x7b540037b9d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:06.083615 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":177884032}
I20260709 13:06:06.092917 16010 rpcz_store.cc:276] 0709 13:06:03.959802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.685306 (+1725504us) service_pool.cc:224] Handling call
0709 13:06:05.693316 (+  8010us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:05.693762 (+   446us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:06:05.693796 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:05.694312 (+   516us) log.cc:844] Serialized 26 byte log entry
0709 13:06:06.017941 (+323629us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:06.018013 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:06.018106 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:06.081910 (+ 63804us) raft_consensus.cc:1685] finished
0709 13:06:06.081934 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:06.082067 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1061,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":137,"thread_start_us":323222,"threads_started":1,"wal-append.queue_time_us":366644}
I20260709 13:06:06.068173 18950 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:06.098795 16535 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:06.097218 16008 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:06:06.061470 19064 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:06.100903 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:06.085142 19024 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.115s	user 0.011s	sys 0.015s
I20260709 13:06:06.103101 18962 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:06.103806 18792 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:06.104374 19064 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:06.095338 18973 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:06.082098 19009 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:06.105500 18586 consensus_queue.cc:1059] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:06.136601 18586 raft_consensus.cc:3055] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:06.136968 18586 raft_consensus.cc:740] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:06.137859 18586 consensus_queue.cc:260] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:06.138378 15857 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:06.139142 15854 raft_consensus.cc:1240] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:06:06.141556 18948 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:06.106703 18929 consensus_queue.cc:1059] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:06.143718 18948 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:06.144462 18929 raft_consensus.cc:3055] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:06.145078 18929 raft_consensus.cc:740] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:06.145991 18929 consensus_queue.cc:260] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:06.103050 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:06.091555 19010 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:06.092834 18970 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:06.149348 18970 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:06.082497 19065 raft_consensus.cc:2764] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:06:06.070022 18966 consensus_queue.cc:1048] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:06:06.094933 18964 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:06.118729 15827 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:06.122551 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:06.153343 18964 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:06.153585 16567 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
I20260709 13:06:06.153772 18964 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:06.118800 15859 raft_consensus.cc:1240] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:06.107626 19016 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:06.114504 18962 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:06.155226 18962 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:06.155807 19016 raft_consensus.cc:2804] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:06.156286 19016 raft_consensus.cc:697] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:06.133026 18163 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:06.133080 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:06.157091 19016 consensus_queue.cc:237] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:06.135210 19064 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:06.135788 18973 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:06.158984 18973 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:06.147359 18929 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:06.154559 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:06.068308 18971 consensus_queue.cc:237] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:06.144420 18948 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:06.169240 18948 raft_consensus.cc:697] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:06.170471 18948 consensus_queue.cc:237] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:06.161044 15827 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:06.179164 15827 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:06.173197 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 241019us
I20260709 13:06:06.139142 18586 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:06.165987 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:06.212975 19064 raft_consensus.cc:2749] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:06.210975 19024 raft_consensus.cc:2749] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:06:06.241041 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 145633us
W20260709 13:06:06.248724 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 177866us
W20260709 13:06:06.230599 18782 consensus_meta.cc:287] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.542s	user 0.009s	sys 0.001s
I20260709 13:06:06.258055 15854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 592) took 3033 ms. Trace:
I20260709 13:06:06.260533 15854 rpcz_store.cc:276] 0709 13:06:03.224421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.818792 (+2594371us) service_pool.cc:224] Handling call
0709 13:06:06.138918 (+320126us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:06.139506 (+   588us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:06.257875 (+118369us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b5400427cd0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:06.258004 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":735462272}
I20260709 13:06:06.261194 15861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 1888) took 1502 ms. Trace:
I20260709 13:06:06.261556 15861 rpcz_store.cc:276] 0709 13:06:04.758745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.076510 (+1317765us) service_pool.cc:224] Handling call
0709 13:06:06.076894 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:06.077160 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:06:06.093830 (+ 16670us) log.cc:844] Serialized 14 byte log entry
0709 13:06:06.260889 (+167059us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:06.260938 (+    49us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:06.260981 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:06.261045 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:06.261144 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":18896,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"thread_start_us":166708,"threads_started":1}
W20260709 13:06:06.258018 15974 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 1.784s (SENT) [suppressed 41 similar messages]
W20260709 13:06:06.273861 15855 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58214 (request call id 586) took 3037 ms (client timeout 2237 ms). Trace:
W20260709 13:06:06.274456 15855 rpcz_store.cc:269] 0709 13:06:03.222629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.937051 (+714422us) service_pool.cc:224] Handling call
0709 13:06:06.259783 (+2322732us) spinlock_profiling.cc:243] Waited 1.87 s on lock 0x7b54003b6854. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:06.259888 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4302611840}
I20260709 13:06:06.279007 15858 raft_consensus.cc:1275] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:06:06.229015 19076 raft_consensus.cc:2804] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:06.284322 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:06.285161 16566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45246 (request call id 739) took 1080 ms. Trace:
I20260709 13:06:06.285470 16566 rpcz_store.cc:276] 0709 13:06:05.204769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.205112 (+   343us) service_pool.cc:224] Handling call
0709 13:06:06.284257 (+1079145us) spinlock_profiling.cc:243] Waited 6.73 ms on lock 0x7b5c003a028c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73edd12161 00007f73edeb3a79 00007f73ef36d9b6 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:06.285097 (+   840us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15475072}
I20260709 13:06:06.177836 18926 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:06.288873 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:06.292343 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:06:06.296145 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 114753us
I20260709 13:06:06.297415 15937 raft_consensus.cc:2393] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 6: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:06.297055 15976 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:06.298100 15855 raft_consensus.cc:1275] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 13:06:06.298527 15855 pending_rounds.cc:85] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8: Aborting all ops after (but not including) 2
I20260709 13:06:06.298758 15855 pending_rounds.cc:107] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:06:06.298996 15855 raft_consensus.cc:2889] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:06.300769 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:06.260066 18966 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.302829 15930 raft_consensus.cc:2393] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 7: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:06.275796 18782 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:06.306283 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:06.307103 15976 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:06.307549 15937 raft_consensus.cc:2393] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 7: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:06.260437 18950 raft_consensus.cc:697] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:06.308521 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:06.309094 15936 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:06.280277 18899 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.312804 15902 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:06.310631 15976 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:06.308993 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:06.305967 15974 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:06.276226 18992 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:06.313181 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:06.312165 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:06.315096 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:06.314026 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:06.321100 15934 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:06.292296 18854 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:06.288653 18949 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:06.376925 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:06.319130 15855 raft_consensus.cc:1240] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:06.377889 15937 raft_consensus.cc:2393] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 5: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:06.317332 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:06.318069 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:06.378593 15933 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:06.378741 15931 raft_consensus.cc:2393] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 5: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
W20260709 13:06:06.350678 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:06.304219 19057 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:06.368583 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 62701.3us
I20260709 13:06:06.302430 19076 raft_consensus.cc:697] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:06:06.369449 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 51501us
I20260709 13:06:06.320703 15935 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:06.417232 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 318462us
I20260709 13:06:06.471101 15853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1783) took 1996 ms. Trace:
I20260709 13:06:06.471760 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:06.441205 15902 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:06.473764 15931 raft_consensus.cc:1240] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:06.475749 15931 raft_consensus.cc:1240] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:06.314030 19000 raft_consensus.cc:697] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:06.359020 18966 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:06.480767 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1781) took 2015 ms. Trace:
I20260709 13:06:06.322218 18899 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:06.452299 19077 log.cc:927] Time spent T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 13:06:06.452857 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:06.453758 19080 log.cc:927] Time spent T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.194s	user 0.000s	sys 0.000s
W20260709 13:06:06.454488 19081 log.cc:927] Time spent T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
I20260709 13:06:06.440934 19016 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.445094 18968 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.416069 19035 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:06.478659 19083 raft_consensus.cc:2749] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:06.463516 18949 raft_consensus.cc:2804] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:06.302110 18932 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:06.465006 18926 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:06.469594 18950 consensus_queue.cc:237] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:06.470147 15859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 1885) took 1814 ms. Trace:
I20260709 13:06:06.470557 15825 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:06.552064 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 81591us
I20260709 13:06:06.553071 18932 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:06.553726 18460 raft_consensus.cc:2749] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:06.445297 15974 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:06.551534 15859 rpcz_store.cc:276] 0709 13:06:04.656018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.902121 (+1246103us) service_pool.cc:224] Handling call
0709 13:06:05.902415 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:06.155024 (+252609us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:06.469930 (+314906us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b540006ed54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:06.470091 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":497203584}
I20260709 13:06:06.511443 18899 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:06.483261 18802 consensus_queue.cc:1059] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
W20260709 13:06:06.473989 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 53650.9us
W20260709 13:06:06.556641 19003 consensus_meta.cc:287] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.712s	user 0.008s	sys 0.002s
I20260709 13:06:06.477221 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:06.557132 15902 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:06.483793 19033 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:06:06.448021 19079 log.cc:927] Time spent T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 13:06:06.481624 16002 rpcz_store.cc:276] 0709 13:06:04.465315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.718589 (+1253274us) service_pool.cc:224] Handling call
0709 13:06:05.770099 (+ 51510us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:05.770306 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:06.036991 (+266685us) log.cc:844] Serialized 14 byte log entry
0709 13:06:06.480250 (+443259us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:06.480359 (+   109us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:06.480434 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:06.480578 (+   144us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:06.480708 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8982,"prepare.queue_time_us":135483,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":77175,"raft.run_cpu_time_us":289,"raft.run_wall_time_us":7059,"thread_start_us":685202,"threads_started":3}
I20260709 13:06:06.479703 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:06.316665 18993 consensus_queue.cc:1048] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.129s
I20260709 13:06:06.471611 15853 rpcz_store.cc:276] 0709 13:06:04.474956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.892392 (+1417436us) service_pool.cc:224] Handling call
0709 13:06:05.892742 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:05.892998 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:05.893025 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:05.893686 (+   661us) log.cc:844] Serialized 26 byte log entry
0709 13:06:06.118227 (+224541us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:06.118283 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:06.119145 (+   862us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:06.466518 (+347373us) spinlock_profiling.cc:243] Waited 7.39 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:06:06.470864 (+  4346us) raft_consensus.cc:1685] finished
0709 13:06:06.470882 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:06.471052 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16990080,"thread_start_us":224032,"threads_started":1,"wal-append.queue_time_us":415612}
I20260709 13:06:06.417770 18984 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:06.491999 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:06.588263 15929 raft_consensus.cc:2393] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 6: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:06.485971 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:06.441188 18861 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:06.569726 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:06.589761 18861 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:06.590479 15853 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:06:06.455435 19076 consensus_queue.cc:237] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:06.591439 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:06.485816 19057 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:06.500838 18898 consensus_meta.cc:287] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.788s	user 0.009s	sys 0.002s
I20260709 13:06:06.485735 18854 raft_consensus.cc:2804] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:06.510123 15856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 591) took 3286 ms. Trace:
I20260709 13:06:06.359987 18992 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:06.473232 19085 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:06.515581 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:06.512022 15932 raft_consensus.cc:1275] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
W20260709 13:06:06.535066 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:06.609542 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:06.458201 18964 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:06.537197 15934 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:06.539225 15936 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:06.614326 15929 raft_consensus.cc:1240] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:06.539322 15900 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:06.546430 18949 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:06.555397 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:06.556463 18957 raft_consensus.cc:3055] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:06:06.482519 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 401660us
I20260709 13:06:06.564507 16564 raft_consensus.cc:1275] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:06:06.567432 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:06.567021 15931 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:06.570570 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:06.618742 15930 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
W20260709 13:06:06.569808 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 124603us
I20260709 13:06:06.515873 19016 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:06.576622 19009 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:06.581831 15933 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:06:06.581415 18968 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:06.621963 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:06.567018 19033 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:06.583480 19003 raft_consensus.cc:2804] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:06.593688 15853 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:06.601313 19057 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:06.602334 15856 rpcz_store.cc:276] 0709 13:06:03.224150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.679730 (+2455580us) service_pool.cc:224] Handling call
0709 13:06:05.721657 (+ 41927us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:05.722093 (+   436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:05.722136 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:05.726780 (+  4644us) log.cc:844] Serialized 26 byte log entry
0709 13:06:06.114574 (+387794us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:06.114616 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:06.114684 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:06.509911 (+395227us) raft_consensus.cc:1685] finished
0709 13:06:06.509932 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:06.510066 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":113,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":7405,"thread_start_us":330444,"threads_started":1,"wal-append.queue_time_us":499660}
I20260709 13:06:06.636019 19057 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:06:06.636540 19057 raft_consensus.cc:697] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:06.636796 15856 raft_consensus.cc:1275] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:06:06.637365 19057 consensus_queue.cc:237] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:06.639750 18728 consensus_queue.cc:1059] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:06.640667 18728 raft_consensus.cc:3055] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:06.641021 18728 raft_consensus.cc:740] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:06.642014 18992 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:06.641834 18728 consensus_queue.cc:260] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:06.631227 19009 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:06.645203 19009 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:06.646411 18966 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:06.648262 18964 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:06.602182 18854 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:06.649068 18854 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:06.650696 15931 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:06:06.604450 18898 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:06.607874 18861 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:06.657418 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:06.643433 18728 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:06.607153 15935 raft_consensus.cc:2410] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:06:06.569188 18899 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:06:06.621333 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:06.511583 19000 consensus_queue.cc:237] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:06.601815 19096 log.cc:927] Time spent T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.223s	user 0.000s	sys 0.001s
I20260709 13:06:06.628250 15974 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:06.624539 16533 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:06.625478 19003 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:06.629113 18952 consensus_queue.cc:1059] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:06:06.629739 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 90495.3us
W20260709 13:06:06.679037 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 50852.6us
I20260709 13:06:06.625753 19033 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:06.695232 15974 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:06.633098 18968 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:06.661867 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:06.603824 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 293295us
W20260709 13:06:06.688632 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:06.622529 18957 raft_consensus.cc:740] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:06.622792 18949 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:06.630867 19016 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:06.677865 15853 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:06:06.704380 16570 raft_consensus.cc:2410] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:06:06.677512 18962 consensus_meta.cc:287] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.522s	user 0.009s	sys 0.000s
I20260709 13:06:06.692512 18998 raft_consensus.cc:2749] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:06.639561 18970 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:06.711791 15976 leader_election.cc:400] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 election: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 5. Current term is 6.
I20260709 13:06:06.719509 15976 leader_election.cc:403] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:06:06.695834 19003 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:06.715886 15974 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:06.716791 15854 raft_consensus.cc:1240] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:06.717358 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:06.705102 19035 consensus_queue.cc:1059] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 13:06:06.674057 18732 consensus_queue.cc:1048] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:06:06.727888 15853 raft_consensus.cc:1275] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:06:06.729141 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:06.749393 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:06.750540 15974 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:06.762405 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:06.780869 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:06.721494 19103 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:06.782110 15853 raft_consensus.cc:2393] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 8: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:06.784996 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:06.723688 19016 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:06.792179 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:06.813792 16009 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:06.720760 18962 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:06.793526 16003 raft_consensus.cc:1240] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:06:06.814334 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:06.794729 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 44295.7us
I20260709 13:06:06.815407 15974 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:06.720199 19102 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:06.796121 18898 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:06.707182 18899 consensus_queue.cc:1048] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 13:06:06.719863 19092 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:06.752449 19098 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:06.813213 16006 raft_consensus.cc:2393] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 9: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:06.816649 15974 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:06.795380 19094 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:06.847851 15974 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:06.820634 16564 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:06:06.828997 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:06.849860 16007 raft_consensus.cc:2393] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 6: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:06.787401 18854 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:06.835398 18966 consensus_queue.cc:1059] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:06.781332 18998 raft_consensus.cc:2749] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:06.869488 18998 raft_consensus.cc:2749] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:06.870604 18998 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:06.871093 18998 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:06.837296 19102 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:06.873039 18998 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:06.873785 19102 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:06.853163 19092 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:06.875077 18966 raft_consensus.cc:3055] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:06.875588 18966 raft_consensus.cc:740] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:06.876474 19092 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:06.876613 18966 consensus_queue.cc:260] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:06.878103 18966 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:06.811942 15860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 1895) took 1117 ms. Trace:
I20260709 13:06:06.879420 15860 rpcz_store.cc:276] 0709 13:06:05.694238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.749083 (+ 54845us) service_pool.cc:224] Handling call
0709 13:06:06.754025 (+1004942us) spinlock_profiling.cc:243] Waited 97.9 ms on lock 0x7b54004268d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:06.811864 (+ 57839us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":225254656}
I20260709 13:06:06.842927 15861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1789) took 1566 ms. Trace:
I20260709 13:06:06.881397 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:06.844719 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:06.882359 16010 raft_consensus.cc:2376] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4. Re-sending same reply.
I20260709 13:06:06.800388 18932 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.795442 19085 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:06:06.845947 15859 raft_consensus.cc:1240] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:06.852239 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:06.834903 15976 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:06.812897 19035 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:06.864497 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:06.889185 16006 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
I20260709 13:06:06.851305 18957 consensus_queue.cc:260] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:06.866401 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:06.866999 18952 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:06.856102 19094 raft_consensus.cc:697] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:06.899255 15934 raft_consensus.cc:1240] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:06.900354 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:06.900648 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 276026us
I20260709 13:06:06.844498 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:06.883533 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:06.903119 16010 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:06.886101 16569 raft_consensus.cc:1275] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:06:06.887080 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:06.902127 16533 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:06.847788 19076 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:06.865939 18962 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:06.891301 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 56557us
I20260709 13:06:06.891474 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:06.892094 15974 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:06.823395 19016 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:06.898070 16570 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
I20260709 13:06:06.898819 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:06.867017 18973 consensus_meta.cc:287] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.708s	user 0.010s	sys 0.000s
I20260709 13:06:06.908780 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:06.857600 18948 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:06:06.915639 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:06.918283 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:06.926982 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:06.927989 15937 raft_consensus.cc:2393] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 5: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:06.880982 15861 rpcz_store.cc:276] 0709 13:06:05.276196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.264662 (+988466us) service_pool.cc:224] Handling call
0709 13:06:06.291552 (+ 26890us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:06.291767 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:06:06.292233 (+   466us) log.cc:844] Serialized 14 byte log entry
0709 13:06:06.525544 (+233311us) spinlock_profiling.cc:243] Waited 5.47 ms on lock 0x7b1800428aa0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f4823a 00007f73ea3b2a2a
0709 13:06:06.826054 (+300510us) log.cc:844] Serialized 14 byte log entry
0709 13:06:06.841638 (+ 15584us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:06.841766 (+   128us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:06.841853 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:06.842010 (+   157us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:06.842487 (+   477us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4391,"prepare.queue_time_us":353,"prepare.run_cpu_time_us":121,"prepare.run_wall_time_us":8346,"raft.queue_time_us":20325,"raft.run_wall_time_us":11440,"spinlock_wait_cycles":76825600,"thread_start_us":299896,"threads_started":1}
I20260709 13:06:06.946547 16535 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:06.902029 18998 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:06.974670 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:06.975840 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:06.976871 15860 raft_consensus.cc:2393] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 6: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:06.873044 19003 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:06:06.913251 19102 raft_consensus.cc:2804] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:06.978273 15937 raft_consensus.cc:1240] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:06.912619 18952 raft_consensus.cc:697] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:06.977240 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:06.956542 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:06.890509 18932 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:06.948849 18957 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:06.986763 15930 raft_consensus.cc:1240] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:07.021459 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:07.027563 15861 raft_consensus.cc:2410] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:06:07.029563 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:06.980698 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:06:06.967773 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 46917.7us
I20260709 13:06:06.946892 18958 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:06:07.044454 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:07.052314 15976 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:07.053763 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:07.053987 15976 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:07.054594 15861 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:06:06.981184 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:06.966133 16010 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:07.015458 19099 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:07.058068 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:06.947921 19085 raft_consensus.cc:2749] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:07.033914 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:07.057039 15860 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:06:07.060256 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:06.906695 19035 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:07.058666 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:06.951907 19076 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:06.948364 18973 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:07.163156 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:07.127349 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:07.164680 16003 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:07.065493 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:07.170744 15930 raft_consensus.cc:1240] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:07.172287 15930 raft_consensus.cc:1240] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:07.154827 15936 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:07.117909 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 171469us
I20260709 13:06:07.074064 15928 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
W20260709 13:06:07.099098 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:07.156625 16010 raft_consensus.cc:2393] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 6: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
W20260709 13:06:07.114801 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 81189.6us
W20260709 13:06:07.157393 19111 log.cc:927] Time spent T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 13:06:07.026271 18898 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:07.270721 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:07.060644 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:07.058980 19110 log.cc:927] Time spent T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:06:07.015861 19097 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.975400 18948 raft_consensus.cc:697] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:07.199867 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:07.279448 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:06:07.112571 19081 log.cc:927] Time spent T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:06:07.112792 19094 consensus_queue.cc:237] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:07.144618 19092 raft_consensus.cc:2804] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:07.210167 16009 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:07.212966 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:07.135541 19053 raft_consensus.cc:2804] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:07.215377 15857 raft_consensus.cc:1240] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:06:07.036037 19102 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:07.222851 18728 consensus_meta.cc:287] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.547s	user 0.009s	sys 0.000s
I20260709 13:06:07.223997 19098 raft_consensus.cc:2749] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 5. Current term is 6.
I20260709 13:06:07.188333 19090 consensus_queue.cc:1048] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:06:07.193850 19033 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:07.028599 18998 consensus_queue.cc:1059] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.176s
I20260709 13:06:07.188310 19101 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:07.157821 18970 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:07.228385 19112 log.cc:927] Time spent T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.170s	user 0.000s	sys 0.000s
W20260709 13:06:07.229784 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:07.232442 16004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1780) took 3204 ms. Trace:
I20260709 13:06:07.238905 16003 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
W20260709 13:06:07.242987 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 189040us
W20260709 13:06:07.253852 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:07.278905 15861 raft_consensus.cc:2393] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 7: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:07.279950 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:07.239748 19003 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:07.239265 18952 consensus_queue.cc:237] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:07.073827 19099 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:07.279790 16006 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:06:07.300913 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:07.537777 16568 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
W20260709 13:06:07.537921 16533 connection.cc:256] RPC call timeout handler was delayed by -2.4787302350000573s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30950000000000005s for a response
W20260709 13:06:07.353040 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 426781us
I20260709 13:06:07.302124 15936 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6.
I20260709 13:06:07.540565 16010 raft_consensus.cc:2393] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 7: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:07.537948 16009 raft_consensus.cc:1240] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 13:06:07.542004 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:07.544747 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:06:07.549001 16533 connection.cc:256] RPC call timeout handler was delayed by -1.5036335180000151s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1607s for a response
I20260709 13:06:07.543516 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:07.543362 16004 rpcz_store.cc:276] 0709 13:06:04.028571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.708961 (+1680390us) service_pool.cc:224] Handling call
0709 13:06:05.752563 (+ 43602us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:05.767006 (+ 14443us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:07.232200 (+1465194us) spinlock_profiling.cc:243] Waited 43.2 ms on lock 0x7b54003ee250. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:07.232370 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":99427200}
W20260709 13:06:07.553545 16533 connection.cc:256] RPC call timeout handler was delayed by -1.4766335179999714s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1637s for a response
W20260709 13:06:07.554082 16533 connection.cc:256] RPC call timeout handler was delayed by -1.0833335180000176s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2074s for a response
I20260709 13:06:07.550853 15860 raft_consensus.cc:2376] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7. Re-sending same reply.
I20260709 13:06:07.549269 15902 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:07.268716 19045 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:07.519968 19114 log.cc:927] Time spent T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.392s	user 0.000s	sys 0.001s
W20260709 13:06:07.518752 19125 log.cc:927] Time spent T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.300s	user 0.000s	sys 0.000s
I20260709 13:06:07.545090 19135 consensus_queue.cc:1059] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:06:07.578404 19135 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:07.579022 19135 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:07.519459 19122 log.cc:927] Time spent T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.314s	user 0.001s	sys 0.000s
I20260709 13:06:07.537077 18796 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:07.362906 19104 log.cc:927] Time spent T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.416s	user 0.000s	sys 0.001s
I20260709 13:06:07.570969 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:07.557049 18728 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:07.596077 15900 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:07.599334 15933 raft_consensus.cc:2368] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 5. Current term is 6.
I20260709 13:06:07.549798 19102 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:07.605178 18932 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:07.606276 18966 consensus_meta.cc:287] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.719s	user 0.010s	sys 0.000s
I20260709 13:06:07.607577 15929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1868) took 1122 ms. Trace:
I20260709 13:06:07.607990 15929 rpcz_store.cc:276] 0709 13:06:06.485241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.588939 (+103698us) service_pool.cc:224] Handling call
0709 13:06:06.614085 (+ 25146us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:06.615846 (+  1761us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:07.607375 (+991529us) spinlock_profiling.cc:243] Waited 24.8 ms on lock 0x7b54003c50d0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:07.607509 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":56953600}
I20260709 13:06:07.572212 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:07.585975 19045 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:07.609553 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:07.610139 16008 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:06:07.610757 19045 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:07.553892 19016 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:07.293830 19097 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:07.571940 19136 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:07.612429 19136 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:07.577499 18840 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:07.581507 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:07.552347 18854 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:07.584532 19103 consensus_meta.cc:287] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.747s	user 0.003s	sys 0.004s
I20260709 13:06:07.558333 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:07.631625 15860 raft_consensus.cc:2410] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:06:07.502206 18968 raft_consensus.cc:3055] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:07.558046 16004 raft_consensus.cc:1240] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 6. Current term is 7. Ops: [4.6-6.8]
I20260709 13:06:07.564633 15902 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:07.582477 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:07.510524 18990 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:07.349242 19035 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:07.545156 18737 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:07.543998 19092 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:07.659165 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:07.605229 18728 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:07.581794 18949 consensus_meta.cc:287] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.865s	user 0.005s	sys 0.005s
I20260709 13:06:07.608384 19135 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:07.557251 19075 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:07.616103 15900 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:07.616495 16007 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6.
W20260709 13:06:07.675144 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 59148.1us
I20260709 13:06:07.676394 15900 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:07.277791 18898 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:07.677563 19076 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:07.665076 19035 raft_consensus.cc:2749] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:07.678736 19035 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:07.679075 18898 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:07.679198 19035 raft_consensus.cc:697] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:07.619572 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
W20260709 13:06:07.619810 16011 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60156 (request call id 553) took 1760 ms (client timeout 1547 ms). Trace:
I20260709 13:06:07.639853 18968 raft_consensus.cc:740] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:07.662308 19092 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:07.680557 16561 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 6. (term mismatch)
I20260709 13:06:07.697542 16561 pending_rounds.cc:85] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting all ops after (but not including) 5
I20260709 13:06:07.697441 15900 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:07.680668 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:06:07.681555 16011 rpcz_store.cc:269] 0709 13:06:05.859646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.865405 (+  5759us) service_pool.cc:224] Handling call
0709 13:06:07.619497 (+1754092us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b54000aa5d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:07.619651 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":362750720}
I20260709 13:06:07.682876 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:07.702414 16568 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5.
I20260709 13:06:07.620308 19097 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:07.685120 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:07.709636 16005 raft_consensus.cc:2376] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7. Re-sending same reply.
I20260709 13:06:07.612838 19136 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:07.625525 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:07.713153 16563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 837) took 1379 ms. Trace:
I20260709 13:06:07.713369 15861 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:06:07.714603 15861 raft_consensus.cc:1275] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:06:07.713670 16563 rpcz_store.cc:276] 0709 13:06:06.321810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.444349 (+122539us) service_pool.cc:224] Handling call
0709 13:06:06.455561 (+ 11212us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:06.455923 (+   362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:06.455951 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:06.456373 (+   422us) log.cc:844] Serialized 26 byte log entry
0709 13:06:06.696168 (+239795us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:06.696208 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:06.696280 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:07.700873 (+1004593us) raft_consensus.cc:1685] finished
0709 13:06:07.700900 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:07.701056 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":992,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":82,"thread_start_us":239444,"threads_started":1,"wal-append.queue_time_us":395893}
I20260709 13:06:07.715134 15861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1799) took 1001 ms. Trace:
I20260709 13:06:07.715499 15861 rpcz_store.cc:276] 0709 13:06:06.713292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:07.714051 (+1000759us) service_pool.cc:224] Handling call
0709 13:06:07.714304 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:07.714912 (+   608us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:07.715083 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:07.716835 15861 raft_consensus.cc:1275] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:06:07.717669 15862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1798) took 1011 ms. Trace:
I20260709 13:06:07.628162 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:07.580675 19115 raft_consensus.cc:3055] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:07.548691 19053 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:07.582000 19101 raft_consensus.cc:697] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 5 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:07.643235 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:06:07.680351 19128 log.cc:927] Time spent T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.369s	user 0.000s	sys 0.001s
W20260709 13:06:07.646941 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:07.647725 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:07.649281 15932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 1787) took 1357 ms. Trace:
I20260709 13:06:07.653594 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:07.727481 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:07.727638 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:07.727963 16565 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5.
I20260709 13:06:07.656013 15855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 1897) took 1935 ms. Trace:
I20260709 13:06:07.728742 15855 rpcz_store.cc:276] 0709 13:06:05.720235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.299979 (+579744us) service_pool.cc:224] Handling call
0709 13:06:06.307476 (+  7497us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:06.378014 (+ 70538us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:07.655703 (+1277689us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b5400219154. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:07.655939 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26225024}
W20260709 13:06:07.661144 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 96619.7us
I20260709 13:06:07.730108 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:07.626159 18732 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:07.664054 16566 raft_consensus.cc:1275] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:06:07.644338 19103 raft_consensus.cc:2749] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:07.587071 18948 consensus_queue.cc:237] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:07.697856 16561 pending_rounds.cc:107] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 6.6
I20260709 13:06:07.559468 16533 leader_election.cc:400] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 election: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 5. Current term is 6.
I20260709 13:06:07.682293 18728 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:07.721911 15860 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:07.686115 18993 raft_consensus.cc:3055] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:07.722332 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:07.659516 18973 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:07.644997 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:07.640417 18932 consensus_queue.cc:1048] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 13:06:07.724537 15862 rpcz_store.cc:276] 0709 13:06:06.706572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:07.663745 (+957173us) service_pool.cc:224] Handling call
0709 13:06:07.711222 (+ 47477us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:07.711440 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:07.711477 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:07.711545 (+    68us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:07.711601 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:07.717442 (+  5841us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:07.717608 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:07.685111 19108 log.cc:927] Time spent T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.705s	user 0.002s	sys 0.000s
I20260709 13:06:07.730813 15902 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:07.727598 15932 rpcz_store.cc:276] 0709 13:06:06.292263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.292374 (+   111us) service_pool.cc:224] Handling call
0709 13:06:06.309789 (+ 17415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:06.608212 (+298423us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:07.648959 (+1040747us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b540044b6d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:07.649202 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":464297344}
I20260709 13:06:07.733515 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:06:07.692349 19118 log.cc:927] Time spent T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.329s	user 0.000s	sys 0.000s
I20260709 13:06:07.680145 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:07.740607 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:07.743862 16561 raft_consensus.cc:2917] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Aborting config change with OpId 6.6: Aborted: Op aborted by new leader
I20260709 13:06:07.762076 15902 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:07.762925 16563 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:07.743356 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:07.681942 19099 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:07.692159 19134 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:07.754387 15974 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:07.721482 19053 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:07.687992 19138 raft_consensus.cc:2749] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:06:07.757851 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 198519us
I20260709 13:06:07.770758 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:07.771524 16533 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:07.758304 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:07.739854 18732 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:07.759248 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:07.772804 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:07.774852 16533 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:07.767030 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:07.721911 19057 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:07.679653 18898 consensus_queue.cc:1048] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.074s
I20260709 13:06:07.762423 16561 raft_consensus.cc:2889] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:07.682015 18949 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:07.719543 19115 raft_consensus.cc:740] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:07.777238 16533 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:07.769891 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:07.802574 15935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1875) took 1022 ms. Trace:
I20260709 13:06:07.802910 15855 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 4.
I20260709 13:06:07.786761 15860 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:07.772538 16562 raft_consensus.cc:2376] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7. Re-sending same reply.
I20260709 13:06:07.755717 18993 raft_consensus.cc:740] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:07.807471 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:07.803175 15935 rpcz_store.cc:276] 0709 13:06:06.780238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.781731 (+  1493us) service_pool.cc:224] Handling call
0709 13:06:06.787780 (+  6049us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:06.788100 (+   320us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:06:06.914210 (+126110us) log.cc:844] Serialized 14 byte log entry
0709 13:06:07.679433 (+765223us) spinlock_profiling.cc:243] Waited 5.98 ms on lock 0x7b180043e760. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e7235c0a 00007f73f0975219 00007f73e72bda40 00007f73e72b84e4 00007f73e72b8455 00007f73edda03c7 00007f73ea3b29a7 00007f73ea3c20bf 00007f73ea3c2038
0709 13:06:07.798497 (+119064us) log.cc:844] Serialized 14 byte log entry
0709 13:06:07.802131 (+  3634us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:07.802210 (+    79us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:07.802271 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:07.802358 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:07.802509 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59883,"prepare.queue_time_us":733694,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":6866,"raft.queue_time_us":306,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":23655,"spinlock_wait_cycles":13756800,"thread_start_us":309979,"threads_started":3,"wal-append.queue_time_us":259156}
I20260709 13:06:07.810600 15860 raft_consensus.cc:1240] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:06:07.816802 15974 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:07.817127 15855 raft_consensus.cc:1275] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:06:07.818603 15974 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:06:07.820068 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 58071.1us
I20260709 13:06:07.821446 15902 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:07.782625 18728 consensus_queue.cc:1048] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:06:07.682857 18968 consensus_queue.cc:260] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:07.692759 19035 consensus_queue.cc:237] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:07.696193 19075 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:07.873654 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 49218us
I20260709 13:06:07.782428 19099 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:07.814100 19022 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:07.806910 19138 raft_consensus.cc:2749] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:06:07.875165 19128 log.cc:927] Time spent T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:06:07.825541 19083 raft_consensus.cc:2749] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:07.872685 18998 consensus_queue.cc:1048] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.078s
I20260709 13:06:07.783455 18732 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:07.825755 16002 raft_consensus.cc:1240] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:07.832177 15855 raft_consensus.cc:1275] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:06:07.797073 19143 raft_consensus.cc:2804] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:07.824864 19137 raft_consensus.cc:2749] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:07.918342 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:07.918464 15860 raft_consensus.cc:1275] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
W20260709 13:06:07.923969 16564 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.005s	sys 0.017s
I20260709 13:06:07.872931 19097 raft_consensus.cc:2749] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:06:07.928894 16564 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.005s	sys 0.017s
W20260709 13:06:07.920864 19126 log.cc:927] Time spent T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:06:07.936913 15855 raft_consensus.cc:1275] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:07.919117 19099 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:07.904527 18782 consensus_queue.cc:1059] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:06:07.928522 19033 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:07.928153 19143 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:07.958986 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:07.954861 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:07.960117 15936 raft_consensus.cc:2393] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 5: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:07.782835 19134 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:07.919073 19103 consensus_queue.cc:1059] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:06:07.819509 19090 consensus_queue.cc:1059] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.114s
I20260709 13:06:07.959309 15932 raft_consensus.cc:1240] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 6. Current term is 7. Ops: []
W20260709 13:06:07.961634 18957 consensus_meta.cc:287] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.906s	user 0.011s	sys 0.000s
I20260709 13:06:07.949630 19099 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:07.961637 19153 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:07.963449 19149 raft_consensus.cc:2749] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:07.963769 19143 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:07.963762 19141 consensus_queue.cc:1059] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.142s
I20260709 13:06:07.963806 19133 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:07.723824 18966 raft_consensus.cc:2955] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:06:07.954092 19138 raft_consensus.cc:2749] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:07.935112 18993 consensus_queue.cc:260] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:07.971956 16009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1913) took 1171 ms. Trace:
I20260709 13:06:07.972169 18796 raft_consensus.cc:3055] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:07.972193 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:08.019760 16011 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:06:07.947932 19102 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:07.974248 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:08.022265 15935 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:07.974892 16006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1907) took 1177 ms. Trace:
I20260709 13:06:08.022964 16006 rpcz_store.cc:276] 0709 13:06:06.797156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:07.542231 (+745075us) service_pool.cc:224] Handling call
0709 13:06:07.542509 (+   278us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:07.542712 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:07.542743 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:07.553503 (+ 10760us) log.cc:844] Serialized 26 byte log entry
0709 13:06:07.754141 (+200638us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:07.754186 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:07.754410 (+   224us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:07.974686 (+220276us) raft_consensus.cc:1685] finished
0709 13:06:07.974708 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:07.974836 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":181882,"threads_started":1,"wal-append.queue_time_us":347373}
I20260709 13:06:07.912341 19101 consensus_queue.cc:237] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:07.975885 18932 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:07.986021 16010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1908) took 1188 ms. Trace:
I20260709 13:06:07.994603 18998 raft_consensus.cc:3055] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:07.996683 19136 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:07.997514 16005 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:06:08.007898 19103 consensus_queue.cc:1243] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Peer 31882606e87447fb9a36bbf625129fe8 log is divergent from this leader: its last log entry 5.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:06:08.011374 18854 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:08.015563 16009 rpcz_store.cc:276] 0709 13:06:06.800319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:07.553265 (+752946us) service_pool.cc:224] Handling call
0709 13:06:07.553577 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:07.642685 (+ 89108us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:07.642717 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:07.658582 (+ 15865us) log.cc:844] Serialized 26 byte log entry
0709 13:06:07.821585 (+163003us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:06:07.893716 (+ 72131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:07.893844 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:07.971732 (+ 77888us) raft_consensus.cc:1685] finished
0709 13:06:07.971752 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:07.971894 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":123665,"raft.queue_time_us":88933,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":8486,"thread_start_us":143536,"threads_started":1,"wal-append.queue_time_us":281972}
I20260709 13:06:08.015842 19053 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:08.027514 16010 rpcz_store.cc:276] 0709 13:06:06.797432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:07.548350 (+750918us) service_pool.cc:224] Handling call
0709 13:06:07.548633 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:07.548818 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:06:07.722676 (+173858us) log.cc:844] Serialized 14 byte log entry
0709 13:06:07.925266 (+202590us) log.cc:844] Serialized 14 byte log entry
0709 13:06:07.985620 (+ 60354us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:07.985688 (+    68us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:07.985747 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:07.985831 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:07.985958 (+   127us) inbound_call.cc:177] Queueing success response
0709 13:06:07.994000 (+  8042us) spinlock_profiling.cc:243] Waited 3.62 ms on lock 0x7b18003b88a0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb522 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd 000000000038f354 000000000038f199 00007f73e729e83a
Metrics: {"mutex_wait_us":55832,"prepare.queue_time_us":271033,"prepare.run_cpu_time_us":133,"prepare.run_wall_time_us":1014,"raft.queue_time_us":47682,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":5063,"spinlock_wait_cycles":13236480,"thread_start_us":363238,"threads_started":3,"wal-append.queue_time_us":249838}
I20260709 13:06:08.037036 16009 raft_consensus.cc:1240] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 6. Current term is 7. Ops: []
W20260709 13:06:08.026659 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 205258us
I20260709 13:06:08.019309 18796 raft_consensus.cc:740] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:08.040905 16009 raft_consensus.cc:1275] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:06:08.043409 16009 raft_consensus.cc:1275] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:06:08.038853 15902 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:08.031898 16011 raft_consensus.cc:1275] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:06:08.045197 16563 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:06:08.045538 15902 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:08.031042 18998 raft_consensus.cc:740] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:08.019451 19092 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:08.049762 16008 raft_consensus.cc:1240] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:08.053927 15854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 1899) took 1796 ms. Trace:
I20260709 13:06:08.054320 15854 rpcz_store.cc:276] 0709 13:06:06.257055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.627248 (+370193us) service_pool.cc:224] Handling call
0709 13:06:06.716627 (+ 89379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:06.721912 (+  5285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:08.053740 (+1331828us) spinlock_profiling.cc:243] Waited 89.1 ms on lock 0x7b540008cd50. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:08.053871 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":204893440}
I20260709 13:06:08.037075 18993 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:08.057444 15902 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:08.056721 19159 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:08.060118 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:08.009686 19153 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:08.033294 18932 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:08.060178 15902 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:08.061698 19102 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:08.050808 19035 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:08.042403 19134 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:08.038841 18854 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:08.015779 19133 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:06:08.065914 18796 consensus_queue.cc:260] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:08.069072 19164 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:08.080958 19164 raft_consensus.cc:697] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 7 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:08.068446 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:08.063756 19141 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:08.062681 19138 raft_consensus.cc:2749] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:08.073453 18782 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:08.070727 18998 consensus_queue.cc:260] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:08.078791 18796 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:08.079005 19053 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:08.091061 18854 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:08.080341 19092 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:08.091857 19092 consensus_queue.cc:1059] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:08.092105 18998 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:08.081116 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:08.093060 16564 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:08.069726 15902 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:08.084957 19141 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:08.099081 15902 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:08.095227 16533 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:08.089922 19057 raft_consensus.cc:2749] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:08.087896 18782 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:08.104283 19102 consensus_queue.cc:1059] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:08.104822 19136 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:08.120278 15862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 1902) took 1214 ms. Trace:
I20260709 13:06:08.121731 15902 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:08.122094 15974 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:08.105823 19153 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:08.081954 19164 consensus_queue.cc:237] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:08.126639 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.068872 19035 raft_consensus.cc:697] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 7 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:08.106321 18957 raft_consensus.cc:3055] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:06:08.108661 19159 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:08.114925 15856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1792) took 1588 ms. Trace:
I20260709 13:06:08.116608 19138 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:06:08.117564 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 503184us
W20260709 13:06:08.119659 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.092633 19092 raft_consensus.cc:3055] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:06:08.142338 19107 log.cc:927] Time spent T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 1.114s	user 0.004s	sys 0.000s
I20260709 13:06:08.123111 15902 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:08.123162 15974 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:08.115787 19024 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:08.144562 19024 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:08.145766 19075 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:08.147047 19024 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:08.134846 15827 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:08.148025 15855 raft_consensus.cc:1275] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:06:08.148828 15827 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:08.149701 15827 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:08.150213 19007 raft_consensus.cc:2764] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:06:08.150369 18990 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:08.151016 18990 raft_consensus.cc:697] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 7 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:08.132922 15856 rpcz_store.cc:276] 0709 13:06:06.526901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.637561 (+110660us) service_pool.cc:224] Handling call
0709 13:06:06.637856 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:06.655778 (+ 17922us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:06.656193 (+   415us) log.cc:844] Serialized 14 byte log entry
0709 13:06:06.902628 (+246435us) log.cc:844] Serialized 14 byte log entry
0709 13:06:07.505107 (+602479us) spinlock_profiling.cc:243] Waited 2.68 ms on lock 0x7b180034c180. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb3f3 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd 000000000038f354 000000000038f199 00007f73e729e83a
0709 13:06:08.110479 (+605372us) spinlock_profiling.cc:243] Waited 3.08 ms on lock 0x7b180034c180. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e7235c0a 00007f73f0975219 00007f73e72bda40 00007f73e72b84e4 00007f73e72b8455 00007f73ea3a0060 00007f73ea3b2716 00007f73ea3c3287 00007f73ea3c3208
0709 13:06:08.110629 (+   150us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:08.110685 (+    56us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:08.110736 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:08.114626 (+  3890us) spinlock_profiling.cc:243] Waited 17.6 ms on lock 0x7b540008b954. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:08.114685 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:08.114854 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1168616,"prepare.queue_time_us":9177,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":84,"raft.queue_time_us":1214703,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":10503,"spinlock_wait_cycles":53720064,"thread_start_us":243598,"threads_started":1,"wal-append.queue_time_us":378708}
I20260709 13:06:08.151906 18990 consensus_queue.cc:237] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:08.152350 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:08.153460 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:08.153983 15856 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:08.155229 15857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1787) took 2879 ms. Trace:
I20260709 13:06:08.120098 18973 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:08.155582 15857 rpcz_store.cc:276] 0709 13:06:05.275616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.139159 (+863543us) service_pool.cc:224] Handling call
0709 13:06:06.139477 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:07.540685 (+1401208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:08.155006 (+614321us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54004263d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:08.155174 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2473783040}
I20260709 13:06:08.158167 15827 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:08.127053 19053 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:08.127488 18898 raft_consensus.cc:2749] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:08.129177 19135 raft_consensus.cc:2804] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:08.129604 18732 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:08.158236 19076 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:08.164434 19076 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:08.164857 18732 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:08.165565 19143 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:08.166533 19076 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:08.167191 19165 consensus_queue.cc:1048] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:08.167790 19101 consensus_queue.cc:1048] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:08.168612 19090 consensus_queue.cc:1048] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:08.169572 18782 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:08.170403 19053 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:08.171238 19053 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:08.167572 19143 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:08.126634 19103 consensus_queue.cc:1059] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:06:08.176296 19103 raft_consensus.cc:3055] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:08.176749 19103 raft_consensus.cc:740] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:08.177292 18973 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:08.177526 19103 consensus_queue.cc:260] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:08.178654 19103 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:08.179737 19094 raft_consensus.cc:2804] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:08.179859 18962 raft_consensus.cc:2804] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:08.180263 18962 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:08.176151 19035 consensus_queue.cc:237] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:08.180817 18962 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:08.183776 15857 raft_consensus.cc:1275] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:06:08.180301 19094 raft_consensus.cc:697] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 7 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:08.184914 19141 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:08.130488 18970 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:08.185274 19094 consensus_queue.cc:237] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:08.185719 18970 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:08.188141 18970 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:08.179397 18973 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:08.193989 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:08.190601 15974 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:08.195647 16009 raft_consensus.cc:1240] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:08.132297 18957 raft_consensus.cc:740] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:08.196815 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:06:08.196952 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 73901us
I20260709 13:06:08.196467 18840 consensus_queue.cc:1048] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:06:08.197386 19165 consensus_queue.cc:1059] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:06:08.201011 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:08.201431 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:08.202235 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:08.202391 16009 raft_consensus.cc:1240] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:08.202934 16011 raft_consensus.cc:2393] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 7: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:08.134399 19138 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:08.203919 19138 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:08.204685 19123 log.cc:927] Time spent T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.754s	user 0.002s	sys 0.000s
I20260709 13:06:08.158862 18971 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:08.205978 18971 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:08.158180 19159 raft_consensus.cc:697] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 7 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:08.208425 19159 consensus_queue.cc:237] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:08.209415 18971 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:08.210762 18932 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:08.212203 18957 consensus_queue.cc:260] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:08.214330 15859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1790) took 1692 ms. Trace:
I20260709 13:06:08.216022 19135 raft_consensus.cc:697] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:08.121384 15862 rpcz_store.cc:276] 0709 13:06:06.905896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:07.756702 (+850806us) service_pool.cc:224] Handling call
0709 13:06:07.774088 (+ 17386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:07.774277 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:07.937962 (+163685us) log.cc:844] Serialized 14 byte log entry
0709 13:06:08.119953 (+181991us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:08.120004 (+    51us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:08.120047 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:08.120111 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:08.120224 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":136072,"prepare.queue_time_us":45551,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":136604,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":79,"thread_start_us":172611,"threads_started":2,"wal-append.queue_time_us":112343}
I20260709 13:06:08.227329 15859 rpcz_store.cc:276] 0709 13:06:06.522222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.628500 (+106278us) service_pool.cc:224] Handling call
0709 13:06:06.628736 (+   236us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:06.888328 (+259592us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:08.214078 (+1325750us) spinlock_profiling.cc:243] Waited 217 ms on lock 0x7b54003b5e54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:08.214270 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":499092736}
I20260709 13:06:08.217314 18968 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:08.218482 16561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 838) took 1657 ms. Trace:
I20260709 13:06:08.136373 19092 raft_consensus.cc:740] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:08.220059 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:08.234826 16563 raft_consensus.cc:2410] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:06:08.220605 19075 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:08.232973 16561 rpcz_store.cc:276] 0709 13:06:06.560944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.561134 (+   190us) service_pool.cc:224] Handling call
0709 13:06:06.561477 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:07.783028 (+1221551us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:08.218235 (+435207us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b540050f0d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:08.218404 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2572651904}
I20260709 13:06:08.234977 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:08.242354 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:08.237788 19090 raft_consensus.cc:2804] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:08.244405 19090 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:08.244650 16004 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:06:08.246083 19162 consensus_queue.cc:1048] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:06:08.246451 19053 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:08.246724 19092 consensus_queue.cc:260] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:08.244796 19090 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:08.247913 19092 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:08.248764 18949 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:08.251888 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.253005 19138 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:08.255898 15928 raft_consensus.cc:1240] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:08.256247 19135 consensus_queue.cc:237] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:08.257437 19090 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:08.261404 15862 raft_consensus.cc:1275] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:06:08.265761 18957 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:06:08.263860 19099 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:08.268352 19099 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:08.271940 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:08.237970 19075 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:06:08.275398 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 48458.3us
I20260709 13:06:08.277995 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:08.281726 15827 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:08.284531 15827 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:08.270291 19103 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:06:08.241364 19101 consensus_queue.cc:1048] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:06:08.255610 19138 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:08.293576 15827 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:08.294956 16533 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:08.304587 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:08.305938 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:08.306797 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:08.321471 15936 raft_consensus.cc:2410] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:06:08.307574 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:08.322422 15930 raft_consensus.cc:2410] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:06:08.318727 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.277220 19075 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:08.326179 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.326956 15827 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:08.330802 15827 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:08.332896 15827 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:08.314489 18973 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:08.270108 18962 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:08.349841 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:08.350378 18962 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:08.350801 16011 raft_consensus.cc:2410] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:06:08.351469 15858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 575) took 5132 ms. Trace:
I20260709 13:06:08.351742 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:08.351975 15858 rpcz_store.cc:276] 0709 13:06:03.218951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.267934 (+2048983us) service_pool.cc:224] Handling call
0709 13:06:06.259933 (+991999us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:06.279482 (+ 19549us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:08.351196 (+2071714us) spinlock_profiling.cc:243] Waited 832 ms on lock 0x7b54003b6850. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:08.351391 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1913094912}
I20260709 13:06:08.353434 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:08.334923 18840 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:08.354128 15858 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:08.333551 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:08.340260 16533 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:08.357097 16533 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:08.357316 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:08.326110 18732 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:08.339708 19076 consensus_queue.cc:1059] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:06:08.339913 18973 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:08.358613 18973 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:08.359192 18994 consensus_queue.cc:1059] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:08.360152 18994 consensus_queue.cc:1048] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:08.361336 19143 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:08.361848 19143 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:08.361692 19115 consensus_queue.cc:260] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:08.362604 19102 raft_consensus.cc:2749] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:08.362726 19115 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:08.344085 18802 consensus_queue.cc:1048] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:08.363657 19076 raft_consensus.cc:3055] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:08.364073 19076 raft_consensus.cc:740] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:08.324982 19136 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:08.364970 19076 consensus_queue.cc:260] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:08.365787 18958 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:08.365602 19136 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:08.366101 19076 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:08.361423 18994 raft_consensus.cc:3055] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:08.367424 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:08.334755 18898 consensus_queue.cc:1048] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:06:08.367835 18994 raft_consensus.cc:740] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:08.368063 15937 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 7.
I20260709 13:06:08.367978 18732 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:06:08.368664 15937 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53636 (request call id 631) took 1331 ms. Trace:
I20260709 13:06:08.369021 15937 rpcz_store.cc:276] 0709 13:06:07.037322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.367191 (+1329869us) service_pool.cc:224] Handling call
0709 13:06:08.368613 (+  1422us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:08.370371 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:08.370589 18732 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
I20260709 13:06:08.334604 18970 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:08.362241 19143 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:08.355882 18899 consensus_queue.cc:1048] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:06:08.376150 18994 consensus_queue.cc:260] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:08.364009 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:08.376147 16533 leader_election.cc:304] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:08.358088 16561 raft_consensus.cc:2393] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 7: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:08.376267 16566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 846) took 1769 ms. Trace:
I20260709 13:06:08.326484 19016 raft_consensus.cc:2804] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:08.381664 16566 rpcz_store.cc:276] 0709 13:06:06.606561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.606665 (+   104us) service_pool.cc:224] Handling call
0709 13:06:06.626269 (+ 19604us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:07.731550 (+1105281us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:08.376040 (+644490us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54007454d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:08.376215 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2374251904}
I20260709 13:06:08.367646 19136 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:08.383249 19092 raft_consensus.cc:2804] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:08.383915 19092 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:08.384326 19092 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:08.376329 18728 consensus_queue.cc:1175] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8430db80a39d4868bcf7641c16de35ea" responder_term: 7 status { last_received { term: 6 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 6. Current term is 7. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:06:08.386158 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:08.386767 15930 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:06:08.333907 19141 consensus_queue.cc:1059] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:06:08.392202 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:08.371418 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:08.401790 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:08.394507 15862 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:06:08.396313 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:08.403168 16565 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:06:08.401494 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:08.395462 16567 raft_consensus.cc:1275] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:06:08.404639 15937 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:06:08.400025 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:08.406857 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:08.407047 16561 raft_consensus.cc:1240] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:08.407375 16009 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:08.335992 19024 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:08.336133 18782 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:08.411415 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:08.411769 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:08.412365 15934 raft_consensus.cc:2393] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 6: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:08.416039 16533 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:08.417387 16533 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:08.419185 16533 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:08.425138 15976 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:08.426644 15976 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:08.379875 15858 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
I20260709 13:06:08.429335 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:08.366209 18958 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:08.429978 16564 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:08.437377 15935 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:06:08.442167 19158 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:08.442592 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:08.442792 19158 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:08.443282 19158 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:08.452245 18796 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:08.378211 18994 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:08.453879 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:08.329375 18952 consensus_queue.cc:1048] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.087s
I20260709 13:06:08.389421 19016 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:08.463764 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:08.416595 19090 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:08.456518 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:08.427789 19102 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:08.467842 16009 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5.
I20260709 13:06:08.509598 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:08.509845 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:08.511116 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:08.511742 16009 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:08.513008 16565 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:08.462926 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:08.383273 18841 consensus_queue.cc:1059] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.076s
I20260709 13:06:08.514679 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:08.448657 18950 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:08.515379 15935 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:08.449790 19024 raft_consensus.cc:697] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 8 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:08.514667 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:08.518683 15827 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:06:08.511576 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 53504.6us
I20260709 13:06:08.416988 18932 consensus_queue.cc:1059] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.084s
I20260709 13:06:08.439958 18971 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:08.452627 16533 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:08.508508 16566 raft_consensus.cc:2393] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 6: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:08.520627 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:08.521445 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:08.415099 18970 consensus_queue.cc:1059] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.086s
W20260709 13:06:08.526737 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 74177.7us
I20260709 13:06:08.532608 16533 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:08.535135 15931 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 1880) took 1497 ms (client timeout 1929 ms). Trace:
I20260709 13:06:08.535310 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:08.491255 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:08.500478 19016 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:08.527350 15936 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:06:08.526474 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:06:08.535532 15931 rpcz_store.cc:269] 0709 13:06:07.037637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.521195 (+1483558us) service_pool.cc:224] Handling call
0709 13:06:08.535077 (+ 13882us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:08.539247 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:06:08.528465 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 190017us
I20260709 13:06:08.528679 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:08.542397 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:06:08.544193 15825 connection.cc:256] RPC call timeout handler was delayed by -1.0203022829999782s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2303s for a response
I20260709 13:06:08.456607 19156 consensus_queue.cc:1048] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 13:06:08.553514 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:08.523532 19161 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:08.554464 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:08.529644 19102 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:08.556113 16535 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:08.529312 18971 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:08.560518 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:08.559620 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:08.561836 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:08.564635 16535 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:08.548914 18802 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:08.568382 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1890) took 1785 ms. Trace:
I20260709 13:06:08.537726 18932 consensus_queue.cc:1243] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer 31882606e87447fb9a36bbf625129fe8 log is divergent from this leader: its last log entry 6.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:06:08.570152 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.571051 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:08.573616 15858 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:08.573944 19135 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:08.574522 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:08.575150 15858 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:08.588419 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:08.590737 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:08.591428 15934 raft_consensus.cc:1240] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:08.592139 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:08.592365 15936 raft_consensus.cc:1275] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:06:08.593181 16009 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 7.
W20260709 13:06:08.593490 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.597064 15935 raft_consensus.cc:1275] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:06:08.572886 18802 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:08.599009 18802 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:08.599933 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:08.572911 16002 rpcz_store.cc:276] 0709 13:06:06.783157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.788798 (+  5641us) service_pool.cc:224] Handling call
0709 13:06:06.789174 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:07.924357 (+1135183us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:08.568170 (+643813us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54001db354. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:08.568332 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2383472384}
I20260709 13:06:08.599593 19133 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:08.605012 16002 raft_consensus.cc:1240] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 3. Current term is 5. Ops: []
I20260709 13:06:08.564016 18971 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:08.562526 19119 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.317s	user 0.015s	sys 0.013s
I20260709 13:06:08.607836 16002 raft_consensus.cc:1240] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 7 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:08.613860 15934 raft_consensus.cc:1275] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:06:08.627519 15933 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 1796) took 1728 ms (client timeout 2186 ms). Trace:
W20260709 13:06:08.629819 15933 rpcz_store.cc:269] 0709 13:06:06.880716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.881923 (+  1207us) service_pool.cc:224] Handling call
0709 13:06:08.608720 (+1726797us) spinlock_profiling.cc:243] Waited 698 ms on lock 0x7b5400328554. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:08.608849 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1605572864}
W20260709 13:06:08.644390 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:08.640815 19119 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.372s	user 0.015s	sys 0.013s
I20260709 13:06:08.647894 15935 raft_consensus.cc:1275] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:06:08.561463 19075 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:08.649359 15935 pending_rounds.cc:85] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 2
I20260709 13:06:08.649442 15934 raft_consensus.cc:1240] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 13:06:08.649653 15935 pending_rounds.cc:107] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:06:08.600366 19102 raft_consensus.cc:2749] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:08.649989 15935 raft_consensus.cc:2889] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:06:08.650270 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 199073us
I20260709 13:06:08.664469 15933 raft_consensus.cc:1275] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:06:08.666162 19172 log.cc:927] Time spent T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:06:08.605792 19092 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:08.594003 19138 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:06:08.576545 19161 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:08.677420 15902 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:08.691939 19161 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:08.695688 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:08.696269 16569 raft_consensus.cc:1240] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:08.696514 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:08.699748 19156 consensus_queue.cc:1059] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:08.700750 19156 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:08.701200 19156 raft_consensus.cc:697] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 7 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:08.701318 18732 raft_consensus.cc:3055] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:08.701735 18732 raft_consensus.cc:740] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:08.702663 16561 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:08.702170 19156 consensus_queue.cc:237] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:08.702514 18732 consensus_queue.cc:260] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:08.703728 18732 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:08.705189 19174 log.cc:927] Time spent T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 13:06:08.707370 19158 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:08.708549 19053 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:08.708734 16009 raft_consensus.cc:2393] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 9: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:08.696074 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:08.678222 15856 raft_consensus.cc:1240] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:06:08.684152 18932 consensus_queue.cc:1048] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:06:08.688107 19168 log.cc:927] Time spent T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 13:06:08.596201 19024 consensus_queue.cc:237] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:08.719321 15902 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:08.722743 16569 raft_consensus.cc:1275] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:06:08.728180 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 163638us
W20260709 13:06:08.740512 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.685649 19102 consensus_queue.cc:1048] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:06:08.749967 16561 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5.
I20260709 13:06:08.750932 16535 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:08.830782 18728 consensus_queue.cc:1059] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:08.831909 18728 consensus_queue.cc:1243] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer 85e997ba5003439781ec4f1f8db0c3ee log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:06:08.832285 19057 raft_consensus.cc:2749] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:08.832540 18728 raft_consensus.cc:3055] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:08.833038 18728 raft_consensus.cc:740] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:08.839840 19143 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:08.844430 18792 raft_consensus.cc:3055] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:06:08.844605 16007 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60156 (request call id 560) took 2013 ms (client timeout 1521 ms). Trace:
I20260709 13:06:08.845145 18792 raft_consensus.cc:740] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:06:08.845362 16007 rpcz_store.cc:269] 0709 13:06:06.831784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.866147 (+ 34363us) service_pool.cc:224] Handling call
0709 13:06:08.844335 (+1978188us) spinlock_profiling.cc:243] Waited 662 ms on lock 0x7b54003edd54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:08.844499 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1522138368}
I20260709 13:06:08.846210 18792 consensus_queue.cc:260] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:08.745590 18737 consensus_queue.cc:1048] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.064s
I20260709 13:06:08.847043 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:08.848671 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:08.849691 16007 raft_consensus.cc:2393] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 9: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:08.851187 16007 raft_consensus.cc:1240] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:06:08.852743 16007 raft_consensus.cc:3055] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:06:08.853116 19175 log.cc:927] Time spent T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.327s	user 0.002s	sys 0.000s
I20260709 13:06:08.853200 16007 raft_consensus.cc:740] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:08.856777 16008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1914) took 1296 ms. Trace:
W20260709 13:06:08.857031 19182 log.cc:927] Time spent T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:06:08.857221 16008 rpcz_store.cc:276] 0709 13:06:07.560764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:07.610684 (+ 49920us) service_pool.cc:224] Handling call
0709 13:06:07.622295 (+ 11611us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:08.051208 (+428913us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:08.605793 (+554585us) spinlock_profiling.cc:243] Waited 427 ms on lock 0x7b54000aa5d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:08.856528 (+250735us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b54000aa5d0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:08.856717 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1008587904}
I20260709 13:06:08.854631 16007 consensus_queue.cc:260] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:08.858669 16007 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:08.859220 19184 log.cc:927] Time spent T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
W20260709 13:06:08.863945 19179 log.cc:927] Time spent T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
W20260709 13:06:08.859969 19176 log.cc:927] Time spent T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 13:06:08.867959 15859 raft_consensus.cc:1275] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:06:08.879297 15976 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:08.862675 16010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1798) took 1191 ms. Trace:
I20260709 13:06:08.881346 16010 rpcz_store.cc:276] 0709 13:06:07.671725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.040806 (+369081us) service_pool.cc:224] Handling call
0709 13:06:08.041072 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:08.415308 (+374236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:08.415342 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:08.415389 (+    47us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:08.415418 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:08.862412 (+446994us) spinlock_profiling.cc:243] Waited 374 ms on lock 0x7b54001b9754. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:08.862464 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:08.862616 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5099,"raft.queue_time_us":4193,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":18196,"spinlock_wait_cycles":859903744}
I20260709 13:06:08.902570 15974 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:08.908377 18782 raft_consensus.cc:2749] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:08.824196 19076 raft_consensus.cc:2749] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:08.809334 19158 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:08.922928 15974 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:08.926935 16007 raft_consensus.cc:1275] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:06:08.928115 18854 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
W20260709 13:06:08.931200 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.936302 16562 raft_consensus.cc:1275] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:06:08.937685 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:08.938647 16562 raft_consensus.cc:2393] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 9: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:08.940066 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:08.956887 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.898921 19119 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:08.960786 16005 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41674 (request call id 1797) took 1293 ms (client timeout 1699 ms). Trace:
W20260709 13:06:08.961292 16005 rpcz_store.cc:269] 0709 13:06:07.671271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:07.744732 (+ 73461us) service_pool.cc:224] Handling call
0709 13:06:08.960578 (+1215846us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b54003831d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:08.960719 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":509164416}
I20260709 13:06:08.962323 16003 raft_consensus.cc:1240] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:08.911303 19085 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:06:08.975250 15855 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.001s	sys 0.010s
I20260709 13:06:08.956399 19129 raft_consensus.cc:2749] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:06:08.975924 15855 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.001s	sys 0.010s
I20260709 13:06:08.998492 15974 leader_election.cc:400] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Vote denied by peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) with higher term. Message: Invalid argument: T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 5. Current term is 6.
W20260709 13:06:08.978890 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 259661us
I20260709 13:06:09.000089 15929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 635) took 1065 ms. Trace:
I20260709 13:06:09.000733 15902 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:08.966955 19158 raft_consensus.cc:2749] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:09.008162 15902 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:09.008566 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:09.009198 15930 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:08.972785 19085 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:09.000700 15929 rpcz_store.cc:276] 0709 13:06:07.934341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:07.934514 (+   173us) service_pool.cc:224] Handling call
0709 13:06:08.512177 (+577663us) spinlock_profiling.cc:243] Waited 5.66 ms on lock 0x7b5800003698. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef381fc0 00007f73ef36c929 00007f73ef36c3d2 00007f73e88e3bd4
0709 13:06:08.512461 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:08.512801 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:06:08.676194 (+163393us) log.cc:844] Serialized 14 byte log entry
0709 13:06:08.999614 (+323420us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:08.999661 (+    47us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:08.999726 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:08.999785 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:08.999957 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22274,"prepare.queue_time_us":23836,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":22689,"spinlock_wait_cycles":13009280,"thread_start_us":463379,"threads_started":2}
I20260709 13:06:09.003507 15974 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:09.022567 15974 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:06:08.976718 18973 consensus_meta.cc:287] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.618s	user 0.010s	sys 0.000s
I20260709 13:06:08.970230 19119 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:09.020989 15936 raft_consensus.cc:1275] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:06:08.991079 18998 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:09.030453 16005 raft_consensus.cc:1275] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:06:08.973522 18950 consensus_queue.cc:1048] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 13:06:09.015045 18737 raft_consensus.cc:2749] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:09.016953 19075 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:09.004187 19094 consensus_queue.cc:1048] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:06:09.021354 19136 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:09.019585 19085 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:08.973359 18952 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:06:09.042290 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:09.043751 16564 raft_consensus.cc:1240] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:09.005934 19090 consensus_queue.cc:1048] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:06:09.045280 16566 raft_consensus.cc:2410] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:06:09.031616 19101 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:09.027803 19053 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:09.059195 18993 consensus_queue.cc:1048] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:09.060923 15931 raft_consensus.cc:1240] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:09.065747 19129 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:06:09.068231 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 549580us
I20260709 13:06:09.044366 19092 leader_election.cc:290] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:09.011286 18968 consensus_queue.cc:1059] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 13:06:09.070164 15825 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:09.023013 19102 consensus_queue.cc:1059] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
W20260709 13:06:09.092370 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:09.092481 16003 raft_consensus.cc:1275] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:06:09.095541 16010 raft_consensus.cc:2955] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:06:09.099076 16011 raft_consensus.cc:1275] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:06:09.102680 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:09.102800 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:09.066385 19115 consensus_meta.cc:287] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.703s	user 0.009s	sys 0.004s
W20260709 13:06:09.071094 18958 consensus_meta.cc:287] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.614s	user 0.003s	sys 0.006s
I20260709 13:06:09.047006 19136 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:09.068455 18841 raft_consensus.cc:3055] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:06:09.067664 18994 consensus_meta.cc:287] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.608s	user 0.008s	sys 0.000s
W20260709 13:06:09.115543 19200 log.cc:927] Time spent T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:06:09.071697 19141 consensus_queue.cc:1048] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:06:09.030125 18973 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:09.038749 18998 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:09.069485 18970 raft_consensus.cc:3055] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:09.151724 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:06:09.156926 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 106224us
I20260709 13:06:09.115935 19119 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:09.076118 19066 raft_consensus.cc:2749] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:09.168797 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:09.041217 19035 consensus_queue.cc:1059] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 13:06:09.180286 16567 raft_consensus.cc:1275] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 13:06:09.229610 16567 pending_rounds.cc:85] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5: Aborting all ops after (but not including) 5
W20260709 13:06:09.181878 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 72192.1us
I20260709 13:06:09.178997 15827 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:09.229931 16567 pending_rounds.cc:107] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:06:09.230338 16567 raft_consensus.cc:2889] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:09.188333 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:09.220043 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:09.167976 19101 raft_consensus.cc:697] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:09.232401 15935 raft_consensus.cc:2410] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:06:09.219784 19198 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:09.223390 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:09.068938 18861 raft_consensus.cc:2804] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:09.177897 19136 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:06:09.183012 19203 log.cc:927] Time spent T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:06:09.116245 19053 raft_consensus.cc:697] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:06:09.116475 19152 log.cc:927] Time spent T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 13:06:09.230983 15902 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:09.266913 16004 raft_consensus.cc:1275] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:06:09.176818 18841 raft_consensus.cc:740] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:09.282761 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:09.291105 15933 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6.
I20260709 13:06:09.183389 19075 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:09.264889 15929 raft_consensus.cc:1240] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:09.183180 18932 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:09.152262 19129 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:09.294791 15935 raft_consensus.cc:1275] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:06:09.299801 15935 raft_consensus.cc:1240] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:09.300797 15861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 1901) took 2412 ms. Trace:
I20260709 13:06:09.224583 18970 raft_consensus.cc:740] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:09.179586 18958 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:09.304136 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 73251.3us
I20260709 13:06:09.221969 18950 consensus_queue.cc:1059] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 13:06:09.324668 15974 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:09.301198 15861 rpcz_store.cc:276] 0709 13:06:06.888111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:07.717582 (+829471us) service_pool.cc:224] Handling call
0709 13:06:07.717809 (+   227us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:07.717975 (+   166us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:07.738469 (+ 20494us) log.cc:844] Serialized 14 byte log entry
0709 13:06:08.846541 (+1108072us) spinlock_profiling.cc:243] Waited 9.65 ms on lock 0x7b1800426400. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e9f504bc 00007f73e729e48b 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f4823a
0709 13:06:09.300474 (+453933us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:09.300525 (+    51us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:09.300568 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:09.300644 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:09.300746 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":228030,"prepare.queue_time_us":20853,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":36,"raft.queue_time_us":268130,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":101,"spinlock_wait_cycles":22203776,"thread_start_us":292836,"threads_started":2,"wal-append.queue_time_us":135224,"wal-append.run_cpu_time_us":1762,"wal-append.run_wall_time_us":1092139}
W20260709 13:06:09.312013 19016 consensus_meta.cc:287] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.759s	user 0.012s	sys 0.000s
I20260709 13:06:09.334707 16563 raft_consensus.cc:2393] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 6: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:09.347107 15974 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:09.281574 19092 consensus_queue.cc:1048] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:06:09.304984 19007 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:06:09.311844 19135 consensus_meta.cc:287] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.737s	user 0.011s	sys 0.000s
I20260709 13:06:09.267666 18994 consensus_queue.cc:1059] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 13:06:09.304673 19204 raft_consensus.cc:3055] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:06:09.422276 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 243338us
I20260709 13:06:09.268683 15858 raft_consensus.cc:2393] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 7: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:06:09.426894 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 356792us
I20260709 13:06:09.316535 18966 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:09.266346 19198 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:09.444583 15930 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 9.
I20260709 13:06:09.354986 18968 raft_consensus.cc:2749] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:09.310571 18932 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:09.356293 18854 raft_consensus.cc:2749] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:09.419538 19102 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:09.465302 19135 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:09.486522 15900 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:09.455250 19204 raft_consensus.cc:740] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:09.491225 15900 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:09.312253 19075 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:09.361954 19016 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:09.501636 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 219519us
W20260709 13:06:09.502239 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:09.454830 18950 consensus_queue.cc:1059] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:06:09.496696 15900 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:09.516747 15900 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:09.517910 15900 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:09.512698 19202 log.cc:927] Time spent T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.273s	user 0.000s	sys 0.001s
I20260709 13:06:09.513139 19206 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:09.513618 19212 raft_consensus.cc:2804] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:09.514068 19201 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:06:09.514510 19195 log.cc:927] Time spent T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.451s	user 0.001s	sys 0.001s
I20260709 13:06:09.478087 18861 raft_consensus.cc:697] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:09.354833 18841 consensus_queue.cc:260] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:06:09.528187 16533 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:09.497426 18966 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:09.538865 19212 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:06:09.532604 19192 log.cc:927] Time spent T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.364s	user 0.001s	sys 0.000s
W20260709 13:06:09.563889 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 97494.7us
I20260709 13:06:09.490499 19102 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:09.530054 18732 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:06:09.567605 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:09.558426 18802 consensus_meta.cc:287] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.959s	user 0.011s	sys 0.002s
I20260709 13:06:09.479377 19101 consensus_queue.cc:237] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:09.575273 15825 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:09.556002 19135 raft_consensus.cc:2804] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:09.565201 18841 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:09.482292 19053 consensus_queue.cc:237] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:09.574684 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:09.564152 18998 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:09.564270 18973 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:09.589509 15861 raft_consensus.cc:1240] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:09.608007 19129 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:09.559012 19212 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:06:09.650797 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 131843us
I20260709 13:06:09.653563 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 578) took 1112 ms. Trace:
I20260709 13:06:09.578218 18732 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:09.654093 16002 rpcz_store.cc:276] 0709 13:06:08.541246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.643878 (+102632us) service_pool.cc:224] Handling call
0709 13:06:08.644175 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:08.646300 (+  2125us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:08.646330 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:08.646798 (+   468us) log.cc:844] Serialized 26 byte log entry
0709 13:06:08.975049 (+328251us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:08.975094 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:08.975161 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:09.653316 (+678155us) raft_consensus.cc:1685] finished
0709 13:06:09.653343 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:09.653493 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1779,"raft.queue_time_us":2498,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":12051,"thread_start_us":277489,"threads_started":1,"wal-append.queue_time_us":333711}
W20260709 13:06:09.660590 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 118368us
I20260709 13:06:09.578483 18993 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:09.663746 16533 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:09.608103 19190 log.cc:927] Time spent T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.294s	user 0.001s	sys 0.000s
I20260709 13:06:09.567538 19115 consensus_queue.cc:1059] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.079s
I20260709 13:06:09.584700 19135 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:09.546279 19206 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:09.657752 18840 consensus_queue.cc:1059] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:06:09.701319 19222 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:09.569878 19214 consensus_queue.cc:1059] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.110s
I20260709 13:06:09.658228 19216 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:09.660717 18958 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:09.661870 18998 raft_consensus.cc:3055] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:09.665803 18732 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:09.681231 19135 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:09.780187 19213 log.cc:927] Time spent T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.277s	user 0.001s	sys 0.000s
I20260709 13:06:09.701952 19222 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:06:09.799183 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 223977us
I20260709 13:06:09.748206 18998 raft_consensus.cc:740] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:09.664384 19129 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:06:09.742586 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:09.583202 18802 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:09.817037 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 149528us
I20260709 13:06:09.720196 19206 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:09.732563 19235 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:09.720937 19204 consensus_queue.cc:260] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:09.790287 18958 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:09.702418 18861 consensus_queue.cc:237] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:09.745628 19216 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:10.378996 19205 log.cc:927] Time spent T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 1.064s	user 0.001s	sys 0.000s
W20260709 13:06:10.379199 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:10.367393 16563 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45244 (request call id 860) took 1976 ms (client timeout 2612 ms). Trace:
I20260709 13:06:10.361987 19149 raft_consensus.cc:3055] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:09.885462 15859 raft_consensus.cc:1240] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:10.307281 19094 raft_consensus.cc:3055] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:10.283105 19075 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:10.554464 19094 raft_consensus.cc:740] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:06:10.130143 19217 log.cc:927] Time spent T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.518s	user 0.000s	sys 0.001s
W20260709 13:06:10.444576 15933 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.776s	user 0.022s	sys 0.084s
I20260709 13:06:10.552914 19149 raft_consensus.cc:740] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:06:10.555665 15933 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.890s	user 0.025s	sys 0.087s
W20260709 13:06:10.518587 15827 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.874s	user 0.005s	sys 0.005s
W20260709 13:06:10.556149 15827 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.912s	user 0.006s	sys 0.005s
I20260709 13:06:09.872653 19129 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:10.558120 15933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 1820) took 1450 ms. Trace:
I20260709 13:06:10.558511 15933 rpcz_store.cc:276] 0709 13:06:09.107327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:09.292585 (+185258us) service_pool.cc:224] Handling call
0709 13:06:09.293063 (+   478us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:09.310996 (+ 17933us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:06:09.363108 (+ 52112us) log.cc:844] Serialized 14 byte log entry
0709 13:06:10.294733 (+931625us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b180032bea0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb3f3 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd 000000000038f354 000000000038f199 00007f73e729e83a
0709 13:06:10.556169 (+261436us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:10.556607 (+   438us) log.cc:844] Serialized 26 byte log entry
0709 13:06:10.557692 (+  1085us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:10.557724 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:10.557779 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:10.557815 (+    36us) raft_consensus.cc:1685] finished
0709 13:06:10.557961 (+   146us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:10.558068 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69232,"prepare.queue_time_us":108043,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":685824,"raft.run_cpu_time_us":322,"raft.run_wall_time_us":34681,"spinlock_wait_cycles":27954048,"thread_start_us":1024446,"threads_started":2,"wal-append.queue_time_us":238391}
I20260709 13:06:10.559454 19094 consensus_queue.cc:260] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:10.551424 16563 rpcz_store.cc:269] 0709 13:06:08.391349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.393570 (+  2221us) service_pool.cc:224] Handling call
0709 13:06:10.367181 (+1973611us) spinlock_profiling.cc:243] Waited 930 ms on lock 0x7b540059bd54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:10.367336 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2138914048}
W20260709 13:06:10.434736 15902 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.875s	user 0.000s	sys 0.005s
I20260709 13:06:10.367744 16562 raft_consensus.cc:1240] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 5. Current term is 6. Ops: []
W20260709 13:06:10.486004 19218 log.cc:927] Time spent T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.878s	user 0.002s	sys 0.000s
I20260709 13:06:10.416850 18728 consensus_queue.cc:260] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:10.308961 16008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1933) took 1208 ms. Trace:
I20260709 13:06:10.486655 15931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 1802) took 2163 ms. Trace:
I20260709 13:06:10.575464 15931 rpcz_store.cc:276] 0709 13:06:08.323138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.538916 (+215778us) service_pool.cc:224] Handling call
0709 13:06:08.539166 (+   250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:09.060774 (+521608us) spinlock_profiling.cc:243] Waited 9.88 ms on lock 0x7b54000dcda0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea31fdc0 00007f73ea3aa1c0 00007f73ea3ab231 00007f73ea3a7d8f 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c
0709 13:06:09.107825 (+ 47051us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:10.486380 (+1378555us) spinlock_profiling.cc:243] Waited 5.26 ms on lock 0x7b54000dcda0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea31fdc0 00007f73ea3aba02 00007f73ea3a7dd6 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2
0709 13:06:10.486596 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34833920}
I20260709 13:06:10.391841 19143 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:10.556731 19149 consensus_queue.cc:260] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
W20260709 13:06:10.378882 16561 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45246 (request call id 760) took 1968 ms (client timeout 2234 ms). Trace:
I20260709 13:06:10.482585 19156 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:09.886214 18971 consensus_meta.cc:287] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 1.245s	user 0.009s	sys 0.002s
W20260709 13:06:10.321893 19136 consensus_meta.cc:287] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 1.033s	user 0.011s	sys 0.001s
W20260709 13:06:09.847149 19208 log.cc:927] Time spent T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.373s	user 0.000s	sys 0.001s
I20260709 13:06:10.384088 18962 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:10.519541 16534 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.004s	sys 0.004s
W20260709 13:06:10.634771 16534 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.737s	user 0.005s	sys 0.005s
I20260709 13:06:10.566037 18854 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:10.394282 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:10.436075 16568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 872) took 1741 ms. Trace:
I20260709 13:06:10.533811 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:10.383558 19225 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:10.376353 16569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 875) took 1660 ms. Trace:
I20260709 13:06:10.123648 15929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 1821) took 1014 ms. Trace:
I20260709 13:06:10.555141 15859 raft_consensus.cc:1275] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:06:10.559988 19129 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:10.643282 16569 rpcz_store.cc:276] 0709 13:06:08.716209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.716329 (+   120us) service_pool.cc:224] Handling call
0709 13:06:08.716573 (+   244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:08.729285 (+ 12712us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:10.375953 (+1646668us) spinlock_profiling.cc:243] Waited 5.75 ms on lock 0x7b54000ce754. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:10.376234 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13218560}
W20260709 13:06:10.365715 19199 log.cc:927] Time spent T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 1.134s	user 0.005s	sys 0.023s
I20260709 13:06:10.289129 16004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1928) took 1197 ms. Trace:
I20260709 13:06:10.618287 19143 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.574657 16008 rpcz_store.cc:276] 0709 13:06:09.100712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:09.100848 (+   136us) service_pool.cc:224] Handling call
0709 13:06:09.101257 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:09.330909 (+229652us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:09.330956 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:09.331022 (+    66us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:09.331084 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:10.308643 (+977559us) spinlock_profiling.cc:243] Waited 229 ms on lock 0x7b54002e1b54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:10.308706 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:10.308886 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":527475584}
I20260709 13:06:10.145864 15825 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:10.518968 15900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.854s	user 0.018s	sys 0.025s
I20260709 13:06:10.579388 16562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 715) took 1645 ms. Trace:
I20260709 13:06:10.487028 19007 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:09.893249 16006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 562) took 2331 ms. Trace:
W20260709 13:06:10.446614 15858 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 1813) took 2041 ms (client timeout 2490 ms). Trace:
W20260709 13:06:10.565217 15862 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.915s	user 0.014s	sys 0.017s
W20260709 13:06:10.415572 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:10.429452 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:09.885576 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:10.513908 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:09.844767 18932 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:10.532634 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:09.792833 19198 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:10.582772 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:10.359318 15928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 646) took 1351 ms. Trace:
I20260709 13:06:10.647738 19232 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:10.551532 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:10.690097 15862 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.040s	user 0.014s	sys 0.026s
W20260709 13:06:10.582191 15902 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.022s	user 0.000s	sys 0.010s
I20260709 13:06:10.698813 15862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 1925) took 1111 ms. Trace:
I20260709 13:06:10.044449 16567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 711) took 1634 ms. Trace:
W20260709 13:06:10.362840 19135 consensus_meta.cc:287] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.581s	user 0.010s	sys 0.000s
I20260709 13:06:10.699160 15862 rpcz_store.cc:276] 0709 13:06:09.587295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:09.587416 (+   121us) service_pool.cc:224] Handling call
0709 13:06:09.587793 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:09.588168 (+   375us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:09.588203 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:09.649395 (+ 61192us) log.cc:844] Serialized 26 byte log entry
0709 13:06:10.698478 (+1049083us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:10.698528 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:10.698602 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:10.698647 (+    45us) raft_consensus.cc:1685] finished
0709 13:06:10.698663 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:10.698768 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":119558,"raft.queue_time_us":109137,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":21581,"thread_start_us":1048720,"threads_started":1,"wal-append.queue_time_us":792571}
W20260709 13:06:10.603315 16561 rpcz_store.cc:269] 0709 13:06:08.409993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.411450 (+  1457us) service_pool.cc:224] Handling call
0709 13:06:10.378657 (+1967207us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b540059c9d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:10.378813 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":665797504}
W20260709 13:06:10.441917 15974 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.082s	user 0.036s	sys 0.070s
W20260709 13:06:10.719978 15974 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.361s	user 0.038s	sys 0.073s
W20260709 13:06:10.720304 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 1.36091e+06us
I20260709 13:06:10.392283 18792 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:10.363776 18841 consensus_meta.cc:287] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.722s	user 0.012s	sys 0.000s
W20260709 13:06:10.356933 19193 log.cc:927] Time spent T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 1.044s	user 0.002s	sys 0.000s
W20260709 13:06:09.890120 19085 consensus_meta.cc:287] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.801s	user 0.007s	sys 0.002s
W20260709 13:06:10.129783 19207 log.cc:927] Time spent T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.528s	user 0.001s	sys 0.000s
I20260709 13:06:10.631527 18971 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:10.609637 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:10.428565 19242 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:06:10.081451 19219 log.cc:927] Time spent T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.495s	user 0.001s	sys 0.000s
W20260709 13:06:10.218089 18993 consensus_meta.cc:287] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.547s	user 0.009s	sys 0.000s
W20260709 13:06:10.520946 19024 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.868s	user 0.010s	sys 0.020s
I20260709 13:06:10.726281 18957 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.317047 18952 raft_consensus.cc:3055] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:06:10.624339 15860 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.128s	user 0.000s	sys 0.014s
W20260709 13:06:10.762598 15860 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.270s	user 0.000s	sys 0.014s
I20260709 13:06:09.887229 18949 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:09.883927 19095 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.737396 19246 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:10.269667 19212 consensus_meta.cc:287] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.590s	user 0.000s	sys 0.010s
I20260709 13:06:10.583030 18728 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:10.404778 15757 catalog_manager.cc:5697] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 4 to 6, leader changed from <none> to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 6 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:10.509675 19191 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.217300 18802 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:10.483543 19201 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.300112 19092 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:09.891157 19102 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:10.492808 16003 raft_consensus.cc:1217] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 7 LEADER]: Deduplicated request from leader. Original: 2.5->[4.6-6.8]   Dedup: 5.7->[6.8-6.8]
I20260709 13:06:10.806690 16003 raft_consensus.cc:1240] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 7 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 6. Current term is 7. Ops: [4.6-6.8]
I20260709 13:06:10.634064 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:10.382619 19214 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:10.807369 15855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 1917) took 2577 ms. Trace:
I20260709 13:06:10.128461 19016 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:10.383553 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:10.810541 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:10.008664 19229 raft_consensus.cc:3055] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:10.265236 19233 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:10.813680 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:10.643653 15929 rpcz_store.cc:276] 0709 13:06:09.109461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:09.297250 (+187789us) service_pool.cc:224] Handling call
0709 13:06:09.313880 (+ 16630us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:09.328698 (+ 14818us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:09.328744 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:09.348989 (+ 20245us) log.cc:844] Serialized 26 byte log entry
0709 13:06:09.575978 (+226989us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:09.576026 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:09.576117 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:10.123334 (+547217us) raft_consensus.cc:1685] finished
0709 13:06:10.123365 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:10.123552 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26382,"raft.queue_time_us":14948,"raft.run_cpu_time_us":259,"raft.run_wall_time_us":87484,"thread_start_us":96136,"threads_started":1,"wal-append.queue_time_us":596892}
I20260709 13:06:10.640697 16568 rpcz_store.cc:276] 0709 13:06:08.694968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.704011 (+  9043us) service_pool.cc:224] Handling call
0709 13:06:08.704296 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:08.704501 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:06:08.984423 (+279922us) log.cc:844] Serialized 14 byte log entry
0709 13:06:10.324821 (+1340398us) spinlock_profiling.cc:243] Waited 6.93 ms on lock 0x7b18003a42c0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e9f504bc 00007f73e729e48b 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f4823a
0709 13:06:10.378181 (+ 53360us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:06:10.435560 (+ 57379us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:10.435642 (+    82us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:10.435728 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:10.435825 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:10.435998 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":110315,"prepare.queue_time_us":340444,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":41,"raft.queue_time_us":111287,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":107,"spinlock_wait_cycles":48360064,"thread_start_us":393466,"threads_started":2,"wal-append.queue_time_us":252856}
I20260709 13:06:10.263540 19234 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:10.650038 15859 raft_consensus.cc:1275] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:06:10.650712 16004 rpcz_store.cc:276] 0709 13:06:09.092002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:09.092397 (+   395us) service_pool.cc:224] Handling call
0709 13:06:09.115564 (+ 23167us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:09.282607 (+167043us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:10.288797 (+1006190us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b5400305a54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:10.289043 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":347337600}
I20260709 13:06:10.767349 18957 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:10.019577 16570 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:10.652834 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:10.655043 15900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.990s	user 0.018s	sys 0.027s
W20260709 13:06:10.655292 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 509480us
W20260709 13:06:10.621810 18966 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.728s	user 0.000s	sys 0.011s
W20260709 13:06:10.656358 15901 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.867s	user 0.002s	sys 0.013s
W20260709 13:06:10.820556 15901 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.032s	user 0.002s	sys 0.013s
W20260709 13:06:10.656653 15856 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.156s	user 0.000s	sys 0.015s
I20260709 13:06:10.280332 19231 consensus_queue.cc:1059] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.363s
I20260709 13:06:10.636183 19204 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:10.631484 19149 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:09.792618 19222 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:10.810053 19092 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:10.651587 19115 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.916s	user 0.042s	sys 0.118s
I20260709 13:06:10.655072 16562 rpcz_store.cc:276] 0709 13:06:08.933962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.939254 (+  5292us) service_pool.cc:224] Handling call
0709 13:06:10.367543 (+1428289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:10.572806 (+205263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:10.579180 (+  6374us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b540059bd50. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:10.579333 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3283924864}
I20260709 13:06:10.083009 18998 consensus_queue.cc:260] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:10.637672 19136 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:10.641981 18962 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:10.834256 15934 raft_consensus.cc:1240] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:06:10.834724 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:10.820335 18966 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.942s	user 0.000s	sys 0.011s
I20260709 13:06:10.756667 19242 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:10.720367 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:10.839560 16561 raft_consensus.cc:2393] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 6: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:10.802126 19244 raft_consensus.cc:3055] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:10.477823 18970 consensus_queue.cc:260] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:10.690353 16006 rpcz_store.cc:276] 0709 13:06:07.561655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.024234 (+462579us) service_pool.cc:224] Handling call
0709 13:06:08.024533 (+   299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:08.029145 (+  4612us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:08.029179 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:08.030491 (+  1312us) log.cc:844] Serialized 26 byte log entry
0709 13:06:08.613096 (+582605us) spinlock_profiling.cc:243] Waited 965 us on lock 0x7b1800271d80. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f480d1 00007f73ea363a0f
0709 13:06:08.977280 (+364184us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:08.977324 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:08.977394 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:09.868032 (+890638us) raft_consensus.cc:1685] finished
0709 13:06:09.868056 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:09.893170 (+ 25114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16909,"raft.queue_time_us":1620,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":5237,"spinlock_wait_cycles":2220288,"thread_start_us":364015,"threads_started":1,"wal-append.queue_time_us":1149346}
W20260709 13:06:10.690090 15858 rpcz_store.cc:269] 0709 13:06:08.405109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.591915 (+186806us) service_pool.cc:224] Handling call
0709 13:06:10.446330 (+1854415us) spinlock_profiling.cc:243] Waited 644 ms on lock 0x7b54003b5954. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:10.446526 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1481639296}
I20260709 13:06:10.699615 16567 rpcz_store.cc:276] 0709 13:06:08.410424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.411280 (+   856us) service_pool.cc:224] Handling call
0709 13:06:08.411530 (+   250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:09.230846 (+819316us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:10.044100 (+813254us) spinlock_profiling.cc:243] Waited 768 ms on lock 0x7b54002d7dd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:10.044364 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1766915456}
I20260709 13:06:10.724288 15862 raft_consensus.cc:1275] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:06:10.700645 16565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 874) took 1985 ms. Trace:
I20260709 13:06:10.725521 16009 raft_consensus.cc:1240] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:06:10.682695 19223 raft_consensus.cc:2749] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:10.658253 19206 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.851972 15827 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:10.831707 19115 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.107s	user 0.043s	sys 0.118s
I20260709 13:06:10.683553 19247 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.700801 19161 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.691579 15928 rpcz_store.cc:276] 0709 13:06:09.007599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:09.007770 (+   171us) service_pool.cc:224] Handling call
0709 13:06:09.021568 (+ 13798us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:09.046449 (+ 24881us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:06:09.056848 (+ 10399us) log.cc:844] Serialized 14 byte log entry
0709 13:06:09.310518 (+253670us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:09.310587 (+    69us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:09.310636 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:10.358978 (+1048342us) spinlock_profiling.cc:243] Waited 24.6 ms on lock 0x7b540034bf54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:10.359040 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:10.359234 (+   194us) inbound_call.cc:177] Queueing success response
0709 13:06:10.376311 (+ 17077us) spinlock_profiling.cc:243] Waited 29.6 ms on lock 0x7b4800002dd8. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ef2d1aa1 00007f73ef2c8b37 00007f73ef2c89ca 00007f73ef3c026f 00007f73ef3bff37 00007f73ef3d455c 00007f73ef3d4508 00007f73ef3d4488 00007f73ef3d4443 00007f73ef3d36ec 00007f73ee2fda96
Metrics: {"mutex_wait_us":14951,"prepare.queue_time_us":12915,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":27549,"raft.run_cpu_time_us":11614,"raft.run_wall_time_us":1048165,"spinlock_wait_cycles":124805760,"thread_start_us":231642,"threads_started":1,"wal-append.queue_time_us":438656}
I20260709 13:06:10.696444 19099 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:10.750780 15853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 1919) took 2520 ms. Trace:
I20260709 13:06:10.838382 19216 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:10.466503 16566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45232 (request call id 714) took 1532 ms. Trace:
I20260709 13:06:10.725769 19083 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:10.685329 19143 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:10.682667 19225 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:10.437351 19119 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.748468 19228 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.751345 19230 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:10.755916 19024 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.103s	user 0.010s	sys 0.021s
I20260709 13:06:10.700700 19007 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:10.750764 19075 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.774336 15902 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:10.762342 18952 raft_consensus.cc:740] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:06:10.778369 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 57980.9us
I20260709 13:06:10.760164 18993 raft_consensus.cc:2749] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:10.763098 19053 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.756201 19158 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.648453 18854 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:10.797505 15759 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58434 (request call id 342) took 13822 ms (client timeout 15000 ms). Trace:
I20260709 13:06:10.727173 19135 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:10.723021 19232 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:10.801288 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:10.955096 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 90745.1us
I20260709 13:06:10.758792 19101 consensus_queue.cc:1048] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:06:10.771076 18737 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:10.968369 16570 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45244 (request call id 867) took 2275 ms (client timeout 1937 ms). Trace:
I20260709 13:06:10.807816 15855 rpcz_store.cc:276] 0709 13:06:08.229657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.311363 (+ 81706us) service_pool.cc:224] Handling call
0709 13:06:08.324074 (+ 12711us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:08.335339 (+ 11265us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:08.335375 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:08.335792 (+   417us) log.cc:844] Serialized 26 byte log entry
0709 13:06:08.999569 (+663777us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:08.999610 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:08.999707 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:10.336813 (+1337106us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:06:10.800438 (+463625us) raft_consensus.cc:1685] finished
0709 13:06:10.800469 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:10.807305 (+  6836us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25356,"raft.queue_time_us":15210,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":86,"spinlock_wait_cycles":31515520,"thread_start_us":596949,"threads_started":1,"wal-append.queue_time_us":753755}
W20260709 13:06:10.797365 18732 consensus_meta.cc:287] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 1.017s	user 0.012s	sys 0.000s
I20260709 13:06:10.959055 19136 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:10.770437 19239 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:10.816704 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 1.15137e+06us
I20260709 13:06:10.817623 15825 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:10.990945 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 173387us
I20260709 13:06:10.991716 15825 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:10.992738 15825 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:10.993770 15825 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:10.822171 15856 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.331s	user 0.000s	sys 0.016s
I20260709 13:06:10.770895 18949 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:06:10.823665 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:10.995458 16003 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:10.738814 19085 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:10.813868 18841 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.009509 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:10.778529 19212 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:10.831111 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:10.829349 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:10.799381 19165 consensus_queue.cc:1059] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:06:10.832847 18998 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:10.605415 19156 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:10.835330 16562 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
I20260709 13:06:10.840505 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:10.690299 16002 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6.
I20260709 13:06:11.149662 16561 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:06:10.823253 19229 raft_consensus.cc:740] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:10.849467 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:10.848402 16565 rpcz_store.cc:276] 0709 13:06:08.715824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.716001 (+   177us) service_pool.cc:224] Handling call
0709 13:06:08.716420 (+   419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:08.735197 (+ 18777us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:08.735228 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:08.737021 (+  1793us) log.cc:844] Serialized 26 byte log entry
0709 13:06:08.987669 (+250648us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:08.987730 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:08.987804 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:10.364924 (+1377120us) spinlock_profiling.cc:243] Waited 4.19 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:06:10.700365 (+335441us) raft_consensus.cc:1685] finished
0709 13:06:10.700410 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:10.700591 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38674,"raft.queue_time_us":30343,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":66510,"spinlock_wait_cycles":9648384,"thread_start_us":244488,"threads_started":1,"wal-append.queue_time_us":351157}
I20260709 13:06:11.151865 15827 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:11.153882 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:10.695626 18958 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.004s	sys 0.011s
I20260709 13:06:10.854162 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:06:11.070335 19252 log.cc:927] Time spent T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.213s	user 0.001s	sys 0.000s
I20260709 13:06:10.825870 18898 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:11.166185 18898 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:11.168856 18898 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:11.169811 16564 raft_consensus.cc:1275] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 13:06:11.170275 16564 pending_rounds.cc:85] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5: Aborting all ops after (but not including) 7
I20260709 13:06:11.170529 16564 pending_rounds.cc:107] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:06:11.170794 16564 raft_consensus.cc:2889] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:10.860028 15860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1818) took 2354 ms. Trace:
I20260709 13:06:11.171568 15860 rpcz_store.cc:276] 0709 13:06:08.505868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.704284 (+198416us) service_pool.cc:224] Handling call
0709 13:06:08.704565 (+   281us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:09.072604 (+368039us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:09.241512 (+168908us) log.cc:844] Serialized 14 byte log entry
0709 13:06:10.367287 (+1125775us) spinlock_profiling.cc:243] Waited 16.9 ms on lock 0x7b48000801d8. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ef2d1aa1 00007f73ef2c8b37 00007f73ef2c89ca 00007f73ef3c026f 00007f73ef3bff37 00007f73ef3d455c 00007f73ef3d4508 00007f73ef3d4488 00007f73ef3d4443 00007f73ef3d36ec 00007f73ee2fda96
0709 13:06:10.485424 (+118137us) spinlock_profiling.cc:243] Waited 37.3 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:06:10.768147 (+282723us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:10.770537 (+  2390us) log.cc:844] Serialized 26 byte log entry
0709 13:06:10.770698 (+   161us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:10.770732 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:10.770811 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:10.859850 (+ 89039us) raft_consensus.cc:1685] finished
0709 13:06:10.859867 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:10.859976 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":420020,"prepare.queue_time_us":256108,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":46,"raft.queue_time_us":537447,"raft.run_cpu_time_us":29164,"raft.run_wall_time_us":1289766,"spinlock_wait_cycles":124667520,"thread_start_us":1612113,"threads_started":3,"wal-append.queue_time_us":290280}
W20260709 13:06:10.740800 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:10.861924 15853 rpcz_store.cc:276] 0709 13:06:08.230759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.641430 (+410671us) service_pool.cc:224] Handling call
0709 13:06:08.643700 (+  2270us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:09.071468 (+427768us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:09.071498 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:09.072039 (+   541us) log.cc:844] Serialized 26 byte log entry
0709 13:06:09.183610 (+111571us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:09.183665 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:09.184418 (+   753us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:10.041243 (+856825us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b48000801d8. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ef2d1aa1 00007f73ef2c8b37 00007f73ef2c89ca 00007f73ef3c026f 00007f73ef3bff37 00007f73ef3d455c 00007f73ef3d4508 00007f73ef3d4488 00007f73ef3d4443 00007f73ef3d36ec 00007f73ee2fda96
0709 13:06:10.735717 (+694474us) raft_consensus.cc:1685] finished
0709 13:06:10.735738 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:10.750711 (+ 14973us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":769787,"raft.queue_time_us":430920,"raft.run_cpu_time_us":2763,"raft.run_wall_time_us":957298,"spinlock_wait_cycles":39400704,"thread_start_us":111196,"threads_started":1,"wal-append.queue_time_us":244331}
I20260709 13:06:11.178630 18802 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:11.179258 18802 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:11.181730 18802 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:11.182273 18802 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:11.182781 18802 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:10.830801 19234 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:10.853035 19198 raft_consensus.cc:3055] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:10.845167 19244 raft_consensus.cc:740] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:10.793563 19246 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:10.860428 19066 raft_consensus.cc:2749] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:10.801117 19095 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:10.886081 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:10.872701 18970 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:10.858560 19206 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:10.872834 19092 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:10.862759 19247 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:10.865038 19090 consensus_queue.cc:1048] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:06:10.804988 19191 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:10.869432 19099 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:10.945520 19251 log.cc:927] Time spent T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
I20260709 13:06:10.816354 19214 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:10.968806 16570 rpcz_store.cc:269] 0709 13:06:08.693420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.695372 (+  1952us) service_pool.cc:224] Handling call
0709 13:06:10.968185 (+2272813us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b5400550ad4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:10.968297 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3042744576}
I20260709 13:06:10.809587 19201 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:10.868768 19083 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:10.950651 19053 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:10.983902 18732 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:10.869323 19161 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:10.891000 19220 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.893564 18966 raft_consensus.cc:2749] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:10.829578 19233 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:10.879652 19119 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:11.077811 19232 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:11.200711 18792 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.995163 19263 consensus_queue.cc:1048] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.091s
I20260709 13:06:10.956703 19158 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:11.233979 18802 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:11.234499 19158 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:11.000252 19115 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.247557 19115 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:11.249703 18861 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.250429 19115 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:11.005268 19239 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:11.250450 18861 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:11.085301 19007 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:11.087456 18949 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:11.088825 18854 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:10.883630 19230 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:11.253364 19239 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:11.254688 19230 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:11.090525 19260 log.cc:927] Time spent T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.229s	user 0.000s	sys 0.001s
I20260709 13:06:11.096379 18962 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:10.996886 19210 raft_consensus.cc:2749] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:11.009546 18948 consensus_queue.cc:1048] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.095s
I20260709 13:06:11.256170 19095 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:10.994962 19249 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:11.105381 19143 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:10.984800 18737 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:11.258126 19083 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:11.041205 18841 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:11.226205 19220 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:06:11.261850 18841 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:11.016065 19016 raft_consensus.cc:2804] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:11.262606 19220 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:11.107273 19135 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:10.890545 19228 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:11.109599 19094 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:11.110212 19085 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:11.111357 19156 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:10.989711 18971 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:11.010269 19235 raft_consensus.cc:2749] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:11.050473 19245 consensus_queue.cc:1048] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 13:06:11.145655 19242 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:10.891625 19075 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:11.069484 19256 log.cc:927] Time spent T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
W20260709 13:06:11.069031 19261 log.cc:927] Time spent T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.190s	user 0.000s	sys 0.000s
W20260709 13:06:11.069950 19257 log.cc:927] Time spent T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
W20260709 13:06:11.068554 19258 log.cc:927] Time spent T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.177s	user 0.000s	sys 0.000s
I20260709 13:06:11.164989 19149 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:11.157706 18958 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.097s	user 0.004s	sys 0.013s
I20260709 13:06:11.158927 19024 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:11.189172 19102 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.191018 19198 raft_consensus.cc:740] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:06:11.204950 19255 log.cc:927] Time spent T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.399s	user 0.000s	sys 0.003s
I20260709 13:06:11.209545 18957 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:11.209431 19099 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:11.211201 19204 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:11.213780 19092 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:11.079366 19225 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:11.236577 19206 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:11.237396 19233 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:11.239385 19246 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:11.241397 19247 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:11.244046 19053 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:11.246755 19161 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:11.258991 19244 consensus_queue.cc:260] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:11.254739 19201 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:11.286480 19119 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:11.287546 19266 log.cc:927] Time spent T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:06:11.290113 19234 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:11.316454 19244 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:11.328346 19222 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:11.300650 19204 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:11.310599 19191 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:11.326171 19214 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:11.326429 19228 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:11.270792 19229 consensus_queue.cc:260] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:11.335109 19229 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:06:10.949087 15759 rpcz_store.cc:269] 0709 13:05:56.948250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.949011 (+   761us) service_pool.cc:224] Handling call
0709 13:05:57.280686 (+331675us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b54003f3c54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea312090 00007f73ea3b0f3a 00007f73f01671cc 00007f73f026537e 00007f73ee7262a4 00007f73ee726232 00007f73ee726161 00007f73ee7260dc 00007f73ee725392 00007f73e86e348c 00007f73e86e28c6
0709 13:06:10.770002 (+13489316us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:02.707759 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:03.026666 (+318907us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:03.026722 (+    56us) write_op.cc:435] Acquired schema lock
0709 13:06:03.026752 (+    30us) tablet.cc:662] Decoding operations
0709 13:06:03.320554 (+293802us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:03.373065 (+ 52511us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:03.439128 (+ 66063us) tablet.cc:685] Acquiring locks for 17 operations
0709 13:06:03.685473 (+246345us) tablet.cc:701] Row locks acquired
0709 13:06:03.685541 (+    68us) write_op.cc:260] PREPARE: finished
0709 13:06:03.685794 (+   253us) write_op.cc:270] Start()
0709 13:06:03.685989 (+   195us) write_op.cc:276] Timestamp: P: 1783602363685699 usec, L: 0
0709 13:06:03.686028 (+    39us) op_driver.cc:348] REPLICATION: starting
0709 13:06:03.960116 (+274088us) log.cc:844] Serialized 6701 byte log entry
0709 13:06:04.891050 (+930934us) op_driver.cc:464] REPLICATION: finished
0709 13:06:05.612519 (+721469us) write_op.cc:301] APPLY: starting
0709 13:06:05.612634 (+   115us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:05.613117 (+   483us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:05.613290 (+   173us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:05.692839 (+ 79549us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:05.763292 (+ 70453us) spinlock_profiling.cc:243] Waited 33.2 ms on lock 0x7b54003f3c54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a3e28 00007f73eddde581 00007f73eddddd58 00007f73edde29f1 00007f73edde29a9 00007f73edde2939 00007f73edde2901 00007f73edde1b2d 000000000038f354 000000000038f199
0709 13:06:05.870100 (+106808us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=17
0709 13:06:05.876028 (+  5928us) write_op.cc:312] APPLY: finished
0709 13:06:06.039327 (+163299us) log.cc:844] Serialized 119 byte log entry
0709 13:06:06.265671 (+226344us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:06.290664 (+ 24993us) write_op.cc:454] Released schema lock
0709 13:06:06.310024 (+ 19360us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:06.468794 (+158770us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:06.468827 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1003597,"reported_tablets":38,"spinlock_wait_cycles":244878720,"child_traces":[["op",{"apply.queue_time_us":721137,"apply.run_cpu_time_us":15990,"apply.run_wall_time_us":763982,"num_ops":17,"prepare.queue_time_us":255118,"prepare.run_cpu_time_us":44497,"prepare.run_wall_time_us":3076817,"raft.queue_time_us":236689,"raft.run_cpu_time_us":55948,"raft.run_wall_time_us":169211,"replication_time_us":1204835,"spinlock_wait_cycles":76330240,"thread_start_us":623537,"threads_started":5,"wal-append.queue_time_us":1083026,"wal-append.run_cpu_time_us":42201,"wal-append.run_wall_time_us":2284545}]]}
I20260709 13:06:10.861007 15757 catalog_manager.cc:5697] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 2 to 5, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 5 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:06:11.366135 15857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 1924) took 1779 ms. Trace:
I20260709 13:06:10.998404 15825 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:11.366930 15757 catalog_manager.cc:5697] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 3 to 5, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 5 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:06:11.372884 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:11.373813 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:11.366914 15857 rpcz_store.cc:276] 0709 13:06:09.586843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:09.586974 (+   131us) service_pool.cc:224] Handling call
0709 13:06:09.658640 (+ 71666us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:10.639218 (+980578us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:10.639256 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:10.647148 (+  7892us) log.cc:844] Serialized 26 byte log entry
0709 13:06:10.769955 (+122807us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:10.769993 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:10.975507 (+205514us) spinlock_profiling.cc:243] Waited 964 ms on lock 0x7b540032e954. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:10.975555 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:11.364954 (+389399us) raft_consensus.cc:1685] finished
0709 13:06:11.365038 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:11.365725 (+   687us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16391,"raft.queue_time_us":17006,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":7942,"spinlock_wait_cycles":2216184704,"thread_start_us":66729,"threads_started":1,"wal-append.queue_time_us":170309}
I20260709 13:06:11.011126 15859 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:11.374440 15757 catalog_manager.cc:5697] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 2 to 4, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 4 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:06:10.885072 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 110819us
I20260709 13:06:10.956749 15854 raft_consensus.cc:2393] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 8: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:10.996773 15856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1820) took 2490 ms. Trace:
W20260709 13:06:11.382820 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 374977us
I20260709 13:06:11.381472 15856 rpcz_store.cc:276] 0709 13:06:08.506418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.713601 (+207183us) service_pool.cc:224] Handling call
0709 13:06:08.714028 (+   427us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:09.073564 (+359536us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:09.073594 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:09.074130 (+   536us) log.cc:844] Serialized 52 byte log entry
0709 13:06:09.452261 (+378131us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:10.994500 (+1542239us) log.cc:844] Serialized 14 byte log entry
0709 13:06:10.996462 (+  1962us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:10.996552 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:10.996590 (+    38us) raft_consensus.cc:1685] finished
0709 13:06:10.996602 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:10.996721 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":360830,"prepare.queue_time_us":40155,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":390071,"raft.run_cpu_time_us":262,"raft.run_wall_time_us":27345,"thread_start_us":1881135,"threads_started":2,"wal-append.queue_time_us":415175}
I20260709 13:06:11.382162 15902 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:11.043427 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:11.396880 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 435300us
I20260709 13:06:11.400596 15757 catalog_manager.cc:5697] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 2 to 3, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 3 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:11.404227 15757 catalog_manager.cc:5697] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 3 to 5, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 5 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:11.407122 15757 catalog_manager.cc:5697] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 3 to 5, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 5 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:06:11.408140 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 450432us
I20260709 13:06:11.411201 15757 catalog_manager.cc:5697] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:11.419297 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:11.420305 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:11.421311 15857 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:11.421823 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:11.417074 15757 catalog_manager.cc:5697] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:11.107931 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:11.150393 16002 raft_consensus.cc:1240] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:06:11.445010 16563 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:11.281061 19102 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:11.200857 18728 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.426236 16006 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
I20260709 13:06:11.426296 16004 raft_consensus.cc:1240] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:10.813009 16011 raft_consensus.cc:2393] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 9: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:11.450546 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:11.450958 19161 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.150581 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:11.461645 16561 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:06:11.151705 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:11.323274 19198 consensus_queue.cc:260] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:11.464603 16562 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:06:11.466174 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:11.466516 15902 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.103.4:43643 timed out after 1.937s (SENT) [suppressed 15 similar messages]
I20260709 13:06:11.467103 16562 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
I20260709 13:06:11.155324 16565 raft_consensus.cc:2410] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:06:11.469669 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:11.470685 16565 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:06:10.867825 16566 rpcz_store.cc:276] 0709 13:06:08.933667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.939761 (+  6094us) service_pool.cc:224] Handling call
0709 13:06:10.466177 (+1526416us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b5400724554. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:10.466404 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":239451520}
I20260709 13:06:11.473140 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:06:11.474648 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 476133us
I20260709 13:06:11.475171 19222 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:11.476238 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:11.476650 19228 raft_consensus.cc:2749] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:11.477555 16003 raft_consensus.cc:2410] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:06:10.856231 16567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 719) took 1921 ms. Trace:
I20260709 13:06:11.536862 15859 raft_consensus.cc:1275] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 13:06:11.538033 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:11.538360 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:11.539338 15854 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 1928) took 1655 ms (client timeout 2193 ms). Trace:
W20260709 13:06:11.539824 15854 rpcz_store.cc:269] 0709 13:06:09.884315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:09.885297 (+   982us) service_pool.cc:224] Handling call
0709 13:06:11.538970 (+1653673us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b54003b5e54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:11.539167 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":611473664}
I20260709 13:06:11.540385 15976 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:11.541126 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:11.541662 15976 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:11.541779 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:11.542791 16009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1929) took 2450 ms. Trace:
I20260709 13:06:11.544404 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:11.545512 15862 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:11.158267 16568 raft_consensus.cc:2393] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 7: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:11.546464 19075 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:11.546626 18970 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:11.274874 18958 raft_consensus.cc:697] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:11.310053 19092 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:11.278900 19149 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:11.451736 16011 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
I20260709 13:06:11.265234 18737 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:11.463467 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:11.652873 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:11.653649 16561 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:06:11.475245 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:11.476799 15857 raft_consensus.cc:1240] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:11.499944 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:11.500113 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:11.500821 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:11.501645 16569 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:11.501087 15757 catalog_manager.cc:5697] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 1 to 6, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 6 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:11.508049 19136 consensus_queue.cc:1048] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:11.476300 15825 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:11.516325 15827 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:11.517994 18728 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:11.527112 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:11.463347 18994 consensus_queue.cc:1059] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 13:06:11.663268 18994 raft_consensus.cc:3055] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:11.663777 18994 raft_consensus.cc:740] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:11.528872 18932 consensus_queue.cc:1048] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:11.532956 19204 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:11.466434 19198 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:11.534475 19161 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:11.671097 16003 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:11.535372 19223 raft_consensus.cc:2749] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:11.542136 15854 raft_consensus.cc:2393] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 6: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:11.544514 19273 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:06:11.545619 16008 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.001s	sys 0.017s
I20260709 13:06:11.262777 19016 raft_consensus.cc:697] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 6 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:11.675767 16002 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:11.677115 19016 consensus_queue.cc:237] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:11.239928 18792 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:11.686196 18792 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:11.276933 19024 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:11.687137 19273 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:11.688242 19024 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:11.690572 19273 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:11.536278 16567 rpcz_store.cc:276] 0709 13:06:08.935106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:10.855526 (+1920420us) service_pool.cc:224] Handling call
0709 13:06:10.855838 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:10.856016 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:10.856048 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:10.856075 (+    27us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:10.856093 (+    18us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:10.856126 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:10.856192 (+    66us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:11.692116 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 219452us
I20260709 13:06:11.692663 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:11.693435 16567 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:11.543517 15976 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:11.553062 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:11.695940 19153 consensus_queue.cc:1048] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:06:11.697116 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:11.698609 18728 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:11.698990 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:11.543255 16009 rpcz_store.cc:276] 0709 13:06:09.092302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:09.092537 (+   235us) service_pool.cc:224] Handling call
0709 13:06:09.092810 (+   273us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:10.849270 (+1756460us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:11.542142 (+692872us) spinlock_profiling.cc:243] Waited 1.63 s on lock 0x7b540010d6d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:11.542546 (+   404us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3754179968}
I20260709 13:06:11.699318 19201 consensus_queue.cc:1048] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:06:11.700569 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:11.701260 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:11.701515 16009 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:06:11.702018 15857 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
I20260709 13:06:11.702641 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:11.547576 18970 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:11.563124 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:11.554639 19228 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:11.705715 18970 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:11.706336 18970 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:11.705986 19228 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:11.532021 19156 raft_consensus.cc:2749] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:11.476647 19222 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.088s
I20260709 13:06:11.557575 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:11.570297 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 103807us
I20260709 13:06:11.615208 15856 raft_consensus.cc:1240] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:06:11.621755 18958 consensus_queue.cc:237] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:11.623175 19133 raft_consensus.cc:2749] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:11.632508 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:11.637109 19078 raft_consensus.cc:3055] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:11.639580 19092 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:11.752410 19078 raft_consensus.cc:740] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:11.639946 19279 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.753937 19279 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:11.754241 19078 consensus_queue.cc:260] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:11.646626 19149 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:11.623647 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:11.662415 16566 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
W20260709 13:06:11.663168 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 187206us
W20260709 13:06:11.531977 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 354098us
I20260709 13:06:11.679970 16565 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:06:11.671643 19161 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:11.683161 16562 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:11.675815 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:11.715920 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:11.700595 19214 raft_consensus.cc:2749] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:11.713532 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:11.691323 15757 catalog_manager.cc:5697] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 2 to 6, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 6 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:06:11.722808 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:11.707087 18970 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:11.728339 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 184984us
I20260709 13:06:11.728668 19129 raft_consensus.cc:2804] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:11.729070 18861 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:11.710553 19136 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:11.777355 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:11.777832 19136 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:11.732128 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:11.780161 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:11.780558 15825 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:11.732915 19273 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:11.781392 19273 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:11.733604 19133 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:11.784247 15854 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:11.738210 19228 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:11.739795 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:11.743109 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:11.744867 15860 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
I20260709 13:06:11.745195 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:11.745716 15859 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
W20260709 13:06:11.789034 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:11.750770 16567 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:11.711866 16568 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:06:11.644085 19099 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:11.790438 19099 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:11.760325 16002 raft_consensus.cc:2410] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:06:11.773893 19095 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.764555 18994 consensus_queue.cc:260] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:11.775163 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:11.769440 19129 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:11.780128 19094 consensus_meta.cc:287] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.511s	user 0.010s	sys 0.005s
I20260709 13:06:11.802497 19024 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:11.784116 19133 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:11.805091 18970 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:11.805423 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:11.781364 16009 raft_consensus.cc:2410] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:06:11.788920 16565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 721) took 2749 ms. Trace:
I20260709 13:06:11.806806 16565 rpcz_store.cc:276] 0709 13:06:09.019442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:11.759100 (+2739658us) service_pool.cc:224] Handling call
0709 13:06:11.759562 (+   462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:11.759917 (+   355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:11.759948 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:11.762675 (+  2727us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:11.768379 (+  5704us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:11.768529 (+   150us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:11.768688 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:11.809052 16565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 879) took 2485 ms. Trace:
I20260709 13:06:11.810134 16004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1821) took 1098 ms. Trace:
I20260709 13:06:11.810528 16004 rpcz_store.cc:276] 0709 13:06:10.711892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:11.106939 (+395047us) service_pool.cc:224] Handling call
0709 13:06:11.107331 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:11.448187 (+340856us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:11.809808 (+361621us) spinlock_profiling.cc:243] Waited 318 ms on lock 0x7b54001ba154. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:11.810046 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":731995520}
I20260709 13:06:11.811436 19238 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.812351 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:11.798640 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:11.800455 15858 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:06:11.790835 15757 catalog_manager.cc:5697] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: config changed from index 2 to 3, term changed from 1 to 4, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:06:11.809388 16565 rpcz_store.cc:276] 0709 13:06:09.323289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:11.808022 (+2484733us) service_pool.cc:224] Handling call
0709 13:06:11.808393 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:11.808667 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:11.808693 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:11.808752 (+    59us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:11.808786 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:11.808848 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:11.808977 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:11.802709 19165 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.734287 19263 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.823868 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:11.790831 19099 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:11.789696 19273 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:11.795480 18998 consensus_meta.cc:287] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.687s	user 0.007s	sys 0.007s
I20260709 13:06:11.790151 19078 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:11.826421 15825 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:11.825867 16562 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:06:11.802541 18994 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:11.845217 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:11.845022 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:11.817549 16568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 878) took 2706 ms. Trace:
I20260709 13:06:11.812134 19238 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:11.863878 19263 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:11.866267 19238 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:11.813648 18861 raft_consensus.cc:2749] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:11.814034 19129 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:11.851816 19095 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:11.853160 15933 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6.
I20260709 13:06:11.856647 19161 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:11.856981 15936 raft_consensus.cc:1240] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 7 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:11.858111 16005 raft_consensus.cc:2410] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:06:11.858037 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:11.860378 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:11.860658 16564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 718) took 2925 ms. Trace:
I20260709 13:06:11.873567 16563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45232 (request call id 725) took 1313 ms. Trace:
I20260709 13:06:11.861322 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:11.861398 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:11.862098 16569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45232 (request call id 724) took 1436 ms. Trace:
I20260709 13:06:11.863214 19136 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:11.873970 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 78655.4us
I20260709 13:06:11.873947 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:11.886574 16565 raft_consensus.cc:1240] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:11.873111 15929 raft_consensus.cc:1240] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 6. Current term is 7. Ops: []
W20260709 13:06:11.887161 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 53982.3us
I20260709 13:06:11.872956 16562 raft_consensus.cc:1240] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:11.876066 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:11.846937 16568 rpcz_store.cc:276] 0709 13:06:09.101739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:11.806583 (+2704844us) service_pool.cc:224] Handling call
0709 13:06:11.806991 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:11.807311 (+   320us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:11.807334 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:11.807405 (+    71us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:11.807466 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:11.807527 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:11.807639 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:11.874686 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:11.874374 15934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 648) took 2865 ms. Trace:
I20260709 13:06:11.888404 16003 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:06:11.878300 15857 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:06:11.889097 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:11.888151 15858 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:11.871572 16008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1818) took 1173 ms. Trace:
I20260709 13:06:11.805092 19133 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:11.844342 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:11.884649 16563 rpcz_store.cc:276] 0709 13:06:10.559800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:10.578046 (+ 18246us) service_pool.cc:224] Handling call
0709 13:06:11.873342 (+1295296us) spinlock_profiling.cc:243] Waited 750 ms on lock 0x7b54006928d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:11.873481 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1725567744}
I20260709 13:06:11.883842 16564 rpcz_store.cc:276] 0709 13:06:08.934834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:09.311058 (+376224us) service_pool.cc:224] Handling call
0709 13:06:09.359523 (+ 48465us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:11.171283 (+1811760us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:11.860191 (+688908us) spinlock_profiling.cc:243] Waited 1.81 s on lock 0x7b5400545e54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:11.860490 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4162159360}
I20260709 13:06:11.894614 15928 raft_consensus.cc:2393] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 6: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:11.888360 15934 rpcz_store.cc:276] 0709 13:06:09.008472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:09.008616 (+   144us) service_pool.cc:224] Handling call
0709 13:06:09.023095 (+ 14479us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:10.834658 (+1811563us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:11.873905 (+1039247us) spinlock_profiling.cc:243] Waited 1.81 s on lock 0x7b5400327b54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:11.874197 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4164723456}
I20260709 13:06:11.894909 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:11.830423 19222 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:11.895217 16562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 762) took 2241 ms. Trace:
I20260709 13:06:11.885195 16569 rpcz_store.cc:276] 0709 13:06:10.425529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:10.652594 (+227065us) service_pool.cc:224] Handling call
0709 13:06:11.861914 (+1209320us) spinlock_profiling.cc:243] Waited 625 ms on lock 0x7b54000ce754. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:11.862029 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1438529408}
I20260709 13:06:11.830798 19165 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:11.898144 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:11.898245 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:11.866267 19263 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:11.895991 16565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 761) took 2243 ms. Trace:
I20260709 13:06:11.917598 16565 rpcz_store.cc:276] 0709 13:06:09.647290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:11.820220 (+2172930us) service_pool.cc:224] Handling call
0709 13:06:11.881533 (+ 61313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:11.890774 (+  9241us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:11.891015 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:11.900655 16562 rpcz_store.cc:276] 0709 13:06:09.647628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:11.872383 (+2224755us) service_pool.cc:224] Handling call
0709 13:06:11.872753 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:11.889201 (+ 16448us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:11.889362 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:11.904724 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:11.906275 16003 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:06:11.899394 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:11.907850 16564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 765) took 1215 ms. Trace:
I20260709 13:06:11.909037 15931 raft_consensus.cc:1240] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:11.908936 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:11.911278 15859 raft_consensus.cc:1240] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:11.911952 15937 raft_consensus.cc:2368] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 7. Current term is 8.
I20260709 13:06:11.912058 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:11.912755 16007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 589) took 1311 ms. Trace:
I20260709 13:06:11.912811 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:11.914412 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:11.912016 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:11.912655 16002 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:06:11.875121 19212 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:11.921329 16564 rpcz_store.cc:276] 0709 13:06:10.692972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:11.906795 (+1213823us) service_pool.cc:224] Handling call
0709 13:06:11.907149 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:11.907466 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:11.907492 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:11.907534 (+    42us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:11.907602 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:11.907657 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:11.907796 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:11.882346 18732 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:11.889932 18861 raft_consensus.cc:2749] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:11.929708 19222 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:11.930763 19099 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:11.931728 15932 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:11.935642 16565 raft_consensus.cc:1240] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:11.891315 16008 rpcz_store.cc:276] 0709 13:06:10.698026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:10.710610 (+ 12584us) service_pool.cc:224] Handling call
0709 13:06:10.737149 (+ 26539us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:10.986354 (+249205us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:06:10.994280 (+  7926us) log.cc:844] Serialized 14 byte log entry
0709 13:06:11.680152 (+685872us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:11.683045 (+  2893us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:11.683153 (+   108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:11.871274 (+188121us) spinlock_profiling.cc:243] Waited 249 ms on lock 0x7b54001dcc54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:11.871327 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:11.871488 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13912,"prepare.queue_time_us":8210,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":8214,"raft.queue_time_us":28425,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":2688,"spinlock_wait_cycles":572410240,"thread_start_us":679262,"threads_started":1,"wal-append.queue_time_us":283221}
I20260709 13:06:11.922628 16007 rpcz_store.cc:276] 0709 13:06:10.601097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:10.601241 (+   144us) service_pool.cc:224] Handling call
0709 13:06:10.601599 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:11.430232 (+828633us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:11.430319 (+    87us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:11.431481 (+  1162us) log.cc:844] Serialized 26 byte log entry
0709 13:06:11.608475 (+176994us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:11.608526 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:11.912249 (+303723us) spinlock_profiling.cc:243] Waited 827 ms on lock 0x7b54003edd54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:11.912295 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:11.912359 (+    64us) raft_consensus.cc:1685] finished
0709 13:06:11.912371 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:11.912600 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":30268,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":281,"spinlock_wait_cycles":1902914816,"thread_start_us":60800,"threads_started":1,"wal-append.queue_time_us":205540}
I20260709 13:06:11.942194 15976 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:11.937314 16005 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60156 (request call id 586) took 1405 ms (client timeout 1557 ms). Trace:
I20260709 13:06:11.938055 15933 raft_consensus.cc:2468] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
W20260709 13:06:11.943075 16005 rpcz_store.cc:269] 0709 13:06:10.532243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:10.532375 (+   132us) service_pool.cc:224] Handling call
0709 13:06:11.937032 (+1404657us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54001f7cd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:11.937214 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2685034368}
W20260709 13:06:11.945123 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:11.941448 15825 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:11.941061 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:11.937367 16010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1819) took 1209 ms. Trace:
W20260709 13:06:11.948084 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 60267.9us
I20260709 13:06:11.948997 16565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 881) took 1217 ms. Trace:
I20260709 13:06:11.949541 16565 rpcz_store.cc:276] 0709 13:06:10.731406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:11.934848 (+1203442us) service_pool.cc:224] Handling call
0709 13:06:11.935343 (+   495us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:11.948665 (+ 13322us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:11.948918 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:11.896710 19279 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:11.956789 15928 raft_consensus.cc:2410] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:06:11.958132 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:11.959052 15928 raft_consensus.cc:2393] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 7: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:11.960464 19024 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:11.956704 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:11.963845 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:11.968358 19273 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:11.969019 19273 raft_consensus.cc:697] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:11.970263 19273 consensus_queue.cc:237] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:11.972513 19222 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:11.973742 19242 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.974545 19242 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:11.977815 19103 consensus_queue.cc:1048] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:06:11.980113 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 145262us
I20260709 13:06:11.948284 16010 rpcz_store.cc:276] 0709 13:06:10.711243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:10.711389 (+   146us) service_pool.cc:224] Handling call
0709 13:06:10.711789 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:10.861310 (+149521us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:10.861341 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:10.869227 (+  7886us) log.cc:844] Serialized 26 byte log entry
0709 13:06:11.010504 (+141277us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:11.010558 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:11.875588 (+865030us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b54001b9754. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:11.875692 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:11.875778 (+    86us) raft_consensus.cc:1685] finished
0709 13:06:11.875796 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:11.920093 (+ 44297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":343321856,"thread_start_us":118932,"threads_started":1,"wal-append.queue_time_us":121172}
I20260709 13:06:11.989835 15827 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:11.991415 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:11.992117 15937 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6.
W20260709 13:06:11.993620 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 68942.1us
I20260709 13:06:11.994781 15825 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:11.995857 19024 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:11.996026 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:11.996673 19024 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:11.997608 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:11.904181 19161 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:11.996752 15936 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:12.013573 19161 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:12.015748 19153 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:12.015951 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:06:12.043862 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 297127us
W20260709 13:06:12.052743 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.017277 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:11.967656 19290 log.cc:927] Time spent T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:06:11.863153 18952 consensus_queue.cc:260] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:12.021495 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.025393 15931 raft_consensus.cc:1240] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:12.026432 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:12.080376 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:12.125321 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:12.125893 15937 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:06:12.126082 15900 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:12.126473 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:12.126967 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:12.127257 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:12.127789 15937 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:12.128356 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:12.129771 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:12.132771 19099 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:12.152603 19024 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:06:12.088408 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 70407.3us
I20260709 13:06:12.123885 15858 raft_consensus.cc:1275] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:06:12.136755 15935 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:12.124455 15933 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
I20260709 13:06:12.145285 16565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 882) took 1402 ms. Trace:
W20260709 13:06:12.123889 19287 log.cc:927] Time spent T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:06:12.153640 16008 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:12.157491 19210 raft_consensus.cc:2804] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:12.158179 19164 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.159430 19149 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.165555 19136 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:06:12.173339 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 183603us
I20260709 13:06:12.135419 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:12.199223 15932 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 7.
W20260709 13:06:12.204145 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.218899 15931 raft_consensus.cc:2393] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 5: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:12.219928 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:12.084793 19161 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:12.209043 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:12.208940 16565 rpcz_store.cc:276] 0709 13:06:10.736567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:11.950444 (+1213877us) service_pool.cc:224] Handling call
0709 13:06:12.010245 (+ 59801us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:12.085141 (+ 74896us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:12.085187 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:12.121731 (+ 36544us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:12.138750 (+ 17019us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:12.138838 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:12.138977 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:12.210942 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 65075.8us
I20260709 13:06:12.144515 18952 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:12.085234 19153 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:12.236109 15825 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:12.237854 15825 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:12.238974 18861 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:12.227319 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:12.240273 15936 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:06:12.239542 15825 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:12.245621 16565 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:12.218004 19136 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:12.251304 16008 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 7.
I20260709 13:06:12.256387 16569 raft_consensus.cc:2410] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:06:12.259058 19136 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:12.261758 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:12.262636 15858 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:12.287130 15934 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
I20260709 13:06:12.288074 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:12.288276 15827 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:12.273727 15902 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:12.294854 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:06:12.298014 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 85495.2us
I20260709 13:06:12.298234 15933 raft_consensus.cc:1275] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:06:12.298727 15933 pending_rounds.cc:85] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 2
I20260709 13:06:12.299969 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:12.239950 18861 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:12.286303 19095 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:12.299002 15933 pending_rounds.cc:107] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:06:12.309901 15933 raft_consensus.cc:2889] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:12.313620 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:12.313159 15902 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:12.253082 19149 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:12.315069 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.315881 19222 raft_consensus.cc:2804] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:12.316370 19222 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:12.316744 19222 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:12.317989 19149 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:12.320659 15902 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:12.322454 19238 raft_consensus.cc:2749] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:12.324456 15902 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:12.332782 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:12.333006 15853 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:12.340138 16569 raft_consensus.cc:2410] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 9 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:06:12.304859 18732 raft_consensus.cc:2804] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:12.254663 19161 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:12.340747 16565 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:12.339285 15974 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:12.308478 18861 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:12.257094 19164 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
W20260709 13:06:12.352414 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 112968us
I20260709 13:06:12.358382 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:12.359076 16569 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:12.359656 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:12.360535 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 548823us
I20260709 13:06:12.361050 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:12.361780 16533 leader_election.cc:336] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 2.873s (SENT)
I20260709 13:06:12.362316 16533 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:12.363781 19229 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:12.363914 19279 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:12.364251 19229 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:12.364776 19229 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:12.369699 19153 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:12.377071 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.377511 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:12.346396 18732 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:12.382926 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 91839.3us
I20260709 13:06:12.352540 19238 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:12.354718 18861 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:12.395807 16568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 723) took 1996 ms. Trace:
W20260709 13:06:12.402014 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.396296 16568 rpcz_store.cc:276] 0709 13:06:10.425083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:11.904349 (+1479266us) service_pool.cc:224] Handling call
0709 13:06:11.936527 (+ 32178us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:11.936987 (+   460us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:11.937030 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:11.937503 (+   473us) log.cc:844] Serialized 26 byte log entry
0709 13:06:12.302648 (+365145us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:12.302695 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:12.302768 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:12.395521 (+ 92753us) raft_consensus.cc:1685] finished
0709 13:06:12.395541 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:12.395716 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1676,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":79,"thread_start_us":363915,"threads_started":1,"wal-append.queue_time_us":456152}
I20260709 13:06:12.389894 18732 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:12.413316 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:12.414251 15902 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:12.414992 19165 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:12.415735 18948 consensus_queue.cc:1048] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:12.416990 19164 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:12.420997 19136 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:12.395871 19099 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:06:12.424305 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.425311 15827 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:12.440755 15902 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:12.440989 15827 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:12.427578 16566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 876) took 3335 ms. Trace:
I20260709 13:06:12.437563 15825 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:12.396385 19238 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:12.421002 19129 consensus_meta.cc:287] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.553s	user 0.007s	sys 0.003s
I20260709 13:06:12.442849 15858 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:06:12.446059 16565 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:06:12.441874 16566 rpcz_store.cc:276] 0709 13:06:09.092392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:11.759589 (+2667197us) service_pool.cc:224] Handling call
0709 13:06:11.762651 (+  3062us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:11.794709 (+ 32058us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:11.794752 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:11.876978 (+ 82226us) log.cc:844] Serialized 26 byte log entry
0709 13:06:12.304641 (+427663us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:12.304689 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:12.304761 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:12.427360 (+122599us) raft_consensus.cc:1685] finished
0709 13:06:12.427379 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:12.427508 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52055,"raft.queue_time_us":16421,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":26180,"thread_start_us":398537,"threads_started":1,"wal-append.queue_time_us":531215}
I20260709 13:06:12.445312 19238 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:12.460189 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 76025.7us
I20260709 13:06:12.462757 19223 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:12.463227 15860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 1829) took 1044 ms. Trace:
I20260709 13:06:12.463737 15902 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:12.463642 15860 rpcz_store.cc:276] 0709 13:06:11.418883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:11.421274 (+  2391us) service_pool.cc:224] Handling call
0709 13:06:12.463070 (+1041796us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b540032d554. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:12.463170 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":741078784}
I20260709 13:06:12.465659 19149 raft_consensus.cc:2749] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:12.466660 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:12.466586 19238 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:12.467907 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:12.469383 19238 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:12.469496 16565 raft_consensus.cc:1275] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:06:12.470207 16565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 726) took 1514 ms. Trace:
I20260709 13:06:12.470507 15827 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:12.470558 16565 rpcz_store.cc:276] 0709 13:06:10.956332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:12.468821 (+1512489us) service_pool.cc:224] Handling call
0709 13:06:12.469168 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:12.469972 (+   804us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:12.470157 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:12.473119 16565 raft_consensus.cc:1240] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:12.473773 16565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 727) took 1517 ms. Trace:
I20260709 13:06:12.474073 16565 rpcz_store.cc:276] 0709 13:06:10.956615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:12.471646 (+1515031us) service_pool.cc:224] Handling call
0709 13:06:12.472033 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:12.473475 (+  1442us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:12.473693 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:12.476263 16565 raft_consensus.cc:1275] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
W20260709 13:06:12.476320 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 250273us
I20260709 13:06:12.468061 16006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41662 (request call id 1939) took 1703 ms. Trace:
I20260709 13:06:12.466508 16563 raft_consensus.cc:1240] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:12.478157 16563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 885) took 1714 ms. Trace:
I20260709 13:06:12.467612 16008 raft_consensus.cc:2393] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 8: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:12.478535 16563 rpcz_store.cc:276] 0709 13:06:10.764163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:12.465942 (+1701779us) service_pool.cc:224] Handling call
0709 13:06:12.466303 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:12.477958 (+ 11655us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:12.478111 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:12.462591 15825 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:12.477227 16006 rpcz_store.cc:276] 0709 13:06:10.764668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:10.845979 (+ 81311us) service_pool.cc:224] Handling call
0709 13:06:12.467854 (+1621875us) spinlock_profiling.cc:243] Waited 272 ms on lock 0x7b54000ee2d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:12.467991 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":626647936}
I20260709 13:06:12.442981 19099 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:06:12.456459 19078 consensus_meta.cc:287] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.612s	user 0.013s	sys 0.000s
I20260709 13:06:12.446616 19212 raft_consensus.cc:2804] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:12.448351 19016 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:12.457469 19164 raft_consensus.cc:2804] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:12.487637 16565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 886) took 1011 ms. Trace:
I20260709 13:06:12.493145 15974 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:12.506048 16565 rpcz_store.cc:276] 0709 13:06:11.470401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:12.475417 (+1005016us) service_pool.cc:224] Handling call
0709 13:06:12.475888 (+   471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:12.476660 (+   772us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:12.481970 (+  5310us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:12.450366 19153 raft_consensus.cc:2749] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:12.492739 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:12.496927 15853 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:06:12.521567 15900 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:12.462548 19222 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:12.526665 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 44577.7us
I20260709 13:06:12.485997 19099 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:12.492749 19024 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:12.531067 16566 raft_consensus.cc:1240] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:12.529524 19149 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.493393 19212 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:12.450206 19129 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:12.520565 19164 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:12.538162 16566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 884) took 1776 ms. Trace:
I20260709 13:06:12.549711 16566 rpcz_store.cc:276] 0709 13:06:10.761760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:12.448762 (+1687002us) service_pool.cc:224] Handling call
0709 13:06:12.519306 (+ 70544us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:12.537820 (+ 18514us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:12.538090 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:12.533269 19149 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:12.546290 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.548332 16563 raft_consensus.cc:1275] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:06:12.555167 16563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 887) took 1084 ms. Trace:
W20260709 13:06:12.533281 19297 log.cc:927] Time spent T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:06:12.529498 19273 raft_consensus.cc:2804] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:12.555650 16563 rpcz_store.cc:276] 0709 13:06:11.470794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:12.479408 (+1008614us) service_pool.cc:224] Handling call
0709 13:06:12.503898 (+ 24490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:12.554794 (+ 50896us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:12.555095 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:12.559093 16566 raft_consensus.cc:1275] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:06:12.560884 16566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 770) took 1031 ms. Trace:
I20260709 13:06:12.561394 16566 rpcz_store.cc:276] 0709 13:06:11.529674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:12.550949 (+1021275us) service_pool.cc:224] Handling call
0709 13:06:12.551507 (+   558us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:12.560570 (+  9063us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:12.560817 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:12.534699 18994 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:12.564496 16004 raft_consensus.cc:1240] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 5. Current term is 6. Ops: []
W20260709 13:06:12.564692 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.545620 19212 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:12.566095 15900 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:12.568318 16565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 892) took 1027 ms. Trace:
I20260709 13:06:12.581911 16565 rpcz_store.cc:276] 0709 13:06:11.529216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:12.523637 (+994421us) service_pool.cc:224] Handling call
0709 13:06:12.529290 (+  5653us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:12.551485 (+ 22195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:12.551528 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:12.551591 (+    63us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:12.556751 (+  5160us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:12.556842 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:12.556951 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:12.553217 19164 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:12.594659 16567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 722) took 2195 ms. Trace:
I20260709 13:06:12.594772 19201 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:12.605015 19201 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:12.605536 19201 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:12.604898 16567 rpcz_store.cc:276] 0709 13:06:10.399431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:11.895149 (+1495718us) service_pool.cc:224] Handling call
0709 13:06:11.939622 (+ 44473us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:11.939940 (+   318us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:12.301300 (+361360us) log.cc:844] Serialized 14 byte log entry
0709 13:06:12.594307 (+293007us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:12.594360 (+    53us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:12.594412 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:12.594474 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:12.594596 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":488037,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":10002,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":115,"thread_start_us":598517,"threads_started":2}
I20260709 13:06:12.597617 19197 consensus_queue.cc:1048] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:12.596405 19222 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:12.603238 15974 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:12.612197 19136 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:12.612334 18952 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:12.613161 18952 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:12.613592 18952 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:12.616778 15974 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:12.612491 16569 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:06:12.617986 18968 raft_consensus.cc:2804] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:12.620857 18968 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:12.621234 18968 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:12.564433 19016 raft_consensus.cc:2804] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:12.626739 19016 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:12.627249 19016 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:12.559828 19273 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:12.636569 16533 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:12.643787 16563 raft_consensus.cc:1275] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:06:12.581287 18948 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.629753 19161 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:12.636109 19273 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:12.611498 19229 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:12.673993 16569 raft_consensus.cc:1275] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:06:12.675901 16570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 720) took 3661 ms. Trace:
I20260709 13:06:12.676349 19149 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:12.676324 16570 rpcz_store.cc:276] 0709 13:06:09.014704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:11.214430 (+2199726us) service_pool.cc:224] Handling call
0709 13:06:11.214717 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:11.528153 (+313436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:11.528215 (+    62us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:11.528293 (+    78us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:11.528344 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:12.672372 (+1144028us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b5400102554. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:12.672417 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:12.675689 (+  3272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":522945408}
I20260709 13:06:12.678071 15974 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
W20260709 13:06:12.680823 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 111291us
I20260709 13:06:12.660424 19165 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.703050 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:12.704006 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:12.703811 19165 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:12.710175 15932 raft_consensus.cc:2393] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 7: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:12.704169 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:12.703276 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:12.706295 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:12.685969 18958 raft_consensus.cc:2804] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:12.716768 18958 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:12.716979 19129 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:12.717306 18958 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:12.716662 18952 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:12.717681 19129 raft_consensus.cc:2804] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:12.718070 19129 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:12.718391 19129 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:12.719218 18952 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:12.721992 16533 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:12.723903 19078 consensus_queue.cc:1059] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:12.726115 18958 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:12.723280 19279 consensus_queue.cc:1059] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 13:06:12.727056 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:12.729756 15933 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 5.
I20260709 13:06:12.722199 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:12.657822 18948 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:12.732519 15937 raft_consensus.cc:2410] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:06:12.735001 15931 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:12.703394 19229 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:12.753028 18948 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:12.753981 18861 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:12.728953 19136 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:12.765039 15854 raft_consensus.cc:2393] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 8: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:12.770119 18958 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:12.770555 18861 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:12.771178 18958 raft_consensus.cc:2804] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:12.771643 18958 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:12.772156 18958 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:12.776000 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.780408 19016 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:06:12.783412 19016 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:12.789489 19165 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:12.793085 19201 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:12.803190 19201 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:12.796985 18968 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:12.806574 18968 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:12.811940 19136 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:12.814841 19136 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:12.844316 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:12.884321 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 413891us
W20260709 13:06:12.898643 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.898717 15827 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:12.914258 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:12.915278 19099 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:12.915645 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:12.917150 16008 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:12.915323 16010 raft_consensus.cc:1240] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:12.903771 19016 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:06:12.937104 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.932217 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:12.942735 15827 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:12.943485 16006 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 7.
I20260709 13:06:12.916731 18861 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:12.948952 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:12.890372 18732 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:12.946669 15861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 1942) took 1411 ms. Trace:
I20260709 13:06:12.954702 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:12.955576 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:12.957705 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:06:12.960601 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:12.962764 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 500224us
W20260709 13:06:12.961761 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 283737us
I20260709 13:06:12.962478 16005 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 7.
I20260709 13:06:12.949602 19210 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:06:12.965920 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 244039us
I20260709 13:06:12.966476 15974 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:12.952399 15861 rpcz_store.cc:276] 0709 13:06:11.534967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:11.543099 (+  8132us) service_pool.cc:224] Handling call
0709 13:06:11.606502 (+ 63403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:11.644009 (+ 37507us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:06:12.304926 (+660917us) log.cc:844] Serialized 14 byte log entry
0709 13:06:12.946331 (+641405us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:12.946396 (+    65us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:12.946449 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:12.946514 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:12.946620 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":318830,"prepare.queue_time_us":319905,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":97859,"raft.run_cpu_time_us":55,"raft.run_wall_time_us":61,"thread_start_us":938275,"threads_started":3,"wal-append.queue_time_us":549473}
I20260709 13:06:12.965773 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:12.950824 19229 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:12.949719 18861 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:12.951086 19317 raft_consensus.cc:2749] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:06:12.970667 19223 consensus_meta.cc:287] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.506s	user 0.010s	sys 0.001s
I20260709 13:06:12.983489 15900 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:12.984639 19136 raft_consensus.cc:2804] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:12.985172 19136 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:12.985591 19136 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:12.985785 15900 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:12.987013 19201 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:12.987619 19201 raft_consensus.cc:697] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:12.988802 19201 consensus_queue.cc:237] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:12.992159 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.995923 15900 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:12.996923 19197 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:12.997488 19197 raft_consensus.cc:697] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:12.998345 19197 consensus_queue.cc:237] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:13.007130 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:13.014693 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:13.015429 16562 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:13.023856 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:06:13.021188 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:13.027594 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:06:13.029503 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 63089.5us
I20260709 13:06:13.029837 16533 leader_election.cc:400] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 election: Vote denied by peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) with higher term. Message: Invalid argument: T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 7. Current term is 8.
I20260709 13:06:13.030230 16533 leader_election.cc:403] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:06:13.030318 15974 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:13.030972 19223 raft_consensus.cc:2749] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:13.034893 19223 consensus_queue.cc:1048] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:06:13.036612 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:13.036394 19136 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:13.037330 19317 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:13.037905 15974 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:13.037952 19317 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:13.037385 16006 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 7: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:13.040010 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:13.040759 16008 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6.
I20260709 13:06:12.970852 19210 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:13.042953 19327 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:13.046164 19327 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.046728 19327 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:13.043166 19263 raft_consensus.cc:2804] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:13.039268 15974 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:12.972947 19165 raft_consensus.cc:2749] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:12.955027 19016 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:12.957636 18958 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:12.979902 18861 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:06:13.052881 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 80782.3us
I20260709 13:06:13.051807 19263 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:13.057065 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:13.057924 19078 raft_consensus.cc:3055] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:13.080193 19263 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:13.080564 19078 raft_consensus.cc:740] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:13.081699 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:13.081506 19078 consensus_queue.cc:260] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:13.082888 19078 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:06:13.059060 19314 log.cc:927] Time spent T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:06:13.084757 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:13.085337 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:13.085644 16009 raft_consensus.cc:2410] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:06:13.085758 16535 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:13.085983 16011 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 7.
I20260709 13:06:13.086555 18948 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:13.059187 19229 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:13.087872 19229 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:13.090354 16533 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:13.090502 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:13.091208 16570 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 8
W20260709 13:06:13.059798 19313 log.cc:927] Time spent T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 13:06:13.059949 16568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 883) took 2302 ms. Trace:
W20260709 13:06:13.060611 19312 log.cc:927] Time spent T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:06:13.060987 19099 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:13.092516 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:13.093406 16561 raft_consensus.cc:2410] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:06:13.093639 15935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 1836) took 1098 ms. Trace:
I20260709 13:06:13.094375 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:13.092103 16568 rpcz_store.cc:276] 0709 13:06:10.757642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:12.411055 (+1653413us) service_pool.cc:224] Handling call
0709 13:06:12.422222 (+ 11167us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:12.422668 (+   446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:12.422694 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:12.423153 (+   459us) log.cc:844] Serialized 26 byte log entry
0709 13:06:12.788382 (+365229us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:12.790306 (+  1924us) log.cc:844] Serialized 14 byte log entry
0709 13:06:12.790531 (+   225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:12.790628 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:13.059733 (+269105us) raft_consensus.cc:1685] finished
0709 13:06:13.059760 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:13.059895 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52581,"prepare.queue_time_us":1152,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":40,"raft.queue_time_us":180752,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":6724,"thread_start_us":364807,"threads_started":1,"wal-append.queue_time_us":527707}
I20260709 13:06:13.064066 19136 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:13.097067 16562 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
W20260709 13:06:13.083555 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 44359.4us
I20260709 13:06:13.101115 15974 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:13.093978 15935 rpcz_store.cc:276] 0709 13:06:11.995447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:12.212726 (+217279us) service_pool.cc:224] Handling call
0709 13:06:12.213103 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:12.226012 (+ 12909us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:12.226041 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:12.289649 (+ 63608us) log.cc:844] Serialized 26 byte log entry
0709 13:06:12.736813 (+447164us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:12.736863 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:12.736945 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:13.093429 (+356484us) raft_consensus.cc:1685] finished
0709 13:06:13.093450 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:13.093584 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4467,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":33429,"thread_start_us":392356,"threads_started":1,"wal-append.queue_time_us":601442}
I20260709 13:06:13.105527 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:13.106185 16562 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:13.111570 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:13.092536 19328 raft_consensus.cc:2804] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:13.122269 15861 raft_consensus.cc:2368] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 7. Current term is 8.
I20260709 13:06:13.127737 15974 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:13.127166 15902 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:13.137023 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:13.141688 16562 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:06:13.142702 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:13.146137 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:13.146973 16568 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:13.159044 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:13.159824 16561 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:13.116669 19331 raft_consensus.cc:2804] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:13.162185 15856 raft_consensus.cc:2368] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 6. Current term is 7.
I20260709 13:06:13.105882 19164 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:13.163130 15902 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:13.132599 19165 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:13.167795 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:13.169337 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:13.170384 15860 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:06:13.171236 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:13.172333 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 44750.8us
I20260709 13:06:13.132385 19328 raft_consensus.cc:697] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:13.175235 15857 raft_consensus.cc:2368] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 6. Current term is 7.
I20260709 13:06:13.173446 15974 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:13.119081 19136 consensus_queue.cc:1059] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:06:13.161562 19149 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:13.180192 15862 raft_consensus.cc:2410] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 8 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:06:13.133081 19099 consensus_queue.cc:1048] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 13:06:13.164631 19331 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.192196 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:13.166441 19265 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.193356 15976 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:13.181608 16535 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:13.158531 19222 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.199498 16565 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:06:13.201138 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:13.170113 19165 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:06:13.181764 19273 consensus_meta.cc:287] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.511s	user 0.011s	sys 0.000s
I20260709 13:06:13.204870 16568 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:13.175259 19095 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.210445 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:13.214696 15855 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:13.181932 19201 raft_consensus.cc:2749] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:13.192822 19149 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.197580 19161 raft_consensus.cc:2804] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:13.217450 15860 raft_consensus.cc:1240] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:13.228724 15974 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:13.236917 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:13.232295 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 63328us
I20260709 13:06:13.236544 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:13.206306 18732 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:13.202461 19331 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:13.213521 19165 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:13.253353 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 74201.3us
W20260709 13:06:13.216302 19212 consensus_meta.cc:287] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.571s	user 0.007s	sys 0.005s
I20260709 13:06:13.256093 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:13.256454 15856 raft_consensus.cc:1275] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:06:13.256803 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:13.263622 15860 raft_consensus.cc:2393] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 8: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:13.256897 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:13.264582 15858 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:13.264914 16570 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:13.258142 16565 raft_consensus.cc:1275] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:06:13.258605 16535 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:13.257666 19099 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:13.266358 19099 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:13.266808 19099 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:13.268047 16535 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
W20260709 13:06:13.268599 19324 log.cc:927] Time spent T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
I20260709 13:06:13.283074 15976 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:13.270401 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:13.216185 19329 raft_consensus.cc:2749] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) with higher term. Message: Invalid argument: T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 7. Current term is 8.
I20260709 13:06:13.215809 19024 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:13.284343 15862 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
I20260709 13:06:13.236898 19161 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:13.288789 16566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 737) took 1293 ms. Trace:
I20260709 13:06:13.212664 19222 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:13.239638 19149 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:13.292766 16566 rpcz_store.cc:276] 0709 13:06:11.995189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:12.581112 (+585923us) service_pool.cc:224] Handling call
0709 13:06:12.632464 (+ 51352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:12.662155 (+ 29691us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:12.662195 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:12.662710 (+   515us) log.cc:844] Serialized 26 byte log entry
0709 13:06:12.907984 (+245274us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:12.908025 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:12.908091 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:13.288581 (+380490us) raft_consensus.cc:1685] finished
0709 13:06:13.288600 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:13.288735 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8551,"raft.queue_time_us":5859,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":5462,"thread_start_us":244896,"threads_started":1,"wal-append.queue_time_us":315666}
I20260709 13:06:13.229285 19273 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
W20260709 13:06:13.316244 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 53905.6us
I20260709 13:06:13.317209 15976 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:13.213086 19265 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:13.287395 18732 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.293234 18948 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:13.292341 18958 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:13.293051 19024 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:13.305606 19161 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:13.330482 15862 raft_consensus.cc:1240] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:13.254035 19095 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:13.330578 15856 raft_consensus.cc:1240] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:06:13.346299 16010 raft_consensus.cc:2393] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 10: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:13.351518 18958 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:13.352155 18958 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:13.352643 18958 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:06:13.355827 19319 log.cc:927] Time spent T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.250s	user 0.000s	sys 0.001s
I20260709 13:06:13.356005 19242 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:13.356703 19099 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:13.358724 19099 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:13.338286 19329 raft_consensus.cc:2804] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:13.364049 19329 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.364656 19329 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:13.365664 19095 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:13.368988 16004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1826) took 1555 ms. Trace:
I20260709 13:06:13.369484 16004 rpcz_store.cc:276] 0709 13:06:11.820077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:11.827067 (+  6990us) service_pool.cc:224] Handling call
0709 13:06:11.889644 (+ 62577us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:12.580898 (+691254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:13.368687 (+787789us) spinlock_profiling.cc:243] Waited 675 ms on lock 0x7b5400306454. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:13.368917 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1551364864}
I20260709 13:06:13.373052 15859 raft_consensus.cc:1240] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:13.340715 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:13.313998 19016 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:13.199695 19317 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:13.372944 15854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 1848) took 1108 ms. Trace:
I20260709 13:06:13.374428 15854 rpcz_store.cc:276] 0709 13:06:12.265096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:12.313169 (+ 48073us) service_pool.cc:224] Handling call
0709 13:06:13.372686 (+1059517us) spinlock_profiling.cc:243] Waited 441 ms on lock 0x7b54000b5c54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:13.372880 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1014962560}
W20260709 13:06:13.376575 19308 log.cc:927] Time spent T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.674s	user 0.002s	sys 0.003s
I20260709 13:06:13.377336 19265 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:13.382270 19238 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:13.383551 19326 raft_consensus.cc:2804] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:13.338131 18952 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:13.384521 18952 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.384979 18952 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:13.389539 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:13.400063 19345 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:13.400691 19345 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.286895 19212 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:13.401087 18728 raft_consensus.cc:3055] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:13.401620 18728 raft_consensus.cc:740] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:13.402428 18728 consensus_queue.cc:260] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:13.403365 18728 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:13.382493 19334 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:13.405681 19334 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:13.406055 19334 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:13.410952 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:13.412559 19344 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:13.412904 15856 raft_consensus.cc:1240] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 5. Current term is 6. Ops: []
W20260709 13:06:13.414799 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:13.418093 19327 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:13.419487 16005 raft_consensus.cc:1240] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:13.423790 19328 consensus_queue.cc:237] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:13.431911 19336 log.cc:927] Time spent T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
W20260709 13:06:13.432354 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:13.437337 19222 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:13.437711 19102 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:13.448297 19164 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:13.449793 16561 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:06:13.456066 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:13.456869 15853 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:13.339283 18732 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:13.475795 18732 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:13.477576 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:13.478511 15862 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
I20260709 13:06:13.481477 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:13.482161 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:13.487346 19334 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:13.514286 19165 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:13.401118 19345 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:13.493386 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:13.497520 19212 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:13.482034 19326 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:13.495798 19273 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:13.508852 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:13.517973 19334 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:13.536640 19129 consensus_meta.cc:287] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.818s	user 0.011s	sys 0.002s
I20260709 13:06:13.546545 15900 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:06:13.597648 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 51196us
I20260709 13:06:13.566622 15855 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:13.618076 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:13.336790 15858 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:13.624029 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:13.619766 15855 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:13.582091 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:13.535908 19210 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:13.502034 19229 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:13.547940 19164 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:13.653733 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:13.663029 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:13.655066 16004 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 7: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:13.676137 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:13.664898 15859 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 9.
I20260709 13:06:13.579993 19326 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:13.684793 15857 raft_consensus.cc:2393] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 7: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:13.500458 19263 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:13.643517 19222 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:13.662971 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:13.695166 15858 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:13.694217 15856 raft_consensus.cc:2393] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 7: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:13.693691 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:06:13.686355 18861 consensus_meta.cc:287] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.582s	user 0.006s	sys 0.005s
I20260709 13:06:13.723001 15862 raft_consensus.cc:2393] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 6: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:06:13.700711 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 440502us
I20260709 13:06:13.692543 15827 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:13.740119 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 472091us
I20260709 13:06:13.740480 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:13.700547 16533 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.030s	sys 0.060s
W20260709 13:06:13.744002 16533 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.031s	sys 0.065s
W20260709 13:06:13.744297 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 648401us
I20260709 13:06:13.751964 15859 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:06:13.752094 16007 raft_consensus.cc:1240] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 5. Current term is 6. Ops: []
W20260709 13:06:13.753434 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 60967.4us
I20260709 13:06:13.754316 15827 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:13.644421 19129 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:13.698685 19222 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.664606 19164 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:13.759759 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:13.763785 15855 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 5.
W20260709 13:06:13.763330 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 446171us
W20260709 13:06:13.760037 15974 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.010s	sys 0.011s
I20260709 13:06:13.765287 15853 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:06:13.769187 15976 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:13.772003 15974 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.010s	sys 0.011s
I20260709 13:06:13.773352 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:13.773680 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 544962us
I20260709 13:06:13.776017 16005 raft_consensus.cc:1240] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:13.777040 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:13.756196 19340 raft_consensus.cc:2804] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:06:13.760210 19223 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.106s	sys 0.190s
I20260709 13:06:13.763697 19222 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:13.789853 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:13.801308 15853 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:13.798763 15858 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:13.785645 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:13.802935 15855 raft_consensus.cc:2393] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 7: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:13.799476 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:13.803450 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:13.780828 19078 consensus_meta.cc:287] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.697s	user 0.007s	sys 0.007s
I20260709 13:06:13.802913 16535 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:13.805518 19347 log.cc:927] Time spent T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:06:13.806577 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:13.751741 18861 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:13.740078 19212 consensus_queue.cc:1059] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:06:13.809528 15827 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:13.810005 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:06:13.801906 16533 connection.cc:256] RPC call timeout handler was delayed by -2.03824247s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19810000000000003s for a response
I20260709 13:06:13.778733 19318 raft_consensus.cc:2749] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:13.746824 18952 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:13.813552 16563 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:13.815646 15862 raft_consensus.cc:2368] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 7. Current term is 8.
W20260709 13:06:13.802032 19223 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.107s	sys 0.191s
I20260709 13:06:13.793030 19340 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:13.797072 19273 raft_consensus.cc:2749] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:06:13.820888 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:13.820958 15857 raft_consensus.cc:1240] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 6. Current term is 7. Ops: []
W20260709 13:06:13.813705 19136 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.012s	sys 0.019s
I20260709 13:06:13.800437 19327 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:13.832222 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:13.832788 15859 raft_consensus.cc:2393] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 8: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:13.824460 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:13.828522 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:13.836192 16564 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:06:13.837646 15853 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:06:13.841506 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:13.814244 19016 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:13.851246 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:13.803277 19332 consensus_queue.cc:1048] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:06:13.852031 19359 consensus_queue.cc:1048] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:13.852474 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:13.853250 15859 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:06:13.854321 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:13.854606 19165 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:13.854835 15976 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:13.856189 19165 raft_consensus.cc:2749] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:13.860044 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:13.860894 15857 raft_consensus.cc:2393] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 8: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:13.861744 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:06:13.862915 19024 consensus_meta.cc:287] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.513s	user 0.006s	sys 0.001s
I20260709 13:06:13.863173 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:13.865198 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:13.886516 15862 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 7: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:13.866250 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:13.866391 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:13.867220 19352 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:13.867368 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:13.869589 19149 consensus_meta.cc:287] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.557s	user 0.008s	sys 0.001s
I20260709 13:06:13.870167 19229 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:13.871752 19265 consensus_queue.cc:1048] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:13.872879 19164 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:13.830616 19136 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.012s	sys 0.019s
I20260709 13:06:13.819283 18732 consensus_queue.cc:1059] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 13:06:13.811690 19317 consensus_queue.cc:1048] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:06:13.876650 18861 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:13.876806 19129 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:06:13.877348 16533 leader_election.cc:336] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.509s (SENT)
I20260709 13:06:13.900566 16533 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:13.901616 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 54387.3us
W20260709 13:06:13.902302 16533 leader_election.cc:336] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.562s (SENT)
I20260709 13:06:13.902685 16533 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:13.829005 19340 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:13.878300 18958 consensus_meta.cc:287] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.525s	user 0.011s	sys 0.000s
I20260709 13:06:13.904018 18958 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:13.906277 19238 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.906342 18958 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:13.907176 18958 raft_consensus.cc:2749] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:13.906950 19238 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:13.879436 15976 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:13.908036 19348 raft_consensus.cc:2749] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:13.890471 15854 raft_consensus.cc:2410] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:06:13.890811 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:13.878700 19016 raft_consensus.cc:697] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 7 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:13.895121 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:13.892488 19149 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:13.897289 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:13.910135 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:13.911145 15857 raft_consensus.cc:2368] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 6. Current term is 7.
I20260709 13:06:13.914163 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:13.915376 15853 raft_consensus.cc:2393] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 6: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:06:13.921453 16533 leader_election.cc:336] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.597s (SENT)
I20260709 13:06:13.914626 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:13.927351 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:13.928082 15853 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6.
I20260709 13:06:13.928005 16533 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:13.929823 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 50498.2us
I20260709 13:06:13.893702 19078 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.931028 15976 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:13.924149 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:13.937189 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:13.938877 15854 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:06:13.938992 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:13.939313 19016 consensus_queue.cc:237] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:13.940411 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:13.940790 19149 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:13.941068 15930 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:13.941519 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:13.941948 19238 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:13.942023 18732 raft_consensus.cc:3055] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:13.938894 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:13.942700 19361 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.942803 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:13.943241 19361 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:13.914106 19210 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:13.943725 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:06:13.926507 19331 consensus_meta.cc:287] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.637s	user 0.005s	sys 0.005s
I20260709 13:06:13.943506 15853 raft_consensus.cc:2393] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 7: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:13.943519 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:13.945235 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:13.939576 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:13.946247 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 177881us
I20260709 13:06:13.946365 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:13.889726 19024 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:13.947846 15856 raft_consensus.cc:2393] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 10: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:13.945318 15976 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:13.921958 19129 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:13.948662 15859 raft_consensus.cc:2368] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 7. Current term is 8.
I20260709 13:06:13.945125 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:13.929886 19328 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:13.942514 18732 raft_consensus.cc:740] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:13.913663 19223 consensus_queue.cc:1059] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:06:13.951479 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:13.933290 19350 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:13.966912 15856 raft_consensus.cc:2410] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
W20260709 13:06:13.967621 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 57064.1us
I20260709 13:06:13.970137 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:13.972749 15976 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:13.949996 19265 raft_consensus.cc:2804] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:06:14.013160 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 65752.8us
I20260709 13:06:13.948592 19078 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:13.973865 19350 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:13.974956 19361 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:13.975430 19210 raft_consensus.cc:3055] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:13.979759 19024 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:13.968479 18968 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:13.982281 19222 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:06:13.983347 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:14.022779 15856 raft_consensus.cc:2393] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 7: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
W20260709 13:06:13.984009 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:13.950153 19331 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:13.986282 19197 raft_consensus.cc:3055] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:13.960739 19263 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:13.993184 18952 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:14.015936 19265 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.965535 19345 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:13.968721 19273 consensus_queue.cc:1048] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:06:14.000804 15862 raft_consensus.cc:2410] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:06:14.001035 18732 consensus_queue.cc:260] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:14.014734 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:13.966730 19129 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:14.019745 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:14.046144 16007 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:14.046396 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:06:14.026918 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 214523us
I20260709 13:06:14.046790 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:14.030151 15860 raft_consensus.cc:1217] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Deduplicated request from leader. Original: 3.5->[5.6-5.6]   Dedup: 5.6->[]
I20260709 13:06:14.033262 15937 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:13.995653 19161 consensus_meta.cc:287] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.552s	user 0.008s	sys 0.001s
I20260709 13:06:14.036324 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:14.017850 19165 raft_consensus.cc:2804] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:14.016959 19210 raft_consensus.cc:740] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:14.048375 15860 raft_consensus.cc:1240] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:06:14.050195 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:14.053195 15900 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:14.058115 15900 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:14.054440 19363 consensus_queue.cc:1048] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 13:06:14.060180 16003 raft_consensus.cc:1240] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:14.060696 19345 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:14.061419 19362 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:14.061966 19362 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:14.062369 19362 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:14.064481 18732 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:14.030196 19197 raft_consensus.cc:740] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:06:14.040078 19329 consensus_meta.cc:287] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.675s	user 0.011s	sys 0.000s
I20260709 13:06:14.066519 15935 raft_consensus.cc:2393] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 7: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:14.072615 16011 raft_consensus.cc:1240] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:06:14.029740 18968 raft_consensus.cc:697] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:14.072665 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:14.079034 16535 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:14.080686 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:14.081559 16563 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:06:14.082681 19328 raft_consensus.cc:2749] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:14.083578 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:14.083877 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:14.084338 16006 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:14.091970 16567 raft_consensus.cc:1240] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 6. Current term is 7. Ops: [5.7-6.8]
I20260709 13:06:14.096947 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:14.099263 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:14.080752 16535 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:14.101249 19328 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:14.101759 19328 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:14.101861 18958 consensus_queue.cc:1059] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:14.103086 18958 raft_consensus.cc:3055] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:14.103597 18958 raft_consensus.cc:740] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:06:14.104506 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:14.099962 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:14.104965 18958 consensus_queue.cc:260] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:14.106011 16567 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:14.106794 18958 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:14.102312 19328 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:14.099377 16006 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:14.116202 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:14.116765 16007 raft_consensus.cc:1240] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:14.044245 19265 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:14.050058 19129 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:14.122855 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:14.123312 15900 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:14.124498 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:14.125099 16009 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:14.125907 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:14.129129 15976 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:14.131616 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:14.132402 16563 raft_consensus.cc:2393] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 7: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:14.126659 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:14.133567 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:06:14.133939 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 109851us
I20260709 13:06:14.129215 15900 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:14.065764 19317 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:14.136602 16003 raft_consensus.cc:2410] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 8 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 13:06:14.137028 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 162273us
I20260709 13:06:14.141356 16567 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:14.116122 19164 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:14.076020 19165 raft_consensus.cc:697] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:14.168741 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:14.169605 16561 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:14.055745 19212 consensus_queue.cc:1059] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.078s
I20260709 13:06:14.134955 15974 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:14.147555 15859 raft_consensus.cc:1275] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:06:14.066126 19161 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:14.156968 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:14.066301 18861 consensus_queue.cc:1059] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 13:06:14.178165 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:14.131738 19361 raft_consensus.cc:2804] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:14.189814 16567 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:06:14.190006 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:14.178597 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:14.191201 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:14.185417 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:14.186226 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 51338.6us
I20260709 13:06:14.194993 15974 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:14.146634 15976 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:14.144526 19345 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:14.200201 15862 raft_consensus.cc:1275] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:06:14.196509 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:14.151892 15827 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:14.190661 15854 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:06:14.202965 16007 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:14.118142 19329 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:14.201277 16533 leader_election.cc:336] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.873s (SENT)
I20260709 13:06:14.203599 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:14.203879 15930 raft_consensus.cc:1275] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:06:14.202203 16006 raft_consensus.cc:2393] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 7: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:14.205653 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:14.207617 16561 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:14.214365 16010 raft_consensus.cc:1240] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:14.217307 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:14.193197 19238 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:14.221525 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:06:14.222640 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 71823.3us
I20260709 13:06:14.223647 15827 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:14.225533 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:14.226531 15930 raft_consensus.cc:2393] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 9: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:14.228073 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:14.144270 19357 consensus_queue.cc:1059] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 13:06:14.231266 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:14.202980 19345 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:14.238161 16569 raft_consensus.cc:2393] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 8: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:14.201287 19164 raft_consensus.cc:697] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 7 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:14.241428 16009 raft_consensus.cc:2393] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 8: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:14.197144 19361 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:14.247819 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:14.254110 16003 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:14.172926 19317 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:14.289583 16564 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:14.276379 16561 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:06:14.278399 16566 raft_consensus.cc:1275] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:06:14.280009 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 56427.6us
I20260709 13:06:14.220083 19024 raft_consensus.cc:3055] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:14.264696 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:14.297024 16006 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:06:14.293406 16566 raft_consensus.cc:1240] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:06:14.294828 15827 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:14.231828 19229 raft_consensus.cc:2804] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:14.298261 18861 raft_consensus.cc:2804] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:14.298635 19229 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:14.298822 18861 raft_consensus.cc:697] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 7 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:14.299126 19229 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:14.299865 18861 consensus_queue.cc:237] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:14.308338 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:14.280943 19341 log.cc:927] Time spent T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:06:14.247848 19345 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:14.277138 19363 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.290354 19357 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:06:14.310904 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 130788us
I20260709 13:06:14.319593 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:14.312448 16568 raft_consensus.cc:1275] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:06:14.291204 19212 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:14.244257 19238 raft_consensus.cc:697] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:14.257840 19361 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:14.313158 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:14.203162 19210 consensus_queue.cc:260] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:14.252506 19273 consensus_queue.cc:1048] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
W20260709 13:06:14.328001 16533 connection.cc:256] RPC call timeout handler was delayed by -1.908642470000018s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21250000000000002s for a response
I20260709 13:06:14.296347 19024 raft_consensus.cc:740] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:14.309016 19358 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:14.313649 16566 raft_consensus.cc:1240] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:06:14.312146 19352 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:14.333467 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:14.345289 15935 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:06:14.341257 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:14.347380 16009 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:14.342337 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:14.349124 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:14.349996 16006 raft_consensus.cc:2393] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 6: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:14.351835 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:14.353027 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:14.241149 18968 consensus_queue.cc:237] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:14.226817 19197 consensus_queue.cc:260] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:14.366719 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 171773us
W20260709 13:06:14.372262 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 59549us
I20260709 13:06:14.323868 19372 raft_consensus.cc:3055] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:14.377854 16563 raft_consensus.cc:1240] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:14.331449 19357 raft_consensus.cc:697] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 9 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:14.378665 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:14.384270 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:14.384650 19365 consensus_queue.cc:1059] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:06:14.385460 19380 consensus_queue.cc:1059] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:14.385991 19372 raft_consensus.cc:740] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:14.393028 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:14.393652 15936 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:14.386134 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:14.389297 15935 raft_consensus.cc:2393] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 8: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:14.386986 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:14.344734 19078 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:14.295732 19165 consensus_queue.cc:237] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:14.401234 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:14.327787 19363 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:14.382161 19326 consensus_meta.cc:287] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.640s	user 0.008s	sys 0.004s
I20260709 13:06:14.396138 15934 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:14.421914 16009 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:06:14.339721 19364 consensus_queue.cc:1059] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:06:14.396139 15974 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:14.402068 16533 leader_election.cc:336] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.662s (SENT)
W20260709 13:06:14.413672 19371 log.cc:927] Time spent T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 13:06:14.414414 19229 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:14.416108 19374 log.cc:927] Time spent T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:06:14.387535 19164 consensus_queue.cc:237] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:14.422786 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:14.387837 19331 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:14.430056 16533 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:14.431725 15902 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:14.436223 15974 leader_election.cc:400] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 6. Current term is 7.
I20260709 13:06:14.437837 16009 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:14.438221 15974 leader_election.cc:403] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:06:14.439453 15902 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:14.444375 15974 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:14.444159 19357 consensus_queue.cc:237] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:14.446683 19327 raft_consensus.cc:2749] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 6. Current term is 7.
I20260709 13:06:14.447526 19327 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:14.447844 19149 raft_consensus.cc:2749] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:14.448064 19327 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:14.448585 19327 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:14.437449 19273 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:14.453039 19317 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:14.453120 19238 consensus_queue.cc:237] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:14.454860 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:14.433387 19359 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.456100 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:14.456221 19359 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:14.456400 19327 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:14.458432 19359 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:14.458638 19327 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:14.441042 15902 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:14.461887 16009 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:06:14.474650 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:14.470340 16003 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:06:14.395305 18952 consensus_queue.cc:1059] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:06:14.387553 19212 consensus_queue.cc:1059] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:06:14.476439 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:14.402550 19380 consensus_queue.cc:1059] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.057s
W20260709 13:06:14.477437 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 75394us
I20260709 13:06:14.440575 19197 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:14.433036 19161 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:14.487293 19318 raft_consensus.cc:3055] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:14.489120 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:14.458797 19358 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:06:14.490232 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:14.511657 15937 raft_consensus.cc:2468] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
I20260709 13:06:14.494668 15934 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
I20260709 13:06:14.498842 19229 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:14.503886 19273 raft_consensus.cc:3055] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:14.398965 19024 consensus_queue.cc:260] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:14.513130 19024 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:14.516803 15935 raft_consensus.cc:1240] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:14.517871 19387 consensus_queue.cc:1059] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 13:06:14.518716 19385 raft_consensus.cc:3055] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:14.519167 19385 raft_consensus.cc:740] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:14.519220 19387 raft_consensus.cc:3055] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:14.519804 19387 raft_consensus.cc:740] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:14.520057 19332 raft_consensus.cc:3055] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:14.520521 19332 raft_consensus.cc:740] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:14.477470 19078 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:06:14.507402 19366 log.cc:927] Time spent T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.124s	user 0.001s	sys 0.001s
I20260709 13:06:14.481400 19359 raft_consensus.cc:2749] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:14.525573 15936 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:06:14.529107 19332 consensus_queue.cc:260] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:14.530711 19380 raft_consensus.cc:3055] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:06:14.489142 19340 consensus_meta.cc:287] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.585s	user 0.011s	sys 0.000s
I20260709 13:06:14.433610 19326 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:14.508477 19382 log.cc:927] Time spent T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
W20260709 13:06:14.538545 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 409366us
I20260709 13:06:14.539757 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:06:14.566459 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 86917.5us
W20260709 13:06:14.542374 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:14.568635 15936 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:14.579129 16003 raft_consensus.cc:2393] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 8: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:14.567201 15931 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5.
I20260709 13:06:14.509604 19318 raft_consensus.cc:740] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
W20260709 13:06:14.581779 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 135618us
I20260709 13:06:14.521677 19273 raft_consensus.cc:740] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:06:14.582885 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 436289us
I20260709 13:06:14.593330 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:14.604732 15931 raft_consensus.cc:2368] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 7. Current term is 8.
I20260709 13:06:14.604638 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:14.605939 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:14.606168 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:14.606627 15931 raft_consensus.cc:2393] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 8: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:14.537211 19149 raft_consensus.cc:2749] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:14.607553 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:14.537644 19380 raft_consensus.cc:740] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:14.613952 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:14.615006 15860 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:14.610612 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:14.615483 15862 raft_consensus.cc:1240] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:14.608494 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:14.613535 15900 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:14.597725 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:06:14.622155 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:14.623766 15934 raft_consensus.cc:2393] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 8: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:14.520855 19387 consensus_queue.cc:260] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:14.624591 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:14.592334 19332 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:14.627358 16566 raft_consensus.cc:1240] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:14.629859 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:14.630716 15934 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:06:14.632193 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:14.634097 15934 raft_consensus.cc:2393] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 8: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:14.636318 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:14.636509 16007 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:06:14.637382 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:14.643151 15936 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:06:14.575786 19359 consensus_queue.cc:1059] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:06:14.635758 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:14.647344 15934 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:14.648098 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:14.648414 15855 raft_consensus.cc:1240] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:14.649044 16003 raft_consensus.cc:2393] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 8: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:14.649472 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:14.650116 15929 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:14.646804 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:14.651981 15936 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:14.552229 19340 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:14.546238 19362 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:14.629278 19385 consensus_queue.cc:260] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:14.768211 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 154746us
W20260709 13:06:14.747792 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 100761us
I20260709 13:06:14.905269 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:14.782251 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:15.053720 15932 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:14.975917 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:14.654421 16009 raft_consensus.cc:1275] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:06:15.053236 15976 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:14.835501 15855 raft_consensus.cc:1240] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 7 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 6. Current term is 7. Ops: []
W20260709 13:06:14.705524 16008 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.001s	sys 0.021s
W20260709 13:06:15.056095 16008 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.852s	user 0.007s	sys 0.026s
I20260709 13:06:14.848745 15936 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 10.
I20260709 13:06:14.689824 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:06:14.746428 16005 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.003s	sys 0.024s
W20260709 13:06:15.057441 16005 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.985s	user 0.011s	sys 0.027s
I20260709 13:06:15.050675 16566 raft_consensus.cc:1240] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:15.001503 15860 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
I20260709 13:06:14.850611 15825 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:14.784652 16533 leader_election.cc:336] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.918s (SENT)
I20260709 13:06:14.646337 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:14.663229 15862 raft_consensus.cc:1240] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 9. Current term is 10. Ops: []
W20260709 13:06:15.056092 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 52962.9us
I20260709 13:06:14.946376 15934 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 7.
I20260709 13:06:15.057441 16007 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:15.007480 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:14.934517 19389 log.cc:927] Time spent T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.265s	user 0.001s	sys 0.000s
W20260709 13:06:14.995492 19393 log.cc:927] Time spent T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.371s	user 0.000s	sys 0.000s
I20260709 13:06:15.065557 16533 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:14.807121 19387 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:14.689644 19238 raft_consensus.cc:2804] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:15.066735 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:15.074272 16568 raft_consensus.cc:1240] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:15.065471 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:15.066310 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:15.070967 15858 raft_consensus.cc:1240] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:15.075008 16566 raft_consensus.cc:2393] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 7: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:15.072454 15930 raft_consensus.cc:2393] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 6: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:14.994837 19329 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:15.072440 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:15.074420 16570 raft_consensus.cc:2393] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 11: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
W20260709 13:06:14.876243 19129 consensus_meta.cc:287] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.704s	user 0.012s	sys 0.000s
I20260709 13:06:14.689838 19164 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:06:14.836339 19382 log.cc:927] Time spent T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 13:06:15.075591 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:15.076987 16008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1950) took 1060 ms. Trace:
W20260709 13:06:14.910591 18958 consensus_meta.cc:287] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.803s	user 0.011s	sys 0.001s
I20260709 13:06:14.850929 19380 consensus_queue.cc:260] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:15.077674 15929 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:14.746376 19364 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:15.079602 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:15.084738 16008 rpcz_store.cc:276] 0709 13:06:14.016487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.020487 (+  4000us) service_pool.cc:224] Handling call
0709 13:06:14.020979 (+   492us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:06:14.030717 (+  9738us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:14.030746 (+    29us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 13:06:14.031471 (+   725us) log.cc:844] Serialized 104 byte log entry
0709 13:06:14.188779 (+157308us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:15.056538 (+867759us) log.cc:844] Serialized 14 byte log entry
0709 13:06:15.076619 (+ 20081us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:15.076688 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:15.076728 (+    40us) raft_consensus.cc:1685] finished
0709 13:06:15.076740 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:15.076914 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19732,"prepare.queue_time_us":120775,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":47,"raft.queue_time_us":23961,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":3544,"thread_start_us":979893,"threads_started":2,"wal-append.queue_time_us":218028}
I20260709 13:06:15.092375 15932 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:15.096896 15932 raft_consensus.cc:1240] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:15.098362 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:15.099483 15936 raft_consensus.cc:2393] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 7: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:15.101989 16007 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 7.
W20260709 13:06:15.134888 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 386585us
I20260709 13:06:15.079241 19388 raft_consensus.cc:2764] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 13:06:15.083534 19352 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:15.062463 19398 consensus_queue.cc:1059] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:06:14.930869 19273 consensus_queue.cc:260] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:15.193828 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 343283us
I20260709 13:06:15.195437 15825 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:15.098630 19392 log.cc:927] Time spent T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.311s	user 0.001s	sys 0.000s
W20260709 13:06:15.093961 19394 log.cc:927] Time spent T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.164s	user 0.000s	sys 0.000s
W20260709 13:06:15.096127 19265 consensus_meta.cc:287] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.965s	user 0.008s	sys 0.007s
I20260709 13:06:15.147879 18958 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.094213 19362 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:15.086229 19165 consensus_queue.cc:1175] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" responder_term: 6 status { last_received { term: 6 index: 7 } last_committed_idx: 7 last_received_current_leader { term: 6 index: 7 } }
W20260709 13:06:15.158299 18732 consensus_meta.cc:287] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 1.093s	user 0.009s	sys 0.000s
I20260709 13:06:15.143239 18968 consensus_queue.cc:1059] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
W20260709 13:06:15.170377 19332 consensus_meta.cc:287] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.532s	user 0.008s	sys 0.000s
I20260709 13:06:15.092746 19129 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:14.834913 19318 consensus_queue.cc:260] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:15.137342 19164 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:15.176564 19340 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:15.179088 15929 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
W20260709 13:06:15.179186 16533 leader_election.cc:336] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 2.125s (SENT)
I20260709 13:06:15.181031 19367 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:06:15.181010 19197 consensus_meta.cc:287] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.685s	user 0.007s	sys 0.001s
W20260709 13:06:15.181916 19361 consensus_meta.cc:287] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.831s	user 0.013s	sys 0.000s
W20260709 13:06:15.140559 19328 consensus_meta.cc:287] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 1.029s	user 0.009s	sys 0.004s
I20260709 13:06:15.305961 16005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1947) took 1291 ms. Trace:
I20260709 13:06:15.305681 19328 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:15.306342 16005 rpcz_store.cc:276] 0709 13:06:14.014966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.015082 (+   116us) service_pool.cc:224] Handling call
0709 13:06:14.019173 (+  4091us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:14.039651 (+ 20478us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:06:15.064489 (+1024838us) log.cc:844] Serialized 14 byte log entry
0709 13:06:15.305615 (+241126us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:15.305671 (+    56us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:15.305725 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:15.305800 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:15.305910 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15786,"prepare.queue_time_us":249530,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":16205,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":101,"thread_start_us":1182057,"threads_started":2,"wal-append.queue_time_us":205502}
I20260709 13:06:15.308113 19242 raft_consensus.cc:2749] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:06:15.138562 19393 log.cc:927] Time spent T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:06:15.187448 19398 raft_consensus.cc:3055] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:15.312106 19398 raft_consensus.cc:740] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:15.312331 16565 raft_consensus.cc:1240] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 13:06:15.313014 19398 consensus_queue.cc:260] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:15.314042 19398 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:15.197964 16564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 917) took 1056 ms. Trace:
I20260709 13:06:15.315815 16564 rpcz_store.cc:276] 0709 13:06:14.141873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.296558 (+154685us) service_pool.cc:224] Handling call
0709 13:06:14.296974 (+   416us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:14.325848 (+ 28874us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:14.325903 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:14.332459 (+  6556us) log.cc:844] Serialized 26 byte log entry
0709 13:06:14.466480 (+134021us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:14.466522 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:14.466609 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:15.197711 (+731102us) raft_consensus.cc:1685] finished
0709 13:06:15.197731 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:15.197867 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10148,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":5852,"thread_start_us":99508,"threads_started":1,"wal-append.queue_time_us":185875}
I20260709 13:06:15.095660 19238 raft_consensus.cc:697] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 7 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:15.317867 15931 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:15.318113 19238 consensus_queue.cc:237] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:15.193383 19024 consensus_meta.cc:287] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.680s	user 0.007s	sys 0.003s
I20260709 13:06:15.141296 19363 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:15.320950 15854 raft_consensus.cc:2410] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 13:06:15.171454 19345 consensus_meta.cc:287] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.845s	user 0.003s	sys 0.009s
I20260709 13:06:15.321599 19345 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:15.233408 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 172644us
W20260709 13:06:15.240847 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 175789us
I20260709 13:06:15.323493 19421 consensus_queue.cc:1048] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:15.324508 15900 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:15.325016 19361 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:15.326907 15900 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:06:15.251559 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:15.330422 19210 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:15.332428 16569 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45244 (request call id 904) took 2635 ms (client timeout 1731 ms). Trace:
W20260709 13:06:15.332810 16569 rpcz_store.cc:269] 0709 13:06:12.697068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:13.327753 (+630685us) service_pool.cc:224] Handling call
0709 13:06:15.332245 (+2004492us) spinlock_profiling.cc:243] Waited 864 ms on lock 0x7b5400714b54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:15.332363 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1987023616}
I20260709 13:06:15.333249 16010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1839) took 1790 ms. Trace:
I20260709 13:06:15.333729 19406 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:15.333678 16010 rpcz_store.cc:276] 0709 13:06:13.542850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:13.550975 (+  8125us) service_pool.cc:224] Handling call
0709 13:06:13.589195 (+ 38220us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:14.220954 (+631759us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:15.333033 (+1112079us) spinlock_profiling.cc:243] Waited 620 ms on lock 0x7b54001b9754. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:15.333193 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1425846528}
I20260709 13:06:15.334527 19406 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:15.335377 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:15.336328 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:15.336699 19406 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:15.337260 19406 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:15.337586 19406 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:15.337934 19406 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:15.339524 19359 raft_consensus.cc:3055] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:15.340040 16567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 782) took 1199 ms. Trace:
I20260709 13:06:15.340310 19359 raft_consensus.cc:740] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:15.340574 16567 rpcz_store.cc:276] 0709 13:06:14.140609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.244608 (+103999us) service_pool.cc:224] Handling call
0709 13:06:14.245022 (+   414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:14.245424 (+   402us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:14.245464 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:14.263980 (+ 18516us) log.cc:844] Serialized 26 byte log entry
0709 13:06:14.536800 (+272820us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:14.536858 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:14.536953 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:15.339823 (+802870us) raft_consensus.cc:1685] finished
0709 13:06:15.339842 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:15.339979 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":603,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":30011,"thread_start_us":210700,"threads_started":1,"wal-append.queue_time_us":324871}
I20260709 13:06:15.339439 15855 raft_consensus.cc:1275] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:06:15.343163 19372 consensus_queue.cc:260] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:15.344995 19372 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:15.346345 19197 consensus_queue.cc:1048] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:15.093089 19364 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:15.349566 19364 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:15.296221 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 46404.2us
I20260709 13:06:15.304986 16004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 1949) took 1289 ms. Trace:
I20260709 13:06:15.350064 19164 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:15.218189 19265 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:15.351046 15825 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:15.350284 16004 rpcz_store.cc:276] 0709 13:06:14.016143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.017741 (+  1598us) service_pool.cc:224] Handling call
0709 13:06:14.038484 (+ 20743us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:14.151443 (+112959us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:14.151461 (+    18us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:14.184970 (+ 33509us) log.cc:844] Serialized 26 byte log entry
0709 13:06:14.622257 (+437287us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:14.622292 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:14.622359 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:15.304747 (+682388us) raft_consensus.cc:1685] finished
0709 13:06:15.304773 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:15.304925 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42620,"raft.queue_time_us":5809,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":13424,"thread_start_us":523785,"threads_started":2,"wal-append.queue_time_us":660389}
I20260709 13:06:15.352406 19326 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:15.292179 16533 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:15.325240 15902 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:15.242960 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 178095us
I20260709 13:06:15.305106 19129 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:15.243652 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 157351us
I20260709 13:06:15.327147 19408 consensus_queue.cc:1059] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:15.327901 15933 raft_consensus.cc:2368] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 8. Current term is 9.
I20260709 13:06:15.290256 19365 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.328410 19413 raft_consensus.cc:2804] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:15.328662 16006 raft_consensus.cc:2393] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 7: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:15.329401 19345 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:15.329238 19363 consensus_queue.cc:1048] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:15.370463 15856 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:06:15.343883 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:15.356068 15902 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:15.376431 15855 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:15.376682 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:15.363610 15974 leader_election.cc:400] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Vote denied by peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) with higher term. Message: Invalid argument: T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 7. Current term is 8.
I20260709 13:06:15.354008 19318 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:15.224386 18958 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:15.354773 19415 raft_consensus.cc:3055] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:15.378134 15825 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:15.327232 19361 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:15.367884 15929 raft_consensus.cc:1240] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:15.389475 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:15.392702 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:15.405402 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:15.406098 16569 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:15.395375 16563 raft_consensus.cc:1240] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 6. Current term is 7. Ops: [5.7-6.8]
I20260709 13:06:15.395967 15974 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:15.362439 19332 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:15.352646 19265 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:15.363729 19328 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:15.364583 19362 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:15.366329 19352 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:15.404819 15825 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:15.426331 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:15.431638 16004 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:15.408813 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:06:15.423022 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 243828us
I20260709 13:06:15.424340 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:15.424650 15974 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:15.435077 16005 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:15.369215 19365 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:15.375720 19413 raft_consensus.cc:697] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:15.398141 19164 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:15.394874 19364 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:15.426445 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:15.434038 16564 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:15.458534 16010 raft_consensus.cc:2393] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 7: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:15.441745 15855 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
I20260709 13:06:15.448758 16567 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:15.403137 19415 raft_consensus.cc:740] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:06:15.468011 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:15.430994 19398 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:15.398532 19197 consensus_queue.cc:1048] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 13:06:15.473863 16569 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:15.426318 19362 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:15.404666 19408 consensus_queue.cc:1048] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:06:15.433981 19422 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:15.449589 19388 raft_consensus.cc:2749] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:15.450057 19164 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:15.486261 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 52548us
I20260709 13:06:15.487106 15825 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:15.482390 16564 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:15.461205 19212 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:15.493407 15825 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:15.494185 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 124134us
I20260709 13:06:15.462450 19340 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:15.496122 16535 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:15.473958 19367 raft_consensus.cc:2804] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:15.505589 15825 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:15.484174 19362 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:06:15.512755 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:15.512784 15828 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.016s	sys 0.027s
W20260709 13:06:15.530325 15828 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.016s	sys 0.028s
I20260709 13:06:15.513576 19329 raft_consensus.cc:2804] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:15.537784 19329 raft_consensus.cc:697] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:15.538498 16533 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:15.509932 19421 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:15.495060 19422 raft_consensus.cc:697] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 8 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:15.500825 19212 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:15.520078 18958 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:15.524084 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:15.524856 19413 consensus_queue.cc:237] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:15.461404 19328 consensus_queue.cc:1048] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.035s
W20260709 13:06:15.532809 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:15.539561 16004 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
I20260709 13:06:15.492342 19164 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:15.540192 19367 raft_consensus.cc:697] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 10 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:15.512017 19340 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:15.495011 19418 consensus_queue.cc:1059] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.125s
I20260709 13:06:15.620357 19421 raft_consensus.cc:697] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 7 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:15.538733 19329 consensus_queue.cc:237] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:15.624635 19365 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:15.624848 19427 raft_consensus.cc:2749] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:15.625407 19419 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:15.625514 19406 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:15.680826 19431 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:15.680487 19367 consensus_queue.cc:237] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:06:15.681385 19431 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:15.682047 19406 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:15.626544 19422 consensus_queue.cc:237] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:15.629141 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:15.632958 15859 raft_consensus.cc:1275] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:06:15.685176 15859 pending_rounds.cc:85] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8: Aborting all ops after (but not including) 7
I20260709 13:06:15.685266 19340 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:15.685456 15859 pending_rounds.cc:107] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:06:15.654067 19349 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.685855 15859 raft_consensus.cc:2889] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:15.686273 19349 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:15.688084 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:15.688463 19349 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:15.500828 19223 raft_consensus.cc:2749] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:15.690280 19418 raft_consensus.cc:3055] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:15.690487 15858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1869) took 2220 ms. Trace:
I20260709 13:06:15.690812 19418 raft_consensus.cc:740] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:15.690934 15858 rpcz_store.cc:276] 0709 13:06:13.470067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:13.815060 (+344993us) service_pool.cc:224] Handling call
0709 13:06:13.878009 (+ 62949us) spinlock_profiling.cc:243] Waited 2.31 ms on lock 0x7b5800005798. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef381fc0 00007f73ef36c929 00007f73ef36c3d2 00007f73e88e3bd4
0709 13:06:13.878209 (+   200us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:15.075009 (+1196800us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:15.690268 (+615259us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400218c54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:15.690435 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2747978240}
I20260709 13:06:15.692342 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:15.692021 19418 consensus_queue.cc:260] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:15.693446 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:15.693544 19418 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:15.694010 15858 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
W20260709 13:06:15.697155 19364 consensus_queue.cc:1175] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8430db80a39d4868bcf7641c16de35ea" responder_term: 5 status { last_received { term: 5 index: 5 } last_committed_idx: 4 last_received_current_leader { term: 5 index: 5 } }
I20260709 13:06:15.696930 19421 consensus_queue.cc:237] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:15.698203 19419 raft_consensus.cc:697] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 10 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
W20260709 13:06:15.700651 19403 log.cc:927] Time spent T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.858s	user 0.003s	sys 0.001s
I20260709 13:06:15.707381 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:15.708930 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:15.671387 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:15.710035 15931 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:06:15.625552 19212 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:15.713511 16567 raft_consensus.cc:1275] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:06:15.636881 19332 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:15.687464 16010 raft_consensus.cc:1240] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:06:15.713904 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:15.710803 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:15.714504 16561 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:06:15.712039 15827 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:15.712163 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:06:15.665334 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 268229us
I20260709 13:06:15.720731 15856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 1978) took 1115 ms. Trace:
I20260709 13:06:15.721196 15856 rpcz_store.cc:276] 0709 13:06:14.605810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.605947 (+   137us) service_pool.cc:224] Handling call
0709 13:06:15.720543 (+1114596us) spinlock_profiling.cc:243] Waited 644 ms on lock 0x7b54004272d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:15.720665 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1481088256}
I20260709 13:06:15.723834 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:15.726097 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:15.728195 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 189779us
I20260709 13:06:15.726788 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:15.730043 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:15.730528 16564 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:15.730649 16568 raft_consensus.cc:2410] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:06:15.732638 16570 raft_consensus.cc:1275] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:06:15.732852 16563 raft_consensus.cc:2393] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 6: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:15.733090 16533 leader_election.cc:304] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:15.683564 19431 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:15.742740 16564 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:06:15.744807 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:15.746554 16010 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:06:15.744535 19350 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:15.753558 15900 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:15.758369 18861 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:15.753113 19365 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:15.758870 18861 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:15.759508 19365 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:15.759541 19431 raft_consensus.cc:2749] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:15.760955 18861 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:15.756969 19318 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.761552 19365 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:15.761801 19318 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:15.725399 19332 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:15.762149 19223 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.764285 19223 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:15.767813 19318 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:15.756639 19210 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:15.769891 19390 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:15.770418 19390 raft_consensus.cc:697] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 9 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:15.770248 19210 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:15.770577 19332 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:15.772282 19210 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:15.773308 16002 raft_consensus.cc:1275] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 13:06:15.773686 16002 pending_rounds.cc:85] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting all ops after (but not including) 7
I20260709 13:06:15.773895 16002 pending_rounds.cc:107] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:06:15.774107 16002 raft_consensus.cc:2889] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:15.774412 16533 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:15.757745 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 45787.9us
I20260709 13:06:15.725630 19362 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:15.699286 19419 consensus_queue.cc:237] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:15.809507 16533 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:15.839896 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:15.782193 15900 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:06:15.848405 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:15.848723 16563 raft_consensus.cc:2393] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 7: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
W20260709 13:06:15.848771 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 66667.5us
I20260709 13:06:15.779244 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:15.839380 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:15.849567 16564 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:06:15.750036 19349 consensus_queue.cc:1059] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:06:15.828060 15827 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:15.814556 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:15.849817 16568 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:06:15.851977 16010 raft_consensus.cc:2393] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 8: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:15.852068 19229 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.854195 19210 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:15.854626 19210 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:15.854440 19229 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:15.855018 19210 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:15.854911 19406 consensus_queue.cc:1059] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:15.855705 19362 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:15.777980 19350 consensus_queue.cc:1059] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.078s
I20260709 13:06:15.860299 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:15.861148 15935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 677) took 1249 ms. Trace:
I20260709 13:06:15.840513 19332 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.862212 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:15.864814 16007 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:06:15.865022 15928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1905) took 1479 ms. Trace:
I20260709 13:06:15.852296 19408 raft_consensus.cc:2804] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:15.865357 15928 rpcz_store.cc:276] 0709 13:06:14.385866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.393381 (+  7515us) service_pool.cc:224] Handling call
0709 13:06:14.581125 (+187744us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:14.602161 (+ 21036us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:14.602795 (+   634us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:14.617540 (+ 14745us) log.cc:844] Serialized 26 byte log entry
0709 13:06:14.850590 (+233050us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:15.290791 (+440201us) log.cc:844] Serialized 14 byte log entry
0709 13:06:15.291291 (+   500us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:15.291373 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:15.291411 (+    38us) raft_consensus.cc:1685] finished
0709 13:06:15.291422 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:15.864300 (+572878us) spinlock_profiling.cc:243] Waited 2.05 ms on lock 0x7b180043eb20. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72892a6 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4
0709 13:06:15.864844 (+   544us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b5400328f50. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:15.864947 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2342,"prepare.queue_time_us":647010,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":27972,"raft.run_cpu_time_us":329,"raft.run_wall_time_us":18863,"spinlock_wait_cycles":435718144,"thread_start_us":422609,"threads_started":2,"wal-append.queue_time_us":526812}
W20260709 13:06:15.867569 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 58185us
I20260709 13:06:15.851440 15900 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:15.869514 19427 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:15.793545 19149 consensus_queue.cc:1059] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.081s
I20260709 13:06:15.875272 19149 raft_consensus.cc:3055] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:15.794798 19442 raft_consensus.cc:2749] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:15.793300 15902 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:15.877938 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 84733.9us
I20260709 13:06:15.878834 15902 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:15.879230 19427 raft_consensus.cc:2749] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:15.861768 19332 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:15.871524 16533 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:15.792629 19352 raft_consensus.cc:2749] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:15.872193 15827 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:15.861577 15935 rpcz_store.cc:276] 0709 13:06:14.612072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.337267 (+725195us) service_pool.cc:224] Handling call
0709 13:06:15.337596 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:15.357565 (+ 19969us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:15.495363 (+137798us) log.cc:844] Serialized 14 byte log entry
0709 13:06:15.755396 (+260033us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:15.755441 (+    45us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:15.755486 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:15.860964 (+105478us) spinlock_profiling.cc:243] Waited 19.7 ms on lock 0x7b54002c2c54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:15.861001 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:15.861101 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25728,"prepare.queue_time_us":166393,"prepare.run_cpu_time_us":23,"prepare.run_wall_time_us":24,"raft.queue_time_us":29303,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":77,"spinlock_wait_cycles":45229440,"thread_start_us":358573,"threads_started":2,"wal-append.queue_time_us":254116}
I20260709 13:06:15.870733 19406 raft_consensus.cc:3055] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:15.881494 19406 raft_consensus.cc:740] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:15.882035 19350 raft_consensus.cc:2749] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:15.882761 19350 raft_consensus.cc:3055] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:15.882587 19406 consensus_queue.cc:260] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:15.883579 19408 raft_consensus.cc:697] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 9 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:15.884560 19408 consensus_queue.cc:237] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:15.886530 19229 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:15.887585 19406 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:15.893039 19164 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:15.893966 19222 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:15.896523 15937 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53636 (request call id 674) took 1958 ms (client timeout 1851 ms). Trace:
W20260709 13:06:15.897130 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 45753.7us
I20260709 13:06:15.881744 16533 leader_election.cc:400] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 7. Current term is 8.
I20260709 13:06:15.883167 19350 raft_consensus.cc:740] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:06:15.896910 15937 rpcz_store.cc:269] 0709 13:06:13.938041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:13.943281 (+  5240us) service_pool.cc:224] Handling call
0709 13:06:15.896330 (+1953049us) spinlock_profiling.cc:243] Waited 87.6 ms on lock 0x7b5400153e54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:15.896430 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":201372032}
I20260709 13:06:15.895383 19164 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:15.875734 19149 raft_consensus.cc:740] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:15.881676 19362 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:15.912472 15827 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:15.915822 15827 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:15.916713 15827 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:15.917336 15827 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:15.922432 15862 raft_consensus.cc:1240] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:15.923035 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:15.923611 16007 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:06:15.908155 19222 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:15.925354 15827 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:15.928001 19222 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:15.928505 19222 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:15.928604 19024 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:15.929117 19024 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:15.929463 19024 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:15.908397 19352 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:06:15.931953 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 50335.3us
I20260709 13:06:15.925792 19431 raft_consensus.cc:2804] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:15.932850 19431 raft_consensus.cc:697] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 7 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:15.933161 19441 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:15.933466 19212 raft_consensus.cc:2804] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:15.933620 19441 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:15.934036 19212 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:15.934122 19441 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:15.934557 19212 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:15.935838 19437 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:15.936395 19437 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:15.781114 19390 consensus_queue.cc:237] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:06:15.908202 19318 raft_consensus.cc:2749] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:15.933010 16533 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:06:15.943531 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:15.909585 19229 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.792418 19136 consensus_queue.cc:1048] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.129s
I20260709 13:06:15.952278 19149 consensus_queue.cc:260] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:15.944171 19340 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:15.962328 16533 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:15.963001 19149 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:15.963533 16533 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:15.964294 19442 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:15.963850 19350 consensus_queue.cc:260] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:15.965067 19350 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:15.962822 19340 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:15.968060 16533 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:15.969676 16533 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:15.943048 19419 raft_consensus.cc:2804] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:15.981055 16533 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:15.982949 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 104204us
I20260709 13:06:15.983765 15902 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:15.952515 19329 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:15.920040 19421 consensus_queue.cc:1048] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:06:15.986120 15902 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:15.924480 18861 consensus_queue.cc:1048] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 13:06:15.970824 19427 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:15.950707 19164 consensus_queue.cc:1059] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
W20260709 13:06:15.993319 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 50613.5us
I20260709 13:06:15.996501 16565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 912) took 1867 ms. Trace:
I20260709 13:06:15.997031 16565 rpcz_store.cc:276] 0709 13:06:14.129419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.158370 (+ 28951us) service_pool.cc:224] Handling call
0709 13:06:14.158719 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:15.312799 (+1154080us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:15.996169 (+683370us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54007157d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:15.996423 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2652449536}
I20260709 13:06:15.979290 19419 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:15.984669 18958 raft_consensus.cc:2749] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:16.002101 15902 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:15.985651 19437 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:15.987470 19362 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:15.960631 19229 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:16.006929 15902 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:16.000099 19349 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:16.003856 19450 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:16.008989 15902 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:16.009940 19408 raft_consensus.cc:2804] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:16.003162 19419 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:16.018287 19329 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.009284 19427 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:16.019301 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:16.017748 19229 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:16.029141 19164 consensus_queue.cc:1059] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:06:16.036414 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 55479.9us
I20260709 13:06:16.037989 16533 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:16.040788 16533 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:16.040110 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:16.042765 16565 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:06:16.043322 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:16.036870 19329 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:16.026471 19437 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.064679 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:16.064563 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:16.066020 15853 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 7.
I20260709 13:06:16.066808 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:16.060113 15827 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:16.060981 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:16.030424 19362 raft_consensus.cc:2749] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:16.022650 19359 consensus_queue.cc:260] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:16.069612 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:16.071099 15856 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:16.072233 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:16.072597 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:16.075901 15853 raft_consensus.cc:2393] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 7: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:16.077014 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:16.079054 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:16.081867 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:16.082520 15854 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 7: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:16.083266 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:16.088090 15935 raft_consensus.cc:1275] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:06:16.089938 15827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:16.090302 15827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:16.062947 18958 raft_consensus.cc:2749] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:16.091845 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:16.100229 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:16.100445 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:16.100834 15856 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6.
I20260709 13:06:16.026783 19431 consensus_queue.cc:237] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:16.102589 19447 log.cc:927] Time spent T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:06:16.103104 19455 raft_consensus.cc:2804] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:16.083210 19229 raft_consensus.cc:2804] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:16.120200 15855 raft_consensus.cc:2410] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:06:16.104646 19329 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:16.073575 19437 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:16.083359 19210 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:16.126335 19212 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:16.145046 16007 raft_consensus.cc:1275] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:06:16.123199 19229 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:16.145246 19210 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:16.145787 19229 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:16.148854 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:16.149333 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:16.149602 16010 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:16.150027 16007 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:16.153964 19367 consensus_queue.cc:1048] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:16.154294 19340 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:16.154385 19418 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:16.157495 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:16.158262 16008 raft_consensus.cc:2368] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 7. Current term is 8.
I20260709 13:06:16.155079 19418 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:16.163393 19455 raft_consensus.cc:697] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:16.164848 19455 consensus_queue.cc:237] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:16.165405 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:16.165714 19380 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:16.166419 19210 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:16.166903 19210 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:16.168282 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:16.168557 15827 leader_election.cc:400] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 7. Current term is 8.
I20260709 13:06:16.168848 15827 leader_election.cc:403] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:16.121803 18958 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:16.132236 19329 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.142495 19024 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:16.163558 19418 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:16.170532 19329 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:16.171912 19024 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:16.170145 19367 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:06:16.172770 19332 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:16.145251 19212 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:16.174638 15854 raft_consensus.cc:2368] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 7. Current term is 9.
I20260709 13:06:16.169503 18958 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:16.178746 19210 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:16.179765 15856 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:43504
I20260709 13:06:16.180184 15859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 1982) took 1547 ms. Trace:
I20260709 13:06:16.180510 15856 raft_consensus.cc:606] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Received request to transfer leadership
I20260709 13:06:16.180927 15856 raft_consensus.cc:614] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:16.180634 15859 rpcz_store.cc:276] 0709 13:06:14.633074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.633201 (+   127us) service_pool.cc:224] Handling call
0709 13:06:14.633458 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:15.686493 (+1053035us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:16.179911 (+493418us) spinlock_profiling.cc:243] Waited 999 ms on lock 0x7b54003b6d54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:16.180117 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2297776896}
I20260709 13:06:16.183637 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:16.184052 15859 raft_consensus.cc:1275] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:06:16.184839 19223 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:16.185194 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:16.185837 16008 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:06:16.188043 19238 consensus_queue.cc:1048] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:16.188966 16007 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
W20260709 13:06:16.189358 19444 log.cc:927] Time spent T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.282s	user 0.000s	sys 0.001s
I20260709 13:06:16.189898 19427 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:06:16.190685 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:16.193521 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:16.194115 15860 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:16.194525 19419 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:16.195825 15933 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 1901) took 1844 ms (client timeout 1916 ms). Trace:
I20260709 13:06:16.196033 19222 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:16.196244 15933 rpcz_store.cc:269] 0709 13:06:14.353010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.366557 (+ 13547us) service_pool.cc:224] Handling call
0709 13:06:16.195587 (+1829030us) spinlock_profiling.cc:243] Waited 931 ms on lock 0x7b54001a72d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:16.195769 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2141461760}
I20260709 13:06:16.196596 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:16.199146 19317 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.199314 19223 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.199584 19349 raft_consensus.cc:2749] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:16.199719 19317 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:16.199868 19223 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:16.212193 15827 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:16.213584 15827 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:06:16.215731 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 292710us
I20260709 13:06:16.223793 15860 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:16.240247 16010 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:06:16.242621 15825 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:16.244585 15825 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:16.245956 15827 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:16.246201 19441 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:16.247648 15862 raft_consensus.cc:2393] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 7: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:16.248373 19332 raft_consensus.cc:2804] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:06:16.248807 19446 log.cc:927] Time spent T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.243s	user 0.000s	sys 0.004s
I20260709 13:06:16.248308 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:16.249852 15860 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:16.250722 19415 consensus_queue.cc:260] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:16.255205 15976 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:16.256307 15930 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 1896) took 1948 ms (client timeout 1826 ms). Trace:
I20260709 13:06:16.264479 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:16.273228 18968 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:16.273729 18968 raft_consensus.cc:697] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:16.273962 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:16.274595 18968 consensus_queue.cc:237] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:16.275068 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:16.275876 18861 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:16.276176 15976 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:16.276392 18861 raft_consensus.cc:697] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 7 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:16.276999 19372 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:16.277513 19372 raft_consensus.cc:697] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 7 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:16.277336 18861 consensus_queue.cc:237] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:16.278142 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:16.278844 15934 raft_consensus.cc:2393] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 10: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:16.278472 19372 consensus_queue.cc:237] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:16.279280 15934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 1859) took 1685 ms. Trace:
I20260709 13:06:16.279626 15934 rpcz_store.cc:276] 0709 13:06:14.594254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.094957 (+500703us) service_pool.cc:224] Handling call
0709 13:06:16.278004 (+1183047us) spinlock_profiling.cc:243] Waited 1.23 ms on lock 0x7b5800003698. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:06:16.279209 (+  1205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2832640}
W20260709 13:06:16.272279 15930 rpcz_store.cc:269] 0709 13:06:14.307763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.308027 (+   264us) service_pool.cc:224] Handling call
0709 13:06:16.256122 (+1948095us) spinlock_profiling.cc:243] Waited 763 ms on lock 0x7b5400099cd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:16.256242 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1755104000}
I20260709 13:06:16.280354 15976 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:16.280604 19419 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:16.280704 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:16.281539 19212 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:16.281836 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:16.281975 19212 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:16.282367 15930 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:16.282402 19212 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:16.257217 19329 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:16.258817 15936 raft_consensus.cc:1240] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:06:16.261459 16561 raft_consensus.cc:1240] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:16.262826 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:16.285038 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:16.285712 15935 raft_consensus.cc:2393] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 6: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:16.285905 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:16.263820 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:16.286471 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:16.286880 15937 raft_consensus.cc:2393] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 8: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:16.287083 15935 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:06:16.287648 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:16.288373 19445 log.cc:927] Time spent T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.378s	user 0.002s	sys 0.000s
I20260709 13:06:16.288760 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:16.289922 15937 raft_consensus.cc:1240] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:16.264454 16562 raft_consensus.cc:1275] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:06:16.264760 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:16.265436 19387 raft_consensus.cc:3055] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:16.265763 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:16.266074 19024 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.266273 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:16.266662 19380 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:16.295022 19387 raft_consensus.cc:740] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:06:16.267439 16006 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41662 (request call id 1972) took 2122 ms (client timeout 2319 ms). Trace:
I20260709 13:06:16.267529 19437 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:16.268034 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:16.268471 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:16.199091 19222 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:16.270002 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:16.270372 19340 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:16.271600 19332 raft_consensus.cc:697] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 8 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:16.268064 19415 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:16.256506 16565 raft_consensus.cc:1275] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:06:16.283566 15857 raft_consensus.cc:1275] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:06:16.280629 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:16.326893 15860 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:16.287096 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:16.341559 15900 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:16.343274 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:16.344080 16565 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:16.345476 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:16.346889 15933 raft_consensus.cc:2393] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 7: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:16.346264 16567 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:06:16.355484 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:16.349887 15974 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:16.359157 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:16.360083 15827 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:16.298035 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:16.361096 15856 raft_consensus.cc:1240] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 7 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 6. Current term is 7. Ops: []
W20260709 13:06:16.266620 19410 log.cc:927] Time spent T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 1.024s	user 0.002s	sys 0.007s
I20260709 13:06:16.306013 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:16.306537 16006 rpcz_store.cc:269] 0709 13:06:14.145603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.378405 (+232802us) service_pool.cc:224] Handling call
0709 13:06:16.267263 (+1888858us) spinlock_profiling.cc:243] Waited 940 ms on lock 0x7b54001f7cd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:16.267387 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2161086720}
I20260709 13:06:16.307102 16569 raft_consensus.cc:2393] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 9: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:16.299619 15930 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:06:16.315611 19024 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:16.316954 15825 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:16.317308 18958 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:16.305925 19387 consensus_queue.cc:260] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:16.319484 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:16.347144 19418 raft_consensus.cc:2764] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:06:16.360173 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:16.355368 19350 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:16.362681 15827 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:16.359800 15928 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:06:16.364260 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:16.242776 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 213058us
I20260709 13:06:16.367331 16006 raft_consensus.cc:1240] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:16.367985 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:16.373832 16533 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:16.373588 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 56740.6us
I20260709 13:06:16.390730 16007 raft_consensus.cc:2393] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 8: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:16.391213 15825 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:16.393182 15825 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:16.394037 15825 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:16.390377 15902 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:16.378173 15933 raft_consensus.cc:1275] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:06:16.379650 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:16.396234 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:16.396378 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:16.319289 19332 consensus_queue.cc:237] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:16.396718 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:16.396982 15930 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:16.397300 15928 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:16.362093 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:16.397855 16570 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:16.372627 19095 raft_consensus.cc:2804] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:16.305444 19380 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:16.380954 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:16.381215 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:16.418979 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:16.399897 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:16.400656 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:16.242956 19450 raft_consensus.cc:697] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:16.371991 19408 raft_consensus.cc:697] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:16.375169 19340 raft_consensus.cc:2804] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:16.419413 15935 raft_consensus.cc:1275] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:06:16.380157 19431 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:06:16.466926 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 72926.6us
I20260709 13:06:16.420347 16003 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:06:16.408142 15930 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:06:16.419013 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:16.467950 15825 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:16.468518 15929 raft_consensus.cc:2393] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 7: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
W20260709 13:06:16.469338 15902 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.103.4:43643 timed out after 1.731s (SENT) [suppressed 22 similar messages]
I20260709 13:06:16.420447 16007 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6.
I20260709 13:06:16.456697 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:16.457582 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:16.458472 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:16.471099 16007 raft_consensus.cc:1275] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:06:16.360183 19437 consensus_queue.cc:1059] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:06:16.453370 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:16.471952 15933 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:16.420923 15931 raft_consensus.cc:2368] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 7. Current term is 8.
I20260709 13:06:16.362983 19327 consensus_queue.cc:1048] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:06:16.392051 19210 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:16.479310 16570 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:06:16.375068 19242 consensus_queue.cc:1048] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 13:06:16.488155 16008 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:16.483989 16564 raft_consensus.cc:1240] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:16.420836 19365 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:16.494855 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:16.472723 19455 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:06:16.526132 15827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:16.460153 19422 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.775s	user 0.012s	sys 0.009s
I20260709 13:06:16.412802 19238 raft_consensus.cc:2804] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:16.400017 19419 raft_consensus.cc:2749] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:16.409603 19364 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:16.485318 15825 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:16.409360 19350 raft_consensus.cc:697] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 7 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:16.630398 16007 raft_consensus.cc:1275] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:06:16.472014 19340 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:16.410229 19383 consensus_queue.cc:1048] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.082s
I20260709 13:06:16.490624 19421 raft_consensus.cc:2749] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:16.599957 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:16.658324 16567 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:16.600970 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:16.486222 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 112450us
W20260709 13:06:16.640419 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 282969us
I20260709 13:06:16.495296 19442 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:06:16.668351 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:16.669692 19329 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.670297 19329 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:16.672549 19329 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:16.503988 19455 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:06:16.606004 19422 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.921s	user 0.016s	sys 0.009s
I20260709 13:06:16.702291 16004 raft_consensus.cc:1240] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:16.607048 19238 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:16.644227 19413 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.949s	user 0.017s	sys 0.013s
I20260709 13:06:16.717378 16008 raft_consensus.cc:2410] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 8 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:06:16.619508 19429 raft_consensus.cc:2804] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:16.647394 19340 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:16.647140 19265 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:16.741899 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 370693us
I20260709 13:06:16.742189 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:06:16.738108 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 77725.4us
I20260709 13:06:16.656487 19441 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:16.743413 16533 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:16.744644 16567 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:16.754891 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:16.755625 16567 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:16.755313 16566 raft_consensus.cc:2393] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 7: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:16.705613 19455 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:16.705075 19419 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:16.722877 19238 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:06:16.726074 19413 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.031s	user 0.017s	sys 0.013s
I20260709 13:06:16.761426 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:16.763650 15974 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:16.775822 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:16.773658 16535 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:16.784469 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:16.785041 16011 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:16.785444 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:16.786149 15974 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:16.786353 16003 raft_consensus.cc:2393] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 6: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:16.793926 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1850) took 1084 ms. Trace:
I20260709 13:06:16.795182 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:16.794322 16002 rpcz_store.cc:276] 0709 13:06:15.709853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.720257 (+ 10404us) service_pool.cc:224] Handling call
0709 13:06:15.732417 (+ 12160us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:15.774525 (+ 42108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:16.793732 (+1019207us) spinlock_profiling.cc:243] Waited 40.5 ms on lock 0x7b54000aafd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:16.793874 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":93100288}
I20260709 13:06:16.749260 19417 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:16.817914 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 74547.7us
I20260709 13:06:16.819875 16533 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:16.820325 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:16.743211 19414 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.811642 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:16.824180 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:16.825304 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:16.778616 18958 raft_consensus.cc:2749] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:16.797816 15860 raft_consensus.cc:1240] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:16.813031 16567 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:06:16.757526 19429 raft_consensus.cc:697] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 7 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:16.829345 16567 raft_consensus.cc:1275] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:06:16.793030 19427 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:16.834322 16002 raft_consensus.cc:1275] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:06:16.841866 16565 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:06:16.842877 16566 raft_consensus.cc:1240] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:06:16.721733 19450 consensus_queue.cc:237] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:16.758579 19265 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:16.807205 19095 raft_consensus.cc:697] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:16.721136 19408 consensus_queue.cc:237] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:16.848140 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:16.853760 16533 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:06:16.857378 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:16.792852 19331 consensus_queue.cc:1059] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:06:16.863780 15928 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:16.864212 16002 raft_consensus.cc:1275] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:06:16.852989 19242 raft_consensus.cc:2749] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:16.842330 19427 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:16.835786 19469 consensus_queue.cc:1048] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:06:16.828915 19417 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:16.808851 19350 consensus_queue.cc:237] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:16.871579 19390 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.908520 19429 consensus_queue.cc:237] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:16.845355 19414 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:16.890069 19329 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:06:16.914758 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 127590us
I20260709 13:06:16.871856 19380 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:16.884296 19413 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:16.888029 19427 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:16.929448 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 75756us
I20260709 13:06:16.945108 16533 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:16.945317 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:16.946450 16565 raft_consensus.cc:2393] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 8: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:16.946727 15854 raft_consensus.cc:1240] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 7 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 6. Current term is 7. Ops: []
W20260709 13:06:16.948830 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:16.927769 19470 log.cc:927] Time spent T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.321s	user 0.001s	sys 0.000s
I20260709 13:06:16.919418 19467 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.922885 19331 raft_consensus.cc:3055] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:06:16.942620 19468 log.cc:927] Time spent T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.247s	user 0.001s	sys 0.000s
I20260709 13:06:16.962611 16002 raft_consensus.cc:1240] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:16.964836 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:16.963150 15974 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:16.939577 19024 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:16.981725 16011 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
W20260709 13:06:16.989698 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 44694.6us
I20260709 13:06:16.990324 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:16.991099 16533 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:16.968799 19467 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:16.968592 19095 consensus_queue.cc:237] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:16.955404 19463 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.009s	sys 0.000s
W20260709 13:06:16.956779 19476 log.cc:927] Time spent T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:06:17.004523 16011 raft_consensus.cc:1240] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:16.961907 19414 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:17.008294 16564 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:16.960235 19329 raft_consensus.cc:2749] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:16.927302 19390 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
W20260709 13:06:17.005378 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 49369.4us
W20260709 13:06:17.010911 15902 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.019s	sys 0.009s
I20260709 13:06:16.962803 19331 raft_consensus.cc:740] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:06:17.006604 19463 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.010s	sys 0.000s
W20260709 13:06:17.022871 15902 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.019s	sys 0.009s
I20260709 13:06:16.954293 19413 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:17.026959 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 557602us
I20260709 13:06:17.041401 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:17.076063 19492 raft_consensus.cc:2764] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
W20260709 13:06:17.076892 19471 log.cc:927] Time spent T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.245s	user 0.000s	sys 0.001s
W20260709 13:06:17.079764 19486 log.cc:927] Time spent T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 13:06:17.079996 19481 raft_consensus.cc:2749] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:17.081213 19490 raft_consensus.cc:2749] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:17.090715 19273 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:17.091027 19390 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
I20260709 13:06:17.092180 19469 consensus_queue.cc:1048] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:17.094049 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:17.094762 15860 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:17.094633 19469 consensus_queue.cc:1048] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:06:17.096539 19367 consensus_meta.cc:287] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.924s	user 0.010s	sys 0.000s
I20260709 13:06:17.096997 19367 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 7. Current term is 8.
I20260709 13:06:17.098351 19317 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:17.106148 19229 consensus_meta.cc:287] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.960s	user 0.009s	sys 0.000s
I20260709 13:06:17.106593 19229 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:17.107581 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:17.109776 19385 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:17.110594 19212 consensus_meta.cc:287] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.828s	user 0.008s	sys 0.002s
I20260709 13:06:17.112368 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:17.113265 16563 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
W20260709 13:06:17.115828 19473 log.cc:927] Time spent T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
I20260709 13:06:17.116014 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:17.116662 16568 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:06:17.111013 19212 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:17.119918 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:17.120154 15859 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:17.120424 16565 raft_consensus.cc:1275] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:06:17.122741 15854 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:17.131331 16003 raft_consensus.cc:1240] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 7. Current term is 8. Ops: []
W20260709 13:06:17.111115 19474 log.cc:927] Time spent T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
W20260709 13:06:17.164283 15975 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.697s	user 0.009s	sys 0.012s
I20260709 13:06:17.121387 19329 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:06:17.166036 15975 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.009s	sys 0.012s
I20260709 13:06:17.172995 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:17.180050 15858 raft_consensus.cc:2468] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
W20260709 13:06:17.189568 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 175200us
I20260709 13:06:17.165723 19491 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.164572 19265 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:17.135872 19349 consensus_queue.cc:1059] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 13:06:17.232016 19349 consensus_queue.cc:1243] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer 31882606e87447fb9a36bbf625129fe8 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:06:17.232631 19349 raft_consensus.cc:3055] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:17.233045 19349 raft_consensus.cc:740] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:17.234050 19349 consensus_queue.cc:260] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:17.178541 19413 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:17.197142 19365 consensus_meta.cc:287] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.700s	user 0.005s	sys 0.006s
I20260709 13:06:17.235931 19365 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:17.205865 19327 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.236873 19327 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
W20260709 13:06:17.237825 19415 consensus_meta.cc:287] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.887s	user 0.004s	sys 0.008s
I20260709 13:06:17.239053 16005 raft_consensus.cc:1275] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:06:17.239125 19327 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:17.240033 15854 raft_consensus.cc:1240] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:17.206310 19238 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:17.177539 19331 consensus_queue.cc:260] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:17.209474 19417 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:17.212152 19340 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:17.213333 19501 log.cc:927] Time spent T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:06:17.242741 19349 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:17.213728 19491 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:17.245714 19491 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:17.245852 19340 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:16.929013 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:17.166234 19455 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:17.199398 19326 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.205899 19478 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.253739 19478 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:17.255962 19478 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:17.218536 19329 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:17.256623 19329 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:17.257206 19455 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:17.258697 19329 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:17.260013 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:17.259949 19326 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:17.261898 19326 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:06:17.218746 19475 log.cc:927] Time spent T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.202s	user 0.000s	sys 0.001s
I20260709 13:06:17.269104 19427 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:17.219911 19359 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:17.231208 19212 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:17.242863 19238 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:17.282326 19419 raft_consensus.cc:2749] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:06:17.290223 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:17.294014 15936 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 1853) took 2717 ms. Trace:
I20260709 13:06:17.294544 15936 rpcz_store.cc:276] 0709 13:06:14.576316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.145637 (+569321us) service_pool.cc:224] Handling call
0709 13:06:16.258567 (+1112930us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:16.284102 (+ 25535us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:17.293794 (+1009692us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400099cd0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:17.293944 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2424806784}
I20260709 13:06:17.298305 16004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1847) took 1614 ms. Trace:
I20260709 13:06:17.298820 16004 rpcz_store.cc:276] 0709 13:06:15.684173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.720448 (+ 36275us) service_pool.cc:224] Handling call
0709 13:06:15.720791 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:16.702951 (+982160us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:17.298045 (+595094us) spinlock_profiling.cc:243] Waited 981 ms on lock 0x7b54002e1dd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:17.298237 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2256713088}
I20260709 13:06:17.300908 15855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58214 (request call id 649) took 1238 ms. Trace:
I20260709 13:06:17.301308 15855 rpcz_store.cc:276] 0709 13:06:16.062608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:16.064976 (+  2368us) service_pool.cc:224] Handling call
0709 13:06:17.300670 (+1235694us) spinlock_profiling.cc:243] Waited 40.2 ms on lock 0x7b540008e154. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:17.300848 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":92522880}
I20260709 13:06:17.302505 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:17.306582 19465 log.cc:927] Time spent T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.802s	user 0.001s	sys 0.000s
I20260709 13:06:17.314926 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:17.315706 15857 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:17.322784 15974 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:17.324052 15853 raft_consensus.cc:1240] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:17.325686 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:17.328291 19357 consensus_queue.cc:1048] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:17.345499 19326 raft_consensus.cc:2749] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:17.346457 15861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1893) took 1300 ms. Trace:
I20260709 13:06:17.346314 19326 consensus_queue.cc:1059] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:17.346845 15861 rpcz_store.cc:276] 0709 13:06:16.045694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:16.068102 (+ 22408us) service_pool.cc:224] Handling call
0709 13:06:16.090753 (+ 22651us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:16.092622 (+  1869us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:16.092675 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:16.093090 (+   415us) log.cc:844] Serialized 52 byte log entry
0709 13:06:16.332480 (+239390us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:06:16.332538 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:16.332626 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:17.305683 (+973057us) spinlock_profiling.cc:243] Waited 91.9 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:06:17.346164 (+ 40481us) raft_consensus.cc:1685] finished
0709 13:06:17.346184 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:17.346396 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1518,"raft.queue_time_us":1627,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":3103,"spinlock_wait_cycles":211434880,"thread_start_us":224104,"threads_started":1,"wal-append.queue_time_us":301820}
I20260709 13:06:17.348094 19326 raft_consensus.cc:3055] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:17.348443 19326 raft_consensus.cc:740] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:17.349170 19326 consensus_queue.cc:260] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:17.350747 19326 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:17.351980 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:17.333716 15974 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:17.354280 15974 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:17.331660 19229 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:17.366096 19357 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:17.366052 19129 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:17.375033 15902 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:17.384503 15861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:58210 (request call id 14) took 1128 ms. Trace:
I20260709 13:06:17.384606 19129 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:17.393436 15861 rpcz_store.cc:276] 0709 13:06:16.245614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:17.350523 (+1104909us) service_pool.cc:224] Handling call
0709 13:06:17.374397 (+ 23874us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:17.393880 19129 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:17.375993 19357 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:17.394769 19357 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:17.374997 19427 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:17.379976 19437 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.883s	user 0.007s	sys 0.007s
W20260709 13:06:17.407642 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 272654us
I20260709 13:06:17.410588 15932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53636 (request call id 681) took 1365 ms. Trace:
I20260709 13:06:17.410975 15932 rpcz_store.cc:276] 0709 13:06:16.045321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:16.060698 (+ 15377us) service_pool.cc:224] Handling call
0709 13:06:17.410442 (+1349744us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b540008acd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:17.410543 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3083886848}
I20260709 13:06:17.412030 15902 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:17.413372 19427 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.413910 19427 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:17.416420 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:17.416711 19223 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:17.418159 19503 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:17.418649 19503 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:17.420948 19503 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:06:17.421217 19472 log.cc:927] Time spent T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.606s	user 0.003s	sys 0.000s
I20260709 13:06:17.423038 19427 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:06:17.426975 19485 log.cc:927] Time spent T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.473s	user 0.001s	sys 0.009s
I20260709 13:06:17.430140 19442 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:17.430594 19442 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:17.430716 16005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1851) took 1720 ms. Trace:
I20260709 13:06:17.431155 16005 rpcz_store.cc:276] 0709 13:06:15.710126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.710527 (+   401us) service_pool.cc:224] Handling call
0709 13:06:15.727050 (+ 16523us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:17.239532 (+1512482us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:17.430508 (+190976us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b54003ece54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:17.430656 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3476229888}
I20260709 13:06:17.407608 19462 raft_consensus.cc:2749] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:17.437837 15856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 1997) took 1244 ms. Trace:
I20260709 13:06:17.438405 15856 rpcz_store.cc:276] 0709 13:06:16.193805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:16.383034 (+189229us) service_pool.cc:224] Handling call
0709 13:06:16.411501 (+ 28467us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:16.661816 (+250315us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:16.661853 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:16.661897 (+    44us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:16.661936 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:17.437606 (+775670us) spinlock_profiling.cc:243] Waited 250 ms on lock 0x7b540008d254. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:17.437650 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:17.437780 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":575164160}
W20260709 13:06:17.448830 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:17.448158 19449 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:17.449980 19449 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.450441 19449 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:17.451596 19201 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.452136 19201 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:17.452734 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:17.452991 19449 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:17.453573 15932 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 9.
I20260709 13:06:17.454219 19201 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:17.456416 15928 raft_consensus.cc:1240] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
W20260709 13:06:17.409420 19437 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.919s	user 0.007s	sys 0.007s
I20260709 13:06:17.462754 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:17.463532 15929 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 7.
I20260709 13:06:17.469305 15902 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:17.481792 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:17.482568 15929 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 7.
I20260709 13:06:17.485302 16533 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:17.485610 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:17.486218 15936 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:17.488912 15932 raft_consensus.cc:1275] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:06:17.470726 19222 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.459663 19263 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:17.491226 16533 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:17.496992 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:17.470932 19380 consensus_queue.cc:1059] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:06:17.498039 19201 consensus_queue.cc:1048] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 13:06:17.498620 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:17.464905 19398 consensus_queue.cc:1059] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 13:06:17.536752 15929 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 7.
I20260709 13:06:17.508951 16569 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0b3676041725488fb6ffad37889b386b"
dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
mode: GRACEFUL
new_leader_uuid: "8430db80a39d4868bcf7641c16de35ea"
 from {username='slave'} at 127.0.0.1:42622
W20260709 13:06:17.500665 15825 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.875s	user 0.020s	sys 0.026s
I20260709 13:06:17.472818 19462 consensus_queue.cc:1059] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:06:17.495078 19408 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:17.513362 19222 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:17.514070 19372 raft_consensus.cc:3055] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:17.515568 19363 consensus_queue.cc:1059] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:17.517652 19129 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:06:17.521485 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:17.521855 19361 consensus_queue.cc:1059] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:17.522145 15862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1895) took 1307 ms. Trace:
I20260709 13:06:17.522200 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:17.522975 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:17.524070 16567 raft_consensus.cc:1240] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:17.525261 15974 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:17.526381 19467 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:17.526743 19165 consensus_queue.cc:1048] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:17.528250 16564 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 LEARNER]: Advancing to term 9
I20260709 13:06:17.527418 15902 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:17.531639 16533 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:17.537846 15825 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.912s	user 0.020s	sys 0.026s
I20260709 13:06:17.562193 15936 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
W20260709 13:06:17.589171 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 1.10387e+06us
I20260709 13:06:17.591292 15825 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:17.537851 16569 raft_consensus.cc:606] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 LEADER]: Received request to transfer leadership to TS 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:17.539710 19442 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:17.540197 19503 consensus_queue.cc:1059] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:17.609191 15854 raft_consensus.cc:1275] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:06:17.497867 19427 raft_consensus.cc:2749] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:06:17.574088 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 48897.5us
I20260709 13:06:17.587325 19273 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:17.605351 19363 raft_consensus.cc:3055] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:17.539156 19495 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:17.621225 19481 consensus_queue.cc:1059] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:06:17.625838 19495 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:17.626235 19495 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:17.539219 19263 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:17.539535 19357 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:17.645772 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:17.646088 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:17.608562 16002 raft_consensus.cc:1275] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:06:17.613545 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 86205.3us
W20260709 13:06:17.615593 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:17.617465 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:17.622357 15856 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 6. (term mismatch)
I20260709 13:06:17.648833 15856 pending_rounds.cc:85] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8: Aborting all ops after (but not including) 5
I20260709 13:06:17.622895 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:17.649775 15932 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:06:17.624562 15933 raft_consensus.cc:1275] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:06:17.627877 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:17.651245 16004 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:06:17.572901 15862 rpcz_store.cc:276] 0709 13:06:16.215130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:16.812200 (+597070us) service_pool.cc:224] Handling call
0709 13:06:16.812558 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:16.812952 (+   394us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:17.016328 (+203376us) log.cc:844] Serialized 14 byte log entry
0709 13:06:17.380352 (+364024us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:17.380708 (+   356us) log.cc:844] Serialized 26 byte log entry
0709 13:06:17.380868 (+   160us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:17.380897 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:17.380957 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:17.521960 (+141003us) raft_consensus.cc:1685] finished
0709 13:06:17.521980 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:17.522095 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10704,"prepare.queue_time_us":363251,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":2326,"raft.run_cpu_time_us":292,"raft.run_wall_time_us":34497,"thread_start_us":424338,"threads_started":2,"wal-append.queue_time_us":489019}
I20260709 13:06:17.655715 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:17.548506 19372 raft_consensus.cc:740] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:17.619437 19408 raft_consensus.cc:697] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:17.659986 15902 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:17.661492 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:17.662705 15902 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:17.632462 15931 raft_consensus.cc:1275] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:06:17.606650 19398 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:17.637686 16570 raft_consensus.cc:2393] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 8: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:17.638379 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:17.668538 16567 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:17.639127 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:17.639909 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:17.643051 15930 raft_consensus.cc:1275] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:06:17.670053 15902 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:17.644809 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:17.645000 15827 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:17.624373 19418 raft_consensus.cc:2804] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:17.654321 15936 raft_consensus.cc:1275] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:06:17.616937 19462 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:06:17.664860 15933 raft_consensus.cc:1275] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:06:17.664602 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:17.665160 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 133601us
I20260709 13:06:17.656473 15862 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:17.672158 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:17.621537 19363 raft_consensus.cc:740] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:06:17.694468 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 49557.2us
I20260709 13:06:17.695349 16533 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:17.716612 16003 raft_consensus.cc:2393] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 8: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:17.717499 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:17.701928 16564 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 LEARNER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:06:17.649165 15856 pending_rounds.cc:107] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 6.6
I20260709 13:06:17.718156 15861 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:06:17.719869 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:17.654012 19514 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:17.733904 16564 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:17.641252 19467 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.721616 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:17.735788 15856 raft_consensus.cc:2917] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Aborting config change with OpId 6.6: Aborted: Op aborted by new leader
I20260709 13:06:17.736140 16533 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:17.736308 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:17.744079 16567 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:17.744113 15856 raft_consensus.cc:2889] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:17.717720 15827 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:17.744303 16007 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:17.735908 19492 consensus_queue.cc:1048] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.004s
I20260709 13:06:17.745894 18732 consensus_queue.cc:1048] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:06:17.746280 15902 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:17.693929 19359 raft_consensus.cc:2749] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:17.748097 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:17.749308 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:17.755020 18732 raft_consensus.cc:2749] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:17.627162 19273 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:17.718789 19457 raft_consensus.cc:2804] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:17.756135 18732 consensus_queue.cc:1048] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:06:17.677508 19361 raft_consensus.cc:2749] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:17.667474 19398 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:17.721272 19450 raft_consensus.cc:2749] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:17.760026 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:17.760864 15860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1901) took 1465 ms. Trace:
I20260709 13:06:17.785000 15860 rpcz_store.cc:276] 0709 13:06:16.295127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:17.142004 (+846877us) service_pool.cc:224] Handling call
0709 13:06:17.192422 (+ 50418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:17.192674 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:17.192979 (+   305us) log.cc:844] Serialized 14 byte log entry
0709 13:06:17.505917 (+312938us) spinlock_profiling.cc:243] Waited 3.21 ms on lock 0x7b1800498d00. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f4823a 00007f73ea3b2a2a
0709 13:06:17.760540 (+254623us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:17.760590 (+    50us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:17.760639 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:17.760700 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:17.760810 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12335,"prepare.queue_time_us":120,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":2007,"raft.queue_time_us":34774,"spinlock_wait_cycles":7384832,"thread_start_us":241527,"threads_started":1}
I20260709 13:06:17.768211 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:17.786404 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:17.786829 15934 raft_consensus.cc:2376] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9. Re-sending same reply.
I20260709 13:06:17.786950 15937 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:17.768358 16564 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:06:17.693617 19513 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:17.770558 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:17.793300 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:17.772262 15808 auto_rebalancer.cc:893] Removing replace marker failed: Replica 31882606e87447fb9a36bbf625129fe8 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 9 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } } pending_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:06:17.771982 19340 consensus_queue.cc:1048] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:17.771914 19467 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:17.773358 19363 consensus_queue.cc:260] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:17.774739 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:17.813449 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:17.699312 19418 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:17.707718 19367 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:17.777753 19273 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:17.721521 19503 raft_consensus.cc:2804] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:17.779472 18732 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.720870 19419 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.722430 19442 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:17.831851 15853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 1999) took 1086 ms. Trace:
W20260709 13:06:17.813100 19516 log.cc:927] Time spent T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:06:17.791003 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:17.794550 15808 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
I20260709 13:06:17.794811 15902 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:17.835435 16563 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:06:17.795986 16535 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:17.736850 19507 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:17.801381 16567 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:06:17.806511 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:17.809957 16006 raft_consensus.cc:1240] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:17.809957 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:17.839076 16004 raft_consensus.cc:2410] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:06:17.839178 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:17.840011 16006 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:17.840332 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:17.840806 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:17.841209 16004 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
W20260709 13:06:17.842034 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:17.841778 19507 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:17.842772 16004 raft_consensus.cc:1275] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 13:06:17.835453 15808 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:06:17.844242 19467 leader_election.cc:290] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:17.844548 19436 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:17.844789 19419 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:17.845263 19363 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:17.846791 19419 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:17.839093 19498 raft_consensus.cc:2749] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:17.827178 19367 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:17.848222 19222 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:17.832258 15853 rpcz_store.cc:276] 0709 13:06:16.745184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:17.197752 (+452568us) service_pool.cc:224] Handling call
0709 13:06:17.323806 (+126054us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:17.324563 (+   757us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:17.831630 (+507067us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b540008e150. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:17.831802 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":288998784}
I20260709 13:06:17.851621 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:17.802060 19513 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:17.813592 15931 raft_consensus.cc:2376] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8. Re-sending same reply.
I20260709 13:06:17.820029 19095 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.814347 15934 raft_consensus.cc:2368] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 7. Current term is 8.
I20260709 13:06:17.816278 15976 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:17.818771 15937 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
W20260709 13:06:17.863330 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 47130.8us
I20260709 13:06:17.862902 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:17.864755 15937 raft_consensus.cc:2376] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8. Re-sending same reply.
I20260709 13:06:17.865962 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:17.866842 15937 raft_consensus.cc:2393] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 7: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:17.825546 19503 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:06:17.868824 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:17.868769 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:17.868206 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:17.871143 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:17.871779 15934 raft_consensus.cc:2376] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6. Re-sending same reply.
I20260709 13:06:17.872776 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:17.873978 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:17.874589 15934 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:17.876036 19359 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:17.876464 19462 consensus_queue.cc:1059] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:17.875494 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:17.819865 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:17.820399 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 87930us
I20260709 13:06:17.878212 15860 raft_consensus.cc:2393] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 7: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:17.878399 16533 leader_election.cc:400] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 election: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 7. Current term is 9.
I20260709 13:06:17.879297 16533 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:17.785111 19398 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:17.880300 19357 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:17.880506 15857 raft_consensus.cc:1240] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 8. Current term is 9. Ops: []
W20260709 13:06:17.881330 19477 log.cc:927] Time spent T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 1.023s	user 0.000s	sys 0.004s
I20260709 13:06:17.882680 15857 raft_consensus.cc:1275] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:06:17.884194 19498 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:06:17.884343 16562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 802) took 1590 ms. Trace:
I20260709 13:06:17.784641 19455 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:17.884850 16562 rpcz_store.cc:276] 0709 13:06:16.296134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:16.296278 (+   144us) service_pool.cc:224] Handling call
0709 13:06:16.296660 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:16.297061 (+   401us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:16.297103 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:16.297622 (+   519us) log.cc:844] Serialized 26 byte log entry
0709 13:06:16.487898 (+190276us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:16.487948 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:16.488042 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:17.880773 (+1392731us) spinlock_profiling.cc:243] Waited 6.34 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:06:17.884062 (+  3289us) raft_consensus.cc:1685] finished
0709 13:06:17.884083 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:17.884282 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7445,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":7738,"spinlock_wait_cycles":14575872,"thread_start_us":189890,"threads_started":1,"wal-append.queue_time_us":523269}
I20260709 13:06:17.887001 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:17.889323 16570 raft_consensus.cc:1240] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:17.825829 18732 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:17.818771 16007 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:06:17.786020 19457 raft_consensus.cc:697] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 9 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:17.806420 19242 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.891913 19242 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:17.892274 19457 consensus_queue.cc:237] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:17.861140 19513 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:17.894004 19242 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:17.810531 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:17.895313 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:17.821300 19408 consensus_queue.cc:237] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:17.896307 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:17.853248 16004 raft_consensus.cc:1275] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:06:17.774731 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:17.746508 19514 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:17.856832 15862 raft_consensus.cc:2393] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 7: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:17.821938 19229 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:17.857955 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:17.860289 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:17.820178 19418 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:17.782145 19372 consensus_queue.cc:260] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:17.826817 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:06:17.912117 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:17.891609 18732 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:17.905686 16006 raft_consensus.cc:1240] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 8. Current term is 9. Ops: [6.4-8.5]
I20260709 13:06:17.915877 16009 raft_consensus.cc:2393] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 6: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:17.915593 15933 raft_consensus.cc:1275] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:06:17.918694 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:17.919584 16562 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:17.913257 16563 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:17.867542 19480 raft_consensus.cc:2804] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:17.928323 16563 raft_consensus.cc:1240] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:17.912540 19492 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:06:17.929911 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:17.915118 19505 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.900197 19503 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:17.867234 19367 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:17.931734 15857 raft_consensus.cc:1240] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:17.937196 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:06:17.950584 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 195572us
I20260709 13:06:17.865841 19095 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:17.943364 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:17.953449 16009 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:17.944367 15931 raft_consensus.cc:1240] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:17.930619 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:17.931351 19372 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:17.932060 19480 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:17.923578 19229 raft_consensus.cc:697] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 9 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:17.965368 16561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 927) took 1505 ms. Trace:
I20260709 13:06:17.962286 16533 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:17.972054 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:17.973726 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:17.957122 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 239463us
I20260709 13:06:17.975791 15827 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:17.938652 15937 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:17.965914 16561 rpcz_store.cc:276] 0709 13:06:16.459635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:16.461061 (+  1426us) service_pool.cc:224] Handling call
0709 13:06:17.221631 (+760570us) spinlock_profiling.cc:243] Waited 5.41 ms on lock 0x7b5800044798. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef381fc0 00007f73ef36c929 00007f73ef36c3d2 00007f73e88e3bd4
0709 13:06:17.221859 (+   228us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:17.479274 (+257415us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:17.479313 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:17.491176 (+ 11863us) log.cc:844] Serialized 26 byte log entry
0709 13:06:17.754015 (+262839us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:17.754049 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:17.754211 (+   162us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:17.957806 (+203595us) raft_consensus.cc:1685] finished
0709 13:06:17.957831 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:17.965297 (+  7466us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":25047,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":104,"spinlock_wait_cycles":12444544,"thread_start_us":478523,"threads_started":2,"wal-append.queue_time_us":344615}
I20260709 13:06:17.979082 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:17.980408 15902 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:17.990974 16535 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:17.983771 19242 consensus_queue.cc:1059] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 13:06:17.993207 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:17.983925 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:17.988369 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:17.992415 19223 consensus_queue.cc:1048] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:17.994793 16561 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:17.974473 15862 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 7.
I20260709 13:06:17.994012 15934 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:06:17.960376 19467 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:17.981649 19480 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:18.010851 16009 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
I20260709 13:06:18.006778 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:18.008983 16561 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:18.022082 15976 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:17.979915 19442 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:17.942164 19492 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:18.024863 19442 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:18.026643 19492 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:18.027547 19442 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:18.028409 19442 raft_consensus.cc:2749] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:06:18.029132 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 53464.7us
I20260709 13:06:18.029671 19357 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:18.033341 19357 raft_consensus.cc:697] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 8 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:18.034214 19357 consensus_queue.cc:237] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:18.030336 19419 consensus_queue.cc:1048] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:18.035526 16010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1868) took 1740 ms. Trace:
I20260709 13:06:18.036837 19419 consensus_queue.cc:1048] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:18.038935 19419 raft_consensus.cc:2804] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:18.039309 19419 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:18.039501 15858 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58214 (request call id 653) took 1975 ms (client timeout 1881 ms). Trace:
I20260709 13:06:18.039652 19419 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:18.042611 19507 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:18.036072 16010 rpcz_store.cc:276] 0709 13:06:16.294715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:16.295252 (+   537us) service_pool.cc:224] Handling call
0709 13:06:16.295551 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:17.456162 (+1160611us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:17.456191 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:17.456227 (+    36us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:17.456257 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:18.035217 (+578960us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b5400382cd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:18.035278 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:18.035455 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2195,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":5020,"spinlock_wait_cycles":2668450688}
W20260709 13:06:18.039922 15858 rpcz_store.cc:269] 0709 13:06:16.064373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:16.064480 (+   107us) service_pool.cc:224] Handling call
0709 13:06:18.039291 (+1974811us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b5400426dd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:18.039408 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2563342080}
I20260709 13:06:18.050096 16569 raft_consensus.cc:1275] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:06:18.051218 19419 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:18.053879 19419 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:18.055061 19263 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:18.056541 15827 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:18.056859 19522 log.cc:927] Time spent T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:06:18.062237 19431 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:18.062705 19431 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:18.063124 19431 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:18.064488 19421 consensus_queue.cc:1048] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:18.065019 19502 raft_consensus.cc:3055] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:18.065434 19502 raft_consensus.cc:740] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:18.066187 19502 consensus_queue.cc:260] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:18.067154 19502 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:18.069430 19229 consensus_queue.cc:237] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:18.043938 15856 raft_consensus.cc:1275] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:06:18.032012 19340 consensus_queue.cc:1048] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:17.942999 19505 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:18.066834 15827 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:17.993532 19242 raft_consensus.cc:3055] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:18.132074 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:18.133749 15932 raft_consensus.cc:2368] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 10. Current term is 11.
I20260709 13:06:18.101274 16561 raft_consensus.cc:1240] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 6. Current term is 7. Ops: []
W20260709 13:06:18.102149 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 121837us
I20260709 13:06:18.106035 15931 raft_consensus.cc:1275] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 13:06:18.106974 19520 log.cc:927] Time spent T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:06:17.989161 19359 consensus_queue.cc:1059] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.119s
I20260709 13:06:18.109246 19513 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:18.144194 15854 raft_consensus.cc:1240] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 10 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:18.077895 19467 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:18.069192 19514 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:18.157685 19502 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:18.158231 16565 raft_consensus.cc:1240] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:18.159307 19513 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:17.994668 19387 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:06:18.121167 15859 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 1988) took 1929 ms (client timeout 1961 ms). Trace:
I20260709 13:06:18.120904 19495 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:18.124689 16568 raft_consensus.cc:1275] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:06:18.102840 19469 raft_consensus.cc:3055] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:18.128628 19436 raft_consensus.cc:2804] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:17.995088 19481 consensus_queue.cc:1059] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.134s
I20260709 13:06:18.139928 15902 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:18.148095 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:06:18.153674 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 86897.8us
W20260709 13:06:18.156368 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.174521 15858 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:18.175246 15974 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:18.176396 19437 raft_consensus.cc:2749] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:18.178258 15974 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:18.179196 19437 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:18.156483 19263 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:18.179747 19437 raft_consensus.cc:697] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:18.157284 19129 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:18.180817 19437 consensus_queue.cc:237] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:18.181212 19263 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:18.157377 19419 raft_consensus.cc:2804] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:18.184139 19419 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:18.184770 19419 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:06:18.162271 15859 rpcz_store.cc:269] 0709 13:06:16.191581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:16.196366 (+  4785us) service_pool.cc:224] Handling call
0709 13:06:18.120967 (+1924601us) spinlock_profiling.cc:243] Waited 900 ms on lock 0x7b54000b5754. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:18.121093 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2070797696}
I20260709 13:06:18.190438 15974 leader_election.cc:400] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Vote denied by peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) with higher term. Message: Invalid argument: T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 7. Current term is 8.
I20260709 13:06:18.190850 15974 leader_election.cc:403] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:06:18.192014 15827 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:18.186358 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:18.200454 16565 raft_consensus.cc:1240] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:18.202423 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:18.130970 19242 raft_consensus.cc:740] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:18.204064 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:18.205193 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 65360.6us
I20260709 13:06:18.205888 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:18.206154 15902 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:18.211282 15858 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 10.
I20260709 13:06:18.213423 16567 raft_consensus.cc:1240] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:18.213950 15862 raft_consensus.cc:1240] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:18.217304 19350 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:06:18.221849 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.223923 19201 raft_consensus.cc:2749] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:18.225739 15902 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:18.227931 16005 raft_consensus.cc:1275] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:06:18.228066 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:18.239430 15931 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:18.190056 19363 raft_consensus.cc:3055] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:18.173501 19455 raft_consensus.cc:2749] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:18.169000 19469 raft_consensus.cc:740] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:18.249189 15932 raft_consensus.cc:1240] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:06:18.228535 19462 consensus_queue.cc:1048] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:06:18.249003 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 58741.7us
I20260709 13:06:18.158241 19502 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:18.258198 15974 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:18.173617 19436 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:18.235386 19481 raft_consensus.cc:3055] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:18.223313 19418 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:18.265482 15853 raft_consensus.cc:1240] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 9 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:18.268458 15935 raft_consensus.cc:1240] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:18.286515 16562 raft_consensus.cc:1240] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:18.269646 15974 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:18.289345 15974 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:18.240129 19350 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:18.242388 19363 raft_consensus.cc:740] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:18.257987 18968 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) with higher term. Message: Invalid argument: T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 7. Current term is 8.
I20260709 13:06:18.236299 19450 consensus_queue.cc:1048] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:06:18.273375 19436 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:18.306464 15974 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:18.322616 16565 raft_consensus.cc:1275] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:06:18.247354 19398 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:18.256656 19523 raft_consensus.cc:2804] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:18.334838 19523 raft_consensus.cc:697] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 7 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:18.335969 19398 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:18.335783 19523 consensus_queue.cc:237] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:18.337379 19136 raft_consensus.cc:3055] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:18.337792 19136 raft_consensus.cc:740] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:18.338649 19136 consensus_queue.cc:260] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:18.303083 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 69265.6us
I20260709 13:06:18.279095 19263 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:18.316627 15861 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 7.
I20260709 13:06:18.285316 19372 raft_consensus.cc:2804] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:18.285882 19481 raft_consensus.cc:740] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:18.300732 19429 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:06:18.332782 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.311975 15931 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 9.
I20260709 13:06:18.301798 19129 raft_consensus.cc:2804] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:06:18.333570 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 371356us
I20260709 13:06:18.293182 19350 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:18.340072 15827 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:18.362094 19385 raft_consensus.cc:2749] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:18.362743 19524 consensus_queue.cc:1048] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:18.315156 19222 raft_consensus.cc:2804] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:18.364152 19524 consensus_queue.cc:1048] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:18.350368 19263 consensus_queue.cc:1048] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 13:06:18.378768 19419 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:18.381145 19495 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:18.333287 19326 raft_consensus.cc:2749] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:18.382076 19419 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:18.382195 19372 raft_consensus.cc:697] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:18.382961 19419 consensus_queue.cc:1048] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:18.354303 18968 consensus_queue.cc:1059] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:18.388684 18968 consensus_queue.cc:1243] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer 31882606e87447fb9a36bbf625129fe8 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:06:18.384934 19418 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:18.386053 19502 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:18.389688 19495 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:18.383301 19372 consensus_queue.cc:237] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:18.390213 19495 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:18.392277 19495 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:18.386780 16565 raft_consensus.cc:1240] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:18.268014 19503 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:18.340205 19480 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:18.326561 19492 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:18.406159 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:18.407974 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.292676 19242 consensus_queue.cc:260] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:18.420437 19481 consensus_queue.cc:260] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:18.399365 19418 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:18.426154 19385 consensus_queue.cc:1059] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:18.406834 19457 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:18.428287 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:06:18.427888 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 70138.6us
I20260709 13:06:18.339788 19136 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:06:18.439870 16011 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.005s	sys 0.016s
I20260709 13:06:18.410651 19429 raft_consensus.cc:697] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:18.411427 19129 raft_consensus.cc:697] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:18.327497 19469 consensus_queue.cc:260] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:18.443054 16011 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.005s	sys 0.016s
I20260709 13:06:18.451566 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:18.443543 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 452619us
I20260709 13:06:18.442376 19242 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:18.414348 19502 consensus_queue.cc:1059] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.040s
W20260709 13:06:18.473835 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 234444us
I20260709 13:06:18.469877 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:06:18.477975 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.480836 16565 raft_consensus.cc:2393] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 7: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:18.462667 19481 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:18.459683 19505 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:18.400714 19363 consensus_queue.cc:260] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:06:18.443575 19457 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:18.486552 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:18.486517 19359 consensus_queue.cc:1048] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:18.487960 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:18.468321 19469 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:18.437724 19418 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:18.491290 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:18.497561 16535 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:18.499653 16565 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:18.487113 19129 consensus_queue.cc:237] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:18.504532 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 258608us
I20260709 13:06:18.532413 19505 consensus_queue.cc:1048] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:18.534706 19505 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:18.486402 19462 consensus_queue.cc:1059] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:06:18.537410 19462 consensus_queue.cc:1048] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:18.538574 19429 consensus_queue.cc:237] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:18.503129 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.503510 15860 raft_consensus.cc:1240] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 9 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 8. Current term is 9. Ops: [8.8-8.8]
W20260709 13:06:18.546224 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.548362 15931 raft_consensus.cc:1275] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:06:18.479323 19450 consensus_queue.cc:1048] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:06:18.561749 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:18.507278 19535 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:18.576407 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 78952.1us
I20260709 13:06:18.504863 19363 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:18.535312 19505 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:18.574011 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:18.568755 15930 raft_consensus.cc:1240] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:06:18.632571 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:18.560222 19514 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:18.637305 16565 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
W20260709 13:06:18.653296 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 243861us
I20260709 13:06:18.649782 19503 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:18.654127 19547 raft_consensus.cc:3055] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:18.649222 19514 raft_consensus.cc:697] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:06:18.738261 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 296015us
I20260709 13:06:18.746332 16006 raft_consensus.cc:1240] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 7. Current term is 8. Ops: []
W20260709 13:06:18.750144 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.608685 19535 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:18.759878 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:18.737928 19513 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:18.741652 15827 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:18.784523 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 210243us
I20260709 13:06:18.785058 16008 raft_consensus.cc:1275] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:06:18.788537 16007 raft_consensus.cc:1240] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 9. Current term is 10. Ops: [9.7-9.7]
W20260709 13:06:18.789196 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 47641.4us
I20260709 13:06:18.790580 15827 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:18.743177 19223 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:18.768126 19431 consensus_meta.cc:287] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.704s	user 0.009s	sys 0.005s
I20260709 13:06:18.743199 19523 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:18.744298 19547 raft_consensus.cc:740] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:18.762440 19429 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:18.797973 16533 leader_election.cc:336] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.881s (SENT)
I20260709 13:06:18.799214 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:18.803443 15974 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:18.805362 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:18.757123 19340 consensus_queue.cc:1048] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:06:18.793579 19229 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:18.788041 19513 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:18.795547 19385 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:18.802350 19503 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:18.815613 19536 log.cc:927] Time spent T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.316s	user 0.001s	sys 0.000s
W20260709 13:06:18.856148 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 309433us
I20260709 13:06:18.871125 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:06:18.872045 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:18.875773 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.801776 19480 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:18.787883 19550 consensus_queue.cc:1059] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
W20260709 13:06:18.895692 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 92302.7us
W20260709 13:06:18.907147 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.908365 16533 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:18.912389 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:18.913743 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:06:18.868060 19541 log.cc:927] Time spent T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:06:18.912279 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:18.831503 19537 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:18.919551 16007 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 7.
W20260709 13:06:18.913018 19528 log.cc:927] Time spent T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.361s	user 0.001s	sys 0.000s
I20260709 13:06:18.946360 16564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 784) took 1153 ms. Trace:
I20260709 13:06:18.946851 16564 rpcz_store.cc:276] 0709 13:06:17.793037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:17.793214 (+   177us) service_pool.cc:224] Handling call
0709 13:06:17.794275 (+  1061us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:17.820548 (+ 26273us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:17.820579 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:17.821385 (+   806us) log.cc:844] Serialized 26 byte log entry
0709 13:06:18.324895 (+503510us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:18.324933 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:18.325001 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:18.946126 (+621125us) raft_consensus.cc:1685] finished
0709 13:06:18.946154 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:18.946305 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11452,"raft.queue_time_us":17655,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":100,"thread_start_us":456543,"threads_started":1,"wal-append.queue_time_us":625484}
I20260709 13:06:18.949487 16564 raft_consensus.cc:1240] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:06:18.950924 16564 raft_consensus.cc:1240] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 13:06:18.802115 19223 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:18.936440 15974 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:18.808810 19429 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:18.801487 19431 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:18.921571 19477 log.cc:927] Time spent T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
W20260709 13:06:18.943723 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:18.967455 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.964816 15859 raft_consensus.cc:2393] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 8: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:18.966691 19431 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:18.969218 19431 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:18.963815 19429 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:18.973196 19431 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:18.976065 19431 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:18.925642 19538 log.cc:927] Time spent T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.324s	user 0.000s	sys 0.001s
W20260709 13:06:18.980933 19540 log.cc:927] Time spent T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.321s	user 0.000s	sys 0.000s
W20260709 13:06:18.981135 19542 log.cc:927] Time spent T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 13:06:18.809165 19523 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:18.935415 19554 log.cc:927] Time spent T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 13:06:18.977655 19431 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:18.991362 15933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1947) took 1021 ms. Trace:
I20260709 13:06:18.898823 19503 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:18.936551 19418 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:18.917356 19450 consensus_queue.cc:1048] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 13:06:18.998921 15933 rpcz_store.cc:276] 0709 13:06:17.970181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:17.984563 (+ 14382us) service_pool.cc:224] Handling call
0709 13:06:17.984887 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:17.985250 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:17.985290 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:17.985733 (+   443us) log.cc:844] Serialized 26 byte log entry
0709 13:06:18.300450 (+314717us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:18.300492 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:18.300558 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:18.991134 (+690576us) raft_consensus.cc:1685] finished
0709 13:06:18.991156 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:18.991305 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10995,"raft.queue_time_us":6606,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":5556,"thread_start_us":314407,"threads_started":1,"wal-append.queue_time_us":495065}
I20260709 13:06:18.941879 19505 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:18.942945 19480 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:06:19.002698 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 92706.2us
I20260709 13:06:18.989529 19502 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:18.830013 19514 consensus_queue.cc:237] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:19.010490 16533 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:18.990232 19350 consensus_meta.cc:287] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.613s	user 0.005s	sys 0.006s
I20260709 13:06:19.000943 19223 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:19.027215 19553 log.cc:927] Time spent T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
W20260709 13:06:19.028540 19539 log.cc:927] Time spent T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.263s	user 0.000s	sys 0.001s
I20260709 13:06:19.015970 19550 raft_consensus.cc:3055] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:19.029248 19550 raft_consensus.cc:740] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:19.029956 16011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 628) took 1163 ms. Trace:
I20260709 13:06:19.030150 19550 consensus_queue.cc:260] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:19.031080 19543 log.cc:927] Time spent T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:06:19.030340 16011 rpcz_store.cc:276] 0709 13:06:17.866334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:17.897063 (+ 30729us) service_pool.cc:224] Handling call
0709 13:06:17.897452 (+   389us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:17.897865 (+   413us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:17.897898 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:17.898527 (+   629us) log.cc:844] Serialized 52 byte log entry
0709 13:06:18.456318 (+557791us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:18.456386 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:18.456478 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:19.029734 (+573256us) raft_consensus.cc:1685] finished
0709 13:06:19.029759 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:19.029897 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6370,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":13403,"thread_start_us":557403,"threads_started":1,"wal-append.queue_time_us":734003}
I20260709 13:06:18.942329 19537 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:19.010993 19418 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:19.020457 19544 log.cc:927] Time spent T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.361s	user 0.001s	sys 0.000s
I20260709 13:06:18.924705 19547 consensus_queue.cc:260] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:19.072379 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:19.025591 19350 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:19.048765 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 173080us
I20260709 13:06:19.032155 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:19.037034 19331 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:06:19.065779 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 55382us
I20260709 13:06:19.031610 19550 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:19.087736 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:19.033139 19566 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:19.088238 16009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 2008) took 1137 ms. Trace:
I20260709 13:06:19.087591 16533 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:19.004531 19431 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:19.090817 16569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 945) took 1256 ms. Trace:
I20260709 13:06:19.100286 16569 rpcz_store.cc:276] 0709 13:06:17.834119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.050893 (+216774us) service_pool.cc:224] Handling call
0709 13:06:18.051194 (+   301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:18.051575 (+   381us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:18.051604 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:18.128851 (+ 77247us) log.cc:844] Serialized 26 byte log entry
0709 13:06:18.432321 (+303470us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:18.432378 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:18.432472 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:19.090484 (+658012us) raft_consensus.cc:1685] finished
0709 13:06:19.090519 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:19.090712 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":47779,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":24225,"thread_start_us":303046,"threads_started":1,"wal-append.queue_time_us":483409}
I20260709 13:06:19.034500 19523 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:19.058653 19545 log.cc:927] Time spent T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.297s	user 0.000s	sys 0.001s
I20260709 13:06:19.037166 19503 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:19.086625 15900 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:19.094338 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 157968us
W20260709 13:06:19.063129 19481 consensus_meta.cc:287] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.574s	user 0.004s	sys 0.006s
I20260709 13:06:19.093920 16009 rpcz_store.cc:276] 0709 13:06:17.951287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.016464 (+ 65177us) service_pool.cc:224] Handling call
0709 13:06:18.016882 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:18.017152 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:18.017189 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:18.100543 (+ 83354us) log.cc:844] Serialized 26 byte log entry
0709 13:06:18.479878 (+379335us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:18.479916 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:18.479983 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:19.088051 (+608068us) raft_consensus.cc:1685] finished
0709 13:06:19.088070 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:19.088187 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":321311,"threads_started":1,"wal-append.queue_time_us":498512}
I20260709 13:06:19.109189 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:19.110113 16009 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 8: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:19.110994 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:19.111553 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:19.094682 19436 consensus_meta.cc:287] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.761s	user 0.000s	sys 0.012s
I20260709 13:06:19.115355 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:19.119181 16006 raft_consensus.cc:2410] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:06:19.119006 16011 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:06:19.121731 15900 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:19.126641 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:19.130091 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:19.135026 16569 raft_consensus.cc:1275] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:06:19.067106 19418 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:19.139696 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:19.138723 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:19.140492 15936 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:19.139010 15929 raft_consensus.cc:1240] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:19.138773 16011 raft_consensus.cc:1275] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:06:19.144255 15936 raft_consensus.cc:1275] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
I20260709 13:06:19.145152 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:19.118981 19547 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:19.106876 19535 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:19.148483 19535 raft_consensus.cc:2749] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:19.148947 19431 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:19.149191 19535 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:19.149734 19535 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:19.106117 19513 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:19.152026 19513 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:06:19.152383 19136 consensus_meta.cc:287] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.700s	user 0.007s	sys 0.004s
I20260709 13:06:19.136518 19450 raft_consensus.cc:2749] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:06:19.153723 19363 consensus_meta.cc:287] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.545s	user 0.008s	sys 0.001s
I20260709 13:06:19.154520 15934 raft_consensus.cc:2410] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:06:19.156061 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:19.141870 15931 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:19.161384 16569 raft_consensus.cc:2393] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 8: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:19.157289 19535 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:19.104110 19566 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:19.179595 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:19.199321 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:19.255752 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:19.483349 15859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 2021) took 1336 ms. Trace:
W20260709 13:06:19.539526 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 451961us
W20260709 13:06:19.225848 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 92145.4us
W20260709 13:06:19.541005 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:19.336784 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:19.234155 15860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1908) took 1608 ms. Trace:
I20260709 13:06:19.353576 15933 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:19.742228 15855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 1917) took 1341 ms. Trace:
I20260709 13:06:19.744010 16010 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:19.581748 16568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 948) took 1693 ms. Trace:
I20260709 13:06:19.129381 19503 raft_consensus.cc:2804] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:06:19.539075 16564 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.005s	sys 0.025s
W20260709 13:06:19.748507 16564 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.765s	user 0.008s	sys 0.032s
I20260709 13:06:19.742166 15859 rpcz_store.cc:276] 0709 13:06:18.147386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.205584 (+ 58198us) service_pool.cc:224] Handling call
0709 13:06:19.483050 (+1277466us) spinlock_profiling.cc:243] Waited 758 ms on lock 0x7b540032e954. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:19.483256 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1742962944}
I20260709 13:06:19.199447 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:19.744971 15855 rpcz_store.cc:276] 0709 13:06:18.401273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:19.255468 (+854195us) service_pool.cc:224] Handling call
0709 13:06:19.742170 (+486702us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:19.745322 16568 rpcz_store.cc:276] 0709 13:06:17.888311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.170885 (+282574us) service_pool.cc:224] Handling call
0709 13:06:18.171217 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:18.171627 (+   410us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:18.171654 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:18.172251 (+   597us) log.cc:844] Serialized 26 byte log entry
0709 13:06:18.475292 (+303041us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:18.475329 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:18.475396 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:19.581419 (+1106023us) raft_consensus.cc:1685] finished
0709 13:06:19.581452 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:19.581663 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1908,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":19732,"thread_start_us":302611,"threads_started":1,"wal-append.queue_time_us":479592}
I20260709 13:06:19.753381 16568 raft_consensus.cc:1275] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 9 index: 11. (index mismatch)
I20260709 13:06:19.251370 16005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1871) took 1674 ms. Trace:
I20260709 13:06:19.341446 16566 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
W20260709 13:06:19.746917 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 557356us
I20260709 13:06:19.753953 16005 rpcz_store.cc:276] 0709 13:06:17.576883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:17.577174 (+   291us) service_pool.cc:224] Handling call
0709 13:06:17.617120 (+ 39946us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:18.239062 (+621942us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:19.251127 (+1012065us) spinlock_profiling.cc:243] Waited 610 ms on lock 0x7b540025a3d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:19.251303 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1403675008}
I20260709 13:06:19.754449 15974 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:19.751530 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:19.745775 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:19.756297 16010 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 9.
I20260709 13:06:19.213824 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:19.756143 15860 rpcz_store.cc:276] 0709 13:06:17.625637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:17.878952 (+253315us) service_pool.cc:224] Handling call
0709 13:06:17.879296 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:18.546238 (+666942us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:19.233414 (+687176us) spinlock_profiling.cc:243] Waited 624 ms on lock 0x7b5400218254. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:19.234073 (+   659us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1434820992}
I20260709 13:06:19.744021 16533 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:19.755692 16569 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:06:19.132513 19575 raft_consensus.cc:2749] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:19.776101 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:19.136210 19511 raft_consensus.cc:3055] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:19.777527 16533 leader_election.cc:304] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:19.460791 19469 consensus_meta.cc:287] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.857s	user 0.010s	sys 0.003s
I20260709 13:06:19.315974 19350 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:19.161418 19480 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:19.682348 19340 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:19.786803 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 43519us
I20260709 13:06:19.232892 19242 raft_consensus.cc:3055] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:19.803756 16009 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:19.801928 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:19.805358 16569 raft_consensus.cc:1240] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 8. Current term is 9. Ops: []
W20260709 13:06:19.232322 19548 log.cc:927] Time spent T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:06:19.582902 19550 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:19.788100 19492 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:19.800297 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:19.755223 19455 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:19.501618 19431 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:19.464458 19212 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:19.297019 19535 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:19.726150 19331 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:19.233019 19387 consensus_meta.cc:287] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 1.059s	user 0.012s	sys 0.000s
I20260709 13:06:19.644045 19223 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:19.317118 19537 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:19.792742 19591 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:19.845283 19492 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:19.809787 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 55419.5us
I20260709 13:06:19.874709 15974 leader_election.cc:400] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 election: Vote denied by peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) with higher term. Message: Invalid argument: T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 10. Current term is 11.
I20260709 13:06:19.875136 15974 leader_election.cc:403] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
W20260709 13:06:19.875468 19570 log.cc:927] Time spent T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.723s	user 0.005s	sys 0.006s
I20260709 13:06:19.876145 19492 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:19.823325 19480 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:19.792755 19361 consensus_queue.cc:1048] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:06:19.813237 16533 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:19.884049 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 70920.7us
I20260709 13:06:19.476230 19585 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:06:19.823247 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 203118us
I20260709 13:06:19.821975 15855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 1916) took 1421 ms. Trace:
I20260709 13:06:19.133266 19436 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:19.788722 19574 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:19.833739 16002 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:06:19.841516 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:19.825078 19579 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:19.823541 19242 raft_consensus.cc:740] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:19.866061 18732 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:19.849026 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:19.878022 15974 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:19.747694 19503 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:19.849630 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:19.889653 15855 rpcz_store.cc:276] 0709 13:06:18.400905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:19.801620 (+1400715us) service_pool.cc:224] Handling call
0709 13:06:19.821896 (+ 20276us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:19.892407 16561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 947) took 2054 ms. Trace:
I20260709 13:06:19.801290 19571 consensus_queue.cc:1048] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.077s
W20260709 13:06:19.898322 16567 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.023s	sys 0.026s
I20260709 13:06:19.804845 19511 raft_consensus.cc:740] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:06:19.924985 16567 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.023s	sys 0.027s
I20260709 13:06:19.799988 19572 consensus_queue.cc:1059] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 13:06:19.831928 19587 raft_consensus.cc:2804] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:19.925683 19587 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:19.866891 19212 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:19.826332 19340 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:19.925915 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:19.926548 15855 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:19.927866 18968 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:19.928256 18968 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:19.928439 19212 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:19.904361 18732 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:19.905468 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:19.929114 19212 consensus_queue.cc:1048] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:19.930487 19212 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:19.930953 19212 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:19.914350 16561 rpcz_store.cc:276] 0709 13:06:17.837767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.140707 (+302940us) service_pool.cc:224] Handling call
0709 13:06:18.141076 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:18.204762 (+ 63686us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:18.204788 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:18.204821 (+    33us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:18.204847 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:19.892178 (+1687331us) spinlock_profiling.cc:243] Waited 63.5 ms on lock 0x7b5400500354. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:19.892224 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:19.892354 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":146112384}
I20260709 13:06:19.842789 19455 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:19.920960 15930 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 8: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:19.847260 19431 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:19.932710 19484 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:19.933180 19484 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:19.934068 19562 log.cc:927] Time spent T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 1.013s	user 0.008s	sys 0.005s
I20260709 13:06:19.934748 19556 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:19.935000 19484 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:19.935359 19556 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:19.935891 15859 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 7.
I20260709 13:06:19.937312 15859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58214 (request call id 670) took 1576 ms. Trace:
I20260709 13:06:19.937824 19556 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:19.898121 19513 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:06:19.892077 19385 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.966s	user 0.035s	sys 0.041s
W20260709 13:06:19.932399 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:19.933938 15855 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
I20260709 13:06:19.937646 15859 rpcz_store.cc:276] 0709 13:06:18.360475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:19.775823 (+1415348us) service_pool.cc:224] Handling call
0709 13:06:19.937136 (+161313us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b5400427cd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:19.937250 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":339466624}
I20260709 13:06:19.942662 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:19.895736 19418 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:19.891932 19555 raft_consensus.cc:2749] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) with higher term. Message: Invalid argument: T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 10. Current term is 11.
I20260709 13:06:19.923007 19503 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:19.928614 18968 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:19.922888 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:19.878611 19331 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:19.943282 15859 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:06:19.939284 16569 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:06:19.944370 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:19.921727 16533 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:19.920692 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:19.951596 15933 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 7.
W20260709 13:06:19.954551 19590 log.cc:927] Time spent T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:06:19.954630 19340 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:19.904291 19579 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:19.955338 19212 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:19.941947 19385 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.026s	user 0.035s	sys 0.042s
I20260709 13:06:19.840633 19550 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:19.867404 19535 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:19.888012 19223 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:19.962508 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:19.962682 15900 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:19.896454 19558 consensus_queue.cc:1048] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 13:06:19.935001 19566 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:19.933990 19431 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:19.844045 19350 consensus_queue.cc:1059] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.114s
I20260709 13:06:19.900727 19585 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:19.903764 19574 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:20.096138 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:20.106400 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:20.106812 15934 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 9.
I20260709 13:06:20.106894 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:20.107259 16561 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:06:20.107717 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:20.108034 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:20.108440 19531 log.cc:927] Time spent T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 1.619s	user 0.000s	sys 0.013s
I20260709 13:06:20.044284 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:20.109086 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:20.110911 15929 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:20.111052 16003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 2006) took 2299 ms. Trace:
I20260709 13:06:20.093328 19480 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:20.091564 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:20.112159 19431 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.112672 19431 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:20.113289 16568 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
W20260709 13:06:20.108381 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 145773us
I20260709 13:06:20.114210 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:20.109418 15855 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:06:20.109611 19361 raft_consensus.cc:2749] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:06:20.094370 16006 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60156 (request call id 641) took 1344 ms (client timeout 1704 ms). Trace:
W20260709 13:06:20.116686 16006 rpcz_store.cc:269] 0709 13:06:18.749910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.759562 (+  9652us) service_pool.cc:224] Handling call
0709 13:06:20.094106 (+1334544us) spinlock_profiling.cc:243] Waited 358 ms on lock 0x7b54000aaad4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:20.094298 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":823960832}
I20260709 13:06:20.117420 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:20.112938 19566 consensus_queue.cc:1059] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:20.095732 19595 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:06:20.111231 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:20.119176 19595 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:20.119997 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:20.120491 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:20.121559 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:20.118199 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:20.121887 16535 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:20.122486 15862 raft_consensus.cc:2393] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 7: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:20.109400 19350 consensus_queue.cc:1243] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer 31882606e87447fb9a36bbf625129fe8 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 13:06:20.105248 16566 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45244 (request call id 937) took 2586 ms (client timeout 1832 ms). Trace:
W20260709 13:06:20.125910 16566 rpcz_store.cc:269] 0709 13:06:17.519159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:17.520044 (+   885us) service_pool.cc:224] Handling call
0709 13:06:20.104995 (+2584951us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b54002364d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:20.105132 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4060522368}
I20260709 13:06:20.126025 15900 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:20.115584 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:20.124217 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:20.123551 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:20.118175 16561 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:06:20.142591 16006 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:06:20.143373 16007 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:06:20.143399 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:20.127835 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:20.144693 16561 raft_consensus.cc:2393] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 9: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
W20260709 13:06:20.145144 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 223520us
I20260709 13:06:20.143599 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:20.144714 15934 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 9.
I20260709 13:06:20.147401 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:20.129495 16535 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:20.149359 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:06:20.006008 19594 log.cc:927] Time spent T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:06:20.111476 16003 rpcz_store.cc:276] 0709 13:06:17.812077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:17.896769 (+ 84692us) service_pool.cc:224] Handling call
0709 13:06:17.897044 (+   275us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:17.897413 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 13:06:17.897441 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:17.897967 (+   526us) log.cc:844] Serialized 26 byte log entry
0709 13:06:18.286353 (+388386us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:06:18.286393 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:18.286452 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:20.107779 (+1821327us) spinlock_profiling.cc:243] Waited 3.83 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:06:20.110767 (+  2988us) raft_consensus.cc:1685] finished
0709 13:06:20.110788 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:20.110993 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3925,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":14587,"spinlock_wait_cycles":8806272,"thread_start_us":387984,"threads_started":1,"wal-append.queue_time_us":558315}
I20260709 13:06:20.150288 16568 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 8: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:20.154314 16535 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:20.150950 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:19.926100 19587 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:20.176538 16566 raft_consensus.cc:1240] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 7. Current term is 8. Ops: []
W20260709 13:06:20.176986 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:20.146699 19575 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:20.146402 19583 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:20.141429 19574 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:20.182489 19588 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.190317 19589 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:20.198715 19589 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:20.149145 19537 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:19.963069 19582 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.201251 19589 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:20.205292 19599 log.cc:927] Time spent T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.117s	user 0.001s	sys 0.001s
I20260709 13:06:20.206701 16564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 955) took 1417 ms. Trace:
I20260709 13:06:20.208506 19367 raft_consensus.cc:3055] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:20.208942 19367 raft_consensus.cc:740] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:20.184029 19331 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:20.184368 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:20.186576 19095 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:20.027120 19572 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.140283 19513 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:20.114203 19436 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:20.191118 16565 raft_consensus.cc:1275] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:06:20.223800 16565 raft_consensus.cc:1240] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 7. Current term is 8. Ops: [5.7-7.8]
I20260709 13:06:20.226665 16565 raft_consensus.cc:1275] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
W20260709 13:06:20.211971 19590 log.cc:927] Time spent T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:06:20.149304 19350 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:20.237941 16004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1884) took 1216 ms. Trace:
I20260709 13:06:20.238394 16004 rpcz_store.cc:276] 0709 13:06:19.021598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:19.781122 (+759524us) service_pool.cc:224] Handling call
0709 13:06:19.781490 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:19.841059 (+ 59569us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:19.841104 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:19.844767 (+  3663us) log.cc:844] Serialized 26 byte log entry
0709 13:06:20.052573 (+207806us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:20.054017 (+  1444us) log.cc:844] Serialized 14 byte log entry
0709 13:06:20.054933 (+   916us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:20.055014 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:20.237706 (+182692us) raft_consensus.cc:1685] finished
0709 13:06:20.237727 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:20.237871 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58287,"prepare.queue_time_us":956,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":62511,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":30751,"thread_start_us":152979,"threads_started":1,"wal-append.queue_time_us":366218}
I20260709 13:06:20.242205 19535 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:20.147655 19385 raft_consensus.cc:697] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 9 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:20.201866 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:20.254782 16006 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 8: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:20.147912 19579 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:20.141108 19340 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:20.198540 19588 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:20.204908 19582 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:19.965415 19418 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:20.207114 16564 rpcz_store.cc:276] 0709 13:06:18.789812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.970620 (+180808us) service_pool.cc:224] Handling call
0709 13:06:18.975415 (+  4795us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:19.748951 (+773536us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:19.748996 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:19.777609 (+ 28613us) log.cc:844] Serialized 26 byte log entry
0709 13:06:19.957086 (+179477us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:19.957137 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:19.957227 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:20.206383 (+249156us) raft_consensus.cc:1685] finished
0709 13:06:20.206408 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:20.206633 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":164301,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":79,"thread_start_us":884318,"threads_started":2,"wal-append.queue_time_us":280254}
I20260709 13:06:20.257828 19340 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:20.260250 19418 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:20.261471 19588 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:20.262022 19340 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:20.262501 19418 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:20.229713 19572 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:20.264874 19572 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:20.265558 19582 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:20.229957 19513 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:20.266816 19513 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:20.151089 19505 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.267922 19505 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:06:20.270048 19505 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:20.148469 19429 consensus_queue.cc:1059] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 13:06:19.959290 19514 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:20.253907 19436 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:20.271785 19429 raft_consensus.cc:3055] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:20.272217 19429 raft_consensus.cc:740] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:20.273020 19429 consensus_queue.cc:260] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:20.202880 19408 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.234786 19331 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:20.027565 19242 consensus_queue.cc:260] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:20.220475 19537 raft_consensus.cc:697] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:20.150988 19578 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.272806 19385 consensus_queue.cc:237] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:20.233582 19367 consensus_queue.cc:260] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:20.228381 19585 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:20.200567 19575 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:20.203276 19583 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:20.274986 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 209695us
I20260709 13:06:20.356376 19095 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:20.356964 19095 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:20.357097 19579 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:20.359228 19095 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:20.359977 16005 raft_consensus.cc:1240] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
W20260709 13:06:20.360242 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 210162us
I20260709 13:06:20.361296 16533 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:20.362905 16533 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:20.324740 15928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 1903) took 1200 ms. Trace:
I20260709 13:06:20.370767 15902 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:20.364771 15928 rpcz_store.cc:276] 0709 13:06:19.124523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:19.124672 (+   149us) service_pool.cc:224] Handling call
0709 13:06:19.125062 (+   390us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:19.138862 (+ 13800us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:19.138894 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:19.204137 (+ 65243us) log.cc:844] Serialized 52 byte log entry
0709 13:06:19.670562 (+466425us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:19.670620 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:19.670719 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:20.273957 (+603238us) raft_consensus.cc:1685] finished
0709 13:06:20.273985 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:20.324657 (+ 50672us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7581,"raft.queue_time_us":7791,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":15910,"thread_start_us":373177,"threads_started":1,"wal-append.queue_time_us":635997}
I20260709 13:06:20.320815 19418 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.364974 19469 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:20.363874 19331 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:20.427979 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 49589.9us
I20260709 13:06:20.434863 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:20.297197 19550 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:20.324525 19429 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:20.438618 15928 raft_consensus.cc:1275] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:06:20.369522 19242 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:20.376664 19450 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:20.396605 19095 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.436589 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:20.436667 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:20.329591 18968 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:20.429226 15974 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:20.372916 19547 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.272238 19436 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:20.369356 19367 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:20.440091 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:20.373456 19588 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.445215 19583 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:20.470207 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:20.471282 15858 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:20.472754 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:20.473579 15858 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:20.446971 19575 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:20.435920 15932 raft_consensus.cc:2410] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:06:20.448966 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:20.380640 16533 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:20.368913 19578 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:20.436281 19331 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:20.367573 19408 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:20.453727 19602 log.cc:927] Time spent T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 13:06:20.450050 19495 raft_consensus.cc:2749] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:20.453948 19558 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:20.455945 19513 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:20.458195 19589 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:20.458138 19537 consensus_queue.cc:237] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:20.435758 19572 consensus_queue.cc:1059] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:06:20.450472 19223 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:20.286667 19514 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:20.432993 19418 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:20.467140 15974 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:20.468758 15928 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 10.
I20260709 13:06:20.479350 19350 raft_consensus.cc:697] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 9 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:20.477952 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:20.510644 15934 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:20.432214 19480 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.450693 19450 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:20.486641 16004 raft_consensus.cc:1240] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:06:20.469503 19326 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:20.488588 15933 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:06:20.467168 19579 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:20.493466 15858 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:06:20.491948 19609 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:20.475878 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 95338.1us
I20260709 13:06:20.479458 19610 raft_consensus.cc:2804] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:20.475845 19165 consensus_queue.cc:1048] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
W20260709 13:06:20.515625 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 361374us
I20260709 13:06:20.476070 19547 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:20.521755 15858 raft_consensus.cc:606] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Received request to transfer leadership to TS 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:20.696739 15858 raft_consensus.cc:614] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:06:20.495235 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:20.503414 19566 raft_consensus.cc:3055] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:20.520342 19357 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:20.645929 19326 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:20.526173 19457 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:20.529124 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:20.531435 19585 consensus_queue.cc:1048] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:20.561406 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:20.567996 15854 raft_consensus.cc:1240] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:20.586814 19431 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:20.597150 18732 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:20.597281 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:20.600062 15935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 1880) took 2841 ms. Trace:
W20260709 13:06:20.602805 19563 log.cc:927] Time spent T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 1.667s	user 0.002s	sys 0.013s
I20260709 13:06:20.607030 19595 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:20.615226 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:20.618152 19455 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:20.628504 15853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1907) took 3005 ms. Trace:
I20260709 13:06:20.756379 15853 rpcz_store.cc:276] 0709 13:06:17.623043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:17.851291 (+228248us) service_pool.cc:224] Handling call
0709 13:06:17.851640 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:18.286387 (+434747us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:20.628159 (+2341772us) spinlock_profiling.cc:243] Waited 414 ms on lock 0x7b54000e9cd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:20.628421 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":951265664}
I20260709 13:06:20.629824 19263 raft_consensus.cc:3055] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:06:20.629982 19600 log.cc:927] Time spent T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.550s	user 0.004s	sys 0.000s
I20260709 13:06:20.636798 16004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1890) took 1481 ms. Trace:
I20260709 13:06:20.770800 16004 rpcz_store.cc:276] 0709 13:06:19.155551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:20.240490 (+1084939us) service_pool.cc:224] Handling call
0709 13:06:20.240835 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:20.515043 (+274208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:20.636557 (+121514us) spinlock_profiling.cc:243] Waited 244 ms on lock 0x7b54000eecd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:20.636742 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":560328064}
I20260709 13:06:20.638397 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:20.640053 19467 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:20.470680 19095 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:20.645320 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:20.645881 16562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 792) took 2816 ms. Trace:
W20260709 13:06:20.772747 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:20.772789 16562 rpcz_store.cc:276] 0709 13:06:17.829571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:17.923930 (+ 94359us) service_pool.cc:224] Handling call
0709 13:06:17.924840 (+   910us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:18.287092 (+362252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:20.645558 (+2358466us) spinlock_profiling.cc:243] Waited 361 ms on lock 0x7b54002d7dd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:20.645797 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":831145856}
I20260709 13:06:20.774538 19095 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:20.479429 19588 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:20.775650 16010 raft_consensus.cc:1275] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 13:06:20.776654 16010 pending_rounds.cc:85] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting all ops after (but not including) 6
I20260709 13:06:20.776976 16010 pending_rounds.cc:107] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 13:06:20.777145 19588 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:20.777284 16010 raft_consensus.cc:2889] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:20.647346 16533 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:20.695112 19551 log.cc:927] Time spent T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 1.927s	user 0.004s	sys 0.008s
W20260709 13:06:20.695637 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 254747us
I20260709 13:06:20.489889 19558 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:20.788890 15974 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:20.523068 15856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 661) took 2805 ms. Trace:
I20260709 13:06:20.789986 15974 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:20.789966 15856 rpcz_store.cc:276] 0709 13:06:17.717896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:17.913236 (+195340us) service_pool.cc:224] Handling call
0709 13:06:18.043571 (+130335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:18.100024 (+ 56453us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:20.522701 (+2422677us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b5400426dd0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:20.522948 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":254035840}
I20260709 13:06:20.709398 15976 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:20.693208 19602 log.cc:927] Time spent T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.189s	user 0.000s	sys 0.000s
I20260709 13:06:20.794986 15974 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:20.726439 19357 raft_consensus.cc:697] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:20.686168 19610 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:20.799813 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 90492.2us
I20260709 13:06:20.729180 16003 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
I20260709 13:06:20.731604 16565 raft_consensus.cc:2410] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:06:20.731192 15935 rpcz_store.cc:276] 0709 13:06:17.758283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:17.895749 (+137466us) service_pool.cc:224] Handling call
0709 13:06:17.895986 (+   237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:18.286836 (+390850us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:20.599612 (+2312776us) spinlock_profiling.cc:243] Waited 370 ms on lock 0x7b540044bbd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:20.599994 (+   382us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":851991296}
I20260709 13:06:20.803721 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:20.804533 16565 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:06:20.805682 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:20.766278 19524 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.970s	user 0.007s	sys 0.027s
W20260709 13:06:20.737869 19608 log.cc:927] Time spent T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.224s	user 0.000s	sys 0.000s
I20260709 13:06:20.747602 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:20.762514 19263 raft_consensus.cc:740] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:20.693761 19609 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:20.705607 19566 raft_consensus.cc:740] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:20.835772 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:20.758798 15857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1915) took 3116 ms. Trace:
I20260709 13:06:20.723079 19326 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:20.644968 19480 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:20.774457 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:20.776211 16561 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:20.520852 19450 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:20.702054 16535 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:20.785725 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:20.808302 19437 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:06:20.524359 15861 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58214 (request call id 665) took 2805 ms (client timeout 1676 ms). Trace:
I20260709 13:06:20.717731 15858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:57012 (request call id 22) took 2295 ms. Trace:
I20260709 13:06:20.807080 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:20.862948 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:20.808135 19572 raft_consensus.cc:2749] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:20.736750 19581 raft_consensus.cc:3055] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:20.802464 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:20.863166 15861 rpcz_store.cc:269] 0709 13:06:17.719033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:17.733045 (+ 14012us) service_pool.cc:224] Handling call
0709 13:06:20.524139 (+2791094us) spinlock_profiling.cc:243] Waited 568 ms on lock 0x7b5400217354. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:20.524264 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1305556352}
W20260709 13:06:20.864185 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 162246us
I20260709 13:06:20.864679 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:20.865414 15935 raft_consensus.cc:2410] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 12 because replica has last-logged OpId of term: 11 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 13:06:20.798297 19478 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:06:20.750855 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:20.803283 19610 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:20.837862 15857 rpcz_store.cc:276] 0709 13:06:17.642239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:17.884526 (+242287us) service_pool.cc:224] Handling call
0709 13:06:17.931507 (+ 46981us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:17.945850 (+ 14343us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:20.758544 (+2812694us) spinlock_profiling.cc:243] Waited 46.7 ms on lock 0x7b540032d550. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:20.758728 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":107465088}
I20260709 13:06:20.794600 19467 raft_consensus.cc:2749] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:20.850668 16011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1885) took 1807 ms. Trace:
W20260709 13:06:20.827761 19524 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.031s	user 0.007s	sys 0.028s
W20260709 13:06:20.811206 19587 consensus_meta.cc:287] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.608s	user 0.010s	sys 0.000s
W20260709 13:06:20.857232 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:20.894049 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:20.872148 19478 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:20.805162 19568 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:20.857692 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:20.863297 15858 rpcz_store.cc:276] 0709 13:06:18.401571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:20.479486 (+2077915us) service_pool.cc:224] Handling call
0709 13:06:20.697298 (+217812us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:20.863531 16565 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:20.903851 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:20.874250 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:20.766624 19350 consensus_queue.cc:237] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:20.794730 19095 consensus_queue.cc:1048] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.058s
I20260709 13:06:20.831544 19455 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:20.884531 16011 rpcz_store.cc:276] 0709 13:06:19.043561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:19.778881 (+735320us) service_pool.cc:224] Handling call
0709 13:06:19.779188 (+   307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:19.827389 (+ 48201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:19.827433 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:19.828033 (+   600us) log.cc:844] Serialized 26 byte log entry
0709 13:06:20.006450 (+178417us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:20.006763 (+   313us) log.cc:844] Serialized 14 byte log entry
0709 13:06:20.006995 (+   232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:20.007072 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:20.850486 (+843414us) raft_consensus.cc:1685] finished
0709 13:06:20.850505 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:20.850615 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32121,"prepare.queue_time_us":159,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":49590,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":11310,"thread_start_us":136497,"threads_started":1,"wal-append.queue_time_us":405033}
I20260709 13:06:20.871913 19581 raft_consensus.cc:740] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:06:20.788981 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 141703us
I20260709 13:06:20.922588 16533 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:20.898135 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:20.924957 15854 raft_consensus.cc:1275] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:06:20.899988 15859 raft_consensus.cc:1240] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:20.904901 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:20.926352 15928 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:06:20.927465 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:20.863116 19437 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:20.858428 19387 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:20.897377 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:20.881891 19588 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.913803 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:20.920150 15860 raft_consensus.cc:1240] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 7 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:06:20.872038 19618 raft_consensus.cc:2749] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:20.909360 18968 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:20.932639 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:20.949048 15856 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 8: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:20.891574 19408 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:20.935374 16533 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:20.937505 19437 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:20.938407 15855 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:20.907466 19478 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:20.938791 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 83325.1us
I20260709 13:06:20.964316 15976 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:20.965695 15976 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:20.905229 19568 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:20.900655 19585 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:20.915457 19566 consensus_queue.cc:260] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:20.929841 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:20.980134 16564 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:06:20.885811 19429 consensus_queue.cc:1048] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:06:20.956159 19418 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:20.907915 19223 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:20.904932 19513 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:20.952836 16533 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:20.955184 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:20.918597 19578 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:20.901688 19587 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:20.940013 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:21.018784 15859 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:06:20.946368 19535 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:20.969650 16570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 956) took 2167 ms. Trace:
I20260709 13:06:20.972290 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:20.944016 19436 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:20.944428 19635 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:20.981865 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:20.983026 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:20.934059 19455 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:20.945904 16565 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:06:20.984845 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:20.985718 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:21.036695 16003 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 7.
I20260709 13:06:21.037559 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:20.985944 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:20.903971 19263 consensus_queue.cc:260] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:20.968233 19614 raft_consensus.cc:2804] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:20.968246 19568 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:20.967631 19547 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:20.945433 19387 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:20.999568 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:20.948594 19588 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:20.945572 19357 consensus_queue.cc:237] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:21.007377 19632 log.cc:927] Time spent T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:06:20.980347 18968 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:21.009099 16007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1886) took 1875 ms. Trace:
W20260709 13:06:21.011292 19619 log.cc:927] Time spent T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:06:21.016970 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:21.002184 19618 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:06:21.028980 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:21.029373 15855 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
W20260709 13:06:20.988938 19503 consensus_meta.cc:287] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.988s	user 0.012s	sys 0.001s
I20260709 13:06:21.029116 16570 rpcz_store.cc:276] 0709 13:06:18.802302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.991770 (+189468us) service_pool.cc:224] Handling call
0709 13:06:18.992369 (+   599us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:19.183520 (+191151us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:19.183562 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:20.184666 (+1001104us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b2c00047e58. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e8ca1a11 00007f73e8cb1372 00007f73ea3e3a01 00007f73ea3a843a 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2
0709 13:06:20.185239 (+   573us) log.cc:844] Serialized 52 byte log entry
0709 13:06:20.388203 (+202964us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:20.388270 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:20.388374 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:20.964043 (+575669us) raft_consensus.cc:1685] finished
0709 13:06:20.964063 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:20.969568 (+  5505us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":155498,"raft.run_cpu_time_us":53,"raft.run_wall_time_us":53,"spinlock_wait_cycles":23772032,"thread_start_us":393074,"threads_started":2,"wal-append.queue_time_us":296814}
I20260709 13:06:21.032226 16564 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:20.982508 19572 raft_consensus.cc:2749] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:21.023154 15825 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:06:21.078135 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 55049.8us
I20260709 13:06:21.040496 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:21.040647 16569 raft_consensus.cc:1240] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:06:21.042778 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:20.981437 19581 consensus_queue.cc:260] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:21.044317 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:20.972970 19585 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:21.082487 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:21.083348 19585 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:21.083999 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:21.084379 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:21.085248 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:21.085826 15856 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6.
I20260709 13:06:21.086781 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:21.084038 16006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1894) took 1074 ms. Trace:
I20260709 13:06:21.088824 16006 rpcz_store.cc:276] 0709 13:06:20.009170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:20.258319 (+249149us) service_pool.cc:224] Handling call
0709 13:06:20.289145 (+ 30826us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:20.299041 (+  9896us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:20.299086 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:20.328096 (+ 29010us) log.cc:844] Serialized 26 byte log entry
0709 13:06:20.515796 (+187700us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:20.515852 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:20.515944 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:21.083858 (+567914us) raft_consensus.cc:1685] finished
0709 13:06:21.083874 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:21.083990 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9614,"raft.queue_time_us":9799,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":12853,"thread_start_us":186716,"threads_started":1,"wal-append.queue_time_us":539576}
I20260709 13:06:21.079908 19503 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:21.092221 16568 raft_consensus.cc:2393] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 8: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:21.092612 15935 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:06:21.093201 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:21.094084 16004 raft_consensus.cc:2410] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 12 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:06:21.095489 19450 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:21.100610 19503 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:06:21.101337 19640 log.cc:927] Time spent T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:06:21.100582 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:21.078213 18968 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:21.104127 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:21.104755 15935 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:06:21.106164 18968 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:21.001055 19222 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:21.029078 19535 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:21.028738 19566 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:21.030550 19641 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:21.062126 16535 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:21.115195 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 53148.9us
I20260709 13:06:21.067058 16007 rpcz_store.cc:276] 0709 13:06:19.133288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:20.223920 (+1090632us) service_pool.cc:224] Handling call
0709 13:06:20.234103 (+ 10183us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:20.234362 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:20.234396 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:20.612602 (+378206us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b2c000e5768. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e8ca1a11 00007f73e8cb1372 00007f73ea3e3a01 00007f73ea3a843a 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2
0709 13:06:20.613229 (+   627us) log.cc:844] Serialized 26 byte log entry
0709 13:06:20.890283 (+277054us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:20.890318 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:20.890382 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:21.008803 (+118421us) raft_consensus.cc:1685] finished
0709 13:06:21.008823 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:21.009040 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23569664,"thread_start_us":276662,"threads_started":1,"wal-append.queue_time_us":305464}
I20260709 13:06:21.064193 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:21.116745 16561 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:21.061218 19645 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:21.069200 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:20.998669 19511 consensus_queue.cc:260] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:21.129209 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:21.002386 19223 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:21.109333 15825 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:21.135687 19624 log.cc:927] Time spent T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.227s	user 0.000s	sys 0.001s
I20260709 13:06:21.136138 19326 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:21.138221 19326 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:21.141270 15857 raft_consensus.cc:1275] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:06:21.110397 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:21.110872 15928 raft_consensus.cc:2393] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 9: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:21.051123 19558 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:21.111130 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:21.058182 19614 raft_consensus.cc:697] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:06:21.079794 19617 log.cc:927] Time spent T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.223s	user 0.001s	sys 0.000s
I20260709 13:06:21.125386 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:21.063128 15976 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:21.039199 19635 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:22.104972 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 995621us
I20260709 13:06:22.105887 15825 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:22.107501 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:22.107689 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 976594us
I20260709 13:06:22.108337 15859 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:21.254670 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:21.203887 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 178028us
I20260709 13:06:21.627856 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:21.381292 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:21.112919 19222 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:22.155807 15860 raft_consensus.cc:2376] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 10. Re-sending same reply.
I20260709 13:06:22.156615 15935 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 4.
I20260709 13:06:21.372318 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:22.118817 15858 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.047s	user 0.017s	sys 0.014s
I20260709 13:06:21.114717 19535 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:21.459008 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:06:21.638890 15929 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.857s	user 0.000s	sys 0.014s
W20260709 13:06:22.193586 15935 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 1976) took 1203 ms (client timeout 1574 ms). Trace:
W20260709 13:06:22.193830 15929 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.412s	user 0.002s	sys 0.022s
W20260709 13:06:22.193996 15935 rpcz_store.cc:269] 0709 13:06:20.990378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.108003 (+117625us) service_pool.cc:224] Handling call
0709 13:06:22.193531 (+1085528us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:21.700438 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:22.173112 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:22.196009 15859 raft_consensus.cc:2468] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
W20260709 13:06:21.699823 15853 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.798s	user 0.001s	sys 0.009s
I20260709 13:06:22.196830 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:22.196923 15853 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.299s	user 0.006s	sys 0.014s
I20260709 13:06:22.197465 15859 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
W20260709 13:06:21.142030 19632 log.cc:927] Time spent T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 13:06:21.175211 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:21.449672 16566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 807) took 1302 ms. Trace:
I20260709 13:06:22.205458 16566 rpcz_store.cc:276] 0709 13:06:20.146921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:20.188396 (+ 41475us) service_pool.cc:224] Handling call
0709 13:06:20.584403 (+396007us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:20.584691 (+   288us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:20.584733 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:20.584809 (+    76us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:20.584859 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:20.584947 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:21.449414 (+864467us) spinlock_profiling.cc:243] Waited 382 ms on lock 0x7b5400695ad0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:21.449588 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":879570816}
I20260709 13:06:21.114660 19641 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:21.665251 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:22.116613 16565 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.074s	user 0.000s	sys 0.021s
W20260709 13:06:22.032016 19649 log.cc:927] Time spent T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.846s	user 0.001s	sys 0.001s
I20260709 13:06:22.215543 15932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 1984) took 1210 ms. Trace:
I20260709 13:06:21.152174 19591 raft_consensus.cc:2804] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:21.185638 19511 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:21.733525 19437 consensus_meta.cc:287] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.764s	user 0.007s	sys 0.002s
I20260709 13:06:21.336480 19326 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:21.670989 19547 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.113984 19513 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:06:21.445283 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:21.627983 15930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 1905) took 2327 ms. Trace:
I20260709 13:06:22.227617 15930 rpcz_store.cc:276] 0709 13:06:19.125327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:19.126319 (+   992us) service_pool.cc:224] Handling call
0709 13:06:21.452891 (+2326572us) spinlock_profiling.cc:243] Waited 786 ms on lock 0x7b54003b8154. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:21.453045 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1807920896}
W20260709 13:06:22.228756 15900 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.764s (SENT) [suppressed 26 similar messages]
I20260709 13:06:22.112365 16569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 812) took 1516 ms. Trace:
W20260709 13:06:21.628459 16009 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60156 (request call id 642) took 2378 ms (client timeout 2815 ms). Trace:
W20260709 13:06:22.229499 16009 rpcz_store.cc:269] 0709 13:06:19.074152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:19.110787 (+ 36635us) service_pool.cc:224] Handling call
0709 13:06:21.452385 (+2341598us) spinlock_profiling.cc:243] Waited 692 ms on lock 0x7b5400305a54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:21.452535 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1591439616}
I20260709 13:06:22.188011 19558 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:06:21.709980 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:22.230875 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:22.231395 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:22.167256 19635 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:21.673326 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:21.110730 19418 raft_consensus.cc:2804] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:21.106242 19450 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:22.170845 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 62225.2us
I20260709 13:06:22.199083 15936 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 6.
I20260709 13:06:21.113829 19480 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:21.283820 19622 log.cc:927] Time spent T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
W20260709 13:06:21.284611 19639 log.cc:927] Time spent T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.177s	user 0.000s	sys 0.000s
I20260709 13:06:22.251080 19450 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.251598 19450 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:22.253885 19450 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:22.254356 19450 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.254745 19450 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:21.290699 19478 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:21.728451 19598 log.cc:927] Time spent T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 1.623s	user 0.002s	sys 0.001s
W20260709 13:06:21.425607 19638 log.cc:927] Time spent T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
W20260709 13:06:21.280721 19629 log.cc:927] Time spent T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.265s	user 0.001s	sys 0.000s
I20260709 13:06:22.032419 19643 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:22.185971 19242 consensus_meta.cc:287] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 1.722s	user 0.013s	sys 0.000s
W20260709 13:06:21.125931 19519 log.cc:927] Time spent T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 3.126s	user 0.012s	sys 0.008s
I20260709 13:06:21.966285 19587 leader_election.cc:290] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:21.152169 19585 raft_consensus.cc:2749] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:22.007756 19588 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:22.103391 19223 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:21.697651 19129 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.283846 19455 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:22.357126 19129 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:21.283766 19367 consensus_meta.cc:287] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.808s	user 0.007s	sys 0.000s
W20260709 13:06:22.199471 15858 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.133s	user 0.020s	sys 0.014s
I20260709 13:06:22.342586 15937 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 1894) took 3795 ms. Trace:
I20260709 13:06:22.198488 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:22.401063 15902 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:22.228665 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:22.409008 15859 raft_consensus.cc:2410] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:06:22.409816 15930 raft_consensus.cc:2393] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 7: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:22.410655 15930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 1975) took 1440 ms. Trace:
I20260709 13:06:22.168380 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:22.412532 15857 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
I20260709 13:06:22.345438 15933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1951) took 1600 ms. Trace:
I20260709 13:06:22.413465 15933 rpcz_store.cc:276] 0709 13:06:20.744652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:20.760987 (+ 16335us) service_pool.cc:224] Handling call
0709 13:06:20.761417 (+   430us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:20.761882 (+   465us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:20.761922 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:20.768539 (+  6617us) log.cc:844] Serialized 26 byte log entry
0709 13:06:20.927326 (+158787us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:20.927367 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:20.927451 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:22.344794 (+1417343us) raft_consensus.cc:1685] finished
0709 13:06:22.344845 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:22.345191 (+   346us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":120,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":9006,"thread_start_us":158431,"threads_started":1,"wal-append.queue_time_us":282780}
I20260709 13:06:22.212450 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:22.416644 15935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 1978) took 1412 ms. Trace:
W20260709 13:06:22.346426 16007 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.158s	user 0.016s	sys 0.023s
I20260709 13:06:22.417261 19478 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:06:22.417529 16007 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.242s	user 0.016s	sys 0.023s
I20260709 13:06:22.234267 15931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 1974) took 1286 ms. Trace:
I20260709 13:06:22.216035 15932 rpcz_store.cc:276] 0709 13:06:21.005434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.664859 (+659425us) service_pool.cc:224] Handling call
0709 13:06:22.215477 (+550618us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:22.217099 16565 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.175s	user 0.000s	sys 0.023s
I20260709 13:06:22.349145 16567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 957) took 3538 ms. Trace:
I20260709 13:06:22.241883 19418 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.235554 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 2016) took 3161 ms. Trace:
I20260709 13:06:22.444155 19418 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:22.444267 16002 rpcz_store.cc:276] 0709 13:06:19.073816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:20.147383 (+1073567us) service_pool.cc:224] Handling call
0709 13:06:20.195083 (+ 47700us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:20.195506 (+   423us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:20.195545 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:20.196329 (+   784us) log.cc:844] Serialized 26 byte log entry
0709 13:06:20.503746 (+307417us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:20.503801 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:20.503893 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:22.235254 (+1731361us) raft_consensus.cc:1685] finished
0709 13:06:22.235282 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:22.235472 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":401,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":10402,"thread_start_us":259639,"threads_started":1,"wal-append.queue_time_us":524432}
I20260709 13:06:22.112283 16533 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:22.241542 19326 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:22.274631 19242 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:22.260947 18968 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:22.262066 19480 consensus_queue.cc:1048] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.017s
I20260709 13:06:22.460955 19455 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:22.461439 19455 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:22.166092 15827 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:22.243228 19558 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:06:22.462706 18968 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:22.464933 19558 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:22.279819 19587 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:22.218619 19591 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.466205 19591 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:22.466846 15929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1952) took 1692 ms. Trace:
I20260709 13:06:21.373296 19610 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:22.214723 19641 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:22.268883 19643 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:22.310702 19651 consensus_queue.cc:1059] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:22.511790 19651 consensus_queue.cc:1243] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Peer 31882606e87447fb9a36bbf625129fe8 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:06:22.260682 19614 consensus_queue.cc:237] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:22.241207 19513 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:22.209633 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:22.520037 19666 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:22.224172 19437 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:22.520748 19666 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.521267 19666 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:22.114243 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:22.232877 19547 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:22.229434 16569 rpcz_store.cc:276] 0709 13:06:20.596434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:20.599266 (+  2832us) service_pool.cc:224] Handling call
0709 13:06:20.599607 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:21.080799 (+481192us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.112070 (+1031271us) spinlock_profiling.cc:243] Waited 441 ms on lock 0x7b54004790d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:22.112309 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1013727488}
I20260709 13:06:22.207540 15862 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 8: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:22.525139 15862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 1925) took 1085 ms. Trace:
I20260709 13:06:22.525491 15862 rpcz_store.cc:276] 0709 13:06:21.439153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.458662 (+ 19509us) service_pool.cc:224] Handling call
0709 13:06:22.524945 (+1066283us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:21.152462 19645 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:22.528079 15934 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:22.192655 15860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 2043) took 1083 ms. Trace:
I20260709 13:06:22.529273 15860 rpcz_store.cc:276] 0709 13:06:21.109476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.109802 (+   326us) service_pool.cc:224] Handling call
0709 13:06:22.192594 (+1082792us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:22.531333 15860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1931) took 1039 ms. Trace:
I20260709 13:06:22.400097 15858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2034) took 1500 ms. Trace:
I20260709 13:06:22.400977 19129 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:22.402249 19385 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.405586 15853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2030) took 1508 ms. Trace:
I20260709 13:06:22.241055 15936 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53636 (request call id 717) took 1213 ms. Trace:
I20260709 13:06:22.411113 15930 rpcz_store.cc:276] 0709 13:06:20.970186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.228416 (+1258230us) service_pool.cc:224] Handling call
0709 13:06:22.410519 (+182103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:22.545279 15936 rpcz_store.cc:276] 0709 13:06:21.027561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.371943 (+344382us) service_pool.cc:224] Handling call
0709 13:06:22.240974 (+869031us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:22.546406 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:22.546536 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:22.547159 15930 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
W20260709 13:06:22.547742 15930 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 1916) took 1652 ms (client timeout 2159 ms). Trace:
W20260709 13:06:22.548192 15930 rpcz_store.cc:269] 0709 13:06:20.895574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.546016 (+1650442us) service_pool.cc:224] Handling call
0709 13:06:22.547665 (+  1649us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:22.549381 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:22.550021 15930 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:22.410995 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:22.552106 15859 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:06:22.552680 15859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 1927) took 1062 ms. Trace:
I20260709 13:06:22.553040 15859 rpcz_store.cc:276] 0709 13:06:21.490212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.410321 (+920109us) service_pool.cc:224] Handling call
0709 13:06:22.552617 (+142296us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:22.555138 15859 raft_consensus.cc:1240] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 9 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:06:22.417052 15935 rpcz_store.cc:276] 0709 13:06:21.003925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.209235 (+1205310us) service_pool.cc:224] Handling call
0709 13:06:22.416435 (+207200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:22.557695 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:22.558728 15935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 1977) took 1555 ms. Trace:
I20260709 13:06:22.559095 15935 rpcz_store.cc:276] 0709 13:06:21.003581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.557242 (+1553661us) service_pool.cc:224] Handling call
0709 13:06:22.558667 (+  1425us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:22.427500 19350 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.081s	user 0.005s	sys 0.020s
W20260709 13:06:22.559947 19350 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.274s	user 0.005s	sys 0.021s
W20260709 13:06:22.431815 19372 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.336s	user 0.005s	sys 0.025s
I20260709 13:06:22.438251 15931 rpcz_store.cc:276] 0709 13:06:20.947842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.448576 (+500734us) service_pool.cc:224] Handling call
0709 13:06:22.234209 (+785633us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:22.561002 19372 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.544s	user 0.005s	sys 0.025s
I20260709 13:06:22.436779 16567 rpcz_store.cc:276] 0709 13:06:18.810329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:19.037646 (+227317us) service_pool.cc:224] Handling call
0709 13:06:19.037963 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:19.262865 (+224902us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:19.262909 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:19.275418 (+ 12509us) log.cc:844] Serialized 26 byte log entry
0709 13:06:19.925345 (+649927us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:19.925385 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:19.925447 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:21.423893 (+1498446us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:06:22.348356 (+924463us) raft_consensus.cc:1685] finished
0709 13:06:22.348403 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:22.348847 (+   444us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":110792,"raft.run_cpu_time_us":65,"raft.run_wall_time_us":65,"spinlock_wait_cycles":38447360,"thread_start_us":873917,"threads_started":2,"wal-append.queue_time_us":829875}
I20260709 13:06:22.561748 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:22.562836 15931 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:06:22.563700 15931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 1967) took 1756 ms. Trace:
I20260709 13:06:22.564208 15931 rpcz_store.cc:276] 0709 13:06:20.807725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.561321 (+1753596us) service_pool.cc:224] Handling call
0709 13:06:22.563541 (+  2220us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:22.453586 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 341524us
I20260709 13:06:22.458555 19326 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:22.461845 19455 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:22.572798 16533 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:22.573401 19547 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:22.466325 19587 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:22.394129 15937 rpcz_store.cc:276] 0709 13:06:18.547331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.547495 (+   164us) service_pool.cc:224] Handling call
0709 13:06:18.547903 (+   408us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:18.555709 (+  7806us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:18.599892 (+ 44183us) log.cc:844] Serialized 14 byte log entry
0709 13:06:18.858282 (+258390us) log.cc:844] Serialized 14 byte log entry
0709 13:06:20.534393 (+1676111us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b18004318c0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72b8455 00007f73ea3a0060 00007f73ea3b2716 00007f73ea3c3287 00007f73ea3c3208
0709 13:06:20.564199 (+ 29806us) spinlock_profiling.cc:243] Waited 2.74 ms on lock 0x7b18004318c0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb522 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd 000000000038f354 000000000038f199 00007f73e729e83a
0709 13:06:20.753366 (+189167us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:20.754259 (+   893us) log.cc:844] Serialized 52 byte log entry
0709 13:06:20.858263 (+104004us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:20.858315 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:20.858909 (+   594us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:22.341623 (+1482714us) raft_consensus.cc:1685] finished
0709 13:06:22.341706 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:22.342194 (+   488us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9770,"prepare.queue_time_us":44827,"prepare.run_cpu_time_us":118,"prepare.run_wall_time_us":21359,"raft.queue_time_us":431983,"raft.run_cpu_time_us":785,"raft.run_wall_time_us":69563,"spinlock_wait_cycles":31655424,"thread_start_us":522978,"threads_started":3,"wal-append.queue_time_us":692060,"wal-append.run_cpu_time_us":9907,"wal-append.run_wall_time_us":1058500}
I20260709 13:06:22.575101 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:22.262315 19635 raft_consensus.cc:2804] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:22.466351 18968 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:22.575979 15937 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 1963) took 1816 ms. Trace:
I20260709 13:06:22.576228 19635 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.576367 15937 rpcz_store.cc:276] 0709 13:06:20.759899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.574772 (+1814873us) service_pool.cc:224] Handling call
0709 13:06:22.575917 (+  1145us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:22.577598 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:22.578138 19513 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:22.579927 19645 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:22.581044 19437 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:22.582834 19222 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:22.542160 15853 rpcz_store.cc:276] 0709 13:06:20.896773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:20.896888 (+   115us) service_pool.cc:224] Handling call
0709 13:06:20.897154 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:20.897346 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:22.197658 (+1300312us) log.cc:844] Serialized 14 byte log entry
0709 13:06:22.404860 (+207202us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:22.404967 (+   107us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:22.405068 (+   101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.405176 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:22.405404 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":181424,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":2813,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":115,"thread_start_us":1503341,"threads_started":2,"wal-append.queue_time_us":300081}
I20260709 13:06:22.493741 19450 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:22.467257 15929 rpcz_store.cc:276] 0709 13:06:20.745026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:20.746315 (+  1289us) service_pool.cc:224] Handling call
0709 13:06:20.780128 (+ 33813us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:20.780586 (+   458us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:06:20.780626 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:20.781155 (+   529us) log.cc:844] Serialized 26 byte log entry
0709 13:06:22.194551 (+1413396us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:22.194667 (+   116us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.198224 (+  3557us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:22.393592 (+195368us) raft_consensus.cc:1685] finished
0709 13:06:22.395279 (+  1687us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:22.436858 (+ 41579us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4319,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":96,"thread_start_us":1413054,"threads_started":1,"wal-append.queue_time_us":1517626}
I20260709 13:06:22.585208 15855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1926) took 1145 ms. Trace:
I20260709 13:06:22.585332 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:22.505254 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:22.586231 15932 raft_consensus.cc:2376] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7. Re-sending same reply.
W20260709 13:06:22.586890 15932 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 1979) took 1582 ms (client timeout 2029 ms). Trace:
W20260709 13:06:22.587335 15932 rpcz_store.cc:269] 0709 13:06:21.004196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.436049 (+1431853us) service_pool.cc:224] Handling call
0709 13:06:22.586796 (+150747us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":987520}
W20260709 13:06:22.550042 16568 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45244 (request call id 960) took 2106 ms (client timeout 1754 ms). Trace:
I20260709 13:06:22.520067 15861 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 4.
W20260709 13:06:22.588246 16568 rpcz_store.cc:269] 0709 13:06:20.448581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:20.448727 (+   146us) service_pool.cc:224] Handling call
0709 13:06:22.549583 (+2100856us) spinlock_profiling.cc:243] Waited 617 ms on lock 0x7b54000cce54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:22.549793 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1419005184}
I20260709 13:06:22.589224 15861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 1932) took 1097 ms. Trace:
I20260709 13:06:22.589672 15861 rpcz_store.cc:276] 0709 13:06:21.491849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.115342 (+623493us) service_pool.cc:224] Handling call
0709 13:06:22.589115 (+473773us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:22.524097 19660 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:06:22.290059 19631 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.260s	user 0.013s	sys 0.061s
I20260709 13:06:22.590621 19660 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:22.276088 19331 consensus_meta.cc:287] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 1.790s	user 0.013s	sys 0.000s
I20260709 13:06:22.591038 19660 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:22.591775 19331 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:22.585585 15855 rpcz_store.cc:276] 0709 13:06:21.439477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.439729 (+   252us) service_pool.cc:224] Handling call
0709 13:06:21.440160 (+   431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:21.440634 (+   474us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:21.440676 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:22.169816 (+729140us) log.cc:844] Serialized 26 byte log entry
0709 13:06:22.455616 (+285800us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:22.492007 (+ 36391us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.498670 (+  6663us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:22.584824 (+ 86154us) raft_consensus.cc:1685] finished
0709 13:06:22.584842 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:22.585038 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":48461,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":111,"thread_start_us":46706,"threads_started":1,"wal-append.queue_time_us":211821}
I20260709 13:06:22.594338 19331 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:22.519806 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 337729us
W20260709 13:06:22.596006 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 1.5327e+06us
I20260709 13:06:22.597100 15827 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:22.599466 16562 raft_consensus.cc:2368] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 9. Current term is 10.
I20260709 13:06:22.577044 19635 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:22.601711 19643 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:06:22.603770 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 497753us
I20260709 13:06:22.604001 19587 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:22.606381 19387 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:22.527545 15862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1930) took 1036 ms. Trace:
I20260709 13:06:22.443157 16565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 821) took 1460 ms. Trace:
I20260709 13:06:22.528787 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:06:22.547485 15936 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 1919) took 1651 ms (client timeout 2198 ms). Trace:
I20260709 13:06:22.611902 19385 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:22.612416 15936 rpcz_store.cc:269] 0709 13:06:20.896307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.546212 (+1649905us) service_pool.cc:224] Handling call
0709 13:06:22.547405 (+  1193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:22.612742 19223 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:22.611339 16565 rpcz_store.cc:276] 0709 13:06:20.982934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.041119 (+ 58185us) service_pool.cc:224] Handling call
0709 13:06:21.041421 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:21.041629 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:21.041653 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:21.042083 (+   430us) log.cc:844] Serialized 26 byte log entry
0709 13:06:22.442050 (+1399967us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:22.442197 (+   147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.442525 (+   328us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:22.442580 (+    55us) raft_consensus.cc:1685] finished
0709 13:06:22.442597 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:22.442884 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":1399215,"threads_started":1,"wal-append.queue_time_us":1260212}
I20260709 13:06:22.561785 19610 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:22.570204 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:22.616006 19223 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:22.616142 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:22.502562 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 101952us
I20260709 13:06:22.616523 19223 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:22.573729 19668 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.607563 15856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2046) took 1483 ms. Trace:
I20260709 13:06:22.620198 19668 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:22.620509 19367 consensus_queue.cc:1059] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:06:22.620360 15856 rpcz_store.cc:276] 0709 13:06:21.123591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.129366 (+  5775us) service_pool.cc:224] Handling call
0709 13:06:21.256677 (+127311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:21.256968 (+   291us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:21.257008 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:22.186110 (+929102us) log.cc:844] Serialized 26 byte log entry
0709 13:06:22.555992 (+369882us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:22.556088 (+    96us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.556221 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:22.607055 (+ 50834us) raft_consensus.cc:1685] finished
0709 13:06:22.607078 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:22.607310 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":129044,"threads_started":1,"wal-append.queue_time_us":148100}
I20260709 13:06:22.624033 19585 raft_consensus.cc:3055] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:22.624639 19585 raft_consensus.cc:740] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:22.537746 15860 rpcz_store.cc:276] 0709 13:06:21.491525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.530154 (+1038629us) service_pool.cc:224] Handling call
0709 13:06:22.530626 (+   472us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:22.530975 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:22.531007 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:22.531051 (+    44us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:22.531081 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.531146 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:22.531258 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:22.590979 19631 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.575s	user 0.014s	sys 0.062s
I20260709 13:06:22.626462 19585 consensus_queue.cc:260] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:22.616266 15931 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 8: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:22.627981 15854 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
I20260709 13:06:22.538056 15858 rpcz_store.cc:276] 0709 13:06:20.899390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:20.903440 (+  4050us) service_pool.cc:224] Handling call
0709 13:06:20.931691 (+ 28251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:20.934062 (+  2371us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:21.042441 (+108379us) log.cc:844] Serialized 14 byte log entry
0709 13:06:22.399225 (+1356784us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:22.399375 (+   150us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:22.399478 (+   103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.399595 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:22.399845 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2060,"prepare.queue_time_us":195036,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":11594,"raft.run_cpu_time_us":393,"raft.run_wall_time_us":5669,"spinlock_wait_cycles":559488,"thread_start_us":1433323,"threads_started":2,"wal-append.queue_time_us":1245606}
I20260709 13:06:22.630254 15937 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:06:22.630987 15937 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 1983) took 1625 ms. Trace:
I20260709 13:06:22.631368 15937 rpcz_store.cc:276] 0709 13:06:21.005150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.577285 (+1572135us) service_pool.cc:224] Handling call
0709 13:06:22.630925 (+ 53640us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:22.632475 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:22.634436 15937 raft_consensus.cc:1240] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:22.635156 15937 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1960) took 1885 ms. Trace:
I20260709 13:06:22.628007 15931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 1955) took 1879 ms. Trace:
I20260709 13:06:22.636125 15931 rpcz_store.cc:276] 0709 13:06:20.748549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.565487 (+1816938us) service_pool.cc:224] Handling call
0709 13:06:22.627929 (+ 62442us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:22.640139 19610 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:06:22.640426 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 67855.5us
I20260709 13:06:22.640820 19610 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:22.641422 19610 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:22.611181 15862 rpcz_store.cc:276] 0709 13:06:21.491230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.526446 (+1035216us) service_pool.cc:224] Handling call
0709 13:06:22.526876 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:22.527191 (+   315us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:22.527215 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:22.527270 (+    55us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:22.527325 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.527378 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:22.527481 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:22.642145 16533 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:22.643774 19585 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:22.644892 19674 consensus_queue.cc:1048] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:06:22.646770 19674 raft_consensus.cc:2749] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:22.646931 19609 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:22.609930 15757 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58966 (request call id 243) took 25448 ms (client timeout 15000 ms). Trace:
I20260709 13:06:22.647727 15859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1933) took 1074 ms. Trace:
I20260709 13:06:22.648355 15859 rpcz_store.cc:276] 0709 13:06:21.492170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.553803 (+1061633us) service_pool.cc:224] Handling call
0709 13:06:22.554590 (+   787us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:22.555612 (+  1022us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.566428 (+ 10816us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:22.635504 15937 rpcz_store.cc:276] 0709 13:06:20.750061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.633676 (+1883615us) service_pool.cc:224] Handling call
0709 13:06:22.634120 (+   444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:22.634857 (+   737us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.635075 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:22.651912 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 47058.9us
I20260709 13:06:22.664556 16007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 2025) took 1531 ms. Trace:
I20260709 13:06:22.639660 19589 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:22.676837 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:22.672890 19668 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:22.684067 15935 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 8: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:22.691556 15935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 1982) took 1682 ms. Trace:
I20260709 13:06:22.699440 15935 rpcz_store.cc:276] 0709 13:06:21.004910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.566163 (+1561253us) service_pool.cc:224] Handling call
0709 13:06:22.686820 (+120657us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:22.668401 16007 rpcz_store.cc:276] 0709 13:06:21.036987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.116382 (+ 79395us) service_pool.cc:224] Handling call
0709 13:06:21.175188 (+ 58806us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:21.175445 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:22.424295 (+1248850us) log.cc:844] Serialized 14 byte log entry
0709 13:06:22.501960 (+ 77665us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:22.502054 (+    94us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:22.502172 (+   118us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.506358 (+  4186us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:22.552495 (+ 46137us) spinlock_profiling.cc:243] Waited 29.3 ms on lock 0x7b18003f0040. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb522 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd 000000000038f354 000000000038f199 00007f73e729e83a
0709 13:06:22.568600 (+ 16105us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":95274,"prepare.queue_time_us":1137949,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":1477,"raft.run_cpu_time_us":372,"raft.run_wall_time_us":11322,"spinlock_wait_cycles":67369344,"thread_start_us":1278511,"threads_started":2,"wal-append.queue_time_us":55088}
I20260709 13:06:22.711086 16533 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:22.713229 16533 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:22.714426 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:22.714872 19591 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:22.721702 19660 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:22.723413 15857 raft_consensus.cc:2393] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 9: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:22.724439 16565 raft_consensus.cc:2393] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 9: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
W20260709 13:06:22.648281 15757 rpcz_store.cc:269] 0709 13:05:57.160895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.161146 (+   251us) service_pool.cc:224] Handling call
0709 13:06:22.609484 (+25448338us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:12.539030 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:12.539986 (+   956us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:12.540052 (+    66us) write_op.cc:435] Acquired schema lock
0709 13:06:12.540110 (+    58us) tablet.cc:662] Decoding operations
0709 13:06:12.541691 (+  1581us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:12.541802 (+   111us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:12.541845 (+    43us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:06:12.543118 (+  1273us) tablet.cc:701] Row locks acquired
0709 13:06:12.543171 (+    53us) write_op.cc:260] PREPARE: finished
0709 13:06:12.543358 (+   187us) write_op.cc:270] Start()
0709 13:06:12.543488 (+   130us) write_op.cc:276] Timestamp: P: 1783602372543297 usec, L: 0
0709 13:06:12.543511 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:06:12.604284 (+ 60773us) log.cc:844] Serialized 4900 byte log entry
0709 13:06:13.863713 (+1259429us) op_driver.cc:464] REPLICATION: finished
0709 13:06:14.064042 (+200329us) write_op.cc:301] APPLY: starting
0709 13:06:14.064149 (+   107us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:14.064484 (+   335us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:14.064636 (+   152us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:14.333414 (+268778us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:14.384197 (+ 50783us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=12
0709 13:06:14.384257 (+    60us) write_op.cc:312] APPLY: finished
0709 13:06:14.384675 (+   418us) log.cc:844] Serialized 89 byte log entry
0709 13:06:14.384859 (+   184us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:14.385002 (+   143us) write_op.cc:454] Released schema lock
0709 13:06:14.385071 (+    69us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:14.385491 (+   420us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:14.385528 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2070838,"reported_tablets":38,"child_traces":[["op",{"apply.queue_time_us":200048,"apply.run_cpu_time_us":5675,"apply.run_wall_time_us":321421,"num_ops":12,"prepare.queue_time_us":527159,"prepare.run_cpu_time_us":25616,"prepare.run_wall_time_us":429768,"raft.queue_time_us":602552,"raft.run_cpu_time_us":9243,"raft.run_wall_time_us":153803,"replication_time_us":1320002,"thread_start_us":1338163,"threads_started":4,"wal-append.queue_time_us":441792,"wal-append.run_cpu_time_us":37376,"wal-append.run_wall_time_us":2347239}]]}
I20260709 13:06:22.727622 19660 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:22.728108 19591 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:22.728737 19587 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:06:22.730548 19636 log.cc:927] Time spent T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 1.693s	user 0.001s	sys 0.013s
I20260709 13:06:22.744396 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:22.728914 19668 raft_consensus.cc:2749] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:22.704197 19609 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:22.732115 19455 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:22.733276 16005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1882) took 3721 ms. Trace:
I20260709 13:06:22.734510 19641 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:22.728587 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 408555us
I20260709 13:06:22.754593 16005 rpcz_store.cc:276] 0709 13:06:19.011565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:19.778821 (+767256us) service_pool.cc:224] Handling call
0709 13:06:19.810528 (+ 31707us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:20.364307 (+553779us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.732696 (+2368389us) spinlock_profiling.cc:243] Waited 549 ms on lock 0x7b54001dd154. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:22.733056 (+   360us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1262902656}
W20260709 13:06:22.736296 19568 consensus_meta.cc:287] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 1.675s	user 0.013s	sys 0.004s
I20260709 13:06:22.737505 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:22.737211 15934 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53636 (request call id 715) took 1710 ms (client timeout 1631 ms). Trace:
I20260709 13:06:22.752242 15930 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:22.755044 15937 raft_consensus.cc:1275] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 9 index: 11. (index mismatch)
W20260709 13:06:22.762794 15930 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 1917) took 1867 ms (client timeout 2238 ms). Trace:
W20260709 13:06:22.763268 15930 rpcz_store.cc:269] 0709 13:06:20.895869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.549047 (+1653178us) service_pool.cc:224] Handling call
0709 13:06:22.762740 (+213693us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:22.753827 19660 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.764715 19222 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:22.765391 19566 consensus_meta.cc:287] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 1.644s	user 0.012s	sys 0.008s
W20260709 13:06:22.766245 15933 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 1973) took 1818 ms (client timeout 2072 ms). Trace:
W20260709 13:06:22.766667 15933 rpcz_store.cc:269] 0709 13:06:20.947531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.427705 (+1480174us) service_pool.cc:224] Handling call
0709 13:06:22.731534 (+303829us) spinlock_profiling.cc:243] Waited 6.44 ms on lock 0x7b5800003698. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:06:22.766161 (+ 34627us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14809344}
I20260709 13:06:22.768181 15933 raft_consensus.cc:1240] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:22.768846 15933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1980) took 1764 ms. Trace:
I20260709 13:06:22.769217 15933 rpcz_store.cc:276] 0709 13:06:21.004453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.767515 (+1763062us) service_pool.cc:224] Handling call
0709 13:06:22.767942 (+   427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:22.768597 (+   655us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.768790 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:22.760640 15934 rpcz_store.cc:269] 0709 13:06:21.026992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.140134 (+113142us) service_pool.cc:224] Handling call
0709 13:06:22.736899 (+1596765us) spinlock_profiling.cc:243] Waited 418 ms on lock 0x7b54002daad4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:22.737069 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":961776000}
I20260709 13:06:22.764976 19660 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:22.768846 16563 raft_consensus.cc:2393] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 10: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:22.776659 15932 raft_consensus.cc:1240] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:06:22.751016 19587 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:22.781046 15937 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1965) took 1987 ms. Trace:
I20260709 13:06:22.781517 15937 rpcz_store.cc:276] 0709 13:06:20.788078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.656389 (+1868311us) service_pool.cc:224] Handling call
0709 13:06:22.674100 (+ 17711us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:22.762811 (+ 88711us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.775483 (+ 12672us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:22.785306 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 436384us
I20260709 13:06:22.789268 19222 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.790009 19222 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:22.792092 19350 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:22.793267 19511 consensus_meta.cc:287] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.569s	user 0.009s	sys 0.005s
I20260709 13:06:22.792861 19350 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:22.793926 19511 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:22.794557 19511 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:22.795864 19201 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.796420 19201 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:22.797127 19660 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:22.762976 19387 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.798585 15929 raft_consensus.cc:2410] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:06:22.800856 15935 raft_consensus.cc:1275] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:06:22.799278 15902 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:22.803655 15935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 1918) took 1907 ms. Trace:
I20260709 13:06:22.804185 15935 rpcz_store.cc:276] 0709 13:06:20.896096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.700443 (+1804347us) service_pool.cc:224] Handling call
0709 13:06:22.757966 (+ 57523us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:22.802512 (+ 44546us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.803601 (+  1089us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:22.806006 19555 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.806566 19555 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:22.807541 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:22.808022 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:22.808436 16564 raft_consensus.cc:2393] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 7: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:22.808810 16568 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:22.810876 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:22.779906 19589 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:22.784677 19587 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:22.814635 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:22.812000 19387 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:22.819707 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:22.818332 16533 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:22.826624 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 50680.6us
I20260709 13:06:22.798585 19674 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.830487 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:22.830612 15936 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1957) took 2078 ms. Trace:
I20260709 13:06:22.807957 19350 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:22.780009 19609 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:22.826385 19222 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:22.843194 19635 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:22.844129 16568 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:06:22.845250 15974 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:22.846271 19455 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:22.848081 19129 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:22.815145 19675 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:22.843483 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:22.851089 19535 consensus_meta.cc:287] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.608s	user 0.013s	sys 0.002s
I20260709 13:06:22.818840 19582 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:06:22.852694 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:22.853614 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:22.780853 19568 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:22.864193 16008 raft_consensus.cc:2393] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 9: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:22.865234 16004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 652) took 1872 ms. Trace:
W20260709 13:06:22.865537 16008 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41662 (request call id 2028) took 1804 ms (client timeout 1610 ms). Trace:
W20260709 13:06:22.865959 16008 rpcz_store.cc:269] 0709 13:06:21.061625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.082193 (+ 20568us) service_pool.cc:224] Handling call
0709 13:06:22.865270 (+1783077us) spinlock_profiling.cc:243] Waited 1.78 s on lock 0x7b54003ed354. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:22.865446 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4095207680}
I20260709 13:06:22.866736 15857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 1940) took 1100 ms. Trace:
I20260709 13:06:22.867138 15857 rpcz_store.cc:276] 0709 13:06:21.766616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.490799 (+724183us) service_pool.cc:224] Handling call
0709 13:06:22.866555 (+375756us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b5400217d54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:22.866676 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":255042304}
I20260709 13:06:22.868582 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:22.859735 16563 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:06:22.843449 19641 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:22.852530 15825 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:22.852499 19511 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:22.832980 15936 rpcz_store.cc:276] 0709 13:06:20.749179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.613189 (+1864010us) service_pool.cc:224] Handling call
0709 13:06:22.613685 (+   496us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:22.672881 (+ 59196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:22.672919 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:22.683373 (+ 10454us) log.cc:844] Serialized 26 byte log entry
0709 13:06:22.778991 (+ 95618us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:22.779046 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.779134 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:22.827037 (+ 47903us) raft_consensus.cc:1685] finished
0709 13:06:22.827059 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:22.827186 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55468,"raft.queue_time_us":67419,"raft.run_wall_time_us":8147,"spinlock_wait_cycles":27055488,"thread_start_us":76066,"threads_started":1,"wal-append.queue_time_us":119981}
I20260709 13:06:22.871959 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:22.875274 19418 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:22.879191 19514 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:22.880394 19418 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:22.880518 15902 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:22.882442 15902 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:22.888347 15933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1981) took 1883 ms. Trace:
I20260709 13:06:22.888808 15933 rpcz_store.cc:276] 0709 13:06:21.004684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.769989 (+1765305us) service_pool.cc:224] Handling call
0709 13:06:22.770350 (+   361us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:22.770843 (+   493us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:22.770874 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:22.771532 (+   658us) log.cc:844] Serialized 52 byte log entry
0709 13:06:22.831286 (+ 59754us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:22.831346 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.831405 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:22.888094 (+ 56689us) raft_consensus.cc:1685] finished
0709 13:06:22.888118 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:22.888277 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7638,"raft.queue_time_us":7576,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":3446,"thread_start_us":57243,"threads_started":1,"wal-append.queue_time_us":112684}
I20260709 13:06:22.893734 19387 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:22.875556 19222 consensus_queue.cc:1048] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:06:22.895539 15937 raft_consensus.cc:1275] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 4. (term mismatch)
I20260709 13:06:22.895828 19588 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:22.865654 16004 rpcz_store.cc:276] 0709 13:06:20.992835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.095337 (+102502us) service_pool.cc:224] Handling call
0709 13:06:21.095552 (+   215us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:22.532290 (+1436738us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:06:22.532906 (+   616us) log.cc:844] Serialized 14 byte log entry
0709 13:06:22.696237 (+163331us) log.cc:844] Serialized 14 byte log entry
0709 13:06:22.696404 (+   167us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:22.696448 (+    44us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:22.697392 (+   944us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.864770 (+167378us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b54001db354. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:22.864824 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:22.865082 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1048,"prepare.run_cpu_time_us":113,"prepare.run_wall_time_us":115,"raft.queue_time_us":833,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":176,"spinlock_wait_cycles":3302788992,"thread_start_us":162552,"threads_started":1,"wal-append.queue_time_us":162283}
W20260709 13:06:22.900520 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 52841.7us
I20260709 13:06:22.901414 19350 raft_consensus.cc:2749] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:22.901960 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:22.902482 16010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 2026) took 1818 ms. Trace:
I20260709 13:06:22.908847 19609 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:22.848680 19589 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:22.918757 15937 pending_rounds.cc:85] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 3
W20260709 13:06:22.921984 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:22.848763 19674 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:22.913084 19635 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:22.937278 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 62444.6us
I20260709 13:06:22.919762 19588 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.872383 19675 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:22.874090 19535 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:22.943805 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 61477.1us
I20260709 13:06:22.936218 15937 pending_rounds.cc:107] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:06:22.942513 16570 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:06:22.949985 15937 raft_consensus.cc:2889] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:22.950317 15937 pending_rounds.cc:107] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:06:22.950532 15937 raft_consensus.cc:2889] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:22.949432 16566 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 10.
I20260709 13:06:22.944327 19588 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:22.922816 16010 rpcz_store.cc:276] 0709 13:06:21.042106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.442260 (+400154us) service_pool.cc:224] Handling call
0709 13:06:22.115803 (+673543us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:22.115999 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:22.668070 (+552071us) log.cc:844] Serialized 14 byte log entry
0709 13:06:22.797145 (+129075us) log.cc:844] Serialized 14 byte log entry
0709 13:06:22.805109 (+  7964us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:22.824112 (+ 19003us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:22.832672 (+  8560us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.848683 (+ 16011us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:22.860086 (+ 11403us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18132,"prepare.queue_time_us":198499,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":72,"raft.queue_time_us":21710,"raft.run_cpu_time_us":240,"raft.run_wall_time_us":10502,"thread_start_us":529523,"threads_started":2,"wal-append.queue_time_us":143927}
I20260709 13:06:22.919633 19582 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:22.957068 19568 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:22.957142 19514 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:06:22.960033 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 107584us
I20260709 13:06:22.955648 19591 raft_consensus.cc:2804] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:22.943585 19418 raft_consensus.cc:2749] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:22.954402 19670 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:23.001295 19582 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:23.001425 19263 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:23.005470 16561 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
W20260709 13:06:22.963608 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:22.991914 15932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1956) took 2243 ms. Trace:
I20260709 13:06:22.978600 19555 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:22.990583 19201 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:23.027410 19514 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:23.029016 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:23.029808 16570 raft_consensus.cc:2393] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 8: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:23.029887 19514 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:22.969810 19385 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:22.963981 19666 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:06:23.034828 19666 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:23.035495 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:23.036369 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:23.037564 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:23.037943 19223 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:23.038930 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:23.039595 16008 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:23.040632 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:23.041373 19558 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:23.041999 19558 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:23.044116 19591 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:23.044775 19591 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:23.045441 19587 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:22.996944 19589 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:23.047984 15930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 1920) took 2151 ms. Trace:
I20260709 13:06:23.048308 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:23.049196 16007 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:23.049435 19641 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:23.049775 19223 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:23.038259 16004 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 7.
W20260709 13:06:23.045928 15827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:22.964679 15902 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:22.992904 15928 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 1962) took 2242 ms (client timeout 2218 ms). Trace:
I20260709 13:06:23.050346 16561 raft_consensus.cc:1275] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:06:22.995589 16562 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45232 (request call id 818) took 2281 ms (client timeout 1682 ms). Trace:
I20260709 13:06:23.009375 15932 rpcz_store.cc:276] 0709 13:06:20.748861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.588187 (+1839326us) service_pool.cc:224] Handling call
0709 13:06:22.762741 (+174554us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:22.777071 (+ 14330us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.991632 (+214561us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b54002cc750. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:22.991817 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":400354688}
I20260709 13:06:22.986308 15929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53636 (request call id 714) took 1959 ms. Trace:
W20260709 13:06:23.053438 15928 rpcz_store.cc:269] 0709 13:06:20.750625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:20.927223 (+176598us) service_pool.cc:224] Handling call
0709 13:06:22.992486 (+2065263us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b54002cdb54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:22.992700 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":412256512}
W20260709 13:06:22.983410 15854 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 1938) took 1217 ms (client timeout 1580 ms). Trace:
I20260709 13:06:22.994766 15931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 1961) took 2244 ms. Trace:
I20260709 13:06:22.968159 15974 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:23.055960 15929 rpcz_store.cc:276] 0709 13:06:21.026681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.584985 (+1558304us) service_pool.cc:224] Handling call
0709 13:06:22.986012 (+401027us) spinlock_profiling.cc:243] Waited 212 ms on lock 0x7b540022cc54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:22.986219 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":488242432}
W20260709 13:06:23.054409 16562 rpcz_store.cc:269] 0709 13:06:20.714512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:20.774149 (+ 59637us) service_pool.cc:224] Handling call
0709 13:06:22.995344 (+2221195us) spinlock_profiling.cc:243] Waited 1.75 s on lock 0x7b54002b0f54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:22.995512 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4016167808}
W20260709 13:06:23.057029 15854 rpcz_store.cc:269] 0709 13:06:21.765805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.113959 (+348154us) service_pool.cc:224] Handling call
0709 13:06:22.983111 (+869152us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b540032e454. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:22.983326 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":241175936}
W20260709 13:06:23.059168 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 94602.7us
I20260709 13:06:23.048360 15930 rpcz_store.cc:276] 0709 13:06:20.896523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.764078 (+1867555us) service_pool.cc:224] Handling call
0709 13:06:22.779748 (+ 15670us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:22.789220 (+  9472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:22.789286 (+    66us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:22.838583 (+ 49297us) log.cc:844] Serialized 26 byte log entry
0709 13:06:22.941484 (+102901us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:22.941522 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.941582 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:23.047774 (+106192us) raft_consensus.cc:1685] finished
0709 13:06:23.047794 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:23.047931 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2565,"raft.queue_time_us":10474,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"thread_start_us":89444,"threads_started":1,"wal-append.queue_time_us":184500}
I20260709 13:06:23.057322 15931 rpcz_store.cc:276] 0709 13:06:20.750344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.636799 (+1886455us) service_pool.cc:224] Handling call
0709 13:06:22.993159 (+356360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:22.993642 (+   483us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:22.993682 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:22.993738 (+    56us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:22.993791 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.993871 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:22.994572 (+   701us) spinlock_profiling.cc:243] Waited 356 ms on lock 0x7b540022cc50. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:22.994703 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5476,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":92,"spinlock_wait_cycles":818589440}
I20260709 13:06:23.061565 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:23.062990 16566 raft_consensus.cc:2393] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 7: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:23.066459 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:23.068213 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:23.068315 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:23.049970 19641 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:22.989948 19674 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:22.991724 19535 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:22.981460 19581 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:23.080015 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:23.081216 19587 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:23.081969 15902 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:23.086596 19429 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:06:23.086809 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 118800us
I20260709 13:06:23.087330 19429 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:23.070848 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:23.089972 15933 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:23.071827 16567 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
W20260709 13:06:23.061213 15827 connection.cc:256] RPC call timeout handler was delayed by -2.4005830349999542s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2029s for a response
I20260709 13:06:22.998456 19675 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:23.092414 16008 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:06:23.092213 19372 consensus_queue.cc:1048] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:23.090956 19648 raft_consensus.cc:2749] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:23.093487 19651 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:23.094039 19429 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:23.094049 19651 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:23.091784 15827 connection.cc:256] RPC call timeout handler was delayed by -2.3816725760000281s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20720000000000002s for a response
W20260709 13:06:23.096815 15827 connection.cc:256] RPC call timeout handler was delayed by -2.058604561999914s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15990000000000001s for a response
W20260709 13:06:23.097126 15827 connection.cc:256] RPC call timeout handler was delayed by -1.8921045619999859s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1784s for a response
W20260709 13:06:23.097409 15827 connection.cc:256] RPC call timeout handler was delayed by -1.860604561999935s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1819s for a response
W20260709 13:06:23.097674 15827 connection.cc:256] RPC call timeout handler was delayed by -1.8371725759999435s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2677s for a response
W20260709 13:06:23.097949 15827 connection.cc:256] RPC call timeout handler was delayed by -1.8003045619999511s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1886s for a response
W20260709 13:06:23.098225 15827 connection.cc:256] RPC call timeout handler was delayed by -1.7177850249998983s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1567s for a response
W20260709 13:06:23.098495 15827 connection.cc:256] RPC call timeout handler was delayed by -1.2488222859999496s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1575s for a response
I20260709 13:06:23.099951 19201 raft_consensus.cc:2764] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:06:23.103447 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:23.104079 16007 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:06:23.096073 19666 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:23.104967 19666 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:23.105322 19666 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:23.107923 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:23.108019 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:23.108443 19651 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:23.108683 15928 raft_consensus.cc:2393] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 8: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:23.108578 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:23.109025 19558 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:23.110577 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:23.111277 15930 raft_consensus.cc:1275] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:06:23.111950 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:23.112383 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:23.113040 15937 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:23.114928 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:23.100761 15934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 716) took 2073 ms. Trace:
I20260709 13:06:23.109072 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:23.116624 15931 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:23.112398 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:23.115541 16566 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:23.117769 15929 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:23.108629 15932 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:22.986572 19610 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:23.119014 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:23.119999 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:23.119016 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:23.120572 16007 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:23.120765 16567 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:23.121783 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:23.119573 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:23.129021 16005 raft_consensus.cc:1275] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:06:23.129294 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:23.130337 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:23.130470 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:23.131261 16002 raft_consensus.cc:2393] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 7: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:23.131438 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:23.132081 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:23.133023 16002 raft_consensus.cc:2393] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 8: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:23.132907 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:23.133574 16004 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 7.
I20260709 13:06:23.133970 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:23.134491 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:23.135180 16009 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 6.
I20260709 13:06:23.132098 16003 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
I20260709 13:06:23.136823 16011 raft_consensus.cc:2410] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:06:23.137553 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:23.137840 19588 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:23.138268 16009 raft_consensus.cc:2393] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 8: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
W20260709 13:06:23.138216 16011 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60156 (request call id 647) took 2156 ms (client timeout 1837 ms). Trace:
W20260709 13:06:23.138757 16011 rpcz_store.cc:269] 0709 13:06:20.981457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:20.981620 (+   163us) service_pool.cc:224] Handling call
0709 13:06:23.137952 (+2156332us) spinlock_profiling.cc:243] Waited 2.1 s on lock 0x7b5400306454. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:23.138127 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4839741696}
I20260709 13:06:23.139561 19429 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:23.140100 19429 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:23.134768 16005 raft_consensus.cc:2393] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 9: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:23.142379 15974 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:23.147809 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:23.143816 19558 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:23.161019 15976 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:23.133991 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:23.165196 16007 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 9.
W20260709 13:06:23.168462 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 51198.9us
I20260709 13:06:23.168509 15825 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:23.169472 15931 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:06:23.176317 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:23.178308 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:23.179420 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:23.179874 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:23.180186 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:23.181919 16567 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:06:23.115696 15934 rpcz_store.cc:276] 0709 13:06:21.027277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.772542 (+1745265us) service_pool.cc:224] Handling call
0709 13:06:22.772918 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:22.773189 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:06:22.779769 (+  6580us) log.cc:844] Serialized 14 byte log entry
0709 13:06:23.100457 (+320688us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:23.100506 (+    49us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:23.100547 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:23.100607 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:23.100710 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":101596,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":25155,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":176,"thread_start_us":300430,"threads_started":2}
I20260709 13:06:23.184049 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:23.184530 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:23.184753 16007 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
I20260709 13:06:23.185231 15931 raft_consensus.cc:2393] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 9: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:23.186805 15929 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 9.
I20260709 13:06:23.188303 16008 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 9.
I20260709 13:06:23.143579 19429 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:23.189445 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:23.191653 16011 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:06:23.191978 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:23.149382 19641 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:23.195662 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:23.192695 16564 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
I20260709 13:06:23.197651 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:23.198343 16566 raft_consensus.cc:2393] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 10: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:23.198513 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:23.203164 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:23.205255 16570 raft_consensus.cc:2393] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 8: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:23.207067 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:23.207876 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:23.220674 15932 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:06:23.207214 15825 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:23.131446 19666 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:23.223948 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:23.168776 19591 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:23.195077 19588 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:23.224113 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:23.249086 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:23.249773 16563 raft_consensus.cc:2393] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 10: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:23.249876 15858 raft_consensus.cc:2393] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 7: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:23.228746 19651 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:23.250684 19651 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:23.229183 16008 raft_consensus.cc:2410] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:06:23.251049 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:23.191483 19558 raft_consensus.cc:697] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:06:23.252081 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:23.252951 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:23.252513 19558 consensus_queue.cc:237] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:06:23.253332 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:23.223011 19585 raft_consensus.cc:2804] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:23.256034 19550 consensus_queue.cc:1048] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:06:23.258198 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 51067.7us
I20260709 13:06:23.258613 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:23.259370 15825 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:23.260942 15825 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:23.251081 19651 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:23.262712 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:23.263237 15976 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:23.263613 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:23.264590 16566 raft_consensus.cc:2393] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 9: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:23.265077 19263 raft_consensus.cc:2764] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:06:23.265242 15976 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:23.265846 19263 consensus_queue.cc:1059] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:23.232002 15759 catalog_manager.cc:5697] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 1 to 6, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 6 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:23.266681 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:23.238917 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:23.267886 15854 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:23.239768 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:23.268008 15759 catalog_manager.cc:5697] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 3 to 8. New cstate: current_term: 8 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:23.269109 15857 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:23.270053 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:23.270627 15862 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:23.269970 15759 catalog_manager.cc:5697] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 5 to 10, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 10 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } }
I20260709 13:06:23.225015 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:23.239907 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:23.240521 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:23.242403 19591 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:23.248167 19610 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:23.266021 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:23.282524 15855 raft_consensus.cc:2393] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 7: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:23.230716 19666 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:23.263257 16564 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:23.275166 16011 raft_consensus.cc:2393] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 9: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:23.275872 15933 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:23.277185 19357 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:23.277310 19667 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:23.278026 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:23.277817 15759 catalog_manager.cc:5697] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 5 to 9, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 9 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:23.281572 15976 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:23.278923 19385 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:23.280462 19585 raft_consensus.cc:697] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 7 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:23.278038 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:23.281126 15859 raft_consensus.cc:2393] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 9: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:23.293891 15929 raft_consensus.cc:1275] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:06:23.282793 16005 raft_consensus.cc:1275] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 13:06:23.280272 15931 raft_consensus.cc:1275] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:06:23.290724 15862 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:23.291954 15933 raft_consensus.cc:1240] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:23.292461 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:23.296497 15932 raft_consensus.cc:2368] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 8. Current term is 9.
I20260709 13:06:23.294569 16005 pending_rounds.cc:85] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting all ops after (but not including) 5
I20260709 13:06:23.345968 16005 pending_rounds.cc:107] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:06:23.346352 16005 raft_consensus.cc:2889] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:06:23.345084 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 46163.6us
I20260709 13:06:23.348361 16002 raft_consensus.cc:1275] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:06:23.287041 19675 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:23.362787 15900 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:23.365542 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:23.295274 19357 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:23.371760 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 110871us
W20260709 13:06:23.373989 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:23.376148 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:23.377393 16563 raft_consensus.cc:1240] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:23.297093 19670 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:06:23.380695 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:23.364744 19550 raft_consensus.cc:2804] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:23.348138 19429 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:23.299384 19350 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:23.381683 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:23.393064 15934 raft_consensus.cc:1240] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:23.393254 15859 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:23.392052 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:23.393999 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:23.388478 15928 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:23.391949 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:23.395087 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:23.397876 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:23.398540 15932 raft_consensus.cc:2393] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 9: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:23.399626 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:23.379243 19357 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:23.405344 15931 raft_consensus.cc:2393] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 9: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
W20260709 13:06:23.409502 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 46799.2us
W20260709 13:06:23.416668 15974 connection.cc:256] RPC call timeout handler was delayed by -1.2718975569999884s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3171s for a response
I20260709 13:06:23.418612 15933 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:23.388226 19550 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:23.390122 19429 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:23.421211 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:23.424585 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:23.429473 15932 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:23.425907 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:23.432912 15976 leader_election.cc:400] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 9. Current term is 10.
I20260709 13:06:23.434043 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:23.440078 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:23.382733 19675 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:23.426896 15759 catalog_manager.cc:5697] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:23.427731 19550 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:23.435504 19429 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:23.390497 19670 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:23.404891 19350 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:23.467087 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:23.467417 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:23.474320 15935 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:23.470120 15827 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:23.479127 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 77403.6us
I20260709 13:06:23.430044 19651 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:23.490900 15827 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:23.492277 15827 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:23.496439 15937 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:23.497402 15974 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:23.504166 15900 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:23.504359 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:23.468065 19095 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:23.472980 19222 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:23.509735 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:23.510609 15935 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:23.512581 15859 raft_consensus.cc:1240] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:06:23.461575 19667 consensus_queue.cc:1048] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:06:23.523324 16563 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:23.529598 15932 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:23.505101 19223 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:23.524813 15902 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:23.547106 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 63555.3us
I20260709 13:06:23.556526 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:23.511317 19660 raft_consensus.cc:2749] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:06:23.561841 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 83462us
I20260709 13:06:23.470507 19585 consensus_queue.cc:237] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:23.527251 19431 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:23.515836 19635 raft_consensus.cc:2749] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:23.541647 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:23.523943 19588 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:06:23.593565 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 73441.4us
I20260709 13:06:23.565867 15935 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
W20260709 13:06:23.574368 19696 log.cc:927] Time spent T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 13:06:23.505693 19575 consensus_queue.cc:1059] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.189s
I20260709 13:06:23.528714 19641 raft_consensus.cc:2804] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:23.515146 19418 consensus_queue.cc:1059] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:06:23.522948 19222 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:23.584977 19685 consensus_queue.cc:1048] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.141s
W20260709 13:06:23.586479 19697 log.cc:927] Time spent T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
I20260709 13:06:23.586625 16535 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:23.588927 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:23.591027 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:23.509910 19095 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:23.613332 15902 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:23.621970 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:23.638015 19223 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:23.647720 16565 raft_consensus.cc:2393] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 8: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:23.593158 19588 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:23.649188 16566 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:23.615590 19387 raft_consensus.cc:2804] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:06:23.660567 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 74152.3us
I20260709 13:06:23.657207 19588 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:23.661895 19575 raft_consensus.cc:2804] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:23.658769 15902 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:23.689103 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 46284.8us
I20260709 13:06:23.690469 15900 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:23.620872 19641 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:23.696059 16009 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:06:23.653087 19223 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:23.646613 19372 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:23.697504 19591 raft_consensus.cc:2804] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:23.715870 19568 raft_consensus.cc:3055] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:23.718433 19201 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:23.724081 19670 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:23.724684 19669 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:06:23.616624 15759 catalog_manager.cc:5697] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 1 to 6. New cstate: current_term: 6 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
I20260709 13:06:23.738101 15937 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
I20260709 13:06:23.740321 15974 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:23.668947 19387 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:23.696771 19641 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:23.735981 19697 log.cc:927] Time spent T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
W20260709 13:06:23.772818 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 114393us
W20260709 13:06:23.777971 19705 log.cc:927] Time spent T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.159s	user 0.000s	sys 0.000s
I20260709 13:06:23.772392 19372 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:23.778358 19609 raft_consensus.cc:3055] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:23.778939 19609 raft_consensus.cc:740] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:06:23.783149 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:23.784044 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 43843.1us
I20260709 13:06:23.784693 19222 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:23.784790 15974 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:23.785518 19555 raft_consensus.cc:2749] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:23.786290 19555 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:23.785983 19095 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:23.786855 19555 raft_consensus.cc:697] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:23.788290 19609 consensus_queue.cc:260] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:23.789508 19675 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:23.789875 19609 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:23.790500 19675 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:23.791014 19675 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:23.774338 19635 raft_consensus.cc:3055] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:23.792325 19635 raft_consensus.cc:740] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:23.792718 19555 consensus_queue.cc:237] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:23.793144 19635 consensus_queue.cc:260] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:23.794476 19635 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:23.763180 19367 consensus_queue.cc:1048] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:06:23.738710 19591 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:23.741236 19568 raft_consensus.cc:740] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:23.742328 19201 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:23.809880 15974 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:23.807188 16563 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:23.760553 19387 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:23.760579 19669 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:06:23.819887 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 323718us
I20260709 13:06:23.774739 19651 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:23.823380 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:23.830499 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:23.830704 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:23.818163 19201 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:23.830499 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:23.833659 16009 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:06:23.839404 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:23.840399 16008 raft_consensus.cc:2393] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 9: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:23.841248 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:23.842484 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:23.843379 16010 raft_consensus.cc:2393] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 9: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:23.832798 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:23.844749 16011 raft_consensus.cc:2393] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 9: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:23.845446 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:23.812021 19591 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:23.852741 15759 catalog_manager.cc:5697] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:06:23.862850 19372 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:23.804455 19263 consensus_queue.cc:1048] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:06:23.880496 19651 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:23.891330 19675 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:23.828148 19669 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:23.896723 19568 consensus_queue.cc:260] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:23.897840 19568 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:23.802326 19550 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:23.901836 19550 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:23.915839 16564 consensus_meta.cc:287] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.618s	user 0.006s	sys 0.006s
I20260709 13:06:23.917486 16563 raft_consensus.cc:1240] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:23.918365 15976 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:23.920117 16564 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
W20260709 13:06:23.923066 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 86459.6us
I20260709 13:06:23.942538 16566 raft_consensus.cc:1240] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:06:23.851202 19350 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:23.979066 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:23.943279 19263 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:23.980588 19263 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:23.980947 19263 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:23.951874 19660 consensus_queue.cc:1059] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:23.952714 16533 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:23.942402 16570 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:23.953454 15759 catalog_manager.cc:5697] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 1 to 6, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 6 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:23.958686 19641 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:23.971792 19201 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:23.977082 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:23.977082 19685 consensus_queue.cc:1048] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:06:23.998422 16564 raft_consensus.cc:1240] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:24.000439 16570 raft_consensus.cc:1240] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 9. Current term is 10. Ops: []
W20260709 13:06:24.024531 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 71905.7us
I20260709 13:06:24.027348 16533 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:24.001880 16566 raft_consensus.cc:1240] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:24.004136 19660 raft_consensus.cc:3055] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:24.004668 19675 raft_consensus.cc:2749] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:23.911582 19588 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:24.037417 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 461611us
I20260709 13:06:24.032174 15759 catalog_manager.cc:5697] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 2 to 7. New cstate: current_term: 7 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:06:24.035760 19558 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.043s	sys 0.072s
I20260709 13:06:24.071069 15759 catalog_manager.cc:5697] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:24.022496 15976 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:23.980834 19591 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:24.079710 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:24.082255 15854 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 13:06:24.079900 16561 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.786s	user 0.037s	sys 0.093s
I20260709 13:06:24.083072 15856 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
W20260709 13:06:24.083304 16561 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.790s	user 0.037s	sys 0.093s
I20260709 13:06:24.019896 19223 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:24.033383 19660 raft_consensus.cc:740] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:24.082564 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:06:24.078917 15828 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.002s	sys 0.027s
W20260709 13:06:24.068111 19558 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.043s	sys 0.073s
I20260709 13:06:24.082463 15827 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:24.089679 15759 catalog_manager.cc:5697] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 1 to 5, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 5 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
W20260709 13:06:24.093912 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 71545.7us
I20260709 13:06:24.093752 15759 catalog_manager.cc:5697] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 3 to 5, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 5 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:06:24.091733 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:06:24.095855 15828 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.002s	sys 0.027s
I20260709 13:06:24.098054 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:24.098515 16564 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 LEARNER]: Advancing to term 10
I20260709 13:06:24.246775 15976 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:24.139971 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 481236us
W20260709 13:06:24.256692 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:24.097941 19648 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:24.271986 15759 catalog_manager.cc:5697] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:06:24.281878 19648 raft_consensus.cc:697] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 8 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:06:24.295737 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 46090.9us
I20260709 13:06:24.288396 15759 catalog_manager.cc:5697] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 6 to 9, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 9 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:24.302534 19385 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:24.303092 19385 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:24.303542 19385 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:24.308152 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:24.308622 15827 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:24.309947 19685 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:24.310225 19223 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:24.310302 15827 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:24.308203 19660 consensus_queue.cc:260] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:24.313344 19588 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:24.313634 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:24.314481 15861 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:24.315622 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:24.315995 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:24.316639 15858 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 7.
I20260709 13:06:24.316949 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:24.317452 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:24.318507 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:06:24.319012 19609 consensus_meta.cc:287] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.529s	user 0.009s	sys 0.000s
W20260709 13:06:24.275326 19575 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.009s	sys 0.045s
I20260709 13:06:24.319340 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:06:24.319851 19575 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.010s	sys 0.045s
I20260709 13:06:24.320230 19575 raft_consensus.cc:697] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:24.320312 19201 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:24.320556 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:24.321384 19372 raft_consensus.cc:2804] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:24.321822 19372 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:24.322167 19372 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:24.322429 19669 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:24.283099 19357 consensus_meta.cc:287] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.853s	user 0.007s	sys 0.004s
I20260709 13:06:24.323933 19357 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:24.324733 19669 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:24.108392 19581 consensus_queue.cc:1059] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.189s
I20260709 13:06:24.325676 19581 raft_consensus.cc:3055] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:24.326135 19581 raft_consensus.cc:740] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:24.326467 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:24.327055 15856 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:24.326592 19575 consensus_queue.cc:237] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:24.297439 19550 consensus_queue.cc:1059] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:06:24.329447 19223 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:24.330277 19591 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:24.329494 19648 consensus_queue.cc:237] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:24.331836 19591 consensus_queue.cc:1048] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:24.335222 19641 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:24.310372 19685 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:24.337040 19685 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:24.337235 19550 raft_consensus.cc:3055] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:24.337805 19550 raft_consensus.cc:740] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:24.326924 19581 consensus_queue.cc:260] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:24.321585 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:24.350123 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:24.352990 15855 raft_consensus.cc:1275] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:06:24.300776 19675 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:24.371551 15855 raft_consensus.cc:1240] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:24.369167 15856 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 7.
I20260709 13:06:24.356832 15759 catalog_manager.cc:5697] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:06:24.369637 16535 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.696s	user 0.034s	sys 0.062s
W20260709 13:06:24.378113 16535 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.034s	sys 0.062s
W20260709 13:06:24.378438 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 704967us
I20260709 13:06:24.383060 15902 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:24.379282 16535 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:24.387959 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:24.388777 15857 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:24.371721 19581 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:24.381932 19429 consensus_meta.cc:287] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.901s	user 0.011s	sys 0.003s
I20260709 13:06:24.390960 15759 catalog_manager.cc:5697] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 5 to 7, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 7 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:24.377028 19357 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:24.393412 19387 consensus_meta.cc:287] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.533s	user 0.005s	sys 0.006s
I20260709 13:06:24.398419 19367 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:24.416033 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 431493us
I20260709 13:06:24.384940 19651 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:06:24.400457 16562 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.690s	user 0.008s	sys 0.012s
W20260709 13:06:24.400996 15974 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.000s	sys 0.015s
W20260709 13:06:24.400569 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:24.417290 16567 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.000s	sys 0.017s
I20260709 13:06:24.419114 15759 catalog_manager.cc:5697] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 6 to 7, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 7 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:06:24.422142 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:06:24.422693 16562 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.724s	user 0.009s	sys 0.012s
W20260709 13:06:24.422252 15974 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.000s	sys 0.015s
W20260709 13:06:24.429831 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 619998us
I20260709 13:06:24.414631 19610 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:24.409483 19660 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:24.430066 16562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 982) took 1255 ms. Trace:
I20260709 13:06:24.431102 15902 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:24.433887 16562 rpcz_store.cc:276] 0709 13:06:23.174280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.180401 (+  6121us) service_pool.cc:224] Handling call
0709 13:06:23.180652 (+   251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:23.180871 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:23.395191 (+214320us) log.cc:844] Serialized 14 byte log entry
0709 13:06:24.429588 (+1034397us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:24.429656 (+    68us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:24.429713 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:24.429842 (+   129us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:24.430001 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":307752,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":102847,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":86,"thread_start_us":1184667,"threads_started":3,"wal-append.queue_time_us":465714}
I20260709 13:06:24.374692 19550 consensus_queue.cc:260] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:24.422607 19585 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.002s	sys 0.016s
I20260709 13:06:24.435747 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:24.436203 19585 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.002s	sys 0.017s
I20260709 13:06:24.436151 19550 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:24.437177 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:24.437863 15932 raft_consensus.cc:2393] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 9: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:24.438585 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:24.401124 19719 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:24.439730 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:24.440965 15974 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:24.441754 19568 raft_consensus.cc:2804] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:24.442157 19568 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:24.442536 19568 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:24.450412 19568 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:24.420653 19367 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:24.452694 19385 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:24.453869 15858 raft_consensus.cc:2393] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 9: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:24.417136 19387 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:24.454756 19385 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:24.405892 16534 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.005s	sys 0.011s
W20260709 13:06:24.457036 16534 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.005s	sys 0.011s
I20260709 13:06:24.458048 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:24.458653 15928 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:06:24.459977 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:24.460559 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:24.461269 15932 raft_consensus.cc:2393] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 7: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:24.423623 15937 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:24.423018 16567 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.001s	sys 0.017s
I20260709 13:06:24.428452 15827 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:24.460566 15929 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:24.421177 16535 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:24.456501 19387 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:24.463609 16567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 983) took 1281 ms. Trace:
W20260709 13:06:24.471118 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 416307us
I20260709 13:06:24.399085 19429 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:24.472536 15759 catalog_manager.cc:5697] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 3 to 4, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 4 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
W20260709 13:06:24.464525 15936 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.732s	user 0.050s	sys 0.116s
W20260709 13:06:24.493311 15936 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.051s	sys 0.116s
I20260709 13:06:24.495064 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:06:24.469619 19555 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.032s	sys 0.056s
I20260709 13:06:24.491297 15930 raft_consensus.cc:1240] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 9. Current term is 10. Ops: [9.11-9.11]
I20260709 13:06:24.492872 15935 raft_consensus.cc:1240] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:24.501052 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:24.509418 15928 raft_consensus.cc:2410] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:06:24.509683 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 89142.5us
W20260709 13:06:24.427520 19418 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.035s	sys 0.067s
I20260709 13:06:24.495873 16533 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:24.469619 19675 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:24.510362 16535 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:24.519629 15853 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.990s	user 0.020s	sys 0.020s
W20260709 13:06:24.538048 15853 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.009s	user 0.020s	sys 0.020s
I20260709 13:06:24.502236 15929 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:06:24.492584 15974 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:24.536829 15827 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:24.453891 19367 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:24.539661 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 47177.5us
I20260709 13:06:24.540310 15759 catalog_manager.cc:5697] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 4 to 6, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 6 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:24.518496 16533 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:24.525043 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:24.527412 15937 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:06:24.499945 19385 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:24.540874 16535 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:24.491973 19222 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:24.544145 15853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2061) took 1283 ms. Trace:
W20260709 13:06:24.521060 19418 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.739s	user 0.037s	sys 0.070s
I20260709 13:06:24.468178 16567 rpcz_store.cc:276] 0709 13:06:23.182000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.182552 (+   552us) service_pool.cc:224] Handling call
0709 13:06:23.196323 (+ 13771us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:23.212024 (+ 15701us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:23.403885 (+191861us) log.cc:844] Serialized 14 byte log entry
0709 13:06:23.708587 (+304702us) log.cc:844] Serialized 14 byte log entry
0709 13:06:23.832263 (+123676us) spinlock_profiling.cc:243] Waited 2.66 ms on lock 0x7b180035d280. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72b8455 00007f73ea3a0060 00007f73ea3b2716 00007f73ea3c3287 00007f73ea3c3208
0709 13:06:24.463312 (+631049us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:24.463360 (+    48us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:24.463396 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:24.463451 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:24.463559 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":498,"prepare.queue_time_us":276946,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":72,"raft.queue_time_us":164045,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"spinlock_wait_cycles":6110464,"thread_start_us":1052625,"threads_started":3,"wal-append.queue_time_us":471855}
I20260709 13:06:24.543735 15974 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:24.550376 15759 catalog_manager.cc:5697] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 4 to 7, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 7 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:24.472074 19610 consensus_queue.cc:1048] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 13:06:24.539762 19675 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:24.553370 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:24.550088 15853 rpcz_store.cc:276] 0709 13:06:23.261037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.281881 (+ 20844us) service_pool.cc:224] Handling call
0709 13:06:23.299146 (+ 17265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:23.360438 (+ 61292us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:06:23.360748 (+   310us) log.cc:844] Serialized 14 byte log entry
0709 13:06:24.538396 (+1177648us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:24.538444 (+    48us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:24.543921 (+  5477us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:24.543987 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:24.544090 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15271,"prepare.queue_time_us":4689,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":8176,"raft.queue_time_us":114317,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":24854,"thread_start_us":1153726,"threads_started":2,"wal-append.queue_time_us":456345}
I20260709 13:06:24.543179 19095 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:24.560573 15902 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:24.558614 15827 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:24.561183 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:24.559890 15759 catalog_manager.cc:5697] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 4 to 6, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 6 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:06:24.545099 19720 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:24.548592 16533 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:24.550407 19635 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:24.571142 15902 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:24.549656 19385 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:24.574771 15937 raft_consensus.cc:1240] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:06:24.569861 19720 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:06:24.560892 19591 raft_consensus.cc:2804] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:24.570077 19685 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:24.597688 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:24.598191 15861 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 8: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:24.598724 19591 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:24.599112 19591 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:24.577992 19372 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:24.560379 19095 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:24.579818 19669 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:24.561092 19675 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:24.579900 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:24.580642 19367 raft_consensus.cc:2804] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:24.584210 19729 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:24.632419 15930 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:06:24.584363 15928 raft_consensus.cc:1240] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:06:24.597352 19610 raft_consensus.cc:2804] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:24.562359 19555 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:24.592628 19635 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:24.626690 15974 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:24.629665 15902 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:24.628489 19550 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:24.594662 15827 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:24.596012 19385 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:24.568591 19723 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:24.563050 19641 consensus_queue.cc:1059] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
W20260709 13:06:24.645617 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 50627us
I20260709 13:06:24.648092 15825 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:24.648880 15902 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:24.631580 19095 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:24.651057 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 80052.9us
I20260709 13:06:24.651499 15902 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:24.631919 19669 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:24.631604 19685 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:24.646926 19263 consensus_meta.cc:287] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.654s	user 0.009s	sys 0.003s
I20260709 13:06:24.637096 19367 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:06:24.651180 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 404431us
I20260709 13:06:24.658552 15902 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:24.659621 16533 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:24.654796 19722 raft_consensus.cc:2749] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:24.653532 19666 raft_consensus.cc:2749] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:24.646615 19635 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:24.676003 15902 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:24.666642 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:24.649761 19651 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:24.663786 15827 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:24.669512 15928 raft_consensus.cc:1240] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:24.671037 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:24.680631 15937 raft_consensus.cc:2393] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 8: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:24.678380 15857 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:24.653753 19201 raft_consensus.cc:2749] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:24.686017 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:24.686671 16562 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:24.651968 19568 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:24.685307 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:24.692389 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:24.685154 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:24.692925 16567 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:24.693069 15858 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 9
W20260709 13:06:24.677706 15900 leader_election.cc:336] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.764s (SENT)
I20260709 13:06:24.690132 15933 raft_consensus.cc:1240] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:06:24.690014 15825 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:24.655371 19609 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:24.694697 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:24.661980 19669 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:24.690973 15827 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:24.705080 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:24.707229 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:24.650032 19558 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:06:24.713652 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:24.667578 19367 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:24.713901 16561 raft_consensus.cc:1240] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:24.715655 15928 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:24.652415 19350 consensus_queue.cc:1048] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:06:24.714507 16533 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:24.663939 19585 raft_consensus.cc:2804] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:24.679400 19550 raft_consensus.cc:2749] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:24.679968 19651 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:24.687608 19719 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:06:24.740883 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 56433.8us
I20260709 13:06:24.741621 16533 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:24.717666 19726 log.cc:927] Time spent T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:06:24.752377 15934 raft_consensus.cc:1240] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 7. Current term is 8. Ops: []
W20260709 13:06:24.742431 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 52490.5us
W20260709 13:06:24.751428 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:24.755538 15825 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:06:24.757248 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 79563.9us
I20260709 13:06:24.699997 19609 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:06:24.769476 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 105742us
I20260709 13:06:24.770407 16567 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
I20260709 13:06:24.707346 19418 raft_consensus.cc:2764] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:06:24.674540 19263 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:24.782313 16533 leader_election.cc:304] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:24.785687 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:24.725268 19727 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:06:24.787958 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 109818us
I20260709 13:06:24.724943 19429 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:24.792761 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 101876us
I20260709 13:06:24.793224 15825 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:24.753238 19651 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:24.729091 19585 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:24.792038 15974 leader_election.cc:400] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Vote denied by peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) with higher term. Message: Invalid argument: T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 8. Current term is 9.
I20260709 13:06:24.706888 19667 consensus_queue.cc:1059] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 8.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 13:06:24.706498 19648 consensus_queue.cc:1059] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.081s
I20260709 13:06:24.716109 19685 consensus_queue.cc:1059] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:06:24.780862 19719 raft_consensus.cc:697] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 9 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:24.803725 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:24.814353 16566 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:24.779892 19609 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:24.779536 19732 raft_consensus.cc:2804] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:24.795198 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:24.771174 19610 raft_consensus.cc:697] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:24.812588 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:24.791957 19729 raft_consensus.cc:697] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:24.813534 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:24.802428 19725 log.cc:927] Time spent T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:06:24.731113 19717 consensus_queue.cc:1048] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.088s
I20260709 13:06:24.786761 19568 consensus_queue.cc:1059] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:06:24.808022 19585 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:24.826190 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:24.806661 19722 raft_consensus.cc:2804] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:24.827764 19372 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:24.828018 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:24.828457 19715 raft_consensus.cc:2804] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:24.830215 19718 raft_consensus.cc:2804] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:24.872289 15902 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:24.830715 19724 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:24.787307 19357 consensus_queue.cc:1059] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:06:24.834024 15858 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:24.835412 16009 raft_consensus.cc:1240] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:24.836087 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:24.836788 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:24.882066 19723 raft_consensus.cc:697] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:24.882568 16002 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:24.882658 16010 raft_consensus.cc:2393] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 9: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:24.856609 15857 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 10.
I20260709 13:06:24.856860 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:24.870677 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 52646.6us
I20260709 13:06:24.857365 15856 raft_consensus.cc:2393] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 9: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:24.878095 16003 raft_consensus.cc:1240] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:06:24.826180 19648 raft_consensus.cc:3055] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:24.884263 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:24.879912 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:24.900906 15859 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
I20260709 13:06:24.887516 15974 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:24.899272 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:24.906384 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:24.908461 16008 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
W20260709 13:06:24.879222 19721 log.cc:927] Time spent T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 13:06:24.905793 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:24.878409 18968 raft_consensus.cc:3055] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:24.797087 19727 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:24.917629 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:24.872789 19718 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:24.879424 19724 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:24.839493 19635 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:24.917140 19610 consensus_queue.cc:237] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:24.907686 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:24.914649 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:24.935250 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:24.973227 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:24.961666 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:24.920931 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 138693us
I20260709 13:06:24.977124 16533 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:24.908286 19725 log.cc:927] Time spent T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:06:24.883010 19723 consensus_queue.cc:237] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:24.967604 15854 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
W20260709 13:06:24.922945 19737 log.cc:927] Time spent T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:06:24.979768 16006 raft_consensus.cc:2393] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 8: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:24.909005 19201 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:24.969272 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:24.983289 15858 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 8: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:24.979372 16010 raft_consensus.cc:2393] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 11: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:24.922354 19558 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:24.883951 19666 consensus_queue.cc:1059] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.088s
I20260709 13:06:24.981685 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:24.993640 16009 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:24.900499 19648 raft_consensus.cc:740] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:24.926008 19263 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:24.999109 15853 raft_consensus.cc:1240] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:24.926847 18968 raft_consensus.cc:740] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:24.918346 19748 raft_consensus.cc:2749] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:24.979408 19720 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
I20260709 13:06:24.968210 19718 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:25.001931 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:25.003782 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:24.972872 19724 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:25.013964 15902 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:25.017999 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:25.018673 15856 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:24.995059 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 107664us
I20260709 13:06:25.008738 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:25.019794 15974 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:25.016743 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:24.994382 19732 raft_consensus.cc:697] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:24.992764 19201 raft_consensus.cc:697] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 10 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:25.021026 16010 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
I20260709 13:06:25.028040 15857 raft_consensus.cc:2410] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:06:24.875948 19729 consensus_queue.cc:237] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:24.987406 19742 consensus_queue.cc:1048] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:06:25.013016 19754 raft_consensus.cc:2749] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:25.059042 15858 raft_consensus.cc:1240] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:25.063239 16562 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:06:25.065632 19751 consensus_queue.cc:1048] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 13:06:24.970758 19719 consensus_queue.cc:237] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:25.066828 19747 log.cc:927] Time spent T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
W20260709 13:06:25.068246 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 48572.6us
I20260709 13:06:25.069746 15928 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:25.070724 16566 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:25.077562 15974 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:25.078078 15937 raft_consensus.cc:1240] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:25.071238 16002 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:25.011272 19675 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:25.035528 19666 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:25.079385 19722 raft_consensus.cc:697] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:25.021947 19720 consensus_queue.cc:1059] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:06:25.081661 19727 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:06:25.076630 19739 log.cc:927] Time spent T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.267s	user 0.000s	sys 0.001s
I20260709 13:06:25.120707 19758 raft_consensus.cc:2749] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:25.035154 19591 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:25.124356 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:25.155927 16565 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:25.128620 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 51132.9us
I20260709 13:06:25.088007 19566 raft_consensus.cc:2749] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:25.129737 16562 pending_rounds.cc:85] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting all ops after (but not including) 6
I20260709 13:06:25.157305 16562 pending_rounds.cc:107] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:06:25.048682 19728 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:25.131705 19666 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:25.136175 16567 raft_consensus.cc:1275] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:06:25.136974 15858 raft_consensus.cc:1240] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:25.122812 19648 consensus_queue.cc:260] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 6.9, Last appended by leader: 9, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:25.121223 18968 consensus_queue.cc:260] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:25.144470 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:25.156003 19660 consensus_meta.cc:287] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.692s	user 0.010s	sys 0.001s
I20260709 13:06:25.074008 19748 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:25.157013 16009 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:06:25.157577 16562 raft_consensus.cc:2889] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:25.160902 15974 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:06:25.162045 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 162078us
I20260709 13:06:25.163618 16563 raft_consensus.cc:2393] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 11: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:25.083947 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:25.177327 16566 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:25.178968 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:25.179318 15862 raft_consensus.cc:2410] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:06:25.154686 19550 raft_consensus.cc:3055] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:06:25.165540 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 188503us
I20260709 13:06:25.182607 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:25.161777 19666 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:25.204341 16533 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:25.207361 15854 raft_consensus.cc:1240] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:25.208041 15858 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:06:25.201717 16535 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:25.206246 15856 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:06:25.208405 16566 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:25.183159 19550 raft_consensus.cc:740] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:25.213507 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:25.214178 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:25.217967 19591 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:25.221339 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:25.222020 15854 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 8
W20260709 13:06:25.224926 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 64105.7us
I20260709 13:06:25.226234 15974 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:25.059122 19732 consensus_queue.cc:237] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:25.228139 16565 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:25.231169 16535 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:25.231321 16533 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:25.233740 15974 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:25.224466 19651 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:25.174880 19372 consensus_queue.cc:1059] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:06:25.240449 15974 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:25.241277 19764 raft_consensus.cc:2749] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:25.241197 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:25.242093 16002 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:25.242151 19767 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:25.239230 16533 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:25.242811 19585 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:25.243664 16533 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:25.244588 19675 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:25.244865 19585 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:25.244753 16533 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:25.245601 19749 raft_consensus.cc:2749] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:25.245924 19767 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:25.246363 19767 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:25.246315 19367 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:25.246850 19767 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:25.238293 19727 raft_consensus.cc:2804] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:25.239243 19635 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:25.248266 19742 raft_consensus.cc:2749] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:25.248880 15974 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:25.249377 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:25.249269 19724 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:25.249938 15974 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:25.250206 16005 raft_consensus.cc:2393] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 11: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:25.250598 15856 raft_consensus.cc:1240] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:25.240182 18968 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:25.251962 15974 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:25.252522 19357 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:25.252764 19429 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:25.253046 19357 raft_consensus.cc:697] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:25.253147 19429 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:25.253595 19429 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:25.253993 19357 consensus_queue.cc:237] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:25.256558 19750 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:25.257114 19750 raft_consensus.cc:697] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:25.257951 19750 consensus_queue.cc:237] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:25.260396 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:25.260951 15853 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:06:25.175855 19728 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:25.207065 19729 consensus_queue.cc:1059] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:06:25.224084 19566 raft_consensus.cc:2749] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:25.266891 16003 raft_consensus.cc:2393] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 8: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:25.266405 15974 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:25.267455 15854 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:06:25.269230 19367 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:25.266402 19739 log.cc:927] Time spent T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:06:25.274906 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:25.280350 15854 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:06:25.281643 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 490425us
I20260709 13:06:25.275396 15976 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:25.284904 15976 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:25.239130 19748 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:25.269974 19743 raft_consensus.cc:3055] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:25.276878 16002 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 9.
I20260709 13:06:25.278066 19769 raft_consensus.cc:2764] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:06:25.278823 19716 raft_consensus.cc:2804] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:25.178496 19201 consensus_queue.cc:237] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:25.263019 19591 raft_consensus.cc:2749] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:25.238874 19669 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:25.279204 19764 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:25.286989 15974 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:25.278565 19550 consensus_queue.cc:260] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:25.266359 19758 consensus_queue.cc:1059] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:06:25.251567 19724 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:25.286762 19263 raft_consensus.cc:2749] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:25.280293 19431 consensus_queue.cc:1059] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:06:25.286199 19748 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:25.305850 19733 consensus_queue.cc:1048] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:06:25.306945 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:25.313481 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:25.264984 19609 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:25.290321 19749 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:25.304250 19566 raft_consensus.cc:2804] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:25.286492 19743 raft_consensus.cc:740] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:25.259793 19722 consensus_queue.cc:237] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:25.311483 15854 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
W20260709 13:06:25.350868 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 63936.2us
I20260709 13:06:25.352825 15934 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:25.363015 15860 raft_consensus.cc:2393] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 10: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:25.352449 15861 raft_consensus.cc:2376] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8. Re-sending same reply.
W20260709 13:06:25.364943 19385 consensus_meta.cc:287] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.712s	user 0.007s	sys 0.004s
W20260709 13:06:25.348016 19736 log.cc:927] Time spent T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.526s	user 0.001s	sys 0.001s
I20260709 13:06:25.376915 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:06:25.374259 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 129572us
W20260709 13:06:25.379761 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 92418.1us
I20260709 13:06:25.352934 19357 raft_consensus.cc:2749] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:25.380046 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:25.379407 19431 consensus_queue.cc:1243] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer 8430db80a39d4868bcf7641c16de35ea log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:06:25.371555 19660 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:25.308845 19555 consensus_meta.cc:287] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.659s	user 0.009s	sys 0.001s
I20260709 13:06:25.384428 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:25.315092 19767 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:25.352121 19748 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:25.387851 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:25.411936 16002 raft_consensus.cc:2393] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 9: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:25.388906 16533 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:25.370704 19550 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:25.413457 19764 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:25.413887 19764 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:25.414261 19764 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:25.362366 19727 raft_consensus.cc:697] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:25.391965 19713 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:25.416149 19713 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:25.416582 19713 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:25.416846 19727 consensus_queue.cc:237] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:25.360620 19758 consensus_queue.cc:1243] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer 8430db80a39d4868bcf7641c16de35ea log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:06:25.420130 19591 consensus_queue.cc:1059] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:25.386121 19385 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:25.421056 19769 raft_consensus.cc:3055] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:25.372937 19749 raft_consensus.cc:697] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:25.421433 19769 raft_consensus.cc:740] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:25.422116 19764 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:25.422786 19385 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:25.422322 19769 consensus_queue.cc:260] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:25.422258 19749 consensus_queue.cc:237] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:25.392330 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:25.423892 15855 raft_consensus.cc:1240] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:25.392758 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:25.425006 16566 raft_consensus.cc:1240] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 7. Current term is 8. Ops: [5.7-7.8]
I20260709 13:06:25.393258 16004 raft_consensus.cc:1240] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:25.372856 19716 raft_consensus.cc:697] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 7 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:25.379544 19566 raft_consensus.cc:697] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:06:25.448293 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:25.448225 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:25.424508 19769 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:25.393544 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:25.429486 15900 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:25.433845 16562 raft_consensus.cc:1240] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:25.434700 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:25.424841 19555 raft_consensus.cc:2749] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:25.436064 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:25.436746 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:06:25.470635 15827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:25.436919 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:25.437501 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:06:25.471457 15827 connection.cc:256] RPC call timeout handler was delayed by -1.3079041890000553s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1574s for a response
I20260709 13:06:25.438645 19669 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:25.410384 16565 raft_consensus.cc:1240] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:25.424237 19764 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:25.443859 15853 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5.
I20260709 13:06:25.479302 15862 raft_consensus.cc:1240] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:25.479661 15857 raft_consensus.cc:1240] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 4. Current term is 5. Ops: []
I20260709 13:06:25.414165 16533 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:25.453222 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:25.481182 16002 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:25.476590 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:25.453469 19716 consensus_queue.cc:237] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:25.454664 19767 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:25.454859 16570 raft_consensus.cc:1240] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 9. Current term is 10. Ops: [9.11-9.11]
I20260709 13:06:25.483207 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:25.396229 19651 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:25.455725 19385 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:25.392091 19263 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:25.456775 15900 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:25.457172 19609 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:25.458326 19350 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:25.461205 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:25.461855 16564 raft_consensus.cc:1240] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:06:25.462952 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:25.463318 15860 raft_consensus.cc:1275] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:06:25.464222 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:25.493655 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:25.469103 15854 raft_consensus.cc:1240] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 10. Current term is 11. Ops: [7.6-10.7]
I20260709 13:06:25.399585 19610 consensus_queue.cc:1048] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.085s
W20260709 13:06:25.472276 15827 connection.cc:256] RPC call timeout handler was delayed by -1.1045041890000675s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18000000000000002s for a response
I20260709 13:06:25.471381 16004 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:06:25.311309 19718 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:25.483628 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:25.386621 19724 consensus_queue.cc:1059] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.065s
I20260709 13:06:25.497613 15900 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:25.498515 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:25.484648 16562 raft_consensus.cc:2393] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 9: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
W20260709 13:06:25.481210 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 67125.8us
W20260709 13:06:25.477614 19095 consensus_meta.cc:287] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.815s	user 0.004s	sys 0.007s
I20260709 13:06:25.488842 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:25.498623 15854 raft_consensus.cc:1275] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 13:06:25.492772 15853 raft_consensus.cc:1240] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 7. Current term is 8. Ops: []
W20260709 13:06:25.495101 15827 connection.cc:256] RPC call timeout handler was delayed by -1.0595041890001085s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18500000000000003s for a response
I20260709 13:06:25.425961 19660 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:25.442493 19429 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:25.473241 19715 raft_consensus.cc:697] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 8 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:25.500129 16533 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:25.500249 15854 pending_rounds.cc:85] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8: Aborting all ops after (but not including) 5
W20260709 13:06:25.521013 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:25.568967 16002 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:06:25.493620 19385 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:25.611955 19730 log.cc:927] Time spent T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.831s	user 0.002s	sys 0.003s
I20260709 13:06:25.498658 19550 raft_consensus.cc:2749] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:25.610816 19609 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:25.614784 19776 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:25.620704 15854 pending_rounds.cc:107] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8: Aborting uncommitted NO_OP operation due to leader change: 7.6
W20260709 13:06:25.637934 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 137899us
I20260709 13:06:25.643038 16561 raft_consensus.cc:1240] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:25.671466 15854 raft_consensus.cc:2889] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:25.637931 15858 raft_consensus.cc:1240] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:25.396862 19743 consensus_queue.cc:260] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:25.671994 16533 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:06:25.674579 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 177025us
I20260709 13:06:25.648315 15934 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
I20260709 13:06:25.677551 15900 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:25.678589 19669 raft_consensus.cc:2749] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:25.687964 19722 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:25.693081 19720 consensus_queue.cc:1059] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:25.495774 19263 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:25.694418 15900 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:25.640324 19781 raft_consensus.cc:3055] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:25.637573 19764 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:25.637728 19350 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.702648 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:25.671404 19776 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:25.706985 15854 raft_consensus.cc:2393] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 8: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:25.615798 19667 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:25.710638 19387 raft_consensus.cc:3055] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:25.711040 19387 raft_consensus.cc:740] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:06:25.712841 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:25.714156 19568 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.714622 19568 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:25.509845 19666 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:25.719749 19776 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:25.720722 19666 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:25.712743 19718 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:25.637163 15860 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:06:25.724472 15860 raft_consensus.cc:606] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Received request to transfer leadership to TS 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:25.724896 15860 raft_consensus.cc:614] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:25.725894 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:25.727314 15854 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:25.726826 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 441942us
W20260709 13:06:25.728477 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:25.729972 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:25.523316 19095 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:25.730619 15976 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:25.732547 15974 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:25.702545 19781 raft_consensus.cc:740] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:25.705935 19669 raft_consensus.cc:2749] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:25.722157 19743 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:25.721740 19429 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:25.729939 19667 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:25.736852 19713 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:25.737068 19263 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:25.751982 15854 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 9.
I20260709 13:06:25.764143 16005 raft_consensus.cc:2393] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 10: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:25.766758 16566 raft_consensus.cc:1240] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:25.640159 19566 consensus_queue.cc:237] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:25.782470 15936 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 1924) took 2436 ms. Trace:
W20260709 13:06:25.799995 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 54568.9us
I20260709 13:06:25.770959 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:25.737566 19651 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:25.803944 15854 raft_consensus.cc:1240] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
W20260709 13:06:25.806308 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 59159.7us
I20260709 13:06:25.763499 19748 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:25.808708 19372 consensus_queue.cc:1059] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:25.810163 19748 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:25.745975 19787 raft_consensus.cc:2804] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:25.801339 15936 rpcz_store.cc:276] 0709 13:06:23.345960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.347124 (+  1164us) service_pool.cc:224] Handling call
0709 13:06:23.369979 (+ 22855us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:23.397565 (+ 27586us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:06:23.686625 (+289060us) log.cc:844] Serialized 14 byte log entry
0709 13:06:24.536885 (+850260us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:24.547667 (+ 10782us) log.cc:844] Serialized 52 byte log entry
0709 13:06:24.555029 (+  7362us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:06:24.555083 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:24.555175 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:25.781125 (+1225950us) spinlock_profiling.cc:243] Waited 5.51 ms on lock 0x7b24000f01e8. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71419c4 00007f73e713facb 00007f73e72fadc9 00007f73e72faa55 00007f73e72f9fb0 00007f73e9f70488 00007f73e9f70be8 00007f73e9f48398 00007f73e9f4376a 00007f73e9f42bb6
0709 13:06:25.782054 (+   929us) raft_consensus.cc:1685] finished
0709 13:06:25.782154 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:25.782327 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54450,"prepare.queue_time_us":35237,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":57,"raft.queue_time_us":49658,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":22528,"spinlock_wait_cycles":12677888,"thread_start_us":1002139,"threads_started":2,"wal-append.queue_time_us":975646}
I20260709 13:06:25.813361 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:25.814128 15853 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:25.768576 19781 consensus_queue.cc:260] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:25.817019 19781 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:25.817144 19719 raft_consensus.cc:3055] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:25.817638 19719 raft_consensus.cc:740] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:25.818457 19719 consensus_queue.cc:260] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:25.819846 19775 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.820515 19775 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:25.822813 19775 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:25.746224 19715 consensus_queue.cc:237] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:25.765900 18968 consensus_meta.cc:287] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.514s	user 0.007s	sys 0.003s
I20260709 13:06:25.827196 15853 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:25.831046 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:25.833365 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:25.722880 19350 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:25.767992 19713 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:25.838466 16011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60156 (request call id 674) took 1004 ms. Trace:
I20260709 13:06:25.771899 19568 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:25.761569 19385 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:25.764976 19095 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:25.751394 19667 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:25.838901 16011 rpcz_store.cc:276] 0709 13:06:24.833723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.835768 (+  2045us) service_pool.cc:224] Handling call
0709 13:06:25.833229 (+997461us) spinlock_profiling.cc:243] Waited 5.19 ms on lock 0x7b5800114398. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283eee 0000000000375939 00007f73ef2f71e0 00007f73ef3c00c4 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:06:25.838410 (+  5181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11949696}
I20260709 13:06:25.836719 15827 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:25.840694 16007 raft_consensus.cc:2410] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:06:25.831068 19222 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:25.846966 19350 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:25.847713 19222 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:25.848966 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:25.853996 15935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2001) took 1334 ms. Trace:
I20260709 13:06:25.834923 19429 consensus_queue.cc:1059] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 13:06:25.840667 19787 raft_consensus.cc:697] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 10 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:25.857291 19787 consensus_queue.cc:237] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:25.859351 19779 log.cc:927] Time spent T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
I20260709 13:06:25.835744 19775 consensus_queue.cc:1059] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:06:25.844551 19651 raft_consensus.cc:697] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 9 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:25.861138 15855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2078) took 1194 ms. Trace:
I20260709 13:06:25.766969 19746 consensus_queue.cc:1059] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.134s
I20260709 13:06:25.856194 16563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45232 (request call id 852) took 1070 ms. Trace:
I20260709 13:06:25.861559 18968 consensus_queue.cc:1059] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 13:06:25.866721 19719 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:25.869418 15861 raft_consensus.cc:3055] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:25.869849 15861 raft_consensus.cc:740] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:25.870579 15861 consensus_queue.cc:260] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.11, Last appended by leader: 11, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:25.872037 19667 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:25.872274 19651 consensus_queue.cc:237] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:25.871562 15861 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:25.868255 16563 rpcz_store.cc:276] 0709 13:06:24.785325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.785455 (+   130us) service_pool.cc:224] Handling call
0709 13:06:25.855963 (+1070508us) spinlock_profiling.cc:243] Waited 377 ms on lock 0x7b54006bf3d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:25.856104 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":867059584}
I20260709 13:06:25.854377 15935 rpcz_store.cc:276] 0709 13:06:24.519352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.519480 (+   128us) service_pool.cc:224] Handling call
0709 13:06:24.543759 (+ 24279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:24.637281 (+ 93522us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:06:24.810164 (+172883us) log.cc:844] Serialized 14 byte log entry
0709 13:06:25.007635 (+197471us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b1800075640. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e9f504bc 00007f73e729e48b 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f4823a
0709 13:06:25.024812 (+ 17177us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:25.024866 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:25.024909 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:25.777154 (+752245us) spinlock_profiling.cc:243] Waited 1.94 ms on lock 0x7b1800075640. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb3f3 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd 000000000038f354 000000000038f199 00007f73e729e83a
0709 13:06:25.852313 (+ 75159us) spinlock_profiling.cc:243] Waited 1.75 ms on lock 0x7b1800075640. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72892a6 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419
0709 13:06:25.853768 (+  1455us) spinlock_profiling.cc:243] Waited 93.3 ms on lock 0x7b54002c1854. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:25.853816 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:25.853938 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27030,"prepare.queue_time_us":179016,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":14921,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":8659,"spinlock_wait_cycles":381895680,"thread_start_us":227161,"threads_started":2,"wal-append.queue_time_us":155630}
I20260709 13:06:25.878975 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:25.879781 16002 raft_consensus.cc:1240] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:25.880041 15827 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:25.882525 16006 raft_consensus.cc:3055] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:06:25.864073 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:25.883037 16006 raft_consensus.cc:740] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:25.883909 16006 consensus_queue.cc:260] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:25.864784 15855 rpcz_store.cc:276] 0709 13:06:24.666513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.680484 (+ 13971us) service_pool.cc:224] Handling call
0709 13:06:24.710662 (+ 30178us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:25.424297 (+713635us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:25.860892 (+436595us) spinlock_profiling.cc:243] Waited 713 ms on lock 0x7b54002486d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:25.861074 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1639779584}
W20260709 13:06:25.863538 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 169201us
I20260709 13:06:25.885113 16006 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:25.888370 15900 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:25.867151 19722 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:25.896955 15900 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:25.885388 19372 raft_consensus.cc:3055] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:25.903486 15827 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:25.907222 15900 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:25.911623 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:25.912292 15900 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:25.887019 19716 raft_consensus.cc:2764] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:06:25.904662 19372 raft_consensus.cc:740] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:25.899487 19350 consensus_queue.cc:1059] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:06:25.881629 19222 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:25.929279 19669 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:06:25.946348 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:25.946796 15854 raft_consensus.cc:3055] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:25.947269 15854 raft_consensus.cc:740] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:25.901648 19558 consensus_queue.cc:1059] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:06:25.913033 19793 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:25.948076 19372 consensus_queue.cc:260] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:25.948702 19793 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:25.949123 19793 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:25.949437 19372 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:25.917847 19776 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:25.952198 19350 raft_consensus.cc:3055] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:25.952632 19350 raft_consensus.cc:740] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:25.954113 19776 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:25.953642 19350 consensus_queue.cc:260] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:25.957741 19387 consensus_queue.cc:260] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:25.959242 19387 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:25.952664 19667 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:25.916435 15827 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:25.964576 16006 raft_consensus.cc:1275] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 10. (term mismatch)
I20260709 13:06:25.925136 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:25.964635 15861 raft_consensus.cc:1275] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 9 index: 11. Preceding OpId from leader: term: 10 index: 11. (term mismatch)
I20260709 13:06:25.965605 15861 pending_rounds.cc:85] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8: Aborting all ops after (but not including) 10
I20260709 13:06:25.929177 16535 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:25.965920 15861 pending_rounds.cc:107] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8: Aborting uncommitted NO_OP operation due to leader change: 9.11
I20260709 13:06:25.931985 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:25.931021 16533 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:25.966243 15861 raft_consensus.cc:2889] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:06:25.942673 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:25.942665 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:25.937705 19713 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.875586 19758 consensus_queue.cc:1048] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:06:25.961098 16563 raft_consensus.cc:1240] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:25.918530 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:06:25.968370 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:25.968696 16006 pending_rounds.cc:85] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting all ops after (but not including) 9
I20260709 13:06:25.968755 15855 raft_consensus.cc:2393] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 8: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:25.969111 16006 pending_rounds.cc:107] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 13:06:25.932590 16564 raft_consensus.cc:1240] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:06:25.969990 16006 raft_consensus.cc:2889] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:25.910894 19263 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.929370 19720 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:25.922612 19648 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:25.972328 16535 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:25.973573 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:25.985541 15928 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:25.974229 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:25.919845 19385 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:06:25.990475 19754 raft_consensus.cc:2749] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:25.991288 19754 consensus_queue.cc:1048] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:25.986574 19568 consensus_queue.cc:1059] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 13:06:25.993837 19568 raft_consensus.cc:3055] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:25.994225 19568 raft_consensus.cc:740] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:25.994576 19222 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:25.976408 15937 raft_consensus.cc:1240] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:25.996096 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:06:25.977447 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 61091.2us
I20260709 13:06:25.978957 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:25.979180 16009 raft_consensus.cc:1240] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:25.979832 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:26.002260 15930 raft_consensus.cc:1275] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:06:26.002169 19742 consensus_queue.cc:1059] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:25.981024 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:26.004038 15936 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:26.005254 19581 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 13:06:25.999421 15827 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:26.008286 15827 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:25.903584 19733 consensus_queue.cc:1048] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:06:25.942139 19743 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.010996 19743 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:26.013412 19743 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:26.003607 15930 raft_consensus.cc:1275] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:06:26.014876 16008 raft_consensus.cc:1240] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:06:26.017234 16533 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:25.978082 19713 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:25.918179 19790 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:25.987201 15861 raft_consensus.cc:2393] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 9: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:25.987704 19722 consensus_queue.cc:1059] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:25.988057 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:25.988363 19764 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:25.981949 15935 raft_consensus.cc:1240] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 13:06:26.032124 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:26.038879 16562 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:26.016254 15827 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:25.985067 19775 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:25.978257 19776 raft_consensus.cc:2804] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:26.040441 19385 consensus_queue.cc:1048] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:26.041213 19385 consensus_queue.cc:1243] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer 85e997ba5003439781ec4f1f8db0c3ee log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:06:26.044963 15976 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:26.045306 16533 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:26.047425 15827 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:25.986008 19781 raft_consensus.cc:2749] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:26.051189 15827 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:26.032181 19685 raft_consensus.cc:2749] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:25.995076 19568 consensus_queue.cc:260] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:26.015990 19720 raft_consensus.cc:697] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:26.037071 19724 raft_consensus.cc:2764] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:06:26.048818 19715 raft_consensus.cc:2749] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:26.042770 19742 raft_consensus.cc:2804] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:26.050150 19758 raft_consensus.cc:2804] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:26.052055 19367 consensus_queue.cc:1059] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 13:06:26.063144 19758 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:26.065394 19758 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:26.066058 15934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 1951) took 1260 ms. Trace:
I20260709 13:06:26.066483 15934 rpcz_store.cc:276] 0709 13:06:24.805936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.827757 (+ 21821us) service_pool.cc:224] Handling call
0709 13:06:26.065802 (+1238045us) spinlock_profiling.cc:243] Waited 478 ms on lock 0x7b5400327b54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:26.065976 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1099342208}
I20260709 13:06:26.066773 15933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 1948) took 1395 ms. Trace:
I20260709 13:06:26.067628 15933 rpcz_store.cc:276] 0709 13:06:24.671256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.714580 (+ 43324us) service_pool.cc:224] Handling call
0709 13:06:24.916315 (+201735us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b5800003698. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef381fc0 00007f73ef36c929 00007f73ef36c3d2 00007f73e88e3bd4
0709 13:06:24.916522 (+   207us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:24.916725 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:25.987835 (+1071110us) log.cc:844] Serialized 14 byte log entry
0709 13:06:26.066411 (+ 78576us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:26.066464 (+    53us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:26.066519 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:26.066588 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:26.066719 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1067645,"prepare.queue_time_us":1071744,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"spinlock_wait_cycles":28140800,"thread_start_us":78180,"threads_started":1}
I20260709 13:06:26.075606 15936 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 9.
I20260709 13:06:25.985755 19263 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:26.081949 16533 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:26.053941 19585 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:26.052733 19775 raft_consensus.cc:697] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:26.050603 19790 raft_consensus.cc:697] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:26.058203 19776 raft_consensus.cc:697] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:26.044613 19764 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:26.063555 19385 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:26.064608 19742 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:26.053731 19727 consensus_queue.cc:1059] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 13:06:26.060858 19720 consensus_queue.cc:237] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:26.085444 19748 raft_consensus.cc:2749] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:26.086169 19610 raft_consensus.cc:2749] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:26.092823 19568 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:26.094858 19742 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:26.104911 19775 consensus_queue.cc:237] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:26.109310 19760 consensus_queue.cc:1059] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 13:06:26.110167 19263 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:26.110744 19776 consensus_queue.cc:237] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:26.111408 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:26.112205 16562 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:26.118927 19641 raft_consensus.cc:3055] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:26.119540 19641 raft_consensus.cc:740] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:26.108069 19722 consensus_queue.cc:1048] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.012s
I20260709 13:06:26.120671 19641 consensus_queue.cc:260] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:06:26.122383 19641 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:26.131083 15976 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:26.090987 19723 consensus_queue.cc:1048] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 13:06:26.119094 19793 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:26.139281 19743 consensus_queue.cc:1059] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
W20260709 13:06:26.146591 19635 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 13:06:26.138269 19372 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:26.152487 19713 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:26.153254 16004 raft_consensus.cc:1240] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:26.155974 15854 consensus_queue.cc:260] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:26.191285 15854 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:26.195847 19790 consensus_queue.cc:237] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:06:26.157068 19651 raft_consensus.cc:3055] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:26.160979 15929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2002) took 1614 ms. Trace:
I20260709 13:06:26.168319 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:26.198441 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 114356us
I20260709 13:06:26.182027 15858 raft_consensus.cc:1240] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 10. Current term is 11. Ops: [7.6-10.7]
I20260709 13:06:26.200951 15854 raft_consensus.cc:1275] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:06:26.184772 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:26.201630 15856 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:26.182545 19754 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:26.182850 19372 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:26.184114 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:26.208410 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 157363us
I20260709 13:06:26.203255 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:26.209578 15856 raft_consensus.cc:2393] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 7: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:26.197671 15929 rpcz_store.cc:276] 0709 13:06:24.546104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.546222 (+   118us) service_pool.cc:224] Handling call
0709 13:06:24.546528 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:24.555927 (+  9399us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:06:24.683383 (+127456us) spinlock_profiling.cc:243] Waited 19.3 ms on lock 0x7b18001bb6e0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72b8455 00007f73edda03c7 00007f73ea3b29a7 00007f73ea3c20bf 00007f73ea3c2038
0709 13:06:24.787050 (+103667us) log.cc:844] Serialized 14 byte log entry
0709 13:06:25.671970 (+884920us) spinlock_profiling.cc:243] Waited 1.31 ms on lock 0x7b18001bb6e0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f4823a 00007f73ea3b2a2a
0709 13:06:25.805112 (+133142us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:25.830790 (+ 25678us) log.cc:844] Serialized 26 byte log entry
0709 13:06:25.830929 (+   139us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:25.830959 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:26.160672 (+329713us) spinlock_profiling.cc:243] Waited 9.02 ms on lock 0x7b54001a68d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:26.160720 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:26.160780 (+    60us) raft_consensus.cc:1685] finished
0709 13:06:26.160796 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:26.160923 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46789,"prepare.queue_time_us":254372,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":2617,"raft.queue_time_us":1258,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":9910,"spinlock_wait_cycles":68256640,"thread_start_us":224256,"threads_started":2,"wal-append.queue_time_us":1077013}
I20260709 13:06:26.206892 15861 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
W20260709 13:06:26.214610 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:26.215194 15937 raft_consensus.cc:1275] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:06:26.215502 15900 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:26.218502 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:26.221666 16006 raft_consensus.cc:1240] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:26.223727 15900 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:26.224205 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:26.225734 16564 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:26.229133 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:26.197577 19651 raft_consensus.cc:740] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:26.232964 15827 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:26.216100 19372 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:26.241114 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:26.242095 16566 raft_consensus.cc:2393] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 9: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:26.236644 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:26.236567 19713 consensus_queue.cc:1059] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 13:06:26.246177 19651 consensus_queue.cc:260] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:26.247974 19651 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:26.220746 19387 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:26.250552 15827 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:26.251247 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:06:26.255733 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:26.260754 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:26.261408 16564 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:06:26.269934 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:26.272899 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:26.273484 15937 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:06:26.243065 19719 raft_consensus.cc:2749] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:06:26.293208 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:26.302217 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:26.304965 16009 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:26.226837 19648 consensus_queue.cc:1048] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:06:26.258065 19222 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.302141 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:06:26.316469 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 65995.1us
I20260709 13:06:26.322217 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:26.323863 15937 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:26.330360 16009 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:06:26.215602 19754 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:26.356099 15827 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:26.357721 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 106407us
I20260709 13:06:26.359426 15929 raft_consensus.cc:1240] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 10 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:06:26.263039 19720 consensus_queue.cc:1059] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:06:26.361784 15825 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:26.263769 19635 consensus_queue.cc:1059] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.078s
I20260709 13:06:26.265375 19666 consensus_queue.cc:1059] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.091s
I20260709 13:06:26.274667 19809 raft_consensus.cc:2749] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:26.342468 19713 raft_consensus.cc:2749] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:26.354671 19716 raft_consensus.cc:3055] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:26.263031 19758 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:26.401850 15937 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:06:26.361131 19367 consensus_queue.cc:1059] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
W20260709 13:06:26.412949 19418 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 13:06:26.416841 19749 raft_consensus.cc:3055] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:26.417395 19749 raft_consensus.cc:740] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:26.417912 19609 raft_consensus.cc:3055] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:26.418327 19609 raft_consensus.cc:740] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:26.418449 19749 consensus_queue.cc:260] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:26.419085 19609 consensus_queue.cc:260] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:26.420045 19817 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:26.420390 19609 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:26.420589 19817 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:26.422703 19817 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:26.424666 19793 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:26.425160 15861 raft_consensus.cc:1275] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:06:26.372972 19715 raft_consensus.cc:2804] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:26.428970 15861 raft_consensus.cc:1275] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:06:26.444661 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 76677.2us
I20260709 13:06:26.445886 15825 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:06:26.449395 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:26.451910 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:26.355091 19222 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:26.380107 19716 raft_consensus.cc:740] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:26.463032 15902 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:26.475636 16563 raft_consensus.cc:1275] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:06:26.431077 19713 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:26.445664 19715 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:26.488353 16563 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239"
dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
mode: GRACEFUL
new_leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:42622
I20260709 13:06:26.492761 16563 raft_consensus.cc:606] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Received request to transfer leadership to TS 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:26.493206 16563 raft_consensus.cc:614] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:06:26.494354 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 113367us
I20260709 13:06:26.478072 19724 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:26.499855 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:26.500860 15854 raft_consensus.cc:2393] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 7: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:26.486219 19749 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:26.503401 15861 raft_consensus.cc:3055] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:26.506168 15861 raft_consensus.cc:740] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:26.495337 19715 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:06:26.504727 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:26.504608 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 148571us
I20260709 13:06:26.444505 19787 consensus_queue.cc:1048] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 13:06:26.441891 19809 consensus_queue.cc:1059] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 13:06:26.508153 15827 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:26.494225 19581 consensus_queue.cc:1048] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 13:06:26.514464 15861 consensus_queue.cc:260] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:26.522928 15854 raft_consensus.cc:1240] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:06:26.508405 19724 raft_consensus.cc:697] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 9 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:26.511870 19095 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:26.529842 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 66898.7us
I20260709 13:06:26.529816 15827 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:26.528483 19385 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:06:26.533962 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:26.526533 19790 raft_consensus.cc:2804] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:26.547046 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:26.528962 19809 consensus_queue.cc:1243] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer 8430db80a39d4868bcf7641c16de35ea log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:06:26.510390 19817 consensus_queue.cc:1048] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 13:06:26.543306 15902 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:26.545486 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:26.548064 15860 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 9: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:26.535725 19820 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:26.551518 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:26.551436 19820 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:26.552325 15858 raft_consensus.cc:2393] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 9: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:26.553125 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:26.553648 19820 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:26.554209 19805 log.cc:927] Time spent T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
I20260709 13:06:26.559908 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:26.559978 15857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2083) took 1108 ms. Trace:
I20260709 13:06:26.560696 15857 rpcz_store.cc:276] 0709 13:06:25.451181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.456280 (+  5099us) service_pool.cc:224] Handling call
0709 13:06:25.479406 (+ 23126us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:25.480398 (+   992us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:26.559792 (+1079394us) spinlock_profiling.cc:243] Waited 22.8 ms on lock 0x7b54002d14d0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:26.559924 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":52482816}
I20260709 13:06:26.561497 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:26.562211 19724 consensus_queue.cc:237] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:26.538098 19827 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:26.565026 19811 log.cc:927] Time spent T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.350s	user 0.003s	sys 0.000s
I20260709 13:06:26.564677 19827 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:26.565820 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:26.566519 16005 raft_consensus.cc:2368] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 10. Current term is 11.
I20260709 13:06:26.566989 19827 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:26.568321 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:26.595530 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:26.551124 15827 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:26.568418 19815 log.cc:927] Time spent T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:06:26.596388 16010 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
W20260709 13:06:26.597813 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 46797.2us
I20260709 13:06:26.527612 19716 consensus_queue.cc:260] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:26.601889 19716 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:26.601805 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:26.603219 16010 raft_consensus.cc:2368] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 8. Current term is 9.
I20260709 13:06:26.568380 19764 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:26.604375 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:26.538669 19095 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:26.605262 15937 raft_consensus.cc:1275] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:06:26.547626 19822 raft_consensus.cc:2804] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:06:26.611724 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:26.609026 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:26.610059 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:26.614946 16003 raft_consensus.cc:1240] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 7. Current term is 8. Ops: [5.7-7.8]
I20260709 13:06:26.594818 19820 raft_consensus.cc:2804] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:26.547139 19790 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:26.606287 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:26.619928 15929 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:26.621165 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:26.604439 15827 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:26.601821 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:26.609030 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:26.618088 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:26.605664 16010 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 9: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:26.617338 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:06:26.618780 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 182783us
I20260709 13:06:26.622394 16007 raft_consensus.cc:2393] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 10: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:26.600591 19758 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:26.623008 15936 raft_consensus.cc:2393] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 10: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:26.625121 15827 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:26.625849 15856 raft_consensus.cc:1275] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:06:26.628595 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:26.605315 19385 raft_consensus.cc:2749] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:26.626130 15857 raft_consensus.cc:1240] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:26.630232 15933 raft_consensus.cc:1217] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Deduplicated request from leader. Original: 6.5->[7.6-10.7]   Dedup: 7.6->[10.7-10.7]
I20260709 13:06:26.630789 15933 raft_consensus.cc:1240] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 10. Current term is 11. Ops: [7.6-10.7]
I20260709 13:06:26.629238 16010 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:26.627629 16005 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:26.632510 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:26.633102 15854 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:06:26.608289 19821 raft_consensus.cc:3055] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:26.634225 16010 raft_consensus.cc:1275] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:06:26.634344 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:26.637303 15929 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:26.619627 19223 raft_consensus.cc:3055] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:26.643782 16563 raft_consensus.cc:1240] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:26.638589 15933 raft_consensus.cc:1240] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 4. Current term is 5. Ops: []
W20260709 13:06:26.640179 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:26.623757 19790 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:26.646018 16007 raft_consensus.cc:1240] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:26.647081 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:26.647364 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:26.650569 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:26.651191 16566 raft_consensus.cc:2393] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 7: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:26.622812 19222 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:26.651245 16563 raft_consensus.cc:2368] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 8. Current term is 9.
I20260709 13:06:26.648545 16567 raft_consensus.cc:1240] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:26.629664 19713 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:26.673895 16007 raft_consensus.cc:1275] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:06:26.674343 16565 raft_consensus.cc:1240] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:26.627956 19820 raft_consensus.cc:697] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:26.627528 19822 raft_consensus.cc:697] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 7 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:26.655406 16570 raft_consensus.cc:2368] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 10. Current term is 11.
I20260709 13:06:26.656666 19806 raft_consensus.cc:3055] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:26.658648 19826 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:26.688992 16570 raft_consensus.cc:1275] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:06:26.658728 19831 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.658905 19844 consensus_queue.cc:1059] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:26.643282 19223 raft_consensus.cc:740] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:26.700253 19823 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:26.701157 19843 raft_consensus.cc:3055] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:26.701325 16570 raft_consensus.cc:1240] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:26.678809 16004 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
I20260709 13:06:26.639580 19675 raft_consensus.cc:3055] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:26.710076 19675 raft_consensus.cc:740] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:26.686173 19806 raft_consensus.cc:740] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:26.712523 19806 consensus_queue.cc:260] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:26.683588 15854 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:26.688613 19826 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:26.690131 15933 raft_consensus.cc:1275] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:06:26.660970 19095 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:26.639245 19821 raft_consensus.cc:740] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:26.726913 19821 consensus_queue.cc:260] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:26.727882 19826 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:26.728062 19760 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:26.728564 19760 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:26.729317 19669 raft_consensus.cc:3055] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:26.729748 19669 raft_consensus.cc:740] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:26.730523 19760 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:26.694224 19831 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:26.695806 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:26.701732 19843 raft_consensus.cc:740] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:26.705435 16561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 848) took 1277 ms. Trace:
W20260709 13:06:26.708112 19641 consensus_meta.cc:287] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.585s	user 0.013s	sys 0.000s
I20260709 13:06:26.708565 19822 consensus_queue.cc:237] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:26.718010 16568 raft_consensus.cc:1275] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
W20260709 13:06:26.719414 19810 log.cc:927] Time spent T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.509s	user 0.002s	sys 0.000s
I20260709 13:06:26.722438 19820 consensus_queue.cc:237] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:26.713537 19806 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:26.732447 16561 rpcz_store.cc:276] 0709 13:06:25.428191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.428322 (+   131us) service_pool.cc:224] Handling call
0709 13:06:25.452865 (+ 24543us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:25.671847 (+218982us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:26.705205 (+1033358us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b54000cce54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:26.705383 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":436731648}
I20260709 13:06:26.732494 19387 raft_consensus.cc:2749] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:26.733980 15933 raft_consensus.cc:1275] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:06:26.734086 19372 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:26.747215 19843 consensus_queue.cc:260] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:26.739500 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 100601us
W20260709 13:06:26.735949 19742 consensus_meta.cc:287] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.599s	user 0.011s	sys 0.000s
I20260709 13:06:26.736558 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:26.748709 19742 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:26.749298 15931 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:26.748191 16006 raft_consensus.cc:2368] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 9. Current term is 10.
I20260709 13:06:26.748657 19843 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:26.750823 19742 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:26.744659 16570 raft_consensus.cc:1240] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:26.711108 19675 consensus_queue.cc:260] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:26.730548 19669 consensus_queue.cc:260] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:26.811744 15931 raft_consensus.cc:1275] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:06:26.899290 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:26.900978 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:26.911087 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:26.900329 19635 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.019s	sys 0.019s
W20260709 13:06:26.915349 19635 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.020s	sys 0.020s
I20260709 13:06:26.916117 19418 consensus_queue.cc:1059] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:06:27.291054 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:26.919071 15931 raft_consensus.cc:1275] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:06:27.372385 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 620332us
I20260709 13:06:26.855636 19742 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.388751 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:27.344029 19716 consensus_meta.cc:287] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.741s	user 0.008s	sys 0.004s
I20260709 13:06:27.390556 16561 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
W20260709 13:06:27.179893 19790 consensus_meta.cc:287] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.505s	user 0.010s	sys 0.000s
W20260709 13:06:27.392408 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:27.033489 19715 consensus_meta.cc:287] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.522s	user 0.004s	sys 0.005s
I20260709 13:06:27.374521 19675 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:27.050364 19831 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:27.395517 16570 raft_consensus.cc:1240] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:06:27.395376 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:27.368410 19635 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:27.387580 19669 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:27.396292 19222 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.409512 19222 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:27.395004 19372 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:27.395395 19790 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:06:27.395743 19715 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:27.420802 19716 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.394313 19742 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:27.412029 19222 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:27.430786 19812 log.cc:927] Time spent T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 1.218s	user 0.009s	sys 0.011s
I20260709 13:06:27.428351 19716 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:27.433647 19748 raft_consensus.cc:3055] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:27.434083 19748 raft_consensus.cc:740] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:27.434902 19748 consensus_queue.cc:260] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:27.438028 15855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2096) took 1278 ms. Trace:
I20260709 13:06:27.421666 19848 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.435422 19716 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:27.437958 19749 consensus_meta.cc:287] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.929s	user 0.012s	sys 0.002s
I20260709 13:06:27.438374 15855 rpcz_store.cc:276] 0709 13:06:26.159467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.179039 (+ 19572us) service_pool.cc:224] Handling call
0709 13:06:26.186959 (+  7920us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:26.187326 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:26.187360 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:26.187816 (+   456us) log.cc:844] Serialized 26 byte log entry
0709 13:06:26.212650 (+ 24834us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:26.212693 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:26.212761 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:27.428270 (+1215509us) spinlock_profiling.cc:243] Waited 2.51 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:06:27.435933 (+  7663us) raft_consensus.cc:1685] finished
0709 13:06:27.435957 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:27.437953 (+  1996us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2069,"raft.run_cpu_time_us":70,"raft.run_wall_time_us":70,"spinlock_wait_cycles":5775232,"thread_start_us":24477,"threads_started":1,"wal-append.queue_time_us":24789}
W20260709 13:06:27.437184 19823 consensus_meta.cc:287] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.736s	user 0.010s	sys 0.003s
I20260709 13:06:27.402801 19635 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:27.446913 19823 raft_consensus.cc:2749] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:27.454092 16006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41674 (request call id 1921) took 1195 ms. Trace:
I20260709 13:06:27.454615 16006 rpcz_store.cc:276] 0709 13:06:26.258295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.260543 (+  2248us) service_pool.cc:224] Handling call
0709 13:06:27.453864 (+1193321us) spinlock_profiling.cc:243] Waited 487 ms on lock 0x7b54001dc754. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:27.454021 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1119330688}
W20260709 13:06:27.454260 19843 consensus_meta.cc:287] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.704s	user 0.009s	sys 0.000s
I20260709 13:06:27.458211 19635 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:27.459033 16008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 2058) took 2460 ms. Trace:
I20260709 13:06:27.459856 16008 rpcz_store.cc:276] 0709 13:06:24.998694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.998835 (+   141us) service_pool.cc:224] Handling call
0709 13:06:25.822296 (+823461us) spinlock_profiling.cc:243] Waited 1.42 ms on lock 0x7b5800114398. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef381fc0 00007f73ef36c929 00007f73ef36c3d2 00007f73e88e3bd4
0709 13:06:26.014612 (+192316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:26.015429 (+   817us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:27.458851 (+1443422us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b54002e1b50. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:27.458969 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":445041280}
I20260709 13:06:27.463406 19790 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
I20260709 13:06:27.445680 19848 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:27.477558 19742 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:27.479146 19806 consensus_meta.cc:287] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.727s	user 0.007s	sys 0.005s
W20260709 13:06:27.482633 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:27.488273 19848 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:27.491195 19748 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:27.492046 19715 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:27.492316 19790 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.494031 19790 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:27.497044 19790 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:27.497517 19790 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.497968 19790 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:27.498260 19660 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:27.500373 15974 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 1.925s (SENT) [suppressed 18 similar messages]
I20260709 13:06:27.500769 19816 consensus_queue.cc:1048] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:27.501196 19263 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.501621 16533 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:27.501842 19263 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:27.503299 19727 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:06:27.503914 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:27.501269 19758 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.494210 15900 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:27.508764 19758 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:27.512171 19790 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:27.513597 19758 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:27.509719 19095 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.514655 19095 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:27.504093 19263 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:27.517314 19095 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:27.514099 19367 raft_consensus.cc:3055] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:27.518059 19367 raft_consensus.cc:740] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:27.518891 19367 consensus_queue.cc:260] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:27.505388 19820 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.520244 19367 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:27.520028 19820 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:27.524718 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:27.536742 15859 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 1948) took 2628 ms (client timeout 2171 ms). Trace:
W20260709 13:06:27.537199 15859 rpcz_store.cc:269] 0709 13:06:24.905808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.906094 (+   286us) service_pool.cc:224] Handling call
0709 13:06:27.534216 (+2628122us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400218c54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:27.534417 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2378639616}
I20260709 13:06:27.541749 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:27.542424 15929 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:06:27.543123 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:27.543740 16009 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
I20260709 13:06:27.533818 19820 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:27.545158 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 51027.4us
I20260709 13:06:27.548923 15900 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:27.549357 15936 raft_consensus.cc:1240] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:27.549628 15935 raft_consensus.cc:1240] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 8. Current term is 9. Ops: []
W20260709 13:06:27.536921 19803 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 1.323s	user 0.014s	sys 0.015s
W20260709 13:06:27.547021 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:27.533812 19641 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:27.547742 16002 raft_consensus.cc:1275] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:06:27.549758 15929 raft_consensus.cc:1275] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:06:27.536456 19651 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:27.546867 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:27.564761 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:27.565419 15935 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:06:27.566222 19820 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:27.564798 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:27.538241 19372 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:27.560346 19555 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.568034 19372 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:27.568303 19555 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:27.570115 19555 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:27.567538 16564 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:06:27.542688 19767 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.570966 19651 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:27.576108 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:27.546034 19790 raft_consensus.cc:2749] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:27.556543 19749 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:27.546504 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 46149.1us
I20260709 13:06:27.542513 19575 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:27.582779 15936 raft_consensus.cc:1275] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:06:27.552614 15827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:27.573308 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:27.574122 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:27.575444 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:27.588464 16002 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
I20260709 13:06:27.550875 19748 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:27.538110 19222 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.600261 19222 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:27.578236 16561 raft_consensus.cc:1275] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:06:27.602207 19651 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:27.555980 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:27.579768 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:27.579874 15825 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:27.606287 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:27.608498 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:06:27.609248 15827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:27.609620 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:06:27.564244 19803 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 1.348s	user 0.016s	sys 0.015s
I20260709 13:06:27.558149 15933 raft_consensus.cc:1240] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:27.611897 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:27.560472 19581 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.614220 19581 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:27.582772 16569 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEARNER]: Advancing to term 11
I20260709 13:06:27.617785 19581 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:27.583060 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:27.618649 15935 raft_consensus.cc:2393] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 12: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:27.619416 19418 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:27.619973 19418 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:27.622682 19418 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:27.623202 19418 consensus_queue.cc:1059] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:27.589937 15934 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 7.
I20260709 13:06:27.590961 16009 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:06:27.590499 19575 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:27.641976 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:27.643090 19808 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.592274 16008 raft_consensus.cc:1275] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:06:27.643626 19808 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:27.644747 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:27.599229 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:27.610121 15859 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:06:27.645750 19808 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:27.646261 15859 raft_consensus.cc:606] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Received request to transfer leadership to TS 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:27.646641 19727 consensus_queue.cc:1059] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:27.646667 15859 raft_consensus.cc:614] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:27.648268 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:27.650298 19651 consensus_queue.cc:1048] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:27.650732 16008 raft_consensus.cc:1240] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:27.651001 19222 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:27.607465 15825 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:27.612462 16561 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:06:27.652743 16003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1926) took 1392 ms. Trace:
I20260709 13:06:27.653156 16003 rpcz_store.cc:276] 0709 13:06:26.259900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.260053 (+   153us) service_pool.cc:224] Handling call
0709 13:06:26.260377 (+   324us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:26.619024 (+358647us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:27.652493 (+1033469us) spinlock_profiling.cc:243] Waited 354 ms on lock 0x7b5400306454. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:27.652690 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":814714752}
I20260709 13:06:27.654847 16003 raft_consensus.cc:1275] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:06:27.623838 19724 consensus_queue.cc:1048] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:27.624850 16010 raft_consensus.cc:1240] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:27.580202 19767 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:27.596565 19749 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:27.578593 19853 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:27.613070 19748 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:27.625795 19715 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:27.626539 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:27.675195 16007 raft_consensus.cc:2410] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:06:27.668632 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:27.574695 15937 raft_consensus.cc:1275] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:06:27.633901 15902 leader_election.cc:400] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 8. Current term is 9.
I20260709 13:06:27.636427 15974 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:27.639845 19816 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.571893 19641 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:06:27.643272 19575 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:27.570128 19372 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:27.657397 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:27.663066 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:27.685015 16563 raft_consensus.cc:2393] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 9: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:27.686038 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:27.686939 16561 raft_consensus.cc:2468] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:06:27.666018 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:27.665892 16535 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:27.669215 16002 raft_consensus.cc:1275] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:06:27.669504 16569 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEARNER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:06:27.670102 16564 raft_consensus.cc:1275] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:06:27.671478 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 64068.4us
I20260709 13:06:27.693037 16006 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:27.693233 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:27.671983 15827 leader_election.cc:400] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 9. Current term is 10.
I20260709 13:06:27.630214 16011 raft_consensus.cc:1240] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:27.676373 16010 raft_consensus.cc:2410] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 9 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:06:27.676636 15902 leader_election.cc:403] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:06:27.694243 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:27.694655 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 60917us
I20260709 13:06:27.694901 16564 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
W20260709 13:06:27.681699 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 45293.6us
I20260709 13:06:27.697778 15974 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:27.697973 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:27.699281 15974 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:27.694118 16568 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:06:27.677282 19713 raft_consensus.cc:2804] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:27.707922 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:27.702589 15902 leader_election.cc:400] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 10. Current term is 11.
I20260709 13:06:27.709259 15902 leader_election.cc:403] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:06:27.703154 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:27.711023 16564 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
I20260709 13:06:27.703999 16011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1925) took 1444 ms. Trace:
I20260709 13:06:27.712146 16011 rpcz_store.cc:276] 0709 13:06:26.259619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.266279 (+  6660us) service_pool.cc:224] Handling call
0709 13:06:26.316540 (+ 50261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:27.630027 (+1313487us) spinlock_profiling.cc:243] Waited 41.6 ms on lock 0x7b5400221b20. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea31fdc0 00007f73ea3aa1c0 00007f73ea3ab231 00007f73ea3a7d8f 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c
0709 13:06:27.703752 (+ 73725us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:27.703938 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":95655168}
I20260709 13:06:27.706140 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:27.713692 16566 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:06:27.700956 15974 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:27.677078 19715 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:27.709053 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:27.729743 16570 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:27.730695 15859 raft_consensus.cc:1275] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:06:27.731213 15859 pending_rounds.cc:85] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8: Aborting all ops after (but not including) 7
I20260709 13:06:27.731487 15859 pending_rounds.cc:107] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:06:27.731803 15859 raft_consensus.cc:2889] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:27.716248 16563 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:27.715776 15974 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:27.737115 15974 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:27.716678 15827 leader_election.cc:403] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:27.717593 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:27.721796 15860 raft_consensus.cc:1275] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:06:27.723311 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:27.725071 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:27.742547 15854 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:27.691092 19787 raft_consensus.cc:2749] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:27.744256 16005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41662 (request call id 2068) took 1166 ms. Trace:
I20260709 13:06:27.749234 16005 rpcz_store.cc:276] 0709 13:06:26.577566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.594519 (+ 16953us) service_pool.cc:224] Handling call
0709 13:06:27.744065 (+1149546us) spinlock_profiling.cc:243] Waited 5.17 ms on lock 0x7b540010efd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:27.744192 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11879680}
W20260709 13:06:27.751470 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 79587.3us
W20260709 13:06:27.752365 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:27.745307 15902 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:27.702713 19808 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:27.745959 15974 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:27.754096 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:27.743479 16562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 998) took 1100 ms. Trace:
I20260709 13:06:27.765470 16006 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:06:27.765585 16562 rpcz_store.cc:276] 0709 13:06:26.643445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.653263 (+  9818us) service_pool.cc:224] Handling call
0709 13:06:26.653584 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:26.728574 (+ 74990us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:26.728610 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:26.728671 (+    61us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:26.728703 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:27.743274 (+1014571us) spinlock_profiling.cc:243] Waited 74.8 ms on lock 0x7b5400714b54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:27.743314 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:27.743420 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":172024704}
I20260709 13:06:27.704377 19651 raft_consensus.cc:2749] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:06:27.701349 19575 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.709085 19372 raft_consensus.cc:2749] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:27.702284 19222 raft_consensus.cc:2749] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 8. Current term is 9.
I20260709 13:06:27.745676 19715 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:27.763795 15855 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:27.743659 19733 raft_consensus.cc:2749] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:27.743979 19581 raft_consensus.cc:2749] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:27.678906 19816 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:27.705191 19790 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.716660 19713 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:27.755204 19859 log.cc:927] Time spent T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:06:27.772442 19568 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:27.748186 19429 raft_consensus.cc:2749] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:27.704509 19742 consensus_queue.cc:1048] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 13:06:27.769840 16563 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 10.
I20260709 13:06:27.701706 19806 consensus_queue.cc:1048] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.067s
W20260709 13:06:27.785770 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 52331.4us
I20260709 13:06:27.774573 19095 consensus_queue.cc:1048] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:06:27.748370 19746 raft_consensus.cc:2749] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 10. Current term is 11.
I20260709 13:06:27.770710 15827 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:27.794902 15859 raft_consensus.cc:1240] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:27.768743 19808 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:27.786556 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:27.759214 19852 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:06:27.778762 15937 raft_consensus.cc:1240] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 10 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:06:27.802564 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:27.746531 19555 consensus_queue.cc:1059] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.043s
I20260709 13:06:27.747951 19758 consensus_queue.cc:1048] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:06:27.808827 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:27.754683 19263 consensus_queue.cc:1048] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:06:27.759002 19827 consensus_queue.cc:1048] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
W20260709 13:06:27.812650 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 108496us
I20260709 13:06:27.814910 16533 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:27.679143 19853 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:27.776990 19575 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:27.816967 15854 raft_consensus.cc:1275] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:06:27.817461 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:27.951056 15854 pending_rounds.cc:85] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8: Aborting all ops after (but not including) 7
I20260709 13:06:27.951333 15854 pending_rounds.cc:107] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:06:27.799973 19713 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:27.820771 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:27.775386 19558 consensus_queue.cc:1048] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:06:27.824265 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:27.799700 19767 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:27.825553 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 54920.9us
I20260709 13:06:27.825839 16562 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:27.826715 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:27.830380 16568 raft_consensus.cc:2410] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
W20260709 13:06:27.839130 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:27.847137 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:27.775776 19566 consensus_queue.cc:1048] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:06:27.802525 19820 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:27.807917 19568 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:27.815528 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:27.826483 19808 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:27.769615 19776 consensus_queue.cc:1048] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:06:27.951638 15854 raft_consensus.cc:2889] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:27.959519 15855 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:27.959403 15859 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:27.978055 15856 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
W20260709 13:06:27.947535 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 202287us
I20260709 13:06:27.985706 15902 leader_election.cc:400] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 8. Current term is 9.
I20260709 13:06:27.826673 19585 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.990096 15827 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:27.820792 19754 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:27.993321 15859 raft_consensus.cc:1275] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:06:27.993155 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:27.977787 19820 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:27.982419 19568 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:27.804450 19790 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:27.935366 19685 raft_consensus.cc:2749] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:28.014425 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:27.996182 15902 leader_election.cc:403] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:06:27.953403 19429 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.023200 16563 raft_consensus.cc:1275] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 9. (term mismatch)
I20260709 13:06:28.023615 16563 pending_rounds.cc:85] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting all ops after (but not including) 8
I20260709 13:06:28.023872 16563 pending_rounds.cc:107] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting uncommitted NO_OP operation due to leader change: 7.9
I20260709 13:06:28.024032 16563 raft_consensus.cc:2889] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:27.999497 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:28.026069 15855 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:27.960103 19733 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.959766 19372 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.003520 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:28.027769 15858 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:27.978835 19769 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.946614 19819 consensus_queue.cc:1048] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:06:28.007913 19418 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:28.038076 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:28.010819 19820 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:28.020957 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:28.019001 19641 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:28.008518 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:28.040865 19609 raft_consensus.cc:2749] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 8. Current term is 9.
I20260709 13:06:28.060335 19816 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:28.062364 19367 raft_consensus.cc:3055] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:28.062801 19367 raft_consensus.cc:740] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
W20260709 13:06:28.054222 19865 log.cc:927] Time spent T 00000000000000000000000000000000 P d33cebb189b14a989836214fdbafb898: Append to log took a long time: real 0.248s	user 0.001s	sys 0.000s
W20260709 13:06:28.054692 19866 log.cc:927] Time spent T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.232s	user 0.001s	sys 0.000s
I20260709 13:06:28.063761 19728 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:28.054951 19555 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:28.065773 19555 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:28.066314 15861 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:06:28.069463 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.070509 19808 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:28.072003 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:28.047168 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:28.076257 15932 raft_consensus.cc:1275] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:06:28.074590 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:28.077479 15937 raft_consensus.cc:1240] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:28.022089 19660 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.125465 19660 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:28.005460 19869 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.127051 19869 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:27.952874 19651 consensus_queue.cc:1048] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:06:28.018668 15857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2106) took 1390 ms. Trace:
W20260709 13:06:28.142956 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:28.138926 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 324085us
I20260709 13:06:28.142496 15857 rpcz_store.cc:276] 0709 13:06:26.628213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.633984 (+  5771us) service_pool.cc:224] Handling call
0709 13:06:26.634239 (+   255us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:26.732280 (+ 98041us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:06:26.766288 (+ 34008us) log.cc:844] Serialized 14 byte log entry
0709 13:06:27.516217 (+749929us) spinlock_profiling.cc:243] Waited 88.5 ms on lock 0x7b18003eefc0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f4823a 00007f73ea3b2a2a
0709 13:06:27.562051 (+ 45834us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:27.575542 (+ 13491us) log.cc:844] Serialized 26 byte log entry
0709 13:06:27.575793 (+   251us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:27.575858 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:27.575969 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:28.006488 (+430519us) raft_consensus.cc:1685] finished
0709 13:06:28.018436 (+ 11948us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:28.018602 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":23212,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":26107,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":11868,"spinlock_wait_cycles":203598208,"thread_start_us":143183,"threads_started":2,"wal-append.queue_time_us":855694}
I20260709 13:06:28.144381 16533 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:28.008582 19754 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.033008 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:06:28.005218 19585 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:28.011394 19749 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:28.152266 19814 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:28.016511 19806 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:28.018666 19748 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:28.073482 16010 raft_consensus.cc:1240] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 10. Current term is 11. Ops: []
W20260709 13:06:28.035027 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 49519.9us
I20260709 13:06:28.007876 19787 consensus_queue.cc:1059] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:06:28.072585 19808 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:28.175439 16562 raft_consensus.cc:2393] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 8: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
W20260709 13:06:28.081739 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.037411 19769 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:28.034440 19372 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:28.098289 19350 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:28.103730 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:28.104674 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:28.105058 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.185283 16566 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
W20260709 13:06:28.107996 16004 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41674 (request call id 1919) took 1890 ms (client timeout 1518 ms). Trace:
W20260709 13:06:28.186156 16004 rpcz_store.cc:269] 0709 13:06:26.218006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.218184 (+   178us) service_pool.cc:224] Handling call
0709 13:06:28.107811 (+1889627us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b54001dbd54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:28.107928 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":660088704}
I20260709 13:06:28.109930 15855 raft_consensus.cc:1240] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:28.116071 15858 raft_consensus.cc:1275] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:06:28.116345 19223 consensus_queue.cc:260] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:28.117682 15859 raft_consensus.cc:1275] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:06:28.119774 15974 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:28.122179 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:28.036648 16005 raft_consensus.cc:1275] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 6. (term mismatch)
I20260709 13:06:28.037083 19733 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:28.077641 19758 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.033707 19429 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:28.129987 19790 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:28.137822 19713 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:28.217473 19853 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:28.219151 19713 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:28.219269 19372 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:28.166947 19754 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:28.219846 19713 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:28.220479 19660 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:28.220396 19713 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:28.221439 19754 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:28.221868 15862 raft_consensus.cc:2368] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 8. Current term is 9.
I20260709 13:06:28.173396 19806 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:28.169865 19418 consensus_queue.cc:1059] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:06:28.223836 19806 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:28.224646 19429 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:28.225315 19869 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:28.225687 19769 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:28.225972 16568 raft_consensus.cc:1275] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:06:28.174816 19715 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:28.226876 19852 raft_consensus.cc:2749] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 9. Current term is 10.
I20260709 13:06:28.140321 16570 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:28.041041 19827 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.151293 19558 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:28.151527 19723 consensus_queue.cc:1048] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:28.151986 19844 consensus_queue.cc:1048] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:28.004529 19722 raft_consensus.cc:2804] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:06:28.155781 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:28.240521 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.167361 15857 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:06:28.169843 15861 raft_consensus.cc:1275] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:06:28.169963 15902 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:28.181386 19727 consensus_queue.cc:1048] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:06:28.177690 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:28.145026 19767 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.178231 15930 raft_consensus.cc:3055] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:28.245836 15930 raft_consensus.cc:740] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:28.245909 19767 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:06:28.236887 19827 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:28.246644 15930 consensus_queue.cc:260] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:28.247570 15930 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:28.178633 15856 raft_consensus.cc:2376] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 5. Re-sending same reply.
I20260709 13:06:28.152819 19814 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:28.213084 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:28.249117 15933 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 7.
I20260709 13:06:28.213147 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:28.250243 15932 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 9.
I20260709 13:06:28.216127 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:28.217226 19749 raft_consensus.cc:2804] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:28.182221 19821 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:28.073455 15928 raft_consensus.cc:1275] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 13:06:28.227409 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 107726us
I20260709 13:06:28.227700 16565 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:28.228925 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:28.215188 19685 consensus_queue.cc:1059] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:06:28.230475 16011 raft_consensus.cc:1240] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:28.233425 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:28.233355 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:28.234114 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:28.181093 19724 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.241890 15861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 712) took 1745 ms. Trace:
I20260709 13:06:28.272601 15861 rpcz_store.cc:276] 0709 13:06:26.496747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.496935 (+   188us) service_pool.cc:224] Handling call
0709 13:06:26.503258 (+  6323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:28.065976 (+1562718us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b58005594d0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3040aa 00007f73ea386447 00007f73ea39efe7 00007f73ea3a07de 00007f73ea3aab49 00007f73ea3ab416 00007f73ea3a7d8f 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4
0709 13:06:28.241637 (+175661us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:28.241837 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2533117696}
I20260709 13:06:28.171386 16533 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
W20260709 13:06:28.242576 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 221121us
W20260709 13:06:28.244915 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.248208 19767 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:28.216878 16005 pending_rounds.cc:85] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting all ops after (but not including) 5
I20260709 13:06:28.302927 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:28.303462 16005 pending_rounds.cc:107] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 6.6
I20260709 13:06:28.303853 16005 raft_consensus.cc:2889] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:28.235705 19660 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:28.266045 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:28.269446 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:28.242439 19722 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:28.274514 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 103190us
I20260709 13:06:28.256242 19575 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:28.287307 15855 raft_consensus.cc:2393] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 8: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:28.287051 15974 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:06:28.259900 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 174210us
I20260709 13:06:28.227026 19758 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:28.262730 19814 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:28.253777 19781 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 13:06:28.262121 19669 raft_consensus.cc:2749] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:28.289534 15930 raft_consensus.cc:1275] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:06:28.239001 19558 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:28.319639 15936 raft_consensus.cc:1240] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 11. Current term is 12. Ops: []
W20260709 13:06:28.657984 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 370983us
I20260709 13:06:28.659071 15974 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:06:28.653429 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 483549us
W20260709 13:06:28.653606 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 346227us
I20260709 13:06:28.660907 15928 raft_consensus.cc:1275] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:06:28.660674 15902 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:28.662806 15902 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:28.315429 15937 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
W20260709 13:06:28.663034 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:28.527407 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:28.462348 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.667285 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:28.668056 15928 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:28.671470 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:28.674100 15827 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:28.288950 19675 raft_consensus.cc:2804] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:28.678596 16533 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:28.704824 15902 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:28.705564 16533 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:28.672220 15933 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
W20260709 13:06:28.707782 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 45633.7us
I20260709 13:06:28.709205 15976 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:28.303776 19685 consensus_queue.cc:1243] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Peer 8430db80a39d4868bcf7641c16de35ea log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:06:28.304370 19769 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:28.405763 19878 log.cc:927] Time spent T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
W20260709 13:06:28.632772 19568 consensus_meta.cc:287] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.610s	user 0.006s	sys 0.005s
I20260709 13:06:28.406162 19767 consensus_queue.cc:1059] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.323s
I20260709 13:06:28.671191 19418 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.713351 16533 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:28.730762 19223 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:28.731267 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.289292 19724 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:28.307880 19722 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:28.416234 19852 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.432256 19713 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:28.736399 15928 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:06:28.671234 19831 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:28.712349 19754 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:28.707732 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 48777.5us
I20260709 13:06:28.776837 15974 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:28.721356 19877 raft_consensus.cc:2764] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:06:28.704627 19844 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.319775 19660 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:28.367115 19869 consensus_queue.cc:1048] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.370s
I20260709 13:06:28.728933 19669 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:28.707367 19675 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:28.433508 19733 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:28.432771 19585 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:28.743489 19350 consensus_meta.cc:287] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.533s	user 0.006s	sys 0.000s
I20260709 13:06:28.713182 19575 consensus_queue.cc:1059] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:06:28.737841 19568 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:28.757231 19758 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:28.759610 19876 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:28.782409 19821 consensus_meta.cc:287] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.510s	user 0.010s	sys 0.000s
I20260709 13:06:28.731877 19808 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:28.432127 19820 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:28.709645 19853 raft_consensus.cc:2749] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:28.824157 15934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2029) took 1126 ms. Trace:
I20260709 13:06:28.824570 15934 rpcz_store.cc:276] 0709 13:06:27.697398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:27.704113 (+  6715us) service_pool.cc:224] Handling call
0709 13:06:27.704520 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:27.733400 (+ 28880us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:27.733450 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:27.769322 (+ 35872us) log.cc:844] Serialized 26 byte log entry
0709 13:06:28.174306 (+404984us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:28.174352 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:28.174421 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:28.823936 (+649515us) raft_consensus.cc:1685] finished
0709 13:06:28.823961 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:28.824096 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11533,"raft.queue_time_us":106,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":44911,"thread_start_us":377882,"threads_started":1,"wal-append.queue_time_us":464645}
I20260709 13:06:28.671358 19790 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.826242 19790 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:28.797428 15860 consensus_meta.cc:287] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.560s	user 0.007s	sys 0.004s
I20260709 13:06:28.733675 19769 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:28.803295 19876 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:28.828286 19877 raft_consensus.cc:2749] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:28.828853 15935 raft_consensus.cc:2393] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 10: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:28.829638 19876 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:28.830157 19568 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:28.783150 19669 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:28.789429 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.807504 19558 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:28.819773 19776 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.785247 19754 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:28.803623 19808 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:28.788450 19861 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.830782 19669 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:28.835304 19776 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:28.835577 19861 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:28.836884 19808 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:28.837653 19754 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:28.838177 19651 consensus_queue.cc:1048] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:28.820794 19887 consensus_queue.cc:1048] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:06:28.821369 19585 raft_consensus.cc:2804] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:28.840456 19585 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:28.840867 19585 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:28.795310 19715 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:28.821655 19581 raft_consensus.cc:3055] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:06:28.842270 19581 raft_consensus.cc:740] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:28.822417 19883 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.822836 19660 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:28.767267 19418 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:28.825603 19820 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:28.783483 19675 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:28.809232 19724 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:28.833508 19733 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:28.842900 19758 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:28.912021 19758 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:28.912424 19758 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:28.772017 19852 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:28.914515 19882 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.915019 19882 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:28.916611 19733 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:28.917065 19733 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:28.833428 15860 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 10.
I20260709 13:06:28.899106 19555 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:28.901377 19827 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:28.902393 19418 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:28.906293 19874 log.cc:927] Time spent T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.729s	user 0.009s	sys 0.009s
I20260709 13:06:28.908401 19769 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:28.880537 19883 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:28.909860 19790 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:28.784752 19844 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:28.868439 19651 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:28.861903 19568 raft_consensus.cc:2804] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:28.920002 19581 consensus_queue.cc:260] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:28.938771 19883 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:28.869421 19263 consensus_queue.cc:1048] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:06:28.939354 19776 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:06:28.947567 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 242825us
I20260709 13:06:28.939477 19568 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:28.939618 19651 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:28.944370 19263 consensus_queue.cc:1243] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer 31882606e87447fb9a36bbf625129fe8 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:06:28.958374 15862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58214 (request call id 719) took 1235 ms. Trace:
W20260709 13:06:28.950538 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 237246us
I20260709 13:06:28.961508 16533 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:28.945519 19749 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.000s	sys 0.008s
I20260709 13:06:28.957695 15902 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:28.961112 15862 rpcz_store.cc:276] 0709 13:06:27.722704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:27.722941 (+   237us) service_pool.cc:224] Handling call
0709 13:06:28.953603 (+1230662us) spinlock_profiling.cc:243] Waited 480 ms on lock 0x7b540008b1d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:28.958308 (+  4705us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1102856448}
I20260709 13:06:28.932495 19814 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:28.966835 19776 raft_consensus.cc:2749] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:06:28.968164 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:28.971263 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 262131us
I20260709 13:06:28.976711 15976 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:28.978009 19418 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.977286 19883 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:28.972074 19749 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.000s	sys 0.009s
I20260709 13:06:28.977351 19820 consensus_queue.cc:1048] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:06:28.971807 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 297788us
I20260709 13:06:28.958948 19568 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:28.977488 15853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 720) took 1254 ms. Trace:
I20260709 13:06:28.962641 19651 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:28.978250 19724 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.974725 19666 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:28.985656 15902 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:28.989259 16533 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:28.963762 19719 consensus_queue.cc:1048] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:06:28.979302 19852 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:28.980415 19581 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:06:29.027705 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 249432us
I20260709 13:06:29.057281 15974 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:28.984455 15827 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:28.983954 15853 rpcz_store.cc:276] 0709 13:06:27.723094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:27.724221 (+  1127us) service_pool.cc:224] Handling call
0709 13:06:27.755824 (+ 31603us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:27.756195 (+   371us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:27.756234 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:27.756761 (+   527us) log.cc:844] Serialized 26 byte log entry
0709 13:06:27.962364 (+205603us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:27.962409 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:27.962487 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:28.905660 (+943173us) spinlock_profiling.cc:243] Waited 4.5 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:06:28.975423 (+ 69763us) raft_consensus.cc:1685] finished
0709 13:06:28.975443 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:28.977430 (+  1987us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3649,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":3522,"spinlock_wait_cycles":10349824,"thread_start_us":170330,"threads_started":1,"wal-append.queue_time_us":258788}
W20260709 13:06:28.994819 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:29.057951 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 68770.5us
I20260709 13:06:28.976002 19790 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.017s
W20260709 13:06:29.059506 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 389465us
I20260709 13:06:28.990423 19769 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.088599 15860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 2108) took 1302 ms. Trace:
I20260709 13:06:29.088956 15860 rpcz_store.cc:276] 0709 13:06:27.786115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:27.786311 (+   196us) service_pool.cc:224] Handling call
0709 13:06:29.088471 (+1302160us) spinlock_profiling.cc:243] Waited 415 ms on lock 0x7b540032e954. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:29.088555 (+    84us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":954304768}
I20260709 13:06:29.089933 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:29.090430 15860 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:29.091250 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:28.982194 19883 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:29.066584 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 81011.8us
I20260709 13:06:29.091816 15860 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:28.983302 19767 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:06:29.078205 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 93821.1us
I20260709 13:06:29.093906 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:29.023231 19861 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:29.072573 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:29.061014 19776 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:29.080200 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:29.112249 15934 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:29.101552 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:29.101933 15861 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:29.081100 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:29.075595 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:29.071120 15825 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:29.092423 15902 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:29.059930 19666 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:28.981092 19418 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:29.095794 15827 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:29.072041 15976 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:29.068704 19899 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:29.098147 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:29.099939 19894 consensus_queue.cc:1048] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:06:28.982084 19749 raft_consensus.cc:697] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 9 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:29.101173 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:29.080468 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:29.072098 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:29.115711 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 44739.4us
I20260709 13:06:29.110710 15858 raft_consensus.cc:1240] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:29.109583 16533 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:29.076821 19790 consensus_queue.cc:1243] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer 85e997ba5003439781ec4f1f8db0c3ee log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:06:29.163582 15862 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
W20260709 13:06:29.165596 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 69931.5us
I20260709 13:06:29.094880 19882 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:29.084043 19660 raft_consensus.cc:2804] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:29.008997 19724 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:29.131147 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 59185.8us
I20260709 13:06:29.193872 19891 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:29.204615 19891 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:29.117568 19769 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:29.132117 15974 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:29.206609 19891 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:29.008874 19575 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.207216 19575 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:29.193966 19660 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:29.208225 19660 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:29.209596 19575 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:29.210781 19844 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:29.211436 19844 consensus_queue.cc:1048] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:06:29.199611 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:29.213604 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:29.199895 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:29.200209 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:29.139343 19903 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:29.202167 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:29.215342 15936 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:06:29.076895 19827 consensus_queue.cc:1048] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:06:29.129784 19901 consensus_queue.cc:1048] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 13:06:29.140799 19418 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:29.163383 15855 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
I20260709 13:06:29.163383 15929 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 7.
W20260709 13:06:29.163530 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 54072.9us
I20260709 13:06:29.123808 15854 raft_consensus.cc:1275] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:06:29.168726 15900 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:29.164182 15825 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:29.171830 19892 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:29.100759 19900 raft_consensus.cc:2804] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:06:29.183809 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:29.111229 19713 consensus_queue.cc:1048] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:06:29.234673 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:29.235289 16569 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:29.235720 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:06:29.235826 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 71704.6us
I20260709 13:06:29.125770 19666 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:29.188563 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:29.190446 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:29.129650 15859 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:29.207520 19350 raft_consensus.cc:3055] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:29.138413 15902 leader_election.cc:400] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 10. Current term is 11.
I20260709 13:06:29.210654 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:29.026947 19669 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:29.201001 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:29.130313 19853 raft_consensus.cc:2804] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:29.201706 16566 raft_consensus.cc:1275] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:06:29.201954 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:29.245455 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 107231us
I20260709 13:06:29.203971 19904 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.195281 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:29.123924 19715 raft_consensus.cc:2804] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:29.214521 16008 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
W20260709 13:06:29.217458 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 85426.2us
I20260709 13:06:29.221480 19769 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:29.135582 19728 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.198472 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:29.216975 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:29.225024 19806 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:29.258621 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:29.264995 16569 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:29.265431 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:29.262457 15976 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:29.266780 15862 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
I20260709 13:06:29.264727 16567 raft_consensus.cc:2410] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:06:29.267113 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:29.267581 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:29.268818 15976 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:29.127017 19767 raft_consensus.cc:697] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 10 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:29.270934 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:29.271518 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:29.272231 15934 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:29.272123 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:29.272732 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:29.234840 19575 raft_consensus.cc:2749] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:29.273331 15931 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5.
W20260709 13:06:29.227861 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 59215.8us
I20260709 13:06:29.135339 19787 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.137720 19899 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:29.234122 15853 raft_consensus.cc:1240] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:29.239493 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:29.186553 19776 raft_consensus.cc:2804] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:29.249013 16570 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:06:29.213662 16564 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:06:29.245980 16004 raft_consensus.cc:2393] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 9: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:29.246711 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:29.203815 19724 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:29.222043 19903 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:29.248000 19891 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:29.274516 19899 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:29.269740 16535 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:29.227816 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:29.274545 19787 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:29.276022 15932 raft_consensus.cc:1240] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:06:29.234015 19900 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:29.278224 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:29.243944 19887 raft_consensus.cc:2804] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:29.279704 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:29.280726 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:29.280779 15900 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:29.239724 19902 consensus_queue.cc:1059] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:06:29.281329 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:29.244946 19350 raft_consensus.cc:740] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:29.253144 19781 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.287657 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:29.254236 16533 leader_election.cc:400] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 election: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 8. Current term is 9.
I20260709 13:06:29.368551 16004 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 7.
W20260709 13:06:29.368718 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 114603us
I20260709 13:06:29.318902 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:29.369472 15933 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:06:29.369501 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:29.324498 16563 raft_consensus.cc:2368] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 9. Current term is 10.
I20260709 13:06:29.370182 16004 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:06:29.271066 15862 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 7.
I20260709 13:06:29.275734 19719 consensus_queue.cc:1048] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:06:29.371063 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:29.371333 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:29.317632 19903 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:29.333526 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:29.334451 19558 raft_consensus.cc:3055] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:29.202191 19367 consensus_queue.cc:260] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:29.385967 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:29.336192 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:29.339841 19776 raft_consensus.cc:697] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:29.275652 19641 raft_consensus.cc:2804] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:29.270298 19852 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.352258 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:29.405167 15935 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:06:29.407954 19852 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:29.354125 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 73408.6us
I20260709 13:06:29.411105 19568 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:29.365396 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:29.282524 15902 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:29.275853 19853 raft_consensus.cc:697] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:29.268726 19904 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:29.372012 15862 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:06:29.376569 19781 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:29.376575 19877 raft_consensus.cc:2804] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:29.377696 19350 consensus_queue.cc:260] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:29.439013 19350 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:29.378786 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:29.380826 15976 leader_election.cc:400] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 9. Current term is 10.
I20260709 13:06:29.440451 15976 leader_election.cc:403] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:06:29.440074 19853 consensus_queue.cc:237] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
W20260709 13:06:29.441094 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 60427.1us
I20260709 13:06:29.386078 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:29.441793 16008 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 12.
I20260709 13:06:29.413757 19719 consensus_queue.cc:1048] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 13:06:29.442703 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:29.440074 16566 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:29.443271 16008 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:29.443486 19719 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:29.398802 19641 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:29.444084 19904 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:29.444219 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:29.276934 19728 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:29.386569 19558 raft_consensus.cc:740] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:29.391616 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:29.386852 15825 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:29.396171 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:29.398671 16011 raft_consensus.cc:1275] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:06:29.463075 16004 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
W20260709 13:06:29.463184 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 76404.7us
I20260709 13:06:29.411944 19896 consensus_queue.cc:1048] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:06:29.413955 19722 consensus_meta.cc:287] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.638s	user 0.006s	sys 0.002s
W20260709 13:06:29.366883 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 97238.7us
I20260709 13:06:29.469420 15825 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:29.410866 16533 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:29.364676 19575 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:29.495221 15931 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:06:29.277458 19806 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:29.352083 19900 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:29.373908 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:29.498100 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:29.498701 15937 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
I20260709 13:06:29.278399 19890 consensus_queue.cc:1059] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.121s
I20260709 13:06:29.431188 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:29.444187 19641 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:29.291965 19894 consensus_queue.cc:1048] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.108s
I20260709 13:06:29.382282 19887 raft_consensus.cc:697] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 9 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:29.321105 19861 consensus_queue.cc:1059] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.085s
I20260709 13:06:29.438161 19883 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:29.441162 19555 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:29.430451 19767 consensus_queue.cc:237] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:29.287006 19893 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:29.529816 19893 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:29.530210 19893 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:29.530006 19903 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:29.416802 19675 consensus_meta.cc:287] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.507s	user 0.011s	sys 0.000s
I20260709 13:06:29.386970 16570 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:29.291603 19814 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:29.444967 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:29.354892 19891 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:29.281641 19749 consensus_queue.cc:237] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:29.270406 19790 consensus_queue.cc:1048] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.094s
I20260709 13:06:29.466454 19558 consensus_queue.cc:260] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:29.492877 19669 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:29.499549 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:29.473809 16535 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:29.525683 19429 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:29.501132 15825 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:06:29.564702 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 63666.6us
I20260709 13:06:29.439297 19781 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:29.515010 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:29.474684 19904 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:06:29.435866 19877 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:29.526371 16009 raft_consensus.cc:2376] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10. Re-sending same reply.
I20260709 13:06:29.536764 15934 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:06:29.468236 19827 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:29.515894 19585 consensus_meta.cc:287] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.675s	user 0.011s	sys 0.000s
I20260709 13:06:29.574682 15934 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:58226
I20260709 13:06:29.549292 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:29.525328 19882 raft_consensus.cc:2804] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:06:29.559374 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 85653.7us
I20260709 13:06:29.570035 19883 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:29.583981 19883 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:29.576145 16566 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:29.431185 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:29.553445 19814 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:06:29.566802 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 156283us
I20260709 13:06:29.528883 19887 consensus_queue.cc:237] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:29.553109 19733 consensus_meta.cc:287] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.636s	user 0.005s	sys 0.007s
I20260709 13:06:29.536849 19890 raft_consensus.cc:3055] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:29.568449 15857 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:06:29.571725 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:29.514123 19722 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:29.515184 19902 raft_consensus.cc:2804] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:29.575919 15934 raft_consensus.cc:606] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Received request to transfer leadership
W20260709 13:06:29.558427 19918 log.cc:927] Time spent T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:06:29.577311 15825 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:29.570631 19429 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:29.553141 19776 consensus_queue.cc:237] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:29.581095 19808 consensus_meta.cc:287] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.744s	user 0.008s	sys 0.002s
I20260709 13:06:29.582463 19728 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:29.575186 19877 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:29.560143 15937 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
W20260709 13:06:29.594177 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:29.554709 19660 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:29.620592 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:29.536187 19675 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:29.555030 19568 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:29.573629 19904 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:29.626446 15934 raft_consensus.cc:614] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:29.607002 19890 raft_consensus.cc:740] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:06:29.626984 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 49803.8us
I20260709 13:06:29.627921 19769 raft_consensus.cc:2804] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:29.630062 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:29.636938 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:29.642635 19844 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.644829 16568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 881) took 1650 ms. Trace:
I20260709 13:06:29.645574 19848 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.647964 19882 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:06:29.655403 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 238247us
W20260709 13:06:29.655804 15860 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.009s	sys 0.005s
I20260709 13:06:29.657310 15928 raft_consensus.cc:1240] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:29.604240 19883 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:29.662710 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:29.674638 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:29.604656 19814 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:29.604740 19585 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:29.604671 19827 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:30.022511 15854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 1982) took 2014 ms. Trace:
I20260709 13:06:29.795987 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:30.202196 15858 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:30.214103 15854 rpcz_store.cc:276] 0709 13:06:28.008394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:28.009957 (+  1563us) service_pool.cc:224] Handling call
0709 13:06:28.021872 (+ 11915us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:29.227113 (+1205241us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:30.022171 (+795058us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400218254. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:30.022430 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2533389056}
I20260709 13:06:29.625856 19902 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:29.914852 15900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.000s	sys 0.013s
W20260709 13:06:30.229904 15900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.816s	user 0.001s	sys 0.025s
W20260709 13:06:30.230300 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 816366us
I20260709 13:06:29.664345 19781 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:30.202730 16568 rpcz_store.cc:276] 0709 13:06:27.994679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:27.995961 (+  1282us) service_pool.cc:224] Handling call
0709 13:06:28.009920 (+ 13959us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:28.226353 (+216433us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:29.644507 (+1418154us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b540059c9d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:29.644754 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":496037632}
I20260709 13:06:30.213434 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:30.237593 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:30.214818 15827 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.934s	user 0.012s	sys 0.009s
I20260709 13:06:30.215919 15862 raft_consensus.cc:2468] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
I20260709 13:06:30.214658 15855 raft_consensus.cc:2410] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 9 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:06:30.218482 15857 raft_consensus.cc:2393] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 10: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:06:30.218518 15861 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.007s	sys 0.009s
W20260709 13:06:29.795040 19651 consensus_meta.cc:287] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.809s	user 0.008s	sys 0.002s
W20260709 13:06:30.156458 16535 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.012s	sys 0.001s
W20260709 13:06:30.248211 16535 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.015s	sys 0.001s
W20260709 13:06:30.248695 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 664544us
I20260709 13:06:30.249526 16535 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:30.022315 15902 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.000s	sys 0.008s
W20260709 13:06:30.224371 15974 leader_election.cc:336] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 1.771s (SENT)
W20260709 13:06:30.251398 15902 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.837s	user 0.006s	sys 0.009s
I20260709 13:06:30.251497 15974 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:30.147780 19350 consensus_meta.cc:287] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.708s	user 0.009s	sys 0.000s
W20260709 13:06:29.794144 19920 log.cc:927] Time spent T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
W20260709 13:06:30.240106 15827 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.960s	user 0.012s	sys 0.009s
W20260709 13:06:30.256989 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 976749us
I20260709 13:06:30.146107 19787 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:30.111552 19853 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.062520 19575 raft_consensus.cc:3055] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:06:29.808593 19581 consensus_meta.cc:287] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.746s	user 0.013s	sys 0.002s
W20260709 13:06:29.811093 19758 consensus_meta.cc:287] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.898s	user 0.008s	sys 0.004s
W20260709 13:06:30.253103 15826 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.000s	sys 0.031s
I20260709 13:06:30.253892 15974 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:06:29.854221 19911 log.cc:927] Time spent T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.358s	user 0.001s	sys 0.000s
I20260709 13:06:30.258843 19869 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:30.257071 16561 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:06:30.089162 19806 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:30.213028 19904 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:30.231775 19848 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:29.621783 19733 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:30.257854 15827 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:30.218618 15860 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.090s	user 0.010s	sys 0.018s
W20260709 13:06:30.251978 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 969445us
I20260709 13:06:30.199923 19769 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:29.839175 19813 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.212018 19814 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:30.220624 19882 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:30.239972 19902 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:30.227088 19903 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:30.250677 16535 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:30.342998 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 92433.9us
W20260709 13:06:30.273761 16533 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.006s	sys 0.014s
W20260709 13:06:30.344589 16533 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.006s	sys 0.014s
W20260709 13:06:30.287107 15826 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.000s	sys 0.032s
I20260709 13:06:30.284881 16003 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 5.
I20260709 13:06:29.859447 19933 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.673954 19808 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:29.796371 19728 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:30.308983 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 55153.1us
I20260709 13:06:30.261780 19568 raft_consensus.cc:2749] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:30.231987 19844 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:30.271765 19575 raft_consensus.cc:740] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:30.285187 19891 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:30.260434 19651 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:30.389676 19651 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:30.319911 19869 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:30.340170 15975 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.970s	user 0.023s	sys 0.014s
W20260709 13:06:30.390614 15975 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.020s	user 0.023s	sys 0.014s
I20260709 13:06:30.335265 19769 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:30.395916 19722 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:30.396144 19651 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.321015 19787 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:30.382622 19728 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:30.390540 19869 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:30.321422 19585 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:30.398955 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 55009.7us
I20260709 13:06:30.400182 16533 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:30.304504 19758 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:30.405678 15974 leader_election.cc:336] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 1.925s (SENT)
I20260709 13:06:30.286902 19853 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:30.424615 15853 raft_consensus.cc:2410] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:06:30.424893 15974 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:30.424357 15856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2119) took 1354 ms. Trace:
I20260709 13:06:30.399752 19806 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:30.425967 15856 rpcz_store.cc:276] 0709 13:06:29.069690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:29.072786 (+  3096us) service_pool.cc:224] Handling call
0709 13:06:29.073228 (+   442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:29.078214 (+  4986us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:29.078264 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:29.096536 (+ 18272us) log.cc:844] Serialized 26 byte log entry
0709 13:06:29.270940 (+174404us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:29.270977 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:29.271040 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:30.424096 (+1153056us) raft_consensus.cc:1685] finished
0709 13:06:30.424123 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:30.424285 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4431,"raft.queue_time_us":8403,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":17797,"thread_start_us":147705,"threads_started":1,"wal-append.queue_time_us":318131}
W20260709 13:06:30.410542 19555 consensus_meta.cc:287] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.882s	user 0.009s	sys 0.001s
I20260709 13:06:30.309216 19890 consensus_queue.cc:260] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:30.457509 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 57433.4us
I20260709 13:06:30.341754 19813 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:30.427127 19920 log.cc:927] Time spent T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:06:30.401544 19787 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:30.484364 16533 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:30.396621 19651 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:30.406486 19722 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:30.429639 19891 raft_consensus.cc:2749] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:30.434654 19806 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:30.427063 19429 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:30.484532 19776 raft_consensus.cc:2749] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:30.430600 19585 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.399425 19350 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 13:06:30.489542 19555 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 9. Current term is 10.
I20260709 13:06:30.508821 16007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 704) took 1304 ms. Trace:
I20260709 13:06:30.512993 16007 rpcz_store.cc:276] 0709 13:06:29.204679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:29.215128 (+ 10449us) service_pool.cc:224] Handling call
0709 13:06:29.226340 (+ 11212us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:29.234208 (+  7868us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:29.234252 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:29.243846 (+  9594us) log.cc:844] Serialized 26 byte log entry
0709 13:06:29.574637 (+330791us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:29.621165 (+ 46528us) log.cc:844] Serialized 14 byte log entry
0709 13:06:29.621452 (+   287us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:29.621550 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:30.508559 (+887009us) raft_consensus.cc:1685] finished
0709 13:06:30.508587 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:30.508751 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":247602,"prepare.queue_time_us":49916,"prepare.run_cpu_time_us":4076,"prepare.run_wall_time_us":8037,"raft.queue_time_us":13365,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":7626,"spinlock_wait_cycles":88085376,"thread_start_us":291516,"threads_started":1,"wal-append.queue_time_us":399529}
W20260709 13:06:30.485076 19900 consensus_meta.cc:287] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.951s	user 0.007s	sys 0.004s
W20260709 13:06:30.426669 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 195615us
I20260709 13:06:30.517760 15900 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:30.497290 19787 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:30.520324 15860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2125) took 1448 ms. Trace:
I20260709 13:06:30.521274 15857 pending_rounds.cc:85] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8: Aborting all ops after (but not including) 6
I20260709 13:06:30.503768 19806 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:30.551539 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 293771us
I20260709 13:06:30.522374 15857 pending_rounds.cc:107] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:06:30.546005 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:30.532780 16010 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
W20260709 13:06:30.523439 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 196195us
I20260709 13:06:30.581622 15857 raft_consensus.cc:2889] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:30.523332 15900 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:06:30.541309 16009 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.745s	user 0.012s	sys 0.023s
W20260709 13:06:30.582511 16009 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.787s	user 0.012s	sys 0.023s
W20260709 13:06:30.532289 15859 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 2121) took 1462 ms (client timeout 1742 ms). Trace:
I20260709 13:06:30.536851 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:30.545409 16564 raft_consensus.cc:1240] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:30.583201 15827 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:30.588286 16564 raft_consensus.cc:1275] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:06:30.584074 15856 raft_consensus.cc:1240] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 6. Current term is 7. Ops: []
W20260709 13:06:30.583088 15859 rpcz_store.cc:269] 0709 13:06:29.070372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:29.082398 (+ 12026us) service_pool.cc:224] Handling call
0709 13:06:30.532063 (+1449665us) spinlock_profiling.cc:243] Waited 16.1 ms on lock 0x7b54003b6d54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:30.532212 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37055744}
I20260709 13:06:30.540796 16565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45246 (request call id 866) took 1352 ms. Trace:
I20260709 13:06:30.591372 16565 rpcz_store.cc:276] 0709 13:06:29.188144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:29.188306 (+   162us) service_pool.cc:224] Handling call
0709 13:06:30.540570 (+1352264us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54006a5754. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:30.540729 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2693905792}
I20260709 13:06:30.544677 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:30.592831 16569 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:30.522477 15860 rpcz_store.cc:276] 0709 13:06:29.071667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:29.107581 (+ 35914us) service_pool.cc:224] Handling call
0709 13:06:29.109130 (+  1549us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:29.109843 (+   713us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:30.325725 (+1215882us) log.cc:844] Serialized 14 byte log entry
0709 13:06:30.519507 (+193782us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:30.519632 (+   125us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:30.519730 (+    98us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:30.519907 (+   177us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:30.520205 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7018,"prepare.queue_time_us":334651,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"thread_start_us":1373618,"threads_started":2}
I20260709 13:06:30.535557 15928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2040) took 1278 ms. Trace:
I20260709 13:06:30.595464 15928 rpcz_store.cc:276] 0709 13:06:29.257066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:29.257270 (+   204us) service_pool.cc:224] Handling call
0709 13:06:29.257571 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:30.219947 (+962376us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:30.535281 (+315334us) spinlock_profiling.cc:243] Waited 399 ms on lock 0x7b54002c1854. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:30.535491 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":918792448}
I20260709 13:06:30.596897 16562 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:30.402467 19933 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:30.524812 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:30.600028 19844 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:30.541297 19919 log.cc:927] Time spent T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.528s	user 0.001s	sys 0.000s
I20260709 13:06:30.515825 19722 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:30.601006 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:30.610108 16568 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:06:30.575649 19916 log.cc:927] Time spent T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:06:30.517624 19675 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:30.550783 19827 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:30.517299 19660 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:30.573968 19923 log.cc:927] Time spent T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 13:06:30.511023 19781 raft_consensus.cc:697] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 9 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:30.610518 19223 raft_consensus.cc:2749] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:30.525502 19852 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:30.521704 19939 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.316s
I20260709 13:06:30.490964 19575 consensus_queue.cc:260] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:30.521893 19940 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.620569 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:30.518339 19585 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:30.600975 19941 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:30.541206 19944 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:30.611361 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:30.541255 19900 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:30.693010 16006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 702) took 1488 ms. Trace:
I20260709 13:06:30.607424 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:30.693395 16006 rpcz_store.cc:276] 0709 13:06:29.204122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:29.212816 (+  8694us) service_pool.cc:224] Handling call
0709 13:06:29.213083 (+   267us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:29.243904 (+ 30821us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:29.243937 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:29.248817 (+  4880us) log.cc:844] Serialized 26 byte log entry
0709 13:06:29.497573 (+248756us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:29.497624 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:29.497724 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:30.692792 (+1195068us) raft_consensus.cc:1685] finished
0709 13:06:30.692815 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:30.692955 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20488,"raft.queue_time_us":20661,"spinlock_wait_cycles":17337088,"thread_start_us":207606,"threads_started":1,"wal-append.queue_time_us":313212}
I20260709 13:06:30.694602 15860 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 10.
I20260709 13:06:30.671433 15856 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:30.689898 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 106748us
I20260709 13:06:30.661749 16562 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 10.
I20260709 13:06:30.698900 15827 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:30.709092 15862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 1998) took 1491 ms. Trace:
I20260709 13:06:30.709668 15862 rpcz_store.cc:276] 0709 13:06:29.217431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:29.620255 (+402824us) service_pool.cc:224] Handling call
0709 13:06:30.708857 (+1088602us) spinlock_profiling.cc:243] Waited 2.79 ms on lock 0x7b5400426dd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:30.709011 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6425344}
W20260709 13:06:30.712224 15936 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53636 (request call id 782) took 1630 ms (client timeout 1718 ms). Trace:
W20260709 13:06:30.712798 15936 rpcz_store.cc:269] 0709 13:06:29.081847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:29.081967 (+   120us) service_pool.cc:224] Handling call
0709 13:06:30.712026 (+1630059us) spinlock_profiling.cc:243] Waited 77.3 ms on lock 0x7b540044cad4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:30.712150 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":177783680}
I20260709 13:06:30.740486 19558 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:30.745426 16003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41674 (request call id 1939) took 1527 ms. Trace:
I20260709 13:06:30.746002 16003 rpcz_store.cc:276] 0709 13:06:29.218152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:29.430867 (+212715us) service_pool.cc:224] Handling call
0709 13:06:30.745174 (+1314307us) spinlock_profiling.cc:243] Waited 770 ms on lock 0x7b54000ee7d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:30.745337 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1771562880}
I20260709 13:06:30.751509 15929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2037) took 1514 ms. Trace:
I20260709 13:06:30.752079 15929 rpcz_store.cc:276] 0709 13:06:29.237320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:29.237465 (+   145us) service_pool.cc:224] Handling call
0709 13:06:29.237768 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:29.238087 (+   319us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:29.238124 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:29.240432 (+  2308us) log.cc:844] Serialized 26 byte log entry
0709 13:06:29.654389 (+413957us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:29.654725 (+   336us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:29.655060 (+   335us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:30.751218 (+1096158us) raft_consensus.cc:1685] finished
0709 13:06:30.751247 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:30.751429 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2623,"raft.queue_time_us":510,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":8722,"thread_start_us":374836,"threads_started":1,"wal-append.queue_time_us":1080592}
I20260709 13:06:30.659660 19917 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:30.681380 19733 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:30.817193 16010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60156 (request call id 706) took 1612 ms. Trace:
I20260709 13:06:30.861079 16010 rpcz_store.cc:276] 0709 13:06:29.205211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:29.395836 (+190625us) service_pool.cc:224] Handling call
0709 13:06:30.816995 (+1421159us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400304654. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:30.817124 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2440263936}
I20260709 13:06:30.785768 19758 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:30.825296 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:30.749260 19675 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.684347 19941 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:30.791304 19946 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:30.811148 15976 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:30.859663 19575 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:30.859186 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:30.813951 15858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 732) took 1542 ms. Trace:
W20260709 13:06:30.862025 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 338741us
I20260709 13:06:30.680490 19350 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:06:30.757619 19948 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.192s
I20260709 13:06:30.819554 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:30.781503 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 82667.3us
W20260709 13:06:30.834717 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 252109us
I20260709 13:06:30.880882 19948 consensus_queue.cc:1243] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer eb1e0493af1e46dbb6dec7e25525bab5 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:06:30.785540 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 701) took 1581 ms. Trace:
W20260709 13:06:30.805579 19950 log.cc:927] Time spent T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
I20260709 13:06:30.690887 19660 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:30.815832 19892 consensus_meta.cc:287] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 1.567s	user 0.008s	sys 0.004s
W20260709 13:06:30.865707 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 54675.6us
I20260709 13:06:30.986265 15976 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:30.780295 19790 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:30.758806 19808 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:30.842588 19813 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:30.811148 19651 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:30.863346 19853 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:30.865214 19946 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:30.864843 19941 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:31.011763 19882 consensus_meta.cc:287] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.657s	user 0.019s	sys 0.000s
I20260709 13:06:30.863456 19933 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:30.865653 19899 consensus_meta.cc:287] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 1.497s	user 0.007s	sys 0.005s
W20260709 13:06:30.963207 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:30.963698 15903 thread.cc:641] server-negotiator [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.005s	sys 0.005s
W20260709 13:06:31.040474 15903 thread.cc:608] server-negotiator [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.005s	sys 0.005s
I20260709 13:06:31.042097 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:30.684336 19940 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:30.881685 16002 rpcz_store.cc:276] 0709 13:06:29.203853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:29.211317 (+  7464us) service_pool.cc:224] Handling call
0709 13:06:29.211718 (+   401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:29.211942 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:29.211968 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:29.212412 (+   444us) log.cc:844] Serialized 26 byte log entry
0709 13:06:29.656251 (+443839us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:29.656298 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:29.656364 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:30.785315 (+1128951us) raft_consensus.cc:1685] finished
0709 13:06:30.785337 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:30.785478 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":443472,"threads_started":1,"wal-append.queue_time_us":875703}
I20260709 13:06:30.869310 15858 rpcz_store.cc:276] 0709 13:06:29.271785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.215447 (+943662us) service_pool.cc:224] Handling call
0709 13:06:30.215829 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:30.342024 (+126195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:30.342076 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:30.342645 (+   569us) log.cc:844] Serialized 26 byte log entry
0709 13:06:30.643859 (+301214us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:30.643885 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:30.643930 (+    45us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:30.813693 (+169763us) raft_consensus.cc:1685] finished
0709 13:06:30.813721 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:30.813880 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34599,"raft.queue_time_us":303380,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":8790,"thread_start_us":357703,"threads_started":2,"wal-append.queue_time_us":412572}
I20260709 13:06:30.765731 19372 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:31.046186 16007 raft_consensus.cc:1240] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:31.045850 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:31.047032 15858 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:30.905460 19953 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:30.883327 19954 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:31.050482 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:30.785938 19959 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.865897 19733 raft_consensus.cc:2804] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:30.863471 15862 raft_consensus.cc:1275] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:06:31.061410 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:31.062258 15862 raft_consensus.cc:2410] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 13:06:30.961016 16570 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.687s	user 0.001s	sys 0.008s
W20260709 13:06:31.062687 16570 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.789s	user 0.001s	sys 0.009s
W20260709 13:06:30.979009 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:30.979350 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 390282us
I20260709 13:06:30.980688 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:31.065003 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:31.065878 16562 raft_consensus.cc:2393] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 10: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
W20260709 13:06:30.981549 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 497223us
I20260709 13:06:30.984902 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:31.066939 16569 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:06:30.987095 15976 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:31.067085 16533 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:31.067886 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 80864.2us
W20260709 13:06:31.037305 19641 consensus_meta.cc:287] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 1.500s	user 0.013s	sys 0.002s
I20260709 13:06:31.069376 15976 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:31.069273 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:31.070286 15976 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:30.759076 19944 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:30.997347 15856 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
W20260709 13:06:30.998602 15900 connection.cc:256] RPC call timeout handler was delayed by -1.3089108579999902s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1648s for a response
I20260709 13:06:31.080448 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:31.001103 16566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 893) took 1767 ms. Trace:
W20260709 13:06:31.004324 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:31.082711 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:31.083150 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:31.083451 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:31.084203 16006 raft_consensus.cc:2393] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 9: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:31.005098 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:30.969786 15825 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:30.788475 19894 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:06:31.017937 15827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:31.087142 15827 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:30.863885 19722 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:31.023335 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:30.976841 19827 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:31.011253 19883 consensus_meta.cc:287] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.764s	user 0.015s	sys 0.005s
I20260709 13:06:30.856158 19367 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:31.012383 19893 consensus_meta.cc:287] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 1.476s	user 0.020s	sys 0.004s
I20260709 13:06:30.971223 19350 raft_consensus.cc:697] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 5 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:31.002271 19892 raft_consensus.cc:2749] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:31.002457 19790 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:31.002246 19685 raft_consensus.cc:3055] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:31.003544 19749 raft_consensus.cc:2804] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:31.037395 16009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1949) took 1668 ms. Trace:
I20260709 13:06:30.864630 19675 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:30.864475 16567 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:30.973376 16568 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:31.031797 19715 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:30.749599 19852 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:31.053234 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:31.172623 16003 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
I20260709 13:06:30.865375 19917 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:31.084373 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:30.791805 19781 consensus_queue.cc:237] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:31.088233 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:31.030625 19946 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:31.186955 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:31.187273 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:31.092445 16533 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:31.092680 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:31.148823 19820 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:31.097587 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:31.091368 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:31.156440 19790 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:31.094592 16564 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:06:31.099293 15827 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
W20260709 13:06:31.194412 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 95212.7us
I20260709 13:06:31.100183 19758 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:31.001799 19660 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:31.109445 15858 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 10.
I20260709 13:06:31.111248 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:31.089650 16566 rpcz_store.cc:276] 0709 13:06:29.233465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:29.606175 (+372710us) service_pool.cc:224] Handling call
0709 13:06:29.664547 (+ 58372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:29.664780 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:29.666209 (+  1429us) log.cc:844] Serialized 14 byte log entry
0709 13:06:30.533744 (+867535us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b1800380680. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f4823a 00007f73ea3b2a2a
0709 13:06:30.695191 (+161447us) log.cc:844] Serialized 14 byte log entry
0709 13:06:30.985251 (+290060us) spinlock_profiling.cc:243] Waited 5.84 ms on lock 0x7b1800380680. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72892a6 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb3f3 00007f73e72bd7c1
0709 13:06:30.999726 (+ 14475us) log.cc:844] Serialized 14 byte log entry
0709 13:06:31.000521 (+   795us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:31.000647 (+   126us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:31.000766 (+   119us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:31.000859 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:31.001017 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23190,"prepare.queue_time_us":9866,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":1663,"raft.run_cpu_time_us":300,"raft.run_wall_time_us":47115,"spinlock_wait_cycles":215902592,"thread_start_us":157793,"threads_started":1,"wal-append.queue_time_us":1155986}
I20260709 13:06:31.051569 19933 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.127908 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:31.111968 19883 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:31.231142 16567 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:31.060498 19733 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:31.145315 15859 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:31.147816 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:31.092072 19894 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:31.045707 19882 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:31.060132 19953 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:31.163409 16009 rpcz_store.cc:276] 0709 13:06:29.369208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:29.652539 (+283331us) service_pool.cc:224] Handling call
0709 13:06:29.652797 (+   258us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:29.653363 (+   566us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:06:29.673713 (+ 20350us) log.cc:844] Serialized 14 byte log entry
0709 13:06:30.591013 (+917300us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:30.591515 (+   502us) log.cc:844] Serialized 26 byte log entry
0709 13:06:30.591702 (+   187us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:30.591749 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:30.591834 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:31.037157 (+445323us) raft_consensus.cc:1685] finished
0709 13:06:31.037177 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:31.037325 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":109733,"prepare.queue_time_us":22835,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":9208,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":3640,"thread_start_us":786868,"threads_started":1,"wal-append.queue_time_us":1028264}
I20260709 13:06:31.060071 19372 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:31.100396 19955 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.092823 19844 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.050325 19899 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:31.230772 19892 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:31.232400 19723 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:31.140962 19971 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:31.294625 19921 log.cc:927] Time spent T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 1.028s	user 0.003s	sys 0.002s
I20260709 13:06:31.294500 19971 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:31.295543 15856 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:31.183194 16004 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
W20260709 13:06:31.093912 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 215277us
I20260709 13:06:31.188508 16006 raft_consensus.cc:2393] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 10: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:31.186955 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:31.300820 15825 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:31.300905 16568 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:06:31.101011 19429 consensus_queue.cc:1059] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 13:06:31.140589 19922 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:31.101207 19651 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 13:06:31.186527 19852 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:31.101902 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:31.309056 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 113768us
I20260709 13:06:31.140708 19893 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:31.163334 19749 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:31.165669 19961 raft_consensus.cc:2749] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:31.100735 19641 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:31.160607 19685 raft_consensus.cc:740] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:31.127556 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:31.311605 19877 consensus_meta.cc:287] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 1.517s	user 0.015s	sys 0.005s
I20260709 13:06:31.059978 19954 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:31.238353 19900 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:31.106082 19827 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:31.114167 19904 consensus_meta.cc:287] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.779s	user 0.007s	sys 0.006s
I20260709 13:06:31.248677 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:31.069260 19959 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:31.250087 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:31.327117 19749 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:31.230551 19758 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:31.278780 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 186408us
I20260709 13:06:31.113000 19585 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:31.282949 15862 raft_consensus.cc:2393] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 11: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:31.245395 19733 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:31.284046 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:31.284279 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:31.249099 19951 log.cc:927] Time spent T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 13:06:31.157104 19666 consensus_meta.cc:287] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 1.893s	user 0.010s	sys 0.003s
I20260709 13:06:31.284904 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:31.248430 19894 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:31.289134 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:31.283244 19955 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:31.292742 19972 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:06:31.293550 19965 log.cc:927] Time spent T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
W20260709 13:06:31.294063 19957 log.cc:927] Time spent T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:06:31.303551 15856 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 10.
I20260709 13:06:31.309077 16565 raft_consensus.cc:2468] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
I20260709 13:06:31.223420 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:31.311247 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:31.312227 16005 raft_consensus.cc:2393] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 8: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:31.311247 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:06:31.316435 19962 log.cc:927] Time spent T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
I20260709 13:06:31.316814 15853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58214 (request call id 734) took 2030 ms. Trace:
I20260709 13:06:31.317119 19975 raft_consensus.cc:2749] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:06:31.318085 19973 log.cc:927] Time spent T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:06:31.128271 19897 consensus_queue.cc:1048] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.081s
I20260709 13:06:31.229596 19820 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:31.279382 19982 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:31.284879 19892 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:31.308871 19814 consensus_meta.cc:287] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.962s	user 0.009s	sys 0.008s
W20260709 13:06:31.335543 16561 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45246 (request call id 872) took 2136 ms (client timeout 2218 ms). Trace:
I20260709 13:06:31.319630 19882 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:31.172878 19715 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:31.308166 19787 consensus_meta.cc:287] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.735s	user 0.012s	sys 0.000s
W20260709 13:06:31.314260 19869 consensus_meta.cc:287] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.885s	user 0.012s	sys 0.001s
I20260709 13:06:31.345111 16010 raft_consensus.cc:2393] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 10: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:31.312100 19922 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:31.375232 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:31.375486 15827 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:31.454743 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 79341.8us
I20260709 13:06:31.455408 15827 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:31.456931 15827 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:31.247965 19933 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:31.251521 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:31.458261 15858 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
I20260709 13:06:31.287786 19844 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:31.392393 16533 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:31.391852 16008 raft_consensus.cc:1240] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:06:31.392730 16567 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
W20260709 13:06:31.394579 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 291384us
I20260709 13:06:31.394697 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:31.463430 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 71118.1us
I20260709 13:06:31.404513 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:31.465780 16533 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:31.326453 19713 consensus_queue.cc:1059] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:06:31.366145 19948 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.491623 15932 raft_consensus.cc:2376] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7. Re-sending same reply.
I20260709 13:06:31.372426 19888 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:31.406036 15931 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:31.481165 15861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2127) took 2406 ms. Trace:
I20260709 13:06:31.373682 19877 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:31.445078 19892 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:31.384732 19900 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:31.427050 16568 raft_consensus.cc:2393] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 9: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:31.427474 15853 rpcz_store.cc:276] 0709 13:06:29.286647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.066218 (+779571us) service_pool.cc:224] Handling call
0709 13:06:31.316604 (+1250386us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b5400218254. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:31.316737 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":428658432}
I20260709 13:06:31.410280 19949 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:06:31.424917 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 340653us
I20260709 13:06:31.382644 19904 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:31.529626 15976 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:31.425752 19978 log.cc:927] Time spent T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.285s	user 0.001s	sys 0.000s
W20260709 13:06:31.445843 16561 rpcz_store.cc:269] 0709 13:06:29.199006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:29.199697 (+   691us) service_pool.cc:224] Handling call
0709 13:06:31.335285 (+2135588us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54000f6ed4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:31.335463 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2397387136}
I20260709 13:06:31.423067 19972 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:31.490290 19893 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:31.451864 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:31.558116 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:31.465775 15937 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:31.468596 16566 raft_consensus.cc:2410] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:06:31.517004 19813 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:31.496322 15974 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:06:31.490486 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:31.308508 19350 consensus_queue.cc:237] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:31.492844 15861 rpcz_store.cc:276] 0709 13:06:29.074264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:29.130415 (+ 56151us) service_pool.cc:224] Handling call
0709 13:06:29.169638 (+ 39223us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:29.264724 (+ 95086us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:29.634098 (+369374us) spinlock_profiling.cc:243] Waited 2.89 ms on lock 0x7b1800425d40. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72b8455 00007f73edda03c7 00007f73ea3b29a7 00007f73ea3c20bf 00007f73ea3c2038
0709 13:06:30.239095 (+604997us) log.cc:844] Serialized 14 byte log entry
0709 13:06:31.030973 (+791878us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:31.031458 (+   485us) log.cc:844] Serialized 26 byte log entry
0709 13:06:31.031607 (+   149us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:31.031647 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:31.343922 (+312275us) spinlock_profiling.cc:243] Waited 94.7 ms on lock 0x7b540032df54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:31.343976 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:31.480933 (+136957us) raft_consensus.cc:1685] finished
0709 13:06:31.480952 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:31.481081 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61426,"prepare.queue_time_us":258740,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":175158,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":33352,"spinlock_wait_cycles":224528512,"thread_start_us":1328183,"threads_started":3,"wal-append.queue_time_us":524974}
W20260709 13:06:31.528839 16005 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60156 (request call id 708) took 2323 ms (client timeout 1648 ms). Trace:
W20260709 13:06:31.488353 19977 log.cc:927] Time spent T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
W20260709 13:06:31.489393 19983 log.cc:927] Time spent T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
W20260709 13:06:31.615808 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 58371.7us
W20260709 13:06:31.489868 19970 log.cc:927] Time spent T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.178s	user 0.000s	sys 0.000s
I20260709 13:06:31.537537 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:31.463852 19961 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:31.435913 16003 raft_consensus.cc:1275] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:06:31.465720 19966 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:31.482301 19974 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:31.468868 19585 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:31.464039 19922 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:31.480356 19984 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:31.551999 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:31.656939 15853 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:31.403941 19666 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:31.485325 19933 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:31.554035 16568 raft_consensus.cc:1240] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 8. Current term is 9. Ops: [8.10-8.10]
I20260709 13:06:31.489400 19959 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:31.452332 19814 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:31.558024 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:31.490901 19940 leader_election.cc:290] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:31.388620 19769 consensus_meta.cc:287] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.996s	user 0.011s	sys 0.002s
I20260709 13:06:31.392028 19758 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:31.580451 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:31.649152 19953 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:31.482882 19967 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:31.554816 19723 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.538371 19713 consensus_queue.cc:1243] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer 8430db80a39d4868bcf7641c16de35ea log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:06:31.560626 19972 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:31.590670 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 94440.3us
I20260709 13:06:31.722926 15974 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:31.529623 19971 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:31.450012 19982 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:31.539816 19949 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:31.590791 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:31.482455 19980 log.cc:927] Time spent T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 13:06:31.533365 19985 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:31.627202 16005 rpcz_store.cc:269] 0709 13:06:29.205662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:29.246492 (+ 40830us) service_pool.cc:224] Handling call
0709 13:06:31.528475 (+2281983us) spinlock_profiling.cc:243] Waited 2.03 s on lock 0x7b54000eddd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:31.528727 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4672794880}
I20260709 13:06:31.638584 15929 raft_consensus.cc:2368] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 9. Current term is 10.
I20260709 13:06:31.470125 19869 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:31.518316 19882 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.075s
W20260709 13:06:31.639850 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 182971us
I20260709 13:06:31.735714 19995 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:31.740067 19963 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.222s
I20260709 13:06:31.740679 19995 raft_consensus.cc:697] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 10 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:31.741717 19995 consensus_queue.cc:237] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:31.647763 16565 raft_consensus.cc:1240] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 9. Current term is 10. Ops: [7.9-9.10]
I20260709 13:06:31.540452 19888 raft_consensus.cc:697] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 10 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:31.661584 19966 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:31.744082 19888 consensus_queue.cc:237] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:31.658216 15902 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:31.748375 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 90255.6us
I20260709 13:06:31.749321 15902 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:31.468598 19787 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
W20260709 13:06:31.666616 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 125194us
I20260709 13:06:31.559058 19917 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:31.637306 19852 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:31.629570 19899 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:31.540132 19900 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:31.686003 15931 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:31.669492 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:31.763827 15861 raft_consensus.cc:2393] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 9: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:31.686954 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:31.688910 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 126399us
I20260709 13:06:31.691141 15928 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 10.
I20260709 13:06:31.529959 19939 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:31.517416 19948 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:31.693764 15930 raft_consensus.cc:2393] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 10: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:31.707285 15935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2048) took 2327 ms. Trace:
W20260709 13:06:31.674965 19806 consensus_meta.cc:287] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 1.064s	user 0.011s	sys 0.003s
I20260709 13:06:31.668408 19641 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:31.671622 19372 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:31.725690 16570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 891) took 2501 ms. Trace:
I20260709 13:06:31.730463 15937 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:31.731057 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:31.778975 15858 raft_consensus.cc:2468] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 12.
I20260709 13:06:31.779222 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:31.700184 19827 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:06:31.717191 19902 consensus_meta.cc:287] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 1.319s	user 0.015s	sys 0.000s
I20260709 13:06:31.734761 19931 consensus_queue.cc:1059] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.097s
I20260709 13:06:31.738575 15936 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
I20260709 13:06:31.750823 15857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2132) took 1233 ms. Trace:
I20260709 13:06:31.687963 19933 consensus_queue.cc:1048] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:06:31.750321 15902 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:31.759110 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:31.760409 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:31.760548 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:31.795360 15902 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:31.761723 15827 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:31.731885 19949 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:31.698228 19769 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:31.668042 19984 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:31.777422 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:31.804255 16008 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:31.777527 19969 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.771138 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:31.778528 16004 raft_consensus.cc:1240] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 11. Current term is 12. Ops: [9.7-11.8]
I20260709 13:06:31.779800 15937 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:31.770179 15935 rpcz_store.cc:276] 0709 13:06:29.380383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:29.405819 (+ 25436us) service_pool.cc:224] Handling call
0709 13:06:29.406149 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:29.406514 (+   365us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:06:29.406542 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:29.406976 (+   434us) log.cc:844] Serialized 26 byte log entry
0709 13:06:30.815874 (+1408898us) spinlock_profiling.cc:243] Waited 7.03 ms on lock 0x7b180037a200. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f480d1 00007f73ea363a0f
0709 13:06:30.969726 (+153852us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:30.969768 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:30.969840 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:31.690372 (+720532us) raft_consensus.cc:1685] finished
0709 13:06:31.690402 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:31.707198 (+ 16796us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":103,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":96,"spinlock_wait_cycles":16168832,"thread_start_us":153660,"threads_started":1,"wal-append.queue_time_us":1622985}
I20260709 13:06:31.735780 19993 raft_consensus.cc:2804] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:31.778178 16570 rpcz_store.cc:276] 0709 13:06:29.224539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:29.536051 (+311512us) service_pool.cc:224] Handling call
0709 13:06:29.564428 (+ 28377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:29.664690 (+100262us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:29.664719 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:29.742363 (+ 77644us) log.cc:844] Serialized 26 byte log entry
0709 13:06:31.063067 (+1320704us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:31.063115 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:31.063198 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:31.725478 (+662280us) raft_consensus.cc:1685] finished
0709 13:06:31.725500 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:31.725627 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":77601,"raft.queue_time_us":90710,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":6980,"thread_start_us":888701,"threads_started":2,"wal-append.queue_time_us":1483571}
I20260709 13:06:31.764310 19963 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:31.788539 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:31.785830 19899 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:31.793041 16003 raft_consensus.cc:2393] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 10: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:31.724730 19723 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:31.798906 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:31.797749 19933 consensus_queue.cc:1243] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer 31882606e87447fb9a36bbf625129fe8 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:06:31.767633 19820 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:31.765494 20010 raft_consensus.cc:2749] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:31.756973 19890 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:31.789660 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:31.788900 19877 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:31.825322 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:31.825531 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:31.788341 15857 rpcz_store.cc:276] 0709 13:06:30.517266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.520622 (+  3356us) service_pool.cc:224] Handling call
0709 13:06:30.521036 (+   414us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:30.976020 (+454984us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:30.976048 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:30.976745 (+   697us) log.cc:844] Serialized 52 byte log entry
0709 13:06:31.169821 (+193076us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:31.169871 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:31.169961 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:31.750602 (+580641us) raft_consensus.cc:1685] finished
0709 13:06:31.750624 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:31.750768 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55326,"raft.queue_time_us":174861,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":11633,"thread_start_us":472634,"threads_started":2,"wal-append.queue_time_us":264278}
W20260709 13:06:31.883186 19998 log.cc:927] Time spent T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.246s	user 0.000s	sys 0.001s
I20260709 13:06:31.800628 19827 raft_consensus.cc:3055] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:31.777889 19939 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:31.721081 19967 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:31.827718 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:31.801403 19999 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:31.801102 19959 raft_consensus.cc:2804] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:31.873669 20003 consensus_queue.cc:1059] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.098s
I20260709 13:06:31.876947 16563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 894) took 1337 ms. Trace:
I20260709 13:06:31.802313 19955 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:31.811254 20000 log.cc:927] Time spent T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.282s	user 0.000s	sys 0.001s
W20260709 13:06:31.960716 19894 consensus_meta.cc:287] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.549s	user 0.010s	sys 0.000s
I20260709 13:06:31.961143 19894 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:31.878679 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 83387.6us
I20260709 13:06:31.808552 19963 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:31.772055 19974 consensus_queue.cc:1048] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.105s
I20260709 13:06:31.909206 19641 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:31.802332 19917 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:06:31.888578 20005 log.cc:927] Time spent T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
W20260709 13:06:31.888808 20002 log.cc:927] Time spent T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
W20260709 13:06:31.889969 19986 log.cc:927] Time spent T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:06:31.890357 19723 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:31.890807 19715 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:31.891299 19869 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:31.811164 19969 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:31.891881 19820 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:31.791123 19902 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:31.902010 15861 raft_consensus.cc:2410] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 10 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:06:31.878914 19904 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:31.904894 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:32.025920 15857 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:06:31.826929 15858 raft_consensus.cc:2393] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 8: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:06:31.906744 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 182963us
I20260709 13:06:31.907076 15935 raft_consensus.cc:2393] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 9: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:31.907119 15933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2046) took 2541 ms. Trace:
W20260709 13:06:31.907271 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:32.027143 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:31.826556 19996 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:31.887923 19993 raft_consensus.cc:697] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:31.913255 16003 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:32.042847 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:32.043921 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:31.943710 15929 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:32.044548 16003 raft_consensus.cc:2393] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 8: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:31.916152 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:31.992849 19787 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:31.903606 19899 raft_consensus.cc:697] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:31.773713 19985 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:31.968320 19900 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:31.957335 19814 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:31.969604 19733 consensus_meta.cc:287] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.565s	user 0.008s	sys 0.001s
I20260709 13:06:31.909974 19666 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:31.784039 19806 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:32.103643 19985 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:32.104434 16569 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 9: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:32.105075 19806 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:31.913868 19827 raft_consensus.cc:740] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:31.956363 19959 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:31.975276 19367 consensus_meta.cc:287] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.848s	user 0.011s	sys 0.000s
I20260709 13:06:32.066011 19984 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:31.980722 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:31.970463 19963 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:31.966002 19999 raft_consensus.cc:697] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:06:31.988899 20012 log.cc:927] Time spent T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:06:31.970808 19982 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:32.013052 15859 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 2139) took 1474 ms (client timeout 1821 ms). Trace:
I20260709 13:06:31.984788 20008 consensus_queue.cc:1048] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:06:32.016184 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:32.018334 16008 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:06:32.128733 15862 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:32.129635 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:32.130409 16008 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
I20260709 13:06:31.974211 19917 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:32.021399 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:06:31.982982 19558 consensus_meta.cc:287] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 1.242s	user 0.010s	sys 0.005s
I20260709 13:06:32.032509 15933 rpcz_store.cc:276] 0709 13:06:29.366106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:29.370138 (+  4032us) service_pool.cc:224] Handling call
0709 13:06:29.372867 (+  2729us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:29.385759 (+ 12892us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:06:29.391171 (+  5412us) log.cc:844] Serialized 14 byte log entry
0709 13:06:30.866214 (+1475043us) spinlock_profiling.cc:243] Waited 1.34 ms on lock 0x7b180037a140. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f4823a 00007f73ea3b2a2a
0709 13:06:31.052320 (+186106us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:31.070321 (+ 18001us) log.cc:844] Serialized 26 byte log entry
0709 13:06:31.070483 (+   162us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:31.070514 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:31.070583 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:31.895092 (+824509us) raft_consensus.cc:1685] finished
0709 13:06:31.906857 (+ 11765us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:31.907037 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4753,"prepare.queue_time_us":5825,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":291620,"raft.run_cpu_time_us":317,"raft.run_wall_time_us":36998,"spinlock_wait_cycles":3074816,"thread_start_us":185903,"threads_started":1,"wal-append.queue_time_us":1888515}
W20260709 13:06:32.036183 15974 connection.cc:256] RPC call timeout handler was delayed by -1.1464011829999663s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2015s for a response
I20260709 13:06:31.975229 19641 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:32.132428 15974 connection.cc:256] RPC call timeout handler was delayed by -1.0705484359999673s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.188s for a response
W20260709 13:06:32.132941 15974 connection.cc:256] RPC call timeout handler was delayed by -1.0705484359999673s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.188s for a response
W20260709 13:06:32.133410 15974 connection.cc:256] RPC call timeout handler was delayed by -1.0375011829999039s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2136s for a response
I20260709 13:06:32.038215 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:32.039243 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:32.045069 20014 log.cc:927] Time spent T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.066s	user 0.002s	sys 0.002s
I20260709 13:06:32.136269 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:32.134930 15974 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:32.137019 16565 raft_consensus.cc:2393] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 8: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:32.046610 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:31.952067 16563 rpcz_store.cc:276] 0709 13:06:30.539629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.543009 (+  3380us) service_pool.cc:224] Handling call
0709 13:06:30.543369 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:30.643220 (+ 99851us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:30.643249 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:30.643288 (+    39us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:30.643320 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:31.876715 (+1233395us) spinlock_profiling.cc:243] Waited 85.1 ms on lock 0x7b54002d7dd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:31.876763 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:31.876892 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":195761536}
I20260709 13:06:32.048913 16567 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:06:32.050405 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:32.053056 15853 raft_consensus.cc:2368] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 7. Current term is 8.
I20260709 13:06:32.049506 19728 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:32.055208 15860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 737) took 1282 ms. Trace:
I20260709 13:06:32.139504 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:32.056689 15858 raft_consensus.cc:1240] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 9 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:32.140533 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:32.141244 15858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2147) took 1105 ms. Trace:
I20260709 13:06:32.063505 19899 consensus_queue.cc:237] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:32.053835 15749 maintenance_manager.cc:419] P d33cebb189b14a989836214fdbafb898: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033359
I20260709 13:06:32.095929 19733 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:31.963868 19848 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:32.161725 16009 raft_consensus.cc:1240] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:32.162436 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:06:32.068802 19575 consensus_meta.cc:287] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 1.202s	user 0.008s	sys 0.004s
I20260709 13:06:32.118723 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:06:32.102232 19946 consensus_meta.cc:287] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.873s	user 0.012s	sys 0.001s
I20260709 13:06:32.165588 16564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45246 (request call id 876) took 1152 ms. Trace:
I20260709 13:06:32.165661 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:32.166317 16568 raft_consensus.cc:1275] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:06:32.067791 19948 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:32.125630 15859 rpcz_store.cc:269] 0709 13:06:30.538338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.591238 (+ 52900us) service_pool.cc:224] Handling call
0709 13:06:31.989281 (+1398043us) spinlock_profiling.cc:243] Waited 454 ms on lock 0x7b5400219654. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:32.012964 (+ 23683us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1043764992}
I20260709 13:06:32.114840 19959 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:32.117089 19887 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.140599 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:32.165993 16564 rpcz_store.cc:276] 0709 13:06:31.013053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:31.013182 (+   129us) service_pool.cc:224] Handling call
0709 13:06:32.165453 (+1152271us) spinlock_profiling.cc:243] Waited 5.08 ms on lock 0x7b54004790d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:32.165543 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11684224}
I20260709 13:06:32.170796 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:32.141844 15858 rpcz_store.cc:276] 0709 13:06:31.036192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.034745 (+998553us) service_pool.cc:224] Handling call
0709 13:06:32.056463 (+ 21718us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:32.140946 (+ 84483us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:32.141180 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:32.045674 19996 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:32.139905 15860 rpcz_store.cc:276] 0709 13:06:30.772504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.870846 (+ 98342us) service_pool.cc:224] Handling call
0709 13:06:30.871196 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:30.991893 (+120697us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:30.991938 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:30.992498 (+   560us) log.cc:844] Serialized 26 byte log entry
0709 13:06:31.224984 (+232486us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:31.225023 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:31.225091 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:32.054920 (+829829us) raft_consensus.cc:1685] finished
0709 13:06:32.054949 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:32.055121 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":109847,"raft.queue_time_us":124323,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":13021,"thread_start_us":74157,"threads_started":1,"wal-append.queue_time_us":572132}
W20260709 13:06:32.132616 19790 consensus_meta.cc:287] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.886s	user 0.009s	sys 0.003s
W20260709 13:06:32.128475 19749 consensus_meta.cc:287] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.733s	user 0.009s	sys 0.012s
I20260709 13:06:32.145750 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:32.146214 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:32.177440 16563 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:32.178357 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:32.135948 19827 consensus_queue.cc:260] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:32.124446 19894 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:32.152168 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:32.154202 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:32.134693 19917 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:32.157583 19733 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:32.159168 19883 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:32.159699 19263 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:32.150460 19939 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:32.106976 19769 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:32.181037 19982 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.166494 16008 raft_consensus.cc:2376] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10. Re-sending same reply.
I20260709 13:06:32.167750 15862 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:32.210350 15862 raft_consensus.cc:1240] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 9 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:32.136746 19372 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.179414 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:32.212857 15935 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:32.153802 19813 consensus_meta.cc:287] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.539s	user 0.011s	sys 0.006s
I20260709 13:06:32.182163 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:06:32.183524 15974 leader_election.cc:336] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 1.541s (SENT)
I20260709 13:06:32.217598 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:32.217718 15934 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:06:32.217790 15974 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:32.218894 19733 raft_consensus.cc:2749] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:32.219105 16564 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:32.219847 20013 log.cc:927] Time spent T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.245s	user 0.000s	sys 0.001s
I20260709 13:06:32.187269 15859 raft_consensus.cc:1240] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 8. Current term is 9. Ops: [8.10-8.10]
I20260709 13:06:32.221793 15859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 2002) took 1135 ms. Trace:
I20260709 13:06:32.222249 15859 rpcz_store.cc:276] 0709 13:06:31.086490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.169702 (+1083212us) service_pool.cc:224] Handling call
0709 13:06:32.180976 (+ 11274us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:32.221421 (+ 40445us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:32.221721 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:32.198757 19941 consensus_meta.cc:287] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 1.154s	user 0.013s	sys 0.000s
I20260709 13:06:32.188227 16003 raft_consensus.cc:1240] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:32.188313 15858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 2003) took 1101 ms. Trace:
I20260709 13:06:32.225883 15858 rpcz_store.cc:276] 0709 13:06:31.086792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.173293 (+1086501us) service_pool.cc:224] Handling call
0709 13:06:32.187825 (+ 14532us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:32.188021 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:32.188044 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:32.188082 (+    38us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:32.188111 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:32.188162 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:32.188260 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:32.180099 19892 consensus_meta.cc:287] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.640s	user 0.009s	sys 0.003s
I20260709 13:06:32.195040 16563 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 10.
I20260709 13:06:32.163113 15928 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:32.206386 16006 raft_consensus.cc:1240] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 10 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 9. Current term is 10. Ops: [9.10-9.10]
W20260709 13:06:32.207367 20018 log.cc:927] Time spent T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.196s	user 0.003s	sys 0.000s
I20260709 13:06:32.214828 16566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 881) took 1064 ms. Trace:
I20260709 13:06:32.198380 19263 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:32.161051 19999 consensus_queue.cc:237] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:32.180480 19887 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:32.148479 19993 consensus_queue.cc:237] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 6.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:32.183817 19790 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:32.220672 20019 log.cc:927] Time spent T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:06:32.197531 19963 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:32.215492 19827 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:32.282548 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:32.288131 16563 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:32.216971 19813 raft_consensus.cc:2749] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:32.187069 19749 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:32.283888 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 303501us
I20260709 13:06:32.297492 16561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 1024) took 1202 ms. Trace:
I20260709 13:06:32.286185 15928 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:32.176294 19946 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:32.263520 15934 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 10.
I20260709 13:06:32.237772 16566 rpcz_store.cc:276] 0709 13:06:31.150367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:31.590662 (+440295us) service_pool.cc:224] Handling call
0709 13:06:31.612314 (+ 21652us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:31.615751 (+  3437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:31.615795 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:31.628896 (+ 13101us) log.cc:844] Serialized 26 byte log entry
0709 13:06:31.725354 (+ 96458us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:31.725395 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:31.725459 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:32.214613 (+489154us) raft_consensus.cc:1685] finished
0709 13:06:32.214632 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:32.214758 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3204,"raft.queue_time_us":35722,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":13153,"thread_start_us":19099,"threads_started":1,"wal-append.queue_time_us":208559}
I20260709 13:06:32.289062 19263 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:32.217144 19949 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:32.298031 16561 rpcz_store.cc:276] 0709 13:06:31.094985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:31.558676 (+463691us) service_pool.cc:224] Handling call
0709 13:06:31.559006 (+   330us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:31.636180 (+ 77174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:31.636217 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:31.667662 (+ 31445us) log.cc:844] Serialized 52 byte log entry
0709 13:06:31.805506 (+137844us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:31.805546 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:31.805615 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:32.297240 (+491625us) raft_consensus.cc:1685] finished
0709 13:06:32.297263 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:32.297426 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56193,"raft.queue_time_us":76664,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":3814,"thread_start_us":80609,"threads_started":1,"wal-append.queue_time_us":194253}
I20260709 13:06:32.301800 15861 raft_consensus.cc:1240] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:32.301995 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:32.303860 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:32.305368 15937 raft_consensus.cc:2368] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 11. Current term is 12.
I20260709 13:06:32.305559 15934 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
W20260709 13:06:32.293136 19972 consensus_meta.cc:287] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.572s	user 0.009s	sys 0.000s
I20260709 13:06:32.310451 16563 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:32.310811 16561 raft_consensus.cc:1240] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:32.307636 15928 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:32.304724 15854 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:32.283927 19902 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:32.216097 19372 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:32.312975 15928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 1989) took 1013 ms. Trace:
I20260709 13:06:32.315629 16002 raft_consensus.cc:1240] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:06:32.316495 15928 rpcz_store.cc:276] 0709 13:06:31.299479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.301760 (+1002281us) service_pool.cc:224] Handling call
0709 13:06:32.312901 (+ 11141us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:32.224138 19982 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:32.320418 15935 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:06:32.239727 19892 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:32.321455 15974 leader_election.cc:336] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 1.754s (SENT)
I20260709 13:06:32.324700 15974 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:32.330615 19963 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:32.332075 16009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 2084) took 1244 ms. Trace:
I20260709 13:06:32.332464 16009 rpcz_store.cc:276] 0709 13:06:31.087888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:31.252700 (+164812us) service_pool.cc:224] Handling call
0709 13:06:31.252988 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:32.162177 (+909189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:32.331839 (+169662us) spinlock_profiling.cc:243] Waited 908 ms on lock 0x7b54000aa5d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:32.332016 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2089474816}
I20260709 13:06:32.327700 15825 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:32.337904 15857 raft_consensus.cc:1240] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:06:32.338645 19996 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:32.339252 15857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 2001) took 1253 ms. Trace:
I20260709 13:06:32.339886 19969 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:32.339624 15857 rpcz_store.cc:276] 0709 13:06:31.086216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.041067 (+954851us) service_pool.cc:224] Handling call
0709 13:06:32.041495 (+   428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:32.338328 (+296833us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:32.338993 (+   665us) spinlock_profiling.cc:243] Waited 296 ms on lock 0x7b540008d254. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:32.339173 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":681165184}
I20260709 13:06:32.345534 15930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 1971) took 1061 ms. Trace:
I20260709 13:06:32.345921 15930 rpcz_store.cc:276] 0709 13:06:31.284248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:31.284666 (+   418us) service_pool.cc:224] Handling call
0709 13:06:32.345335 (+1060669us) spinlock_profiling.cc:243] Waited 299 ms on lock 0x7b540044bbd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:32.345475 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":687503232}
I20260709 13:06:32.346700 19902 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:32.347298 19902 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:32.347739 19902 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:32.356897 16533 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:32.357390 16004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 2088) took 1268 ms. Trace:
I20260709 13:06:32.358373 16533 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:32.359098 15856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 2143) took 1348 ms. Trace:
I20260709 13:06:32.359591 15856 rpcz_store.cc:276] 0709 13:06:31.022109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:31.077663 (+ 55554us) service_pool.cc:224] Handling call
0709 13:06:32.358904 (+1281241us) spinlock_profiling.cc:243] Waited 66 ms on lock 0x7b5400217d54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:32.359028 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":151868288}
I20260709 13:06:32.366870 15974 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:32.357967 16004 rpcz_store.cc:276] 0709 13:06:31.089029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:31.296470 (+207441us) service_pool.cc:224] Handling call
0709 13:06:31.296790 (+   320us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:31.806787 (+509997us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:32.356929 (+550142us) spinlock_profiling.cc:243] Waited 481 ms on lock 0x7b54001db854. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:32.357186 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1107201408}
I20260709 13:06:32.346866 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:32.370033 15930 raft_consensus.cc:2393] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 11: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:32.368777 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:32.371541 15928 raft_consensus.cc:2393] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 10: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:32.360997 19733 raft_consensus.cc:2749] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:32.391628 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:32.401064 15934 raft_consensus.cc:2410] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:06:32.486662 15934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 2004) took 1001 ms. Trace:
I20260709 13:06:32.401708 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:06:32.431362 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:32.488670 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:32.488854 15856 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 9: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:32.238013 19941 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:32.378469 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:32.489776 15934 rpcz_store.cc:276] 0709 13:06:31.485395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.328336 (+842941us) service_pool.cc:224] Handling call
0709 13:06:32.486593 (+158257us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:32.490506 15930 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:06:32.478610 16564 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:32.377839 19853 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.319206 19972 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:32.387539 16533 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:32.493188 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 105715us
I20260709 13:06:32.494217 16533 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:32.496197 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:32.496984 15930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 1977) took 1211 ms. Trace:
I20260709 13:06:32.497390 15930 rpcz_store.cc:276] 0709 13:06:31.285956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.495907 (+1209951us) service_pool.cc:224] Handling call
0709 13:06:32.496923 (+  1016us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:32.501035 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:32.501693 15930 raft_consensus.cc:2393] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 10: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:32.499415 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:32.504019 15928 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
I20260709 13:06:32.502487 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:32.512768 15930 raft_consensus.cc:2368] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 9. Current term is 10.
I20260709 13:06:32.516722 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:06:32.526110 15974 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 3.996s (SENT) [suppressed 26 similar messages]
W20260709 13:06:32.378080 19922 consensus_meta.cc:287] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.708s	user 0.009s	sys 0.001s
I20260709 13:06:32.378432 19568 raft_consensus.cc:2749] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:32.498533 16533 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:32.528533 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:32.529139 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:32.402141 19996 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.485430 19963 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.487516 19641 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:32.402336 19769 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:32.536294 15930 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:06:32.521135 15902 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:32.542817 15857 raft_consensus.cc:2393] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 10: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:32.543140 16562 raft_consensus.cc:2393] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 11: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:32.396822 19969 raft_consensus.cc:2749] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:32.514302 19723 raft_consensus.cc:2804] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:32.521061 19853 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:32.548822 19996 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:32.554841 19575 raft_consensus.cc:2804] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:32.569005 19575 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:32.569401 19575 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:06:32.556650 16562 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45244 (request call id 1021) took 1520 ms (client timeout 1808 ms). Trace:
W20260709 13:06:32.571832 16562 rpcz_store.cc:269] 0709 13:06:31.035982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:31.091126 (+ 55144us) service_pool.cc:224] Handling call
0709 13:06:32.556483 (+1465357us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b54002ff154. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:32.556597 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3103185280}
I20260709 13:06:32.557523 16567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45246 (request call id 879) took 1413 ms. Trace:
I20260709 13:06:32.572808 16567 rpcz_store.cc:276] 0709 13:06:31.144431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:31.527498 (+383067us) service_pool.cc:224] Handling call
0709 13:06:32.557374 (+1029876us) spinlock_profiling.cc:243] Waited 490 ms on lock 0x7b54006a45d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:32.557470 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1126975360}
I20260709 13:06:32.549172 19869 consensus_queue.cc:1059] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:06:32.578958 19575 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:32.582393 16565 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 10.
I20260709 13:06:32.568437 19996 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:32.579939 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:32.585424 19715 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:32.585752 19892 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:32.586303 19675 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:32.587873 19749 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:32.598462 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:32.599085 15930 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:32.600265 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:32.601018 15928 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:32.588196 19575 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:32.536887 19946 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:32.581828 19660 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:32.579591 19853 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:32.551838 15902 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:32.558614 19963 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:32.579659 19869 consensus_queue.cc:1243] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer 31882606e87447fb9a36bbf625129fe8 log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:06:32.565404 19723 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:32.557627 19769 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:06:32.612648 16569 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45244 (request call id 1020) took 1581 ms (client timeout 1987 ms). Trace:
I20260709 13:06:32.613296 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:32.616077 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 64316.2us
I20260709 13:06:32.549320 19990 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:32.607753 19917 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:32.616789 19996 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:32.623059 19723 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:32.627879 19769 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:32.637529 15930 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:06:32.625641 15974 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:32.639393 15974 leader_election.cc:400] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Vote denied by peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) with higher term. Message: Invalid argument: T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 9. Current term is 10.
I20260709 13:06:32.618101 19790 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:32.616693 15975 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.844s	user 0.019s	sys 0.059s
W20260709 13:06:32.644822 15975 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.872s	user 0.020s	sys 0.059s
I20260709 13:06:32.619237 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:32.645349 15928 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:32.645403 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:32.621690 16569 rpcz_store.cc:269] 0709 13:06:31.031834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:31.097286 (+ 65452us) service_pool.cc:224] Handling call
0709 13:06:32.612411 (+1515125us) spinlock_profiling.cc:243] Waited 911 ms on lock 0x7b54002b5f54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:32.612579 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2094754560}
W20260709 13:06:32.617937 19758 consensus_meta.cc:287] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.902s	user 0.008s	sys 0.003s
I20260709 13:06:32.640894 15974 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:06:32.646507 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:32.660318 19963 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:32.634287 19996 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:32.632815 19990 raft_consensus.cc:697] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:32.672258 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:32.657469 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:32.673280 15934 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:32.674446 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:32.597661 19922 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:32.679890 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:32.644181 19749 raft_consensus.cc:2749] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:32.677788 19728 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.660351 19558 raft_consensus.cc:2804] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:32.691596 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:32.661015 19944 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:32.696832 19814 raft_consensus.cc:3055] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:32.672722 19996 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:32.677585 19888 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:32.693533 19890 consensus_meta.cc:287] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.789s	user 0.012s	sys 0.000s
I20260709 13:06:32.691188 19558 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:32.663388 19758 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:32.704402 19972 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:32.701982 19814 raft_consensus.cc:740] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:32.724486 19814 consensus_queue.cc:260] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:32.725736 19814 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:32.726155 19966 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:32.711335 19893 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.702237 19728 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:32.727689 19893 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:32.728541 19949 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:32.729285 19728 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:32.714002 19558 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:32.730202 19893 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:32.708606 19888 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:32.716992 19922 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:32.724520 20025 log.cc:927] Time spent T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:06:32.712296 19941 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:32.741606 19954 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:32.745940 19982 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:32.756407 19917 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:32.739073 19990 consensus_queue.cc:237] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:32.765103 19996 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:31.686815 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:06:32.796692 15860 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.005s	sys 0.035s
W20260709 13:06:32.797467 15860 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.005s	sys 0.035s
I20260709 13:06:32.799464 15860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 740) took 1288 ms. Trace:
W20260709 13:06:32.801257 15862 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.011s	sys 0.007s
W20260709 13:06:32.801766 15862 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.011s	sys 0.008s
I20260709 13:06:32.809705 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:32.812230 16009 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
I20260709 13:06:32.811748 16011 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 9: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:32.814186 15974 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:32.811578 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:32.800027 15860 rpcz_store.cc:276] 0709 13:06:31.510924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.173750 (+662826us) service_pool.cc:224] Handling call
0709 13:06:32.174086 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:32.174332 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:32.188873 (+ 14541us) log.cc:844] Serialized 14 byte log entry
0709 13:06:32.798451 (+609578us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:32.798592 (+   141us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:32.798817 (+   225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:32.798985 (+   168us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:32.799325 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16272,"prepare.queue_time_us":14894,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"thread_start_us":608743,"threads_started":1,"wal-append.queue_time_us":541882}
I20260709 13:06:32.815707 15934 raft_consensus.cc:2393] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 9: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:32.810884 16004 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:32.824208 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:06:32.825557 16563 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.001s	sys 0.011s
W20260709 13:06:32.827545 16570 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.711s	user 0.002s	sys 0.011s
W20260709 13:06:32.829910 16570 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.003s	sys 0.011s
I20260709 13:06:32.830878 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:32.832906 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:32.833003 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:06:32.827872 16563 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.001s	sys 0.011s
I20260709 13:06:32.834338 15856 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:32.748330 19844 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:32.836222 19806 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:32.841919 19555 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.842864 19758 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:32.848898 19758 raft_consensus.cc:2804] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:32.842769 19555 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:32.846529 19685 consensus_queue.cc:260] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:06:32.865168 19685 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:32.866106 19555 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:32.854211 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:32.866881 19758 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:32.856689 15825 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:32.867367 19758 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:32.869199 15825 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:32.870537 15935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 1993) took 1498 ms. Trace:
I20260709 13:06:32.860868 15853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58214 (request call id 742) took 1346 ms. Trace:
W20260709 13:06:32.859568 15937 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 1978) took 1573 ms (client timeout 1925 ms). Trace:
W20260709 13:06:32.856873 15936 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53636 (request call id 784) took 1777 ms (client timeout 2227 ms). Trace:
W20260709 13:06:32.866137 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 1.10134e+06us
I20260709 13:06:32.873689 15935 rpcz_store.cc:276] 0709 13:06:31.372052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.853480 (+1481428us) service_pool.cc:224] Handling call
0709 13:06:32.870273 (+ 16793us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:32.873788 15853 rpcz_store.cc:276] 0709 13:06:31.513872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:31.770868 (+256996us) service_pool.cc:224] Handling call
0709 13:06:32.860354 (+1089486us) spinlock_profiling.cc:243] Waited 246 ms on lock 0x7b540008b954. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:32.860615 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":565918080}
W20260709 13:06:32.875177 15937 rpcz_store.cc:269] 0709 13:06:31.286287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:31.915839 (+629552us) service_pool.cc:224] Handling call
0709 13:06:32.858903 (+943064us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b5400327b54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:32.859272 (+   369us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":596880128}
I20260709 13:06:32.886494 16004 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 6.
W20260709 13:06:32.875619 15936 rpcz_store.cc:269] 0709 13:06:31.078795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:31.078947 (+   152us) service_pool.cc:224] Handling call
0709 13:06:32.856305 (+1777358us) spinlock_profiling.cc:243] Waited 397 ms on lock 0x7b540008a7d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:32.856575 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":912592640}
I20260709 13:06:32.873284 19367 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:32.897228 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 83510.9us
I20260709 13:06:32.897410 19904 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:32.898140 15928 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:06:32.899559 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:32.900578 15928 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:32.900462 19367 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:32.902077 15825 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:32.902478 19888 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:06:32.904505 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:32.915354 15856 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
I20260709 13:06:32.921715 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:32.922686 16010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41674 (request call id 1950) took 1883 ms. Trace:
W20260709 13:06:32.923956 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:32.924314 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:32.925531 16011 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41662 (request call id 2086) took 1837 ms (client timeout 1987 ms). Trace:
I20260709 13:06:32.925645 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:06:32.926026 16011 rpcz_store.cc:269] 0709 13:06:31.088481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:31.186997 (+ 98516us) service_pool.cc:224] Handling call
0709 13:06:32.925278 (+1738281us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b54003ee754. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:32.925477 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3731782528}
I20260709 13:06:32.926972 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:32.923306 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:32.930128 19917 raft_consensus.cc:2804] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:32.930687 19917 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:32.931151 19917 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:32.931414 20010 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:32.932533 20010 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:32.929352 15930 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
W20260709 13:06:32.936476 19937 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.028s	sys 0.045s
I20260709 13:06:32.924008 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:32.923172 16010 rpcz_store.cc:276] 0709 13:06:31.038925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:31.041748 (+  2823us) service_pool.cc:224] Handling call
0709 13:06:32.922229 (+1880481us) spinlock_profiling.cc:243] Waited 302 ms on lock 0x7b54002e1dd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:32.922478 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":694680960}
I20260709 13:06:32.944777 15936 raft_consensus.cc:2410] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:06:32.946259 15937 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 1992) took 1604 ms. Trace:
I20260709 13:06:32.949136 15937 rpcz_store.cc:276] 0709 13:06:31.341520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.922920 (+1581400us) service_pool.cc:224] Handling call
0709 13:06:32.946196 (+ 23276us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:32.951664 15928 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:06:32.948922 15936 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 1975) took 1663 ms. Trace:
I20260709 13:06:32.953195 15936 rpcz_store.cc:276] 0709 13:06:31.285495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.914070 (+1628575us) service_pool.cc:224] Handling call
0709 13:06:32.948849 (+ 34779us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:32.962472 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:32.968171 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:32.970517 15936 raft_consensus.cc:2410] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:06:32.935820 19996 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:32.973369 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 79155us
W20260709 13:06:32.971879 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 87524.1us
I20260709 13:06:32.974129 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:32.971222 15930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 2001) took 1486 ms. Trace:
I20260709 13:06:32.974915 15928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 1972) took 1690 ms. Trace:
I20260709 13:06:32.974521 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:32.979048 19894 raft_consensus.cc:2804] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:32.979554 19894 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:32.979658 16569 raft_consensus.cc:1275] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:06:32.980019 19894 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:06:32.981379 19263 consensus_meta.cc:287] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.667s	user 0.006s	sys 0.012s
I20260709 13:06:32.981894 19263 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:32.969918 19904 raft_consensus.cc:697] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:32.987700 19904 consensus_queue.cc:237] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:32.989890 19367 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:32.971982 15935 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:06:32.991955 15935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 2059) took 1482 ms. Trace:
I20260709 13:06:32.992324 15935 rpcz_store.cc:276] 0709 13:06:31.509629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.926613 (+1416984us) service_pool.cc:224] Handling call
0709 13:06:32.991878 (+ 65265us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:32.993502 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:32.993397 20010 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:32.995016 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:32.995370 19902 consensus_meta.cc:287] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.647s	user 0.005s	sys 0.013s
I20260709 13:06:32.996201 19263 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:32.997378 19372 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:32.975096 15930 rpcz_store.cc:276] 0709 13:06:31.484402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.947833 (+1463431us) service_pool.cc:224] Handling call
0709 13:06:32.971163 (+ 23330us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:32.975270 15928 rpcz_store.cc:276] 0709 13:06:31.284560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.952462 (+1667902us) service_pool.cc:224] Handling call
0709 13:06:32.974856 (+ 22394us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:32.976527 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:32.973603 15900 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:32.996371 19902 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:33.018746 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:33.019635 15930 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 9: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:33.025105 19558 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:33.026806 15935 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:33.026633 19758 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:33.028126 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:33.028371 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:33.028820 15935 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 10.
I20260709 13:06:33.029047 16567 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:33.029624 19758 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:33.030206 15935 raft_consensus.cc:1240] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 7. Current term is 8. Ops: [5.3-7.4]
I20260709 13:06:33.030893 15935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 1981) took 1733 ms. Trace:
I20260709 13:06:33.032023 15936 raft_consensus.cc:1240] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:06:33.034009 15861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2142) took 2153 ms. Trace:
I20260709 13:06:33.034143 19894 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:33.034432 15861 rpcz_store.cc:276] 0709 13:06:30.880234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.033040 (+1152806us) service_pool.cc:224] Handling call
0709 13:06:32.048352 (+ 15312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:32.303349 (+254997us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.033507 (+730158us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b54002481d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:33.033808 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":582363392}
I20260709 13:06:33.031222 15935 rpcz_store.cc:276] 0709 13:06:31.297493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:33.029510 (+1732017us) service_pool.cc:224] Handling call
0709 13:06:33.029903 (+   393us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:33.030624 (+   721us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.030826 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:33.037914 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 88894us
I20260709 13:06:33.041090 19894 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:33.044037 15825 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:33.044247 16566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 906) took 1009 ms. Trace:
I20260709 13:06:33.039373 19350 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:33.044996 19350 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:33.045008 19894 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:33.045650 19350 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:33.045763 19894 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:33.045994 15825 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:33.046423 19894 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:33.047190 19223 raft_consensus.cc:2804] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:33.036995 19558 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:33.047861 19223 raft_consensus.cc:697] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 8 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:33.044781 16566 rpcz_store.cc:276] 0709 13:06:32.034334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.300298 (+265964us) service_pool.cc:224] Handling call
0709 13:06:32.302359 (+  2061us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:32.302771 (+   412us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:32.302811 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:32.303393 (+   582us) log.cc:844] Serialized 52 byte log entry
0709 13:06:32.826664 (+523271us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:32.826786 (+   122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:32.827040 (+   254us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:33.043749 (+216709us) raft_consensus.cc:1685] finished
0709 13:06:33.043777 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:33.044032 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":101,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":10309,"thread_start_us":499159,"threads_started":1,"wal-append.queue_time_us":683608}
I20260709 13:06:33.039422 19967 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:33.051595 19223 consensus_queue.cc:237] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:33.047258 15825 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:33.062265 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:33.078235 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 56022.4us
I20260709 13:06:33.061635 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:33.082125 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:33.083316 15862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 750) took 1200 ms. Trace:
I20260709 13:06:33.081622 19937 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:33.095197 15928 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
I20260709 13:06:33.102576 16567 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
I20260709 13:06:33.085486 15862 rpcz_store.cc:276] 0709 13:06:31.882518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.211463 (+328945us) service_pool.cc:224] Handling call
0709 13:06:32.212100 (+   637us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:32.212371 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:06:32.802488 (+590117us) log.cc:844] Serialized 14 byte log entry
0709 13:06:33.016282 (+213794us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:33.018643 (+  2361us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:33.053712 (+ 35069us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.060986 (+  7274us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:33.083256 (+ 22270us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46208,"prepare.queue_time_us":37033,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":4248,"raft.queue_time_us":61706,"spinlock_wait_cycles":3147520,"thread_start_us":709174,"threads_started":2,"wal-append.queue_time_us":202343}
I20260709 13:06:33.115631 19806 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:33.116365 19806 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:33.088491 19769 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:33.045466 19987 consensus_queue.cc:1059] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 13:06:33.118759 19769 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:33.119310 19987 raft_consensus.cc:3055] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:33.119798 19987 raft_consensus.cc:740] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:33.121135 19987 consensus_queue.cc:260] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:33.122782 19987 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:33.124217 16570 raft_consensus.cc:2393] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 13: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:33.130832 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:33.131636 16003 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:33.135012 19959 consensus_meta.cc:287] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.948s	user 0.010s	sys 0.004s
W20260709 13:06:33.138612 15854 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 2134) took 2603 ms (client timeout 1880 ms). Trace:
W20260709 13:06:33.139078 15854 rpcz_store.cc:269] 0709 13:06:30.535456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.536527 (+  1071us) service_pool.cc:224] Handling call
0709 13:06:33.138247 (+2601720us) spinlock_profiling.cc:243] Waited 1.72 s on lock 0x7b54000b5c54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:33.138409 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3956250368}
I20260709 13:06:33.109237 16569 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:33.111075 16007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 718) took 2028 ms. Trace:
I20260709 13:06:33.113057 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:33.144477 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:33.145174 16562 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 9: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:33.145483 16004 raft_consensus.cc:2393] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 12: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:33.112668 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:33.148600 16566 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 10.
I20260709 13:06:33.148628 19887 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:33.152040 20039 log.cc:927] Time spent T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:06:33.124264 19723 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:33.155187 19558 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.155793 19558 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:33.158093 19558 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:33.159633 15825 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:33.161296 19974 raft_consensus.cc:2804] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:33.161782 19974 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:33.162173 19974 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:33.165864 15825 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:33.113833 19937 raft_consensus.cc:697] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 8 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:33.113317 16568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 904) took 1090 ms. Trace:
I20260709 13:06:33.168766 19937 consensus_queue.cc:237] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:33.169046 16568 rpcz_store.cc:276] 0709 13:06:32.023343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.023548 (+   205us) service_pool.cc:224] Handling call
0709 13:06:32.166007 (+142459us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:32.166674 (+   667us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.112875 (+946201us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b54004790d0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:33.113132 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":306926848}
I20260709 13:06:33.108961 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 2087) took 2020 ms. Trace:
I20260709 13:06:33.170449 16005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41674 (request call id 1954) took 1411 ms. Trace:
I20260709 13:06:33.170892 16005 rpcz_store.cc:276] 0709 13:06:31.758582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:31.758774 (+   192us) service_pool.cc:224] Handling call
0709 13:06:33.170107 (+1411333us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54000ed8d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:33.170288 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2578400000}
I20260709 13:06:33.143869 16007 rpcz_store.cc:276] 0709 13:06:31.082148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:31.082280 (+   132us) service_pool.cc:224] Handling call
0709 13:06:31.082610 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:31.310456 (+227846us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:31.419312 (+108856us) log.cc:844] Serialized 14 byte log entry
0709 13:06:31.449984 (+ 30672us) log.cc:844] Serialized 14 byte log entry
0709 13:06:31.450154 (+   170us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:31.450191 (+    37us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:31.450238 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:32.555277 (+1105039us) spinlock_profiling.cc:243] Waited 1.52 ms on lock 0x7b1800358180. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb3f3 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd 000000000038f354 000000000038f199 00007f73e729e83a
0709 13:06:33.110637 (+555360us) spinlock_profiling.cc:243] Waited 228 ms on lock 0x7b5400305a54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:33.110725 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:33.110910 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":973,"prepare.queue_time_us":663390,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":68,"raft.queue_time_us":707,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":12729,"spinlock_wait_cycles":526925824,"thread_start_us":126678,"threads_started":2,"wal-append.queue_time_us":251538,"wal-append.run_cpu_time_us":1657,"wal-append.run_wall_time_us":1054489}
I20260709 13:06:33.133136 15928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 1995) took 1713 ms. Trace:
I20260709 13:06:33.167091 20010 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:33.170632 16002 rpcz_store.cc:276] 0709 13:06:31.088756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:31.287353 (+198597us) service_pool.cc:224] Handling call
0709 13:06:31.287687 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:32.316313 (+1028626us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.108363 (+792050us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54000ee2d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:33.108737 (+   374us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2363277568}
I20260709 13:06:33.173147 20010 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:33.176146 20010 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:33.116393 15974 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:33.136175 15936 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 1976) took 1850 ms. Trace:
W20260709 13:06:33.166844 20020 log.cc:927] Time spent T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.888s	user 0.003s	sys 0.000s
W20260709 13:06:33.114099 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 111594us
I20260709 13:06:33.180039 19651 raft_consensus.cc:3055] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:33.180517 19651 raft_consensus.cc:740] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:33.135524 19959 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:33.182945 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:33.184388 15855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 738) took 2411 ms. Trace:
I20260709 13:06:33.185998 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:06:33.186121 15929 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 1998) took 1702 ms (client timeout 1541 ms). Trace:
I20260709 13:06:33.111110 19902 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:33.189294 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:33.189724 20007 raft_consensus.cc:2749] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:33.180330 16535 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:33.178102 15936 rpcz_store.cc:276] 0709 13:06:31.285727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.977934 (+1692207us) service_pool.cc:224] Handling call
0709 13:06:33.031768 (+ 53834us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:33.032392 (+   624us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.135884 (+103492us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b54002cc750. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:33.136058 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27132160}
I20260709 13:06:33.178280 15928 rpcz_store.cc:276] 0709 13:06:31.419217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:33.095927 (+1676710us) service_pool.cc:224] Handling call
0709 13:06:33.121071 (+ 25144us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:33.126855 (+  5784us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:33.127959 (+  1104us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:33.128863 (+   904us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:33.129440 (+   577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.130598 (+  1158us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:33.132261 (+  1663us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:33.177906 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 62363.1us
I20260709 13:06:33.190765 16565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45232 (request call id 909) took 1146 ms. Trace:
I20260709 13:06:33.203657 16565 rpcz_store.cc:276] 0709 13:06:32.044513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.152633 (+108120us) service_pool.cc:224] Handling call
0709 13:06:33.190417 (+1037784us) spinlock_profiling.cc:243] Waited 397 ms on lock 0x7b54000ce754. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:33.190591 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":912837888}
W20260709 13:06:33.197134 15929 rpcz_store.cc:269] 0709 13:06:31.483321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:31.760334 (+277013us) service_pool.cc:224] Handling call
0709 13:06:33.185732 (+1425398us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b5400327654. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:33.185928 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":345208704}
W20260709 13:06:33.210049 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 44609.3us
I20260709 13:06:33.190594 19723 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:33.217425 16569 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:33.213636 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:33.222540 16005 raft_consensus.cc:2393] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 10: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
W20260709 13:06:33.209906 15932 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 1968) took 1927 ms (client timeout 1771 ms). Trace:
I20260709 13:06:33.228555 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:33.229264 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:06:33.228623 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 48427.9us
I20260709 13:06:33.202250 19967 consensus_queue.cc:1059] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
W20260709 13:06:33.223022 15932 rpcz_store.cc:269] 0709 13:06:31.282529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:31.282675 (+   146us) service_pool.cc:224] Handling call
0709 13:06:33.209529 (+1926854us) spinlock_profiling.cc:243] Waited 64.2 ms on lock 0x7b54002c2c54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:33.209719 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":147630464}
I20260709 13:06:33.228152 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:33.228241 19975 raft_consensus.cc:2764] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:06:33.243251 15856 raft_consensus.cc:2393] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 10: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:33.238947 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:33.243286 16003 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
I20260709 13:06:33.243643 15854 raft_consensus.cc:1275] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:06:33.248214 15933 raft_consensus.cc:2368] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 12. Current term is 13.
I20260709 13:06:33.191582 20010 consensus_queue.cc:1059] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:06:33.246773 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:33.251009 16002 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:33.252075 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:33.256570 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:33.259325 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:33.196702 15855 rpcz_store.cc:276] 0709 13:06:30.772834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.996553 (+223719us) service_pool.cc:224] Handling call
0709 13:06:31.022058 (+ 25505us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:31.247261 (+225203us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:06:31.309993 (+ 62732us) log.cc:844] Serialized 14 byte log entry
0709 13:06:31.548487 (+238494us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:31.548546 (+    59us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:31.548589 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.183889 (+1635300us) spinlock_profiling.cc:243] Waited 225 ms on lock 0x7b5400217354. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:33.183980 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:33.184212 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3574,"prepare.queue_time_us":162970,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":83434,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":17216,"spinlock_wait_cycles":517406464,"thread_start_us":291550,"threads_started":2,"wal-append.queue_time_us":404770,"wal-append.run_cpu_time_us":1050,"wal-append.run_wall_time_us":1020536}
I20260709 13:06:33.263504 15862 raft_consensus.cc:2393] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 10: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:33.270704 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:33.271405 16005 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:33.287919 15861 raft_consensus.cc:2393] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 8: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:33.296289 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:33.297367 15928 raft_consensus.cc:1275] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:06:33.258958 19769 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:33.317855 15928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2057) took 1842 ms. Trace:
W20260709 13:06:33.319880 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:33.323434 15928 rpcz_store.cc:276] 0709 13:06:31.456280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:33.203410 (+1747130us) service_pool.cc:224] Handling call
0709 13:06:33.248597 (+ 45187us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:33.297918 (+ 49321us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.298171 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:33.326768 16004 raft_consensus.cc:1240] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:33.316574 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:33.331137 15934 raft_consensus.cc:2393] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 10: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:33.327850 15853 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:33.341801 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:33.342267 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:33.323370 19769 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:33.344856 19769 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:33.345227 15936 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2054) took 1889 ms. Trace:
I20260709 13:06:33.345661 15936 rpcz_store.cc:276] 0709 13:06:31.455569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:33.203229 (+1747660us) service_pool.cc:224] Handling call
0709 13:06:33.253070 (+ 49841us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:33.344838 (+ 91768us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:33.344868 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:33.344945 (+    77us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:33.344986 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.345070 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:33.345178 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:33.346740 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:06:33.347299 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:33.347491 15934 raft_consensus.cc:2368] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 9. Current term is 10.
I20260709 13:06:33.348470 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:33.348851 15854 raft_consensus.cc:1240] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 8 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:33.351418 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:33.352073 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:33.353084 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:33.353309 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:33.354249 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:06:33.352114 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 63251.7us
W20260709 13:06:33.355492 20042 log.cc:927] Time spent T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:06:33.355392 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:33.356098 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:33.356441 15862 raft_consensus.cc:1240] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:33.356352 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:33.357087 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:33.354669 19575 consensus_queue.cc:1059] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.116s
I20260709 13:06:33.357824 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:33.358974 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:33.346823 16002 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
W20260709 13:06:33.390252 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 166224us
I20260709 13:06:33.377985 16535 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:33.394435 15857 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:33.399313 16006 raft_consensus.cc:2393] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 10: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:33.401693 15860 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:33.394604 15861 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:33.404197 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:33.405613 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:33.415283 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:33.419728 15859 raft_consensus.cc:2393] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 8: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:06:33.420418 15933 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 2051) took 1965 ms (client timeout 1870 ms). Trace:
I20260709 13:06:33.423262 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:33.422508 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:33.422809 16011 raft_consensus.cc:1275] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:06:33.423744 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:33.424301 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:33.428874 16008 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:06:33.424029 15933 rpcz_store.cc:269] 0709 13:06:31.454848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.140313 (+685465us) service_pool.cc:224] Handling call
0709 13:06:33.415200 (+1274887us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400153e54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:33.420354 (+  5154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2487529728}
I20260709 13:06:33.431928 16002 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:33.434556 15932 raft_consensus.cc:2410] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 8 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:06:33.402467 19575 consensus_queue.cc:1243] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer 85e997ba5003439781ec4f1f8db0c3ee log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:06:33.442497 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 64645.4us
W20260709 13:06:33.443163 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 201317us
I20260709 13:06:33.426841 19959 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:33.442327 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:33.448810 15933 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:06:33.402691 20010 consensus_queue.cc:1243] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer 85e997ba5003439781ec4f1f8db0c3ee log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:06:33.450440 15862 raft_consensus.cc:2393] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 6: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:33.460932 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:33.461745 16006 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:33.460855 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:33.464951 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:33.466652 15857 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 9: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:33.427507 19769 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:33.391717 19651 consensus_queue.cc:260] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:33.479419 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:33.480800 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:33.484553 15934 raft_consensus.cc:2393] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 12: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:33.485666 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:33.486411 15934 raft_consensus.cc:2393] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 13: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:33.487196 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:33.488395 16011 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:06:33.488986 19967 raft_consensus.cc:3055] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:33.489358 19967 raft_consensus.cc:740] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:33.490346 19967 consensus_queue.cc:260] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:06:33.492519 19967 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:33.496572 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:33.497458 19974 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:33.499503 16002 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:06:33.499627 15933 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
I20260709 13:06:33.500559 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:33.500751 16002 raft_consensus.cc:1240] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:33.501328 15933 raft_consensus.cc:2393] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 10: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:33.502024 16002 raft_consensus.cc:1240] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:33.505769 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:33.513779 15932 raft_consensus.cc:2393] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 13: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:33.515260 16005 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
I20260709 13:06:33.502085 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:33.512679 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:33.504302 19939 raft_consensus.cc:3055] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:33.526193 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:33.528976 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:33.531332 16004 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:06:33.529285 15933 raft_consensus.cc:2393] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 10: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:33.445814 19350 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:33.533735 15853 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:33.535794 19813 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.541206 19651 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:33.541395 19813 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:33.458763 19575 consensus_queue.cc:1059] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.075s
W20260709 13:06:33.548753 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:33.550064 19813 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:33.551187 19869 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:33.556030 15936 raft_consensus.cc:2393] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 8: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
W20260709 13:06:33.559031 20043 log.cc:927] Time spent T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 13:06:33.560163 15974 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:33.561256 19993 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:33.545454 19806 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:33.564174 19995 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.564690 19995 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:06:33.565724 19806 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:33.566708 19995 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:33.563588 15974 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:33.536165 19223 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:33.581468 19223 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:33.581938 15976 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:33.583868 15902 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:33.530998 19939 raft_consensus.cc:740] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:33.585332 19939 consensus_queue.cc:260] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:33.586723 19939 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:33.589018 19350 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:33.591886 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 86757us
I20260709 13:06:33.592778 16533 leader_election.cc:400] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 election: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 7. Current term is 8.
I20260709 13:06:33.593035 16533 leader_election.cc:403] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:06:33.593564 19869 raft_consensus.cc:697] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:33.593717 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:33.594839 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:33.594578 19869 consensus_queue.cc:237] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:33.595494 15932 raft_consensus.cc:2368] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 12. Current term is 13.
I20260709 13:06:33.597905 15928 raft_consensus.cc:2368] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 7. Current term is 8.
I20260709 13:06:33.598223 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:33.598888 16005 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
I20260709 13:06:33.599136 15928 raft_consensus.cc:1275] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:06:33.602535 19806 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.603247 16569 raft_consensus.cc:1275] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:06:33.603632 16569 pending_rounds.cc:85] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting all ops after (but not including) 8
I20260709 13:06:33.603876 16569 pending_rounds.cc:107] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:06:33.604179 16569 raft_consensus.cc:2889] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:33.604230 15827 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:33.605789 15827 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:33.606184 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:33.607990 19660 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:33.608111 15930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 1987) took 2309 ms. Trace:
I20260709 13:06:33.608588 19660 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:33.609057 19660 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:33.608729 15930 rpcz_store.cc:276] 0709 13:06:31.298973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:33.057196 (+1758223us) service_pool.cc:224] Handling call
0709 13:06:33.102909 (+ 45713us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:33.125735 (+ 22826us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:33.126426 (+   691us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:33.135238 (+  8812us) log.cc:844] Serialized 26 byte log entry
0709 13:06:33.403898 (+268660us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:33.403957 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.405883 (+  1926us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:33.607808 (+201925us) raft_consensus.cc:1685] finished
0709 13:06:33.607832 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:33.607990 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1344,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":133,"thread_start_us":256840,"threads_started":1,"wal-append.queue_time_us":468172}
I20260709 13:06:33.611371 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:33.605737 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:33.611901 19937 raft_consensus.cc:2749] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:33.612072 15930 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:06:33.613278 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:33.614396 15930 raft_consensus.cc:2393] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 11: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:33.615252 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:33.616436 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:33.617137 15930 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:06:33.619643 15930 raft_consensus.cc:1240] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:33.620146 20035 consensus_queue.cc:1059] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:33.620561 15937 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 1980) took 2280 ms. Trace:
I20260709 13:06:33.621456 19813 raft_consensus.cc:3055] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:33.621907 19813 raft_consensus.cc:740] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:33.622651 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:33.624193 15931 raft_consensus.cc:2410] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 9 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:06:33.620945 15937 rpcz_store.cc:276] 0709 13:06:31.297199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.977754 (+1680555us) service_pool.cc:224] Handling call
0709 13:06:33.022952 (+ 45198us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:33.023643 (+   691us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:06:33.023721 (+    78us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:33.024413 (+   692us) log.cc:844] Serialized 26 byte log entry
0709 13:06:33.082511 (+ 58098us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:33.082552 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.082649 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:33.569186 (+486537us) raft_consensus.cc:1685] finished
0709 13:06:33.569206 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:33.569343 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9451,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":199,"thread_start_us":57646,"threads_started":1,"wal-append.queue_time_us":317771}
I20260709 13:06:33.625908 19769 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:33.627939 19917 consensus_meta.cc:287] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.696s	user 0.008s	sys 0.005s
I20260709 13:06:33.628372 20051 raft_consensus.cc:3055] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:33.628470 19917 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:33.630625 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 67135.4us
I20260709 13:06:33.631227 19917 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:33.632901 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:33.632174 19993 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:33.629182 20051 raft_consensus.cc:740] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:33.631462 15974 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:33.607329 15827 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:33.656852 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 75136.3us
I20260709 13:06:33.603083 19806 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:33.632858 15934 raft_consensus.cc:2393] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 11: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:33.652799 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:33.655988 16566 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 9: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
W20260709 13:06:33.656082 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 63504.1us
I20260709 13:06:33.622997 19813 consensus_queue.cc:260] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:33.655193 19993 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:33.667847 19993 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:33.668360 19806 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:33.669768 19813 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:33.633814 19894 consensus_meta.cc:287] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.562s	user 0.010s	sys 0.000s
I20260709 13:06:33.671082 19894 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:33.671931 19350 raft_consensus.cc:2749] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:33.673280 19894 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:33.674585 19996 raft_consensus.cc:3055] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:33.674942 19996 raft_consensus.cc:740] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:06:33.676370 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 69135.4us
I20260709 13:06:33.676123 19996 consensus_queue.cc:260] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:33.677397 19917 raft_consensus.cc:2804] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:33.677693 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:33.677984 15854 raft_consensus.cc:1240] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:33.661530 19776 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:33.677423 19758 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:33.678866 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:33.679627 16568 raft_consensus.cc:2393] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 11: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:33.679247 19758 raft_consensus.cc:697] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:33.677856 19917 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:33.680397 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:33.680527 19917 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:33.686980 15861 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:06:33.696592 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:33.697242 15860 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
W20260709 13:06:33.698760 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:33.698328 16533 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:33.676076 19881 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:33.677376 19996 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:33.709995 16008 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:06:33.710587 15857 raft_consensus.cc:1240] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:33.707993 19881 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:33.714648 19881 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:33.716032 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:33.716655 16562 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:06:33.718483 15756 catalog_manager.cc:5697] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 6 to 7, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 7 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:33.721315 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:33.721220 15756 catalog_manager.cc:5697] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 3 to 9. New cstate: current_term: 9 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:33.722067 15860 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:33.723969 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:33.724617 16566 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:33.726790 15976 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:33.727692 20007 raft_consensus.cc:2804] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:33.728760 15976 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:33.729755 19728 raft_consensus.cc:2804] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:33.730288 19728 raft_consensus.cc:697] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 9 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:06:33.732025 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 54565.2us
I20260709 13:06:33.731266 19728 consensus_queue.cc:237] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:33.734617 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:33.734835 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:33.735402 15858 raft_consensus.cc:2393] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 11: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:33.736150 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:33.736606 15861 raft_consensus.cc:1240] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 11. Current term is 12. Ops: [7.6-11.8]
I20260709 13:06:33.736898 15856 raft_consensus.cc:2393] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 11: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:33.737998 15861 raft_consensus.cc:1275] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:06:33.736223 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:33.739199 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:33.744025 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 203520us
I20260709 13:06:33.744539 15853 raft_consensus.cc:2393] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 11: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:33.740091 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:33.742846 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:33.745690 20051 consensus_queue.cc:260] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:33.746956 20051 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:33.738049 15857 raft_consensus.cc:1240] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:33.689603 19223 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:33.752813 19223 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:33.753553 19223 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:33.754321 19955 consensus_queue.cc:1048] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:06:33.719621 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:33.758524 19223 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:33.764780 16006 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:06:33.711481 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:33.761394 15756 catalog_manager.cc:5697] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:06:33.767174 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 103210us
I20260709 13:06:33.680855 19758 consensus_queue.cc:237] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:33.776516 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:33.777989 15857 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 9: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:33.713547 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:33.782554 15933 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 9: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:06:33.783797 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:33.772195 19814 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:33.789929 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:33.810727 15854 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:33.811523 16562 raft_consensus.cc:2393] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 10: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:33.812798 16566 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
I20260709 13:06:33.814273 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:33.823382 15860 raft_consensus.cc:2468] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
I20260709 13:06:33.817919 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:06:33.827761 19990 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.747s	user 0.009s	sys 0.028s
W20260709 13:06:33.828166 19990 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.893s	user 0.009s	sys 0.028s
I20260709 13:06:33.828541 15929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2058) took 2349 ms. Trace:
I20260709 13:06:33.828977 15929 rpcz_store.cc:276] 0709 13:06:31.479525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:33.218518 (+1738993us) service_pool.cc:224] Handling call
0709 13:06:33.258634 (+ 40116us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:33.356719 (+ 98085us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:33.357003 (+   284us) log.cc:844] Serialized 14 byte log entry
0709 13:06:33.828213 (+471210us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:33.828269 (+    56us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:33.828313 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.828374 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:33.828483 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33986,"prepare.queue_time_us":1454,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"thread_start_us":428030,"threads_started":1}
W20260709 13:06:33.831895 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 45027.7us
I20260709 13:06:33.832726 15825 leader_election.cc:400] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 election: Vote denied by peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) with higher term. Message: Invalid argument: T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 12. Current term is 13.
I20260709 13:06:33.833011 15825 leader_election.cc:403] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:06:33.833578 19660 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:33.834393 15825 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:33.809718 19814 raft_consensus.cc:697] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:06:33.843161 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:33.835651 15825 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:33.851053 15854 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 9.
I20260709 13:06:33.853794 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:33.854624 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:33.857123 16007 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.037s	sys 0.019s
W20260709 13:06:33.857667 16007 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.037s	sys 0.019s
I20260709 13:06:33.869424 16568 raft_consensus.cc:1240] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:33.906342 15862 raft_consensus.cc:1240] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 10. Current term is 11. Ops: [8.8-10.9]
I20260709 13:06:33.907299 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:33.909744 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:33.910547 15853 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:33.913463 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:33.814100 19769 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:33.916558 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:33.920006 16566 raft_consensus.cc:2368] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 7. Current term is 8.
I20260709 13:06:33.919355 16562 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:06:33.841789 19995 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:33.836566 19814 consensus_queue.cc:237] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:33.923039 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:33.840283 19955 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) with higher term. Message: Invalid argument: T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 12. Current term is 13.
I20260709 13:06:33.877780 19894 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:33.841625 19937 consensus_queue.cc:1048] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.081s
I20260709 13:06:33.940260 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:33.941731 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:33.937328 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:33.945425 16562 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
I20260709 13:06:33.873893 19660 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:33.922442 19995 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:33.939321 19806 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.934924 19894 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:33.966560 19937 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:33.967247 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:33.987025 15853 raft_consensus.cc:2393] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 12: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:06:33.968309 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:33.972002 16569 pending_rounds.cc:85] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting all ops after (but not including) 6
I20260709 13:06:33.966220 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:33.988108 16569 pending_rounds.cc:107] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:06:33.988467 16569 raft_consensus.cc:2889] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:33.988734 15929 raft_consensus.cc:2410] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 9 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:06:33.988734 16533 leader_election.cc:400] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Vote denied by peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) with higher term. Message: Invalid argument: T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 9. Current term is 10.
I20260709 13:06:33.989745 16533 leader_election.cc:403] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:06:33.989545 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:33.991021 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:33.916517 19881 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:34.001317 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:34.015774 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:34.016297 19974 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:34.016642 16561 raft_consensus.cc:2368] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 12. Current term is 13.
I20260709 13:06:34.017664 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:34.018281 15929 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:34.019381 20048 log.cc:927] Time spent T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 13:06:34.020010 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:34.021962 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:34.026181 15756 catalog_manager.cc:5697] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:34.027347 16568 raft_consensus.cc:2393] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 8: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:34.025446 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:34.023010 16567 raft_consensus.cc:1240] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:34.029232 16562 raft_consensus.cc:2393] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 12: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:34.027104 16564 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:34.028386 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:33.985749 19776 raft_consensus.cc:2749] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 7. Current term is 8.
I20260709 13:06:33.988767 19894 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:34.028376 15756 catalog_manager.cc:5697] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 4 to 7, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 7 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:34.031773 16533 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:33.975912 19995 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:34.031555 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:34.053434 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:34.049788 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:34.054772 15853 raft_consensus.cc:2393] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 9: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:06:34.048813 15859 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.000s	sys 0.021s
W20260709 13:06:34.056354 15859 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.000s	sys 0.022s
I20260709 13:06:34.053547 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:33.962265 19769 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:34.062979 16563 raft_consensus.cc:2393] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 8: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:34.068109 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:34.069609 16561 raft_consensus.cc:1240] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:34.091145 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:33.996187 19993 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:34.098205 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:34.100057 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 68388.3us
I20260709 13:06:34.003007 19806 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:34.089915 16570 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:06:34.104691 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:34.108959 16563 raft_consensus.cc:2393] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 10: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:34.107326 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:34.101751 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:34.110975 16561 raft_consensus.cc:2393] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 13: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
W20260709 13:06:34.111157 16565 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.019s	sys 0.054s
I20260709 13:06:34.106175 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:34.112424 16567 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:06:34.113880 15859 raft_consensus.cc:2368] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 7. Current term is 8.
I20260709 13:06:34.116207 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:34.117197 16566 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 9: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
W20260709 13:06:34.108665 19939 consensus_meta.cc:287] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.519s	user 0.003s	sys 0.014s
I20260709 13:06:34.120527 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:34.108383 16011 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.014s	sys 0.015s
W20260709 13:06:34.128176 16011 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.014s	sys 0.015s
W20260709 13:06:34.111852 16565 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.020s	sys 0.054s
I20260709 13:06:34.125481 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:34.106158 15756 catalog_manager.cc:5697] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 4 to 8, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 8 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
I20260709 13:06:34.105923 20035 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:34.127414 16562 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:34.056609 19937 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:34.145563 15936 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.016s	sys 0.015s
W20260709 13:06:34.146225 15936 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.016s	sys 0.016s
I20260709 13:06:34.147195 15936 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2063) took 1768 ms. Trace:
I20260709 13:06:34.147584 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:34.150864 16568 raft_consensus.cc:2410] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:06:34.153120 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:34.152281 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:34.156486 16561 raft_consensus.cc:2393] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 8: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
W20260709 13:06:34.157974 15932 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.024s	sys 0.028s
I20260709 13:06:34.106294 19917 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:34.159410 15932 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.024s	sys 0.028s
I20260709 13:06:34.153857 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:34.168686 15932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2071) took 1602 ms. Trace:
W20260709 13:06:34.149139 19869 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.010s	sys 0.021s
I20260709 13:06:34.175824 15859 raft_consensus.cc:2393] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 10: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:34.169096 15932 rpcz_store.cc:276] 0709 13:06:32.566323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:33.596231 (+1029908us) service_pool.cc:224] Handling call
0709 13:06:33.596567 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:33.596879 (+   312us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:06:33.597162 (+   283us) log.cc:844] Serialized 14 byte log entry
0709 13:06:33.725513 (+128351us) spinlock_profiling.cc:243] Waited 8.41 ms on lock 0x7b180037a5c0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb3f3 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd 000000000038f354 000000000038f199 00007f73e729e83a
0709 13:06:34.168375 (+442862us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:34.168427 (+    52us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:34.168467 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:34.168541 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:34.168642 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":30973,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"spinlock_wait_cycles":19343104,"thread_start_us":524530,"threads_started":1}
I20260709 13:06:34.183079 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:34.155383 20035 raft_consensus.cc:697] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:06:34.188256 15928 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.005s	sys 0.048s
I20260709 13:06:34.147831 15936 rpcz_store.cc:276] 0709 13:06:32.379308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:33.556845 (+1177537us) service_pool.cc:224] Handling call
0709 13:06:33.557286 (+   441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:33.557577 (+   291us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:33.557973 (+   396us) log.cc:844] Serialized 14 byte log entry
0709 13:06:34.146819 (+588846us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:34.146872 (+    53us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:34.146910 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:34.146975 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:34.147129 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7770,"prepare.queue_time_us":21454,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":1872,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":4172,"thread_start_us":588300,"threads_started":1}
I20260709 13:06:34.189359 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:06:34.191684 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 431353us
W20260709 13:06:34.189639 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 405475us
I20260709 13:06:34.192515 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:34.204339 15928 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.005s	sys 0.048s
I20260709 13:06:34.204934 15974 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:06:34.187850 19869 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.010s	sys 0.021s
W20260709 13:06:34.208185 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 87961.2us
I20260709 13:06:34.212049 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:34.216797 15932 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:34.212399 15756 catalog_manager.cc:5697] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 4 to 7, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 7 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:06:34.221071 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:34.215564 16533 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:34.222666 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:34.224349 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:34.225369 15933 raft_consensus.cc:2393] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 12: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
W20260709 13:06:34.227300 15934 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.009s	sys 0.026s
I20260709 13:06:34.209944 19881 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:34.236366 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:34.232803 15934 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.009s	sys 0.027s
I20260709 13:06:34.247831 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:34.235937 19813 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:34.250819 15928 raft_consensus.cc:1275] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:06:34.252475 15928 raft_consensus.cc:1275] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:06:34.253134 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:34.253856 15857 raft_consensus.cc:2393] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 13: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
W20260709 13:06:34.255887 20053 log.cc:927] Time spent T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 13:06:34.256119 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:34.257021 15862 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
W20260709 13:06:34.258060 20054 log.cc:927] Time spent T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 13:06:34.258467 15858 raft_consensus.cc:1240] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 5. Current term is 6. Ops: []
I20260709 13:06:34.259478 16007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 2106) took 1032 ms. Trace:
I20260709 13:06:34.260040 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:34.259981 16007 rpcz_store.cc:276] 0709 13:06:33.226828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:33.232728 (+  5900us) service_pool.cc:224] Handling call
0709 13:06:33.249693 (+ 16965us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:33.322791 (+ 73098us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:33.322840 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:33.353804 (+ 30964us) log.cc:844] Serialized 26 byte log entry
0709 13:06:33.859280 (+505476us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:33.859342 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.859444 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:34.259207 (+399763us) raft_consensus.cc:1685] finished
0709 13:06:34.259233 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:34.259391 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":19722,"raft.run_cpu_time_us":297,"raft.run_wall_time_us":21091,"thread_start_us":503893,"threads_started":1,"wal-append.queue_time_us":740569}
I20260709 13:06:34.261677 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:34.262692 15861 raft_consensus.cc:2393] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 11: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
W20260709 13:06:34.264881 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:34.266562 16010 raft_consensus.cc:1240] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:34.273694 19881 consensus_queue.cc:1059] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:06:34.279805 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:34.284159 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 68699.2us
I20260709 13:06:34.285727 16533 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:34.286831 19813 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:34.290270 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:34.291185 15853 raft_consensus.cc:2393] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 11: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:34.289362 16533 leader_election.cc:400] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 7 election: Vote denied by peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) with higher term. Message: Invalid argument: T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 7. Current term is 8.
I20260709 13:06:34.298379 15929 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:06:34.299876 15932 raft_consensus.cc:1240] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:34.303241 16564 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
I20260709 13:06:34.275106 19881 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:34.226653 20007 raft_consensus.cc:697] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 9 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:06:34.306185 15930 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.021s	sys 0.034s
W20260709 13:06:34.304462 20057 log.cc:927] Time spent T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
W20260709 13:06:34.309156 19728 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.012s	sys 0.020s
W20260709 13:06:34.321656 16003 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.020s	sys 0.030s
I20260709 13:06:34.321976 15827 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:34.305636 19990 raft_consensus.cc:2749] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) with higher term. Message: Invalid argument: T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 9. Current term is 10.
W20260709 13:06:34.311902 15930 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.021s	sys 0.034s
W20260709 13:06:34.326592 16003 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.020s	sys 0.030s
I20260709 13:06:34.311244 19806 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:34.311652 19993 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:34.308460 19813 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:06:34.336000 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:34.341969 16003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1965) took 1078 ms. Trace:
W20260709 13:06:34.324147 19728 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.013s	sys 0.021s
I20260709 13:06:34.342458 16003 rpcz_store.cc:276] 0709 13:06:33.263253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:33.263407 (+   154us) service_pool.cc:224] Handling call
0709 13:06:33.263751 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:33.288635 (+ 24884us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:06:33.655238 (+366603us) log.cc:844] Serialized 14 byte log entry
0709 13:06:34.341548 (+686310us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:34.341612 (+    64us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:34.341671 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:34.341765 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:34.341912 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":621123,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":3287,"thread_start_us":972134,"threads_started":2}
I20260709 13:06:34.338531 19974 raft_consensus.cc:2749] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:06:34.394716 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 79379us
I20260709 13:06:34.375869 19223 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:34.387482 19993 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:34.329008 19894 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:34.413762 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:34.322925 20035 consensus_queue.cc:237] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:34.392876 19917 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:34.416258 19993 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:34.410661 19728 raft_consensus.cc:3055] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:34.400630 19223 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:34.400748 19813 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:34.448172 15860 raft_consensus.cc:2393] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 9: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:06:34.452740 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:34.454649 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:34.456981 20056 log.cc:927] Time spent T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.351s	user 0.001s	sys 0.001s
I20260709 13:06:34.457876 19769 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:34.437175 19993 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:34.435163 19728 raft_consensus.cc:740] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:34.457458 15756 catalog_manager.cc:5697] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 5 to 8, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 8 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:34.510186 15937 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 806) took 1146 ms. Trace:
I20260709 13:06:34.513284 15935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 1982) took 3206 ms. Trace:
W20260709 13:06:34.517621 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:34.512595 15937 rpcz_store.cc:276] 0709 13:06:33.366707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:33.627332 (+260625us) service_pool.cc:224] Handling call
0709 13:06:33.627620 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:33.655344 (+ 27724us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:06:33.960227 (+304883us) log.cc:844] Serialized 14 byte log entry
0709 13:06:34.509844 (+549617us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:34.509893 (+    49us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:34.509940 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:34.510014 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:34.510133 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":70755,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":27,"raft.queue_time_us":110,"thread_start_us":774115,"threads_started":2}
I20260709 13:06:34.522454 15756 catalog_manager.cc:5697] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:06:34.525393 16533 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:06:34.531083 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 338687us
I20260709 13:06:34.513733 15935 rpcz_store.cc:276] 0709 13:06:31.297728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:33.038508 (+1740780us) service_pool.cc:224] Handling call
0709 13:06:33.039017 (+   509us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:33.039661 (+   644us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:33.302726 (+263065us) log.cc:844] Serialized 14 byte log entry
0709 13:06:33.819328 (+516602us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:33.870268 (+ 50940us) log.cc:844] Serialized 26 byte log entry
0709 13:06:33.870412 (+   144us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:33.870441 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.870501 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:34.503998 (+633497us) raft_consensus.cc:1685] finished
0709 13:06:34.504017 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:34.504145 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27117,"prepare.queue_time_us":48242,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":66741,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":196,"thread_start_us":628138,"threads_started":2,"wal-append.queue_time_us":646447}
W20260709 13:06:34.539790 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 80690.3us
W20260709 13:06:34.530376 19651 consensus_meta.cc:287] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.989s	user 0.014s	sys 0.000s
W20260709 13:06:34.533036 19996 consensus_meta.cc:287] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.801s	user 0.010s	sys 0.006s
I20260709 13:06:34.555486 16533 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:34.520690 19937 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:34.554277 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 349421us
I20260709 13:06:34.426056 20007 consensus_queue.cc:237] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:34.577730 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 482871us
I20260709 13:06:34.557891 19869 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:34.632426 15934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 807) took 1251 ms. Trace:
I20260709 13:06:34.649129 16011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1977) took 1192 ms. Trace:
W20260709 13:06:34.680863 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:34.701663 15931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:58236 (request call id 0) took 2549 ms. Trace:
W20260709 13:06:34.699388 20069 log.cc:927] Time spent T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
W20260709 13:06:34.688992 15900 leader_election.cc:336] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.648s (SENT)
W20260709 13:06:34.666785 20063 log.cc:927] Time spent T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.223s	user 0.001s	sys 0.001s
I20260709 13:06:34.664654 15756 catalog_manager.cc:5697] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 4 to 8, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 8 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:34.680401 19813 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:34.665967 20061 log.cc:927] Time spent T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
I20260709 13:06:34.690536 16010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 1968) took 1340 ms. Trace:
W20260709 13:06:34.665160 20073 log.cc:927] Time spent T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
W20260709 13:06:34.675874 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 143952us
W20260709 13:06:34.664331 20059 log.cc:927] Time spent T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.272s	user 0.000s	sys 0.001s
W20260709 13:06:34.698426 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 50774.8us
I20260709 13:06:34.733942 16565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 882) took 1131 ms. Trace:
W20260709 13:06:34.697561 20066 log.cc:927] Time spent T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.177s	user 0.002s	sys 0.010s
W20260709 13:06:34.685328 15759 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58448 (request call id 309) took 15434 ms (client timeout 15000 ms). Trace:
I20260709 13:06:34.579120 19728 consensus_queue.cc:260] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:34.700286 19894 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:34.684289 19223 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:34.633213 19769 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:06:34.697520 19995 consensus_meta.cc:287] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.605s	user 0.006s	sys 0.006s
W20260709 13:06:34.702246 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:34.667898 20062 log.cc:927] Time spent T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
W20260709 13:06:34.695313 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 113128us
W20260709 13:06:34.721505 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 79695.5us
I20260709 13:06:34.722919 16010 rpcz_store.cc:276] 0709 13:06:33.350353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:33.371180 (+ 20827us) service_pool.cc:224] Handling call
0709 13:06:34.266261 (+895081us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:34.267286 (+  1025us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:34.690352 (+423066us) spinlock_profiling.cc:243] Waited 884 ms on lock 0x7b54000aa0d0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:34.690481 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2032632064}
I20260709 13:06:34.719244 15931 rpcz_store.cc:276] 0709 13:06:32.152161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:33.717596 (+1565435us) service_pool.cc:224] Handling call
0709 13:06:34.701396 (+983800us) spinlock_profiling.cc:243] Waited 792 ms on lock 0x7b54003b8154. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ae735 00007f73ef36ed29 00007f73e88ed1d4 00007f73e88ed162 00007f73e88ed091 00007f73e88ed00c 00007f73e88ec2c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:34.701567 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1821741824}
I20260709 13:06:34.720470 15900 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:34.729233 15930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 797) took 1368 ms. Trace:
W20260709 13:06:34.718341 20064 log.cc:927] Time spent T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
I20260709 13:06:34.717803 16011 rpcz_store.cc:276] 0709 13:06:33.456919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:33.489209 (+ 32290us) service_pool.cc:224] Handling call
0709 13:06:33.489538 (+   329us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:33.489978 (+   440us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:33.490017 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:33.490622 (+   605us) log.cc:844] Serialized 52 byte log entry
0709 13:06:34.128782 (+638160us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:34.183611 (+ 54829us) log.cc:844] Serialized 14 byte log entry
0709 13:06:34.520912 (+337301us) log.cc:844] Serialized 14 byte log entry
0709 13:06:34.521424 (+   512us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:34.521529 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:34.648835 (+127306us) raft_consensus.cc:1685] finished
0709 13:06:34.648864 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:34.649051 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":158,"prepare.queue_time_us":52164,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":5345,"raft.queue_time_us":93145,"raft.run_cpu_time_us":309,"raft.run_wall_time_us":7543,"thread_start_us":974363,"threads_started":2,"wal-append.queue_time_us":848322}
W20260709 13:06:34.741485 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 195082us
I20260709 13:06:34.734344 16565 rpcz_store.cc:276] 0709 13:06:33.602199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:33.602943 (+   744us) service_pool.cc:224] Handling call
0709 13:06:33.603274 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:33.603783 (+   509us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:33.603818 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:33.604334 (+   516us) log.cc:844] Serialized 26 byte log entry
0709 13:06:34.129294 (+524960us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:34.129351 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:34.129447 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:34.733740 (+604293us) raft_consensus.cc:1685] finished
0709 13:06:34.733758 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:34.733889 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1056,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":84,"thread_start_us":524626,"threads_started":1,"wal-append.queue_time_us":787557}
W20260709 13:06:34.741107 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 337514us
I20260709 13:06:34.753695 15756 catalog_manager.cc:5697] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 6 to 9, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 9 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:06:34.753620 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:34.755466 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:34.645714 15934 rpcz_store.cc:276] 0709 13:06:33.381179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:33.693932 (+312753us) service_pool.cc:224] Handling call
0709 13:06:33.704534 (+ 10602us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:33.720440 (+ 15906us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:34.309442 (+589002us) log.cc:844] Serialized 14 byte log entry
0709 13:06:34.632078 (+322636us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:34.632134 (+    56us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:34.632176 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:34.632244 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:34.632365 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":5867,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":48805,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":3879,"thread_start_us":789210,"threads_started":2,"wal-append.queue_time_us":394525}
I20260709 13:06:34.759018 15930 rpcz_store.cc:276] 0709 13:06:33.360726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:33.620393 (+259667us) service_pool.cc:224] Handling call
0709 13:06:33.620714 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:33.621092 (+   378us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:06:34.326155 (+705063us) log.cc:844] Serialized 14 byte log entry
0709 13:06:34.728868 (+402713us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:34.728920 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:34.728999 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:34.729065 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:34.729182 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":104595,"prepare.run_cpu_time_us":22,"prepare.run_wall_time_us":22,"raft.queue_time_us":17950,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":14884,"thread_start_us":1039018,"threads_started":2}
I20260709 13:06:34.763329 16533 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:34.769379 15827 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:34.765290 15825 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:34.771301 15976 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:34.764777 19894 consensus_queue.cc:1048] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:06:34.771720 15756 catalog_manager.cc:5697] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 6 to 7, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 7 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
I20260709 13:06:34.778074 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:34.778738 15934 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:34.757318 19995 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:34.796411 15859 raft_consensus.cc:2393] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 10: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:34.777580 20051 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:34.777055 20075 raft_consensus.cc:2804] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:06:34.786360 15808 auto_rebalancer.cc:893] Removing replace marker failed: Replica 8430db80a39d4868bcf7641c16de35ea is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 9 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:06:34.786615 15974 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:34.787926 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 98980.6us
I20260709 13:06:34.767482 19223 consensus_queue.cc:1048] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:06:34.789891 15976 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:34.773783 19660 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:06:34.768160 19939 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:34.768242 20010 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:34.773468 19728 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:06:34.807523 15808 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 13:06:34.736164 15759 rpcz_store.cc:269] 0709 13:06:19.250661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:19.250874 (+   213us) service_pool.cc:224] Handling call
0709 13:06:34.685044 (+15434170us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:25.329765 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:25.330547 (+   782us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:25.330604 (+    57us) write_op.cc:435] Acquired schema lock
0709 13:06:25.330627 (+    23us) tablet.cc:662] Decoding operations
0709 13:06:25.432902 (+102275us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:25.433013 (+   111us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:25.433051 (+    38us) tablet.cc:685] Acquiring locks for 21 operations
0709 13:06:25.434704 (+  1653us) tablet.cc:701] Row locks acquired
0709 13:06:25.434759 (+    55us) write_op.cc:260] PREPARE: finished
0709 13:06:25.434968 (+   209us) write_op.cc:270] Start()
0709 13:06:25.435142 (+   174us) write_op.cc:276] Timestamp: P: 1783602385434873 usec, L: 0
0709 13:06:25.435166 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:06:25.614681 (+179515us) log.cc:844] Serialized 8290 byte log entry
0709 13:06:26.213516 (+598835us) op_driver.cc:464] REPLICATION: finished
0709 13:06:27.492853 (+1279337us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e729e0ac 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73edddeef0
0709 13:06:27.614135 (+121282us) write_op.cc:301] APPLY: starting
0709 13:06:27.614228 (+    93us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:27.614671 (+   443us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:27.614710 (+    39us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:27.617038 (+  2328us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:27.617408 (+   370us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=21
0709 13:06:27.617451 (+    43us) write_op.cc:312] APPLY: finished
0709 13:06:27.617889 (+   438us) log.cc:844] Serialized 145 byte log entry
0709 13:06:27.703461 (+ 85572us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:27.756522 (+ 53061us) write_op.cc:454] Released schema lock
0709 13:06:27.779890 (+ 23368us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:27.824004 (+ 44114us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:27.935113 (+111109us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":819780,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":1400318,"apply.run_cpu_time_us":15967,"apply.run_wall_time_us":193491,"num_ops":21,"prepare.queue_time_us":264942,"prepare.run_cpu_time_us":19562,"prepare.run_wall_time_us":560202,"raft.queue_time_us":222444,"raft.run_cpu_time_us":31548,"raft.run_wall_time_us":1415769,"replication_time_us":778171,"spinlock_wait_cycles":26825088,"thread_start_us":1968697,"threads_started":5,"wal-append.queue_time_us":461332,"wal-append.run_cpu_time_us":22148,"wal-append.run_wall_time_us":2555680}]]}
W20260709 13:06:34.814568 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 53912us
I20260709 13:06:34.810779 15756 catalog_manager.cc:5697] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 4 to 7, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 7 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:34.818111 15756 catalog_manager.cc:5697] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 3 to 7, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 7 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:34.815176 16533 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:34.820972 15808 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:06:34.822620 20080 raft_consensus.cc:2749] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:34.779457 20083 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:34.824525 20051 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:34.828559 15827 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:34.832134 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:34.799075 20091 raft_consensus.cc:2749] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:34.796424 19993 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:34.846593 15756 catalog_manager.cc:5697] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 2 to 6, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 6 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:06:34.848034 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:34.827965 20075 raft_consensus.cc:697] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:34.828115 19881 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:34.852375 15976 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:34.814006 19939 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:34.841852 19769 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:06:34.852411 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 46270.5us
W20260709 13:06:34.863436 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 76896.5us
I20260709 13:06:34.865692 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:34.869057 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:34.869891 15859 raft_consensus.cc:2393] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 10: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:34.870038 15860 raft_consensus.cc:1275] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:06:34.871865 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:34.872746 15860 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:34.846462 20083 raft_consensus.cc:697] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:34.877678 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:34.865154 15756 catalog_manager.cc:5697] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 4 to 9. New cstate: current_term: 9 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:34.881081 15756 catalog_manager.cc:5697] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 3 to 7, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 7 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:34.874677 15928 raft_consensus.cc:2393] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 12: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:34.876451 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:34.841409 19813 consensus_queue.cc:1059] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:06:34.883700 15930 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 9.
I20260709 13:06:34.862041 19894 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:06:34.896700 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:34.864316 19769 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:34.896642 15931 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:34.905244 15860 raft_consensus.cc:1240] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:34.851158 20071 consensus_queue.cc:1048] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:06:34.910496 16533 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:34.892614 19881 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:34.920578 19660 raft_consensus.cc:697] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 6 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
W20260709 13:06:34.921604 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 105466us
I20260709 13:06:34.923002 16533 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:34.925757 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 73457us
I20260709 13:06:34.906495 19769 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:34.930996 16533 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:34.931619 15756 catalog_manager.cc:5697] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 7 to 8, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 8 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
I20260709 13:06:34.912503 20051 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:34.936306 19993 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:34.936759 19814 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:34.936880 19869 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:34.937189 19814 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:34.938580 15756 catalog_manager.cc:5697] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 9 to 10, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 10 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
I20260709 13:06:34.945014 15827 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:34.946987 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:34.923755 19881 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:34.949707 15902 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:34.929584 19990 raft_consensus.cc:2804] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:34.955475 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:34.954807 15853 raft_consensus.cc:1240] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 8. Current term is 9. Ops: []
W20260709 13:06:34.957252 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:34.958513 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:34.954577 19996 consensus_queue.cc:1059] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:06:34.963657 20035 raft_consensus.cc:3055] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:34.963503 20083 consensus_queue.cc:237] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:34.966347 19995 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:34.875763 20075 consensus_queue.cc:237] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:34.964109 20035 raft_consensus.cc:740] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:34.924993 20091 consensus_queue.cc:1059] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:06:34.969137 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:06:34.971108 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:34.958743 16568 raft_consensus.cc:1275] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:06:34.937578 19814 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:34.982364 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:34.967291 15756 catalog_manager.cc:5697] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 6 to 8, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 8 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:34.980906 19993 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:34.986178 15902 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:34.986346 15900 leader_election.cc:336] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.863s (SENT)
W20260709 13:06:34.981361 15974 connection.cc:256] RPC call timeout handler was delayed by -1.53436753699998s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15180000000000002s for a response
I20260709 13:06:34.992471 15900 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:34.992494 15974 connection.cc:256] RPC call timeout handler was delayed by -1.4875675370000181s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15700000000000003s for a response
W20260709 13:06:34.993141 15974 connection.cc:256] RPC call timeout handler was delayed by -1.4209675370000241s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1644s for a response
W20260709 13:06:34.993412 15974 connection.cc:256] RPC call timeout handler was delayed by -1.3327675369999952s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17420000000000002s for a response
W20260709 13:06:34.993717 15974 connection.cc:256] RPC call timeout handler was delayed by -1.1743293470000253s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.31930000000000003s for a response
W20260709 13:06:34.993974 15974 connection.cc:256] RPC call timeout handler was delayed by -1.0681675370000221s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2036s for a response
W20260709 13:06:34.994230 15974 connection.cc:256] RPC call timeout handler was delayed by -1.0219809300000406s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1821s for a response
I20260709 13:06:34.988415 16569 raft_consensus.cc:2393] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 12: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:34.956514 19881 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:34.996973 15902 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:35.002264 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 57295us
I20260709 13:06:35.002887 16568 raft_consensus.cc:1240] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:35.003289 15827 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:34.994794 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:35.006229 15931 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:35.007535 16569 raft_consensus.cc:1275] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:06:35.005290 15756 catalog_manager.cc:5697] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 6 to 8, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 8 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:34.983615 20010 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:34.978771 19813 consensus_queue.cc:1048] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:06:34.995190 19996 raft_consensus.cc:3055] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:34.992650 20007 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:35.020021 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:34.990533 19990 raft_consensus.cc:697] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:35.041437 19939 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:35.006788 19993 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:35.051927 20097 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.022367 20010 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:35.044044 19939 raft_consensus.cc:2804] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:06:35.140862 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 115137us
I20260709 13:06:35.057753 20088 raft_consensus.cc:2749] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:06:35.143947 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 140714us
I20260709 13:06:35.024176 19996 raft_consensus.cc:740] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:06:35.178905 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 238558us
W20260709 13:06:35.143242 20090 log.cc:927] Time spent T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:06:35.175058 15974 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:35.142305 19869 raft_consensus.cc:697] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:35.142860 20098 raft_consensus.cc:2749] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:35.142627 19758 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:06:35.188247 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:35.142818 19939 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:35.142712 20097 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:35.202112 20087 log.cc:927] Time spent T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.191s	user 0.000s	sys 0.001s
W20260709 13:06:35.208328 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 58222.9us
I20260709 13:06:35.022897 19660 consensus_queue.cc:237] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:35.141451 20007 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:35.225471 15934 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
W20260709 13:06:35.227504 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 230579us
I20260709 13:06:35.197957 19939 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:35.202831 20101 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:35.251921 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:35.253954 15827 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
W20260709 13:06:35.254773 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 79831.3us
I20260709 13:06:35.228116 20051 raft_consensus.cc:2764] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:06:35.261894 15827 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:35.199664 19758 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:06:35.233107 20081 raft_consensus.cc:2749] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:35.265085 20097 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:35.273932 19955 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:35.281072 15974 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:35.283044 20070 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:35.283710 20070 raft_consensus.cc:697] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:35.284327 19651 consensus_queue.cc:1059] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:35.284621 20110 raft_consensus.cc:2749] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:35.285394 20110 raft_consensus.cc:2749] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:35.285742 20071 raft_consensus.cc:3055] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:35.286190 20071 raft_consensus.cc:740] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:35.286391 20110 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.286895 20110 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:35.287163 20071 consensus_queue.cc:260] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:35.288440 20071 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:35.284840 20070 consensus_queue.cc:237] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:35.236418 19917 consensus_queue.cc:1059] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:06:35.283766 20098 raft_consensus.cc:2804] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:35.287353 15974 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:35.282593 19881 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:35.192804 20035 consensus_queue.cc:260] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:35.309535 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:35.297755 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:35.250631 20088 consensus_queue.cc:1048] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
W20260709 13:06:35.300154 20103 log.cc:927] Time spent T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 13:06:35.336427 19814 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:35.340291 19881 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:35.341101 19814 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:35.342638 19758 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:06:35.345577 20112 log.cc:927] Time spent T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:06:35.345961 19769 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:35.283713 19937 consensus_queue.cc:1059] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:06:35.348117 19937 raft_consensus.cc:3055] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:35.348474 19937 raft_consensus.cc:740] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
W20260709 13:06:35.349215 19728 consensus_meta.cc:287] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.520s	user 0.007s	sys 0.002s
W20260709 13:06:35.336186 20104 log.cc:927] Time spent T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.300s	user 0.001s	sys 0.000s
I20260709 13:06:35.349275 19937 consensus_queue.cc:260] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:35.304024 19955 raft_consensus.cc:697] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:35.350797 19937 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:35.351181 19955 consensus_queue.cc:237] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:35.260775 19996 consensus_queue.cc:260] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:35.357019 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 89340.4us
W20260709 13:06:35.368232 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 108116us
I20260709 13:06:35.263096 20101 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:35.371732 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:35.374718 20007 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:35.375842 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:35.376479 20035 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:35.266222 19990 consensus_queue.cc:237] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:35.379241 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:35.380226 16569 raft_consensus.cc:2368] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 7. Current term is 8.
I20260709 13:06:35.380268 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:35.380985 16561 raft_consensus.cc:2376] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10. Re-sending same reply.
I20260709 13:06:35.183197 16533 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:35.382486 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 199410us
I20260709 13:06:35.338256 20098 raft_consensus.cc:697] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 8 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:35.376524 16568 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
I20260709 13:06:35.362879 20051 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.398108 16568 raft_consensus.cc:1275] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:06:35.370910 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 84336.4us
W20260709 13:06:35.394093 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 407759us
I20260709 13:06:35.395306 15931 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
I20260709 13:06:35.396914 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:35.414076 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:35.379163 20076 raft_consensus.cc:3055] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:35.421429 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:35.422562 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:35.422991 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:35.423460 16011 raft_consensus.cc:2410] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:06:35.422592 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:35.427307 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:35.428850 19814 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.429381 19814 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:35.430088 20105 log.cc:927] Time spent T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.282s	user 0.001s	sys 0.000s
I20260709 13:06:35.437085 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:35.441075 16568 raft_consensus.cc:2368] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 9. Current term is 10.
I20260709 13:06:35.382408 19996 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:35.423619 16010 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
W20260709 13:06:35.449664 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 57730.6us
I20260709 13:06:35.450647 16533 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:35.456079 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:35.457607 15900 leader_election.cc:336] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.895s (SENT)
I20260709 13:06:35.306126 19869 consensus_queue.cc:237] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:35.458319 16003 raft_consensus.cc:1275] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:06:35.462628 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:35.462941 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:35.399358 19993 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:35.431555 19814 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:35.465107 16011 raft_consensus.cc:1275] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:06:35.467268 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:35.481302 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:35.482985 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:35.484580 15853 raft_consensus.cc:2393] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 9: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:35.438083 20076 raft_consensus.cc:740] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:35.487509 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:35.468782 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:35.486021 16005 raft_consensus.cc:2393] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 8: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:35.485889 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:35.497064 15933 raft_consensus.cc:2410] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 12 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:06:35.497294 16002 raft_consensus.cc:2393] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 13: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:35.486191 16010 raft_consensus.cc:1240] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:35.486123 20123 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:35.486434 20113 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.486915 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:35.487430 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:35.468784 16533 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:35.487905 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:35.398054 19813 consensus_queue.cc:1059] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 13:06:35.466691 15825 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:35.495441 15900 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:35.453038 20107 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.502422 16004 raft_consensus.cc:2393] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 8: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:35.468629 20091 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:35.489212 20117 log.cc:927] Time spent T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:06:35.507335 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:35.459158 20007 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:35.507174 19723 raft_consensus.cc:2804] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:35.449426 19769 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:35.481539 20111 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:35.510051 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:35.528827 16002 raft_consensus.cc:2393] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 12: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:35.535131 16562 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
W20260709 13:06:35.536942 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:35.510681 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:35.465792 20010 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:35.543236 16003 raft_consensus.cc:1240] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:35.513916 16010 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 9.
I20260709 13:06:35.442557 20051 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:06:35.515254 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:35.510699 15825 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:35.513641 19955 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:35.517432 20124 raft_consensus.cc:2749] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:35.514791 20091 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:35.491055 20101 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:35.545478 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:35.559185 15937 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:35.548504 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:35.550506 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:35.551286 16003 raft_consensus.cc:1240] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 10. Current term is 11. Ops: []
W20260709 13:06:35.557420 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 46782us
I20260709 13:06:35.552268 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:35.562227 16008 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 9: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:35.563586 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:35.566152 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:35.567224 15930 raft_consensus.cc:2410] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 10 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:06:35.573813 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:35.579099 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:35.579962 16570 raft_consensus.cc:2393] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 9: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:35.580717 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:35.582121 19990 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.527091 19723 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:35.593590 19723 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:35.593827 16566 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:35.596841 20122 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:35.583461 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:35.582420 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:35.587549 15934 raft_consensus.cc:1275] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 10. (term mismatch)
I20260709 13:06:35.590945 15931 raft_consensus.cc:1275] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:06:35.550685 19223 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:35.587123 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 165055us
I20260709 13:06:35.602025 16566 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
I20260709 13:06:35.602223 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:35.583461 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:35.590303 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:35.542766 20111 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:06:35.607409 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:35.605715 15934 pending_rounds.cc:85] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 9
I20260709 13:06:35.611018 15934 pending_rounds.cc:107] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 13:06:35.611336 15934 raft_consensus.cc:2889] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:35.612200 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:35.554768 20068 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:35.592983 15936 raft_consensus.cc:1275] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 6 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:06:35.615346 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:35.624214 15930 raft_consensus.cc:2393] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 12: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:35.506150 20123 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:35.621999 19223 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:35.625655 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:35.626704 20123 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:35.630146 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:35.630826 15929 raft_consensus.cc:2393] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 13: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:35.631575 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:35.632164 16002 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 10
W20260709 13:06:35.632460 20121 log.cc:927] Time spent T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 13:06:35.634068 15902 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:06:35.635303 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:35.507910 20113 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:35.644272 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:35.583047 19813 raft_consensus.cc:3055] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:06:35.646117 15900 leader_election.cc:336] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.987s (SENT)
I20260709 13:06:35.646543 15900 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:35.652258 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:35.527529 20007 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:35.653667 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:06:35.653409 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 139212us
W20260709 13:06:35.627643 20125 log.cc:927] Time spent T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 13:06:35.656525 15936 raft_consensus.cc:2468] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:06:35.643147 16002 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
I20260709 13:06:35.657164 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:35.584192 20110 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:35.602751 20100 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:35.737779 20100 raft_consensus.cc:697] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 10 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:35.738804 20100 consensus_queue.cc:237] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:35.589810 19776 raft_consensus.cc:2804] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:35.739542 20007 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:35.741314 19776 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:35.741789 19776 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:35.596288 20119 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.613588 20111 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:35.679333 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 129629us
I20260709 13:06:35.680358 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:35.682106 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:35.627601 19223 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:35.622519 19955 raft_consensus.cc:697] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 9 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:35.711652 16563 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
W20260709 13:06:35.714604 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 61512.7us
I20260709 13:06:35.597365 20076 consensus_queue.cc:260] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:35.655190 19813 raft_consensus.cc:740] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:06:35.726208 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:35.728693 15937 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 10.
I20260709 13:06:35.730829 16004 raft_consensus.cc:1275] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:06:35.555497 20107 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:35.731848 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:35.733103 16011 raft_consensus.cc:2393] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 9: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:35.735896 15825 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:35.810834 16009 raft_consensus.cc:2393] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 8: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:35.743080 20120 raft_consensus.cc:2764] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:06:35.744042 20106 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:35.744863 20091 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:35.745791 19723 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:35.748085 15974 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:35.814054 19955 consensus_queue.cc:237] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:35.510216 20098 consensus_queue.cc:237] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:35.774158 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:35.815879 16002 raft_consensus.cc:2393] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 8: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:35.609692 19990 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:35.822983 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 87185.6us
W20260709 13:06:35.787969 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:35.826066 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:35.826769 16562 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
I20260709 13:06:35.827032 19723 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:35.827275 15861 raft_consensus.cc:1240] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:35.737231 16010 raft_consensus.cc:2393] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 13: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:35.813786 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:35.816582 15827 connection.cc:256] RPC call timeout handler was delayed by -1.6651754359999131s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1515s for a response
W20260709 13:06:35.830312 15827 connection.cc:256] RPC call timeout handler was delayed by -1.6340290990000312s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1602s for a response
W20260709 13:06:35.830655 15827 connection.cc:256] RPC call timeout handler was delayed by -1.6096376439999176s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22260000000000002s for a response
W20260709 13:06:35.830937 15827 connection.cc:256] RPC call timeout handler was delayed by -1.2232754359999944s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2006s for a response
W20260709 13:06:35.831198 15827 connection.cc:256] RPC call timeout handler was delayed by -1.0324754359999133s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2218s for a response
W20260709 13:06:35.831453 15827 connection.cc:256] RPC call timeout handler was delayed by -1.0223887219999597s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2826s for a response
I20260709 13:06:35.710306 20010 consensus_queue.cc:1059] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.124s
I20260709 13:06:35.829527 16561 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
I20260709 13:06:35.836123 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:35.826284 15902 leader_election.cc:400] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 7. Current term is 8.
I20260709 13:06:35.836933 15902 leader_election.cc:403] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:06:35.836875 16008 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:35.807835 20071 consensus_meta.cc:287] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.514s	user 0.007s	sys 0.003s
I20260709 13:06:35.614344 20122 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:35.840629 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 92688.7us
I20260709 13:06:35.844453 16006 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:06:35.849906 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:06:35.850123 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 203989us
W20260709 13:06:35.851624 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:35.834538 20068 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:35.838613 19917 raft_consensus.cc:2804] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:35.849236 20106 raft_consensus.cc:2749] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 7. Current term is 8.
I20260709 13:06:35.876716 15936 raft_consensus.cc:1275] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:06:35.880151 15931 raft_consensus.cc:1240] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:35.881297 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:35.801512 20119 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:35.872237 19917 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:35.889714 19917 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:35.892577 15974 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:35.893596 20007 consensus_queue.cc:1059] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:35.894601 20007 raft_consensus.cc:2749] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:35.895351 20007 raft_consensus.cc:3055] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:35.899001 20007 raft_consensus.cc:740] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:35.899883 20007 consensus_queue.cc:260] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:35.901090 20007 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:35.849510 20076 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:35.905973 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:35.906738 15930 raft_consensus.cc:2393] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 9: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:35.897627 15974 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:35.898628 16002 raft_consensus.cc:1275] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:06:35.917307 15937 raft_consensus.cc:1275] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:06:35.841065 19723 consensus_queue.cc:1059] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:06:35.922161 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:35.925541 15936 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 9.
I20260709 13:06:35.928483 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:35.929059 15933 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:35.929865 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:35.931665 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 105473us
W20260709 13:06:35.933290 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 75585.9us
I20260709 13:06:35.934507 15827 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:35.938800 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:35.923030 19939 consensus_meta.cc:287] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.670s	user 0.006s	sys 0.004s
I20260709 13:06:35.849436 19974 consensus_queue.cc:1048] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 13:06:35.970849 15931 raft_consensus.cc:1275] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:06:35.971227 16008 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:06:35.950018 15902 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:35.972406 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 74824.8us
I20260709 13:06:35.973466 15974 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:35.857863 20071 consensus_queue.cc:1048] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.093s
W20260709 13:06:35.980707 20132 log.cc:927] Time spent T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 13:06:35.981534 20127 log.cc:927] Time spent T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:06:35.985015 20113 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:35.986100 19990 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:35.931111 20068 raft_consensus.cc:2764] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:06:35.986129 19917 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:35.987208 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:35.976055 20129 consensus_queue.cc:1059] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.012s
I20260709 13:06:35.988458 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:35.989118 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:36.053092 16004 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
W20260709 13:06:36.051450 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 47014.1us
I20260709 13:06:35.976579 16533 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:35.989753 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:35.969673 19660 raft_consensus.cc:2749] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:35.991706 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:35.992409 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:35.996316 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:35.997277 16005 raft_consensus.cc:1240] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 9 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:35.998086 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:36.058561 15861 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:36.031020 15827 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:36.037336 20134 log.cc:927] Time spent T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:06:36.004617 20051 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:36.047328 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 114776us
I20260709 13:06:36.054241 16009 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 9: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:35.990389 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:06:36.055024 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 81678.6us
W20260709 13:06:36.068115 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 91618.5us
I20260709 13:06:35.962863 20035 consensus_queue.cc:1048] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.087s
W20260709 13:06:36.037447 19937 consensus_meta.cc:287] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.686s	user 0.014s	sys 0.001s
I20260709 13:06:36.070024 15974 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:36.073625 16533 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:36.074621 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:36.077363 15933 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 9.
I20260709 13:06:36.081748 15902 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:36.084355 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:36.085166 16008 raft_consensus.cc:2376] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10. Re-sending same reply.
I20260709 13:06:36.086473 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:36.087464 16005 raft_consensus.cc:2393] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 10: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:36.090026 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:36.090847 16004 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:36.091948 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:35.974357 19939 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:36.053752 19728 raft_consensus.cc:2749] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:36.053577 20120 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:36.112766 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:06:36.113423 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 134805us
I20260709 13:06:36.114687 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:36.104557 16007 raft_consensus.cc:2368] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 12. Current term is 13.
W20260709 13:06:36.108022 15902 leader_election.cc:336] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 1.987s (SENT)
W20260709 13:06:36.117329 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 47390.5us
I20260709 13:06:36.115362 16011 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:06:36.126683 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:36.080678 20113 raft_consensus.cc:2749] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:36.131700 15935 raft_consensus.cc:1240] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:36.104897 20051 raft_consensus.cc:2804] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:36.089638 19955 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:36.109009 20091 raft_consensus.cc:2804] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:36.139396 16009 raft_consensus.cc:2368] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 7. Current term is 8.
I20260709 13:06:36.117851 20068 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:06:36.143896 19996 consensus_meta.cc:287] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.690s	user 0.006s	sys 0.006s
I20260709 13:06:36.137542 19769 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:36.137992 20051 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:06:36.144330 20138 log.cc:927] Time spent T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:06:36.129822 20120 raft_consensus.cc:697] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 10 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:36.144810 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:36.177238 16007 raft_consensus.cc:2393] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 11: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:36.145402 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:06:36.177440 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:36.177827 16005 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
I20260709 13:06:36.146734 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:36.148236 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:36.149014 15827 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:36.150097 16567 raft_consensus.cc:1240] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 9. Current term is 10. Ops: [7.8-9.9]
I20260709 13:06:36.152125 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:36.153314 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:36.154386 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:36.154649 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:36.181519 16002 raft_consensus.cc:2376] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12. Re-sending same reply.
I20260709 13:06:36.156622 20113 consensus_queue.cc:1059] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:36.158449 20133 consensus_queue.cc:1059] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:36.160383 16535 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:36.163115 16565 raft_consensus.cc:1275] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:06:36.105615 19723 consensus_queue.cc:1048] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
W20260709 13:06:36.174157 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 66192.2us
I20260709 13:06:36.142861 20091 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:36.181016 15976 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:36.179527 16010 raft_consensus.cc:2376] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8. Re-sending same reply.
I20260709 13:06:36.187180 15974 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:36.178860 16004 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
I20260709 13:06:36.184027 19728 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:36.067420 16563 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1"
dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
mode: GRACEFUL
new_leader_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:42622
I20260709 13:06:36.191550 16563 raft_consensus.cc:606] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Received request to transfer leadership to TS 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:36.192770 16563 raft_consensus.cc:614] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:36.195515 19728 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:36.196547 19806 raft_consensus.cc:2749] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:06:36.176352 20051 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:36.129120 19776 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:36.203974 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 55034.6us
I20260709 13:06:36.206774 15827 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:36.194813 16535 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:36.189113 20075 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:36.192055 20091 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:36.212646 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 119322us
I20260709 13:06:36.213631 16533 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:36.215199 15902 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:36.220811 15902 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:36.202802 20107 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:36.222054 19728 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:36.212898 19814 consensus_queue.cc:1059] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 13:06:36.222599 19728 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:36.223425 19728 consensus_queue.cc:1048] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:36.231127 15827 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:36.223534 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:36.232347 15931 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:36.180397 19769 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:36.219616 16566 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 10.
I20260709 13:06:36.234135 19996 raft_consensus.cc:2804] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:06:36.238967 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:36.209707 20007 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:36.191288 20123 consensus_queue.cc:1059] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:06:36.247982 20098 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:06:36.254846 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 60113.6us
I20260709 13:06:36.264286 15902 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:36.258172 19814 raft_consensus.cc:2804] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:06:36.268230 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 81123.7us
I20260709 13:06:36.268579 19814 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:36.269074 15974 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:36.269124 19814 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:36.276285 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 45213.3us
I20260709 13:06:36.241293 19996 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:36.277046 15827 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:36.252758 19937 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:36.254918 20035 raft_consensus.cc:2749] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:36.255645 20098 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:36.252823 20007 raft_consensus.cc:697] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:36.279881 15902 leader_election.cc:400] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 12. Current term is 13.
I20260709 13:06:36.282356 15902 leader_election.cc:403] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:36.277820 15931 raft_consensus.cc:2468] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:06:36.276208 20110 raft_consensus.cc:2749] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:36.266031 20123 raft_consensus.cc:3055] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:36.301155 15902 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:36.296726 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 43516.9us
I20260709 13:06:36.275996 19917 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:36.280355 19996 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:36.261219 20071 consensus_queue.cc:1059] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:06:36.281924 20097 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:36.286933 19937 raft_consensus.cc:697] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 10 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:36.286774 20098 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:36.310387 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:36.312558 16566 raft_consensus.cc:2410] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:06:36.282975 20106 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:36.303812 20115 raft_consensus.cc:2749] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 12. Current term is 13.
I20260709 13:06:36.313709 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:36.314507 20152 raft_consensus.cc:2804] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:36.261165 20120 consensus_queue.cc:237] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:36.299470 19939 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:36.310087 15902 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:36.315834 19769 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:36.320861 20115 raft_consensus.cc:2804] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:36.320973 15902 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:36.321313 20115 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:36.321715 20115 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:36.321774 20107 raft_consensus.cc:2804] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:36.322271 20107 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:36.322597 20107 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:36.324200 20145 log.cc:927] Time spent T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:06:36.316474 20106 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:36.328590 20106 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:36.311748 20110 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:36.330415 20107 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:36.301640 20123 raft_consensus.cc:740] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:36.331602 20110 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:36.332198 20110 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:36.332680 20107 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:36.332460 20123 consensus_queue.cc:260] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:36.333882 20123 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:36.334774 20110 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:36.336670 20119 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:36.340792 20122 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:36.343122 15827 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:36.300256 19723 consensus_queue.cc:1048] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:06:36.345261 15827 leader_election.cc:400] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 9. Current term is 10.
I20260709 13:06:36.345564 15827 leader_election.cc:403] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:36.347124 20100 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:36.347384 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:36.347142 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:36.348904 19990 raft_consensus.cc:3055] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:36.349305 19937 consensus_queue.cc:237] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:36.350008 19955 raft_consensus.cc:2749] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:36.357384 15827 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:36.350610 19814 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:36.349300 19990 raft_consensus.cc:740] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:36.335462 20091 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:36.354903 20115 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:36.359459 15827 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:36.316689 20153 consensus_queue.cc:1048] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:06:36.363545 20141 raft_consensus.cc:2749] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:06:36.365314 20142 log.cc:927] Time spent T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.113s	user 0.001s	sys 0.002s
I20260709 13:06:36.363945 15827 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:36.367363 20141 raft_consensus.cc:2804] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:36.368081 20141 raft_consensus.cc:697] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 10 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:36.369992 20115 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:36.370422 20111 consensus_meta.cc:287] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.565s	user 0.008s	sys 0.000s
I20260709 13:06:36.361050 19223 raft_consensus.cc:2749] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:36.358476 20152 raft_consensus.cc:697] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 9 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:36.360139 20100 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:06:36.315219 20007 consensus_queue.cc:237] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:06:36.379053 16561 raft_consensus.cc:2393] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 10: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:36.380962 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:36.381708 16563 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:36.384869 15825 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:06:36.386698 20151 log.cc:927] Time spent T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:06:36.387365 19776 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:36.391115 16563 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
W20260709 13:06:36.391803 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 121162us
I20260709 13:06:36.392601 15827 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:36.369227 20141 consensus_queue.cc:237] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:36.358790 19917 consensus_queue.cc:1048] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:06:36.398962 15902 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:36.413813 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:36.418252 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:36.401832 19955 raft_consensus.cc:2804] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:36.420233 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:36.441617 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:36.442925 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:36.444023 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:36.445521 15934 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:36.446350 15937 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:36.408301 20115 raft_consensus.cc:2804] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:36.376495 20111 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:36.454885 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:36.449337 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 167644us
I20260709 13:06:36.456951 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:36.457998 15929 raft_consensus.cc:2393] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 10: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:36.460906 15934 raft_consensus.cc:1275] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:06:36.420050 20158 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:36.448750 19814 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:36.460057 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:36.467473 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:36.461493 16535 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:36.479166 15853 raft_consensus.cc:1275] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:06:36.481473 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:36.481667 20165 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:36.481839 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:36.482506 15859 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
I20260709 13:06:36.461890 20160 consensus_queue.cc:1059] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:06:36.483639 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:36.467136 20152 consensus_queue.cc:237] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:36.484433 15854 raft_consensus.cc:2393] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 9: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:36.486552 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:36.487004 20098 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:36.489192 20098 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:36.490424 15855 raft_consensus.cc:1240] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:36.459138 20115 raft_consensus.cc:697] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 10 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:06:36.498343 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 105833us
I20260709 13:06:36.500072 15827 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:36.500723 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:36.501080 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 43625.4us
I20260709 13:06:36.478539 20051 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:36.511981 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:36.514220 16011 raft_consensus.cc:1240] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:06:36.514048 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:36.515489 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 212513us
I20260709 13:06:36.489946 20098 consensus_queue.cc:1048] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:06:36.516577 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:36.517671 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:36.519284 16011 raft_consensus.cc:1275] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:06:36.522009 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:36.522477 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:36.523773 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:36.524714 16563 raft_consensus.cc:2393] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 10: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:36.526328 16561 raft_consensus.cc:1275] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:06:36.526593 15933 raft_consensus.cc:1240] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:36.522130 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:36.527107 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:36.528684 15862 raft_consensus.cc:2368] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 9. Current term is 10.
I20260709 13:06:36.506157 20129 raft_consensus.cc:2749] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:36.533973 16011 raft_consensus.cc:1275] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:06:36.530048 16535 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:36.530819 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:36.531742 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:36.534871 16568 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:36.533126 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:36.507030 19660 raft_consensus.cc:2804] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:36.538770 19813 consensus_queue.cc:260] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:36.539775 20156 raft_consensus.cc:3055] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:36.539875 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:36.540181 19813 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:36.470453 20158 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:36.541198 15858 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
I20260709 13:06:36.542040 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:36.542261 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:36.540252 20156 raft_consensus.cc:740] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:36.544891 15853 raft_consensus.cc:2393] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 12: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:36.546286 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:36.544394 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:36.544569 15862 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
I20260709 13:06:36.540802 19814 raft_consensus.cc:2749] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:36.548379 16564 raft_consensus.cc:1240] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:36.537457 20091 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:36.534428 19955 raft_consensus.cc:697] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:36.531839 20115 consensus_queue.cc:237] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:36.537207 19660 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:06:36.555424 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 134656us
I20260709 13:06:36.555475 19660 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:36.554926 15936 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:36.556533 20051 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:36.556757 20148 log.cc:927] Time spent T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.303s	user 0.001s	sys 0.000s
I20260709 13:06:36.543300 20172 consensus_queue.cc:1048] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:06:36.579859 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:36.580564 16567 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:36.581574 15974 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:36.585544 15827 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:36.586678 16535 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:36.587901 16535 leader_election.cc:304] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:36.589066 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 70883.4us
I20260709 13:06:36.590155 20177 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:36.587875 15827 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:36.558559 20147 log.cc:927] Time spent T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.362s	user 0.002s	sys 0.000s
I20260709 13:06:36.584707 20172 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:06:36.594050 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 630089us
W20260709 13:06:36.604413 20164 log.cc:927] Time spent T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:06:36.610224 20111 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:36.806674 16567 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
I20260709 13:06:36.594317 20091 consensus_queue.cc:1048] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:36.581213 19937 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:36.593533 19814 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:36.738670 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:36.808272 16011 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 9.
I20260709 13:06:36.741731 16568 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
W20260709 13:06:36.738165 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:36.803110 15827 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:36.586072 20088 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:36.812426 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 72410.5us
I20260709 13:06:36.812870 15902 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:36.813472 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 225612us
I20260709 13:06:36.814167 16535 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:36.815506 16535 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:36.738101 19869 raft_consensus.cc:2749] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:36.825646 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:36.742368 20051 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:36.828177 20051 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:36.831907 16004 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:06:36.833264 20119 consensus_queue.cc:1059] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:36.834044 20119 consensus_queue.cc:1243] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer 31882606e87447fb9a36bbf625129fe8 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:06:36.834271 20051 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:36.835289 20051 raft_consensus.cc:3055] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:36.835850 20051 raft_consensus.cc:740] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:36.837291 20051 consensus_queue.cc:260] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:36.840164 20166 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:36.840737 20166 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:36.842973 20166 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:36.844105 20169 log.cc:927] Time spent T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.224s	user 0.004s	sys 0.001s
I20260709 13:06:36.846123 20158 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:36.825572 20110 raft_consensus.cc:2804] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:36.847805 20175 raft_consensus.cc:2749] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:36.852064 20100 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 9. Current term is 10.
W20260709 13:06:36.853192 20171 log.cc:927] Time spent T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:06:36.864645 20170 log.cc:927] Time spent T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 13:06:36.825946 19814 raft_consensus.cc:697] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:36.866315 19814 consensus_queue.cc:237] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:36.870690 20162 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:06:36.876765 20173 log.cc:927] Time spent T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:06:36.826294 19955 consensus_queue.cc:237] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:36.741273 19939 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:36.883275 16566 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:06:36.886516 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:36.804714 20098 raft_consensus.cc:2749] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:06:36.943717 20167 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:36.886391 20100 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:36.946055 19996 consensus_meta.cc:287] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.635s	user 0.009s	sys 0.011s
W20260709 13:06:36.946981 20174 log.cc:927] Time spent T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 13:06:36.821553 20129 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:36.801990 20107 consensus_queue.cc:1048] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.147s
I20260709 13:06:36.951408 19660 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:36.814324 20075 consensus_queue.cc:1048] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.139s
I20260709 13:06:36.958015 15935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2111) took 1393 ms. Trace:
I20260709 13:06:36.847539 20110 raft_consensus.cc:697] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 10 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:06:36.885126 20136 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.729s	user 0.079s	sys 0.142s
W20260709 13:06:36.971957 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 168966us
I20260709 13:06:36.975247 15935 rpcz_store.cc:276] 0709 13:06:35.564194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:35.568046 (+  3852us) service_pool.cc:224] Handling call
0709 13:06:35.568392 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:36.137720 (+569328us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:36.957715 (+819995us) spinlock_profiling.cc:243] Waited 563 ms on lock 0x7b54003b9054. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:36.957941 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1294973184}
I20260709 13:06:36.968554 19769 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:36.979763 20162 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:36.981428 19769 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:36.981765 20162 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:36.971179 20100 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:36.984057 19769 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:36.985122 20106 consensus_meta.cc:287] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.656s	user 0.007s	sys 0.002s
I20260709 13:06:36.985651 20100 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:36.985526 20106 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:36.808858 20156 consensus_queue.cc:260] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:36.820600 19937 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:36.974802 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 392292us
I20260709 13:06:36.973968 20129 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:36.991778 15974 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:36.979351 20136 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.823s	user 0.095s	sys 0.181s
I20260709 13:06:36.968843 19939 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:36.995283 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 174773us
I20260709 13:06:36.821013 20088 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:36.972046 19996 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:36.996250 15930 raft_consensus.cc:1240] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:36.992187 20175 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:36.989069 20115 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.024523 15974 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:36.995379 20129 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:36.969931 20167 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:37.031481 15900 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:37.032462 15861 raft_consensus.cc:1240] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:37.032469 15827 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:37.080906 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 48521.4us
I20260709 13:06:37.082279 15827 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:37.034109 20156 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:37.034747 20110 consensus_queue.cc:237] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:37.025214 20123 raft_consensus.cc:2749] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:06:37.038936 16562 raft_consensus.cc:1275] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:06:37.027352 19894 consensus_queue.cc:1059] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:06:37.046430 19660 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:37.048715 19996 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:37.050279 20146 log.cc:927] Time spent T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.854s	user 0.001s	sys 0.012s
I20260709 13:06:37.024859 20162 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:36.995045 20100 consensus_queue.cc:1048] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:06:37.043448 20088 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:37.028926 19917 consensus_queue.cc:1048] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
W20260709 13:06:37.093261 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 68852.2us
I20260709 13:06:37.088361 20158 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.092392 20075 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:37.103644 19996 consensus_queue.cc:1048] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:37.109310 15827 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:37.110728 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 79298us
I20260709 13:06:37.138258 19894 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:37.138391 15900 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:37.139832 19894 raft_consensus.cc:697] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 10 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:37.140009 15900 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:37.140758 19894 consensus_queue.cc:237] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:37.142465 20075 consensus_queue.cc:1048] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:37.144251 16004 raft_consensus.cc:1275] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:06:37.144371 19937 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:37.146463 20106 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:37.146813 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:37.147049 20124 raft_consensus.cc:3055] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:06:37.147535 20124 raft_consensus.cc:740] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:37.147610 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:37.148379 16562 raft_consensus.cc:2368] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 9. Current term is 10.
I20260709 13:06:37.107985 20186 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:37.126936 15974 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:37.135839 20192 log.cc:927] Time spent T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:06:37.129194 20123 consensus_queue.cc:1048] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 13:06:37.131134 20158 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:37.133355 20167 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:37.133049 16009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41662 (request call id 2125) took 1141 ms. Trace:
I20260709 13:06:37.090955 20115 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:37.147138 20106 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:06:37.149263 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:37.167766 15900 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:37.168458 16568 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:06:37.089772 20184 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:37.150911 16561 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:06:37.152258 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:06:37.158402 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 58465.7us
I20260709 13:06:37.097509 20188 consensus_queue.cc:1048] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:06:37.159490 15974 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:37.148487 20124 consensus_queue.cc:260] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:37.161288 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:37.174644 16008 raft_consensus.cc:2393] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 9: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:37.174757 15974 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:37.161734 20068 raft_consensus.cc:2749] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:37.162453 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:37.159094 16009 rpcz_store.cc:276] 0709 13:06:35.991346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.080248 (+ 88902us) service_pool.cc:224] Handling call
0709 13:06:37.132856 (+1052608us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b54001dcc54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:37.132990 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38295424}
I20260709 13:06:37.162847 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:37.163620 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:37.164657 16010 raft_consensus.cc:1275] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:06:37.178563 16002 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:06:37.148526 19939 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:37.175751 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:37.181502 16008 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:06:37.182479 15974 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:37.184126 15974 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:37.171772 20186 raft_consensus.cc:2749] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:37.185209 20161 raft_consensus.cc:2804] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:37.186692 16561 raft_consensus.cc:1275] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:06:37.186951 20189 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:37.188263 20172 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:37.178014 20007 raft_consensus.cc:2749] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:37.169831 20088 consensus_queue.cc:1048] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 13:06:37.202957 15935 raft_consensus.cc:1240] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:37.197813 19806 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.193724 20110 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:37.207000 16007 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:06:37.181008 20107 consensus_queue.cc:1059] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 13:06:37.212985 15974 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:37.202544 20172 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:37.215265 15974 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:37.215852 20158 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:37.207918 19806 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:37.224130 15974 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:37.216354 20172 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:37.216535 20156 raft_consensus.cc:2749] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:37.214040 20161 raft_consensus.cc:697] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 10 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:37.222795 20152 raft_consensus.cc:2804] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:37.234002 20152 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:37.234565 20152 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:37.230422 20068 raft_consensus.cc:2749] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:37.238054 20115 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:37.223264 20007 consensus_queue.cc:1048] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:06:37.242455 20152 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:06:37.246438 20184 raft_consensus.cc:2749] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:37.250659 19776 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:06:37.264971 20190 log.cc:927] Time spent T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.231s	user 0.002s	sys 0.000s
I20260709 13:06:37.233501 19806 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:37.251359 19776 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:37.269626 16564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 923) took 2330 ms. Trace:
I20260709 13:06:37.269119 20161 consensus_queue.cc:237] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:37.272528 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:37.279022 16564 rpcz_store.cc:276] 0709 13:06:34.939169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.939268 (+    99us) service_pool.cc:224] Handling call
0709 13:06:34.939528 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:36.548800 (+1609272us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.269362 (+720562us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b54002d7dd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:37.269547 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3699517312}
I20260709 13:06:37.285494 16006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41662 (request call id 2126) took 1293 ms. Trace:
I20260709 13:06:37.285892 16006 rpcz_store.cc:276] 0709 13:06:35.991643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.272227 (+1280584us) service_pool.cc:224] Handling call
0709 13:06:37.285433 (+ 13206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:37.279004 20153 raft_consensus.cc:3055] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:37.286181 20129 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:37.288029 20153 raft_consensus.cc:740] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:06:37.297464 20149 log.cc:927] Time spent T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 1.043s	user 0.004s	sys 0.003s
W20260709 13:06:37.299829 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 132142us
I20260709 13:06:37.296974 20153 consensus_queue.cc:260] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:37.300906 15900 leader_election.cc:400] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 9. Current term is 10.
I20260709 13:06:37.301052 20153 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:37.301311 15900 leader_election.cc:403] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:37.307585 20120 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 9. Current term is 10.
I20260709 13:06:37.303194 19776 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:37.305274 20113 raft_consensus.cc:3055] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:37.310269 15930 raft_consensus.cc:1240] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:37.317034 16565 raft_consensus.cc:1240] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:06:37.317736 20113 raft_consensus.cc:740] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:37.324021 19995 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.323889 20184 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.326392 15936 raft_consensus.cc:1275] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:06:37.329788 15856 raft_consensus.cc:1240] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:37.329937 20129 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:37.330698 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:37.339293 15936 raft_consensus.cc:1275] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:06:37.340240 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:37.340873 15856 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:37.336119 20184 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:37.325544 19995 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:37.345865 20184 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:37.349231 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:37.350721 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:37.352020 15853 raft_consensus.cc:2410] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 12 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:06:37.353381 15900 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:37.354825 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:37.355482 15932 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:06:37.355106 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:37.357082 15860 raft_consensus.cc:1275] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:06:37.358394 15900 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:37.348796 19995 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:37.350878 20051 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:37.355063 20204 log.cc:927] Time spent T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:06:37.360728 20120 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:37.363080 19776 consensus_queue.cc:1048] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:06:37.363261 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:37.364166 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:37.364620 16533 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:37.365234 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:37.381922 15857 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
W20260709 13:06:37.359460 20101 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.361s	user 0.001s	sys 0.013s
W20260709 13:06:37.366025 15902 leader_election.cc:336] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 1.584s (SENT)
I20260709 13:06:37.366441 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:37.386788 15902 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:37.387012 15858 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
W20260709 13:06:37.387494 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:37.346843 20113 consensus_queue.cc:260] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:37.389125 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:37.389592 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:37.390266 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:37.390733 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:37.392531 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:37.392747 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:37.393229 15902 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:37.393695 16002 raft_consensus.cc:1240] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:06:37.394112 20186 raft_consensus.cc:2749] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:37.394816 20186 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:37.395241 20186 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:37.395650 20186 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:37.396039 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:37.396806 15931 raft_consensus.cc:2393] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 8: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:37.397779 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:37.398761 15902 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:37.371770 15932 raft_consensus.cc:1275] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:06:37.372773 20122 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.373098 20124 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:06:37.373409 20115 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.374500 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:37.400497 15934 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:06:37.378695 16568 raft_consensus.cc:1240] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:06:37.383337 16565 raft_consensus.cc:1275] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:06:37.399550 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:37.365368 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:37.377790 16535 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:37.384490 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:37.360316 20211 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:37.388705 16564 raft_consensus.cc:1240] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:06:37.389256 20113 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:37.365386 20176 log.cc:927] Time spent T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.558s	user 0.000s	sys 0.002s
I20260709 13:06:37.360319 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:37.371512 20206 log.cc:927] Time spent T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:06:37.414297 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:37.394133 16006 raft_consensus.cc:1275] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:06:37.391393 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:37.399300 16567 raft_consensus.cc:1240] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:37.414562 15934 raft_consensus.cc:1275] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:06:37.361878 15856 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:06:37.417178 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:37.418205 15931 raft_consensus.cc:2393] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 10: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:37.422590 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:37.423300 15936 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:06:37.424091 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:37.424651 15931 raft_consensus.cc:1240] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:37.425099 16565 raft_consensus.cc:1240] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:37.423508 16010 raft_consensus.cc:2410] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
W20260709 13:06:37.399452 20101 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.388s	user 0.001s	sys 0.013s
I20260709 13:06:37.445808 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:37.428593 20106 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:37.429068 15902 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:37.429543 15931 raft_consensus.cc:1240] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 13. Current term is 14. Ops: [13.13-13.13]
I20260709 13:06:37.431716 15900 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:37.433362 16533 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:37.434434 20155 raft_consensus.cc:2804] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:37.427506 19806 consensus_queue.cc:1048] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:06:37.435079 16567 raft_consensus.cc:1275] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:06:37.413311 20211 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:37.436340 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:37.442028 15936 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 8
W20260709 13:06:37.451485 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:37.451347 15825 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:37.458410 15900 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:37.456194 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:37.409255 20158 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:37.456445 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:37.448309 16006 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:37.511245 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:37.518324 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:37.422345 20115 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:37.457674 16533 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:37.462189 20211 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:37.521102 15862 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
W20260709 13:06:37.521526 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 63864.9us
I20260709 13:06:37.456290 19917 consensus_queue.cc:1059] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:06:37.522285 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:37.523084 19917 raft_consensus.cc:3055] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:37.489910 20198 raft_consensus.cc:2804] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:37.489714 20167 consensus_queue.cc:1048] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:06:37.523756 19723 consensus_queue.cc:1059] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:37.508376 19728 consensus_queue.cc:1059] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:06:37.524605 20115 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:37.524941 19728 consensus_queue.cc:1048] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:37.525473 20159 raft_consensus.cc:3055] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:37.525846 20159 raft_consensus.cc:740] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:37.514559 15933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2118) took 1391 ms. Trace:
I20260709 13:06:37.510350 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:06:37.490617 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 61623.8us
I20260709 13:06:37.527104 15853 raft_consensus.cc:2393] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 10: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:37.526625 20159 consensus_queue.cc:260] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:37.528081 15902 leader_election.cc:400] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 7. Current term is 8.
I20260709 13:06:37.528373 15902 leader_election.cc:403] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:37.528414 20162 consensus_queue.cc:1059] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:06:37.516765 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:37.529250 15902 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:37.529672 20129 raft_consensus.cc:3055] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:37.530189 20129 raft_consensus.cc:740] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:37.531060 16533 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:37.531005 20129 consensus_queue.cc:260] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:37.526713 15933 rpcz_store.cc:276] 0709 13:06:36.123018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.123208 (+   190us) service_pool.cc:224] Handling call
0709 13:06:36.123590 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:36.526977 (+403387us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.514329 (+987352us) spinlock_profiling.cc:243] Waited 403 ms on lock 0x7b540008a2d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:37.514503 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":926419200}
I20260709 13:06:37.532095 20129 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:37.532230 16533 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:37.489763 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:37.533393 20111 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:37.534243 20159 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:37.489317 20075 raft_consensus.cc:2804] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:37.535894 15931 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:06:37.500481 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:37.528627 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:37.537741 15853 raft_consensus.cc:2393] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 9: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:37.500874 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:37.538424 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:37.538805 16002 raft_consensus.cc:2393] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 10: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:37.523430 19917 raft_consensus.cc:740] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:37.529680 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:37.540683 19917 consensus_queue.cc:260] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:37.541986 19917 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:37.535959 20075 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:37.544087 20075 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:37.539425 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:37.524108 20198 raft_consensus.cc:697] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:37.552687 20198 consensus_queue.cc:237] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:37.545753 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:37.555058 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:37.557242 15853 raft_consensus.cc:2393] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 10: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:37.557642 20224 raft_consensus.cc:2804] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:37.564996 15825 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:37.565766 20097 consensus_queue.cc:1059] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:37.566571 20097 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:37.566900 20097 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:37.567027 20188 raft_consensus.cc:3055] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:37.567273 20097 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:37.421914 20122 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:37.567508 20188 raft_consensus.cc:740] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:37.568030 20129 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.568665 20129 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:37.568882 20188 consensus_queue.cc:260] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:37.570050 20188 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:37.570652 20129 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:37.563031 20186 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:37.523465 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:37.578370 20186 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:37.511147 15976 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
W20260709 13:06:37.581619 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 70544.4us
I20260709 13:06:37.582029 20224 raft_consensus.cc:697] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:37.583015 20160 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:37.583647 20160 raft_consensus.cc:697] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:37.584677 20160 consensus_queue.cc:237] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:37.586498 15854 raft_consensus.cc:1240] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 10 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:06:37.586597 20224 consensus_queue.cc:237] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:37.587852 20007 consensus_queue.cc:1059] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:37.589068 20157 raft_consensus.cc:3055] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:37.589448 20157 raft_consensus.cc:740] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:37.589709 20163 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:37.589824 15855 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
I20260709 13:06:37.590335 20157 consensus_queue.cc:260] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:37.591952 20157 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:37.570878 20122 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:37.416602 19776 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:37.459450 19894 consensus_queue.cc:1048] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.112s
I20260709 13:06:37.513787 15936 raft_consensus.cc:1275] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:06:37.594833 19776 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:37.596210 20119 raft_consensus.cc:2749] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 7. Current term is 8.
I20260709 13:06:37.597182 19776 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:37.572052 20120 raft_consensus.cc:2749] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:06:37.606634 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 75068.5us
I20260709 13:06:37.607429 15902 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:37.675215 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 67897.4us
I20260709 13:06:37.413916 19974 consensus_queue.cc:1059] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.101s
I20260709 13:06:37.422521 20088 consensus_queue.cc:1059] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.189s
W20260709 13:06:37.678313 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 48970.4us
I20260709 13:06:37.707984 15932 raft_consensus.cc:1240] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:37.489363 20110 raft_consensus.cc:2804] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:37.715935 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:37.716543 16006 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:37.710922 20161 consensus_queue.cc:1059] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 13:06:37.720383 20156 consensus_queue.cc:1048] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:37.721318 20110 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:37.721343 20068 raft_consensus.cc:3055] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:37.721870 20110 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:37.721917 20068 raft_consensus.cc:740] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:37.722229 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:37.722867 20068 consensus_queue.cc:260] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:37.724267 20068 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:37.724782 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:37.496483 20010 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:37.492758 20106 raft_consensus.cc:697] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 12 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:37.551337 20075 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:37.460897 19814 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:37.708933 19939 raft_consensus.cc:2749] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:06:37.625572 20176 log.cc:927] Time spent T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.179s	user 0.000s	sys 0.000s
I20260709 13:06:37.503786 20155 raft_consensus.cc:697] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:37.501128 20071 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.741478 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:37.768064 16002 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 9: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:37.747951 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:37.673784 20070 consensus_queue.cc:1059] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.084s
I20260709 13:06:37.602572 19894 consensus_queue.cc:1048] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.140s
I20260709 13:06:37.489547 20175 consensus_queue.cc:1048] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.254s
I20260709 13:06:37.459546 20184 consensus_queue.cc:1048] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.286s
I20260709 13:06:37.734050 20159 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.678205 20158 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:37.672878 20115 consensus_queue.cc:1048] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 13:06:37.723193 20211 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:06:37.773326 16006 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
W20260709 13:06:37.757237 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 225049us
I20260709 13:06:37.625319 19728 consensus_queue.cc:1048] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.131s
I20260709 13:06:37.716873 20120 consensus_queue.cc:1048] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:06:37.776327 19806 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:06:37.801997 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:37.795696 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:37.804126 16002 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:06:37.799136 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:37.799218 15902 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:37.813261 16535 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:37.818145 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:06:37.795213 20172 consensus_meta.cc:287] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.561s	user 0.009s	sys 0.001s
I20260709 13:06:37.822932 20225 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:37.772405 20071 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:37.921607 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:37.788528 20159 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:37.816759 20010 raft_consensus.cc:2804] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:06:37.858230 20176 log.cc:927] Time spent T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:06:37.943846 20156 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.980883 19728 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.977574 20203 consensus_queue.cc:1059] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.304s
I20260709 13:06:37.981719 20071 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:37.981635 19728 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:37.810359 20106 consensus_queue.cc:237] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:37.982645 20071 raft_consensus.cc:3055] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:37.983006 20071 raft_consensus.cc:740] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:37.983853 20071 consensus_queue.cc:260] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:37.984962 20071 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:37.985497 20214 log.cc:927] Time spent T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.241s	user 0.000s	sys 0.001s
W20260709 13:06:37.988375 20217 log.cc:927] Time spent T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:06:37.988770 15935 raft_consensus.cc:1240] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 8. Current term is 9. Ops: [8.10-8.10]
I20260709 13:06:37.990173 15935 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:37.945551 20195 raft_consensus.cc:2749] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:37.831933 20076 consensus_queue.cc:1059] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.137s
W20260709 13:06:38.006012 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:37.963550 16006 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:06:37.966076 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:37.944314 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:38.008045 15932 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:06:37.857271 20172 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:37.967564 20225 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.971890 20010 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:37.972476 20107 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:37.968606 20129 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:06:38.017956 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 248537us
I20260709 13:06:37.825834 20155 consensus_queue.cc:237] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:38.027016 16004 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
I20260709 13:06:38.025322 16533 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:38.020612 20010 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:06:38.035154 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:37.980924 20156 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:38.037468 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:38.024940 20075 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:38.042228 20153 consensus_meta.cc:287] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.741s	user 0.009s	sys 0.002s
I20260709 13:06:38.069654 16009 raft_consensus.cc:2410] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:06:38.034942 19974 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.102674 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:38.103108 19974 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:38.104288 15855 raft_consensus.cc:1275] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:06:38.104664 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:38.105226 16011 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:06:38.105264 19974 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:38.030650 20107 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:38.106227 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:38.108059 20107 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:38.109819 15933 raft_consensus.cc:1240] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:38.110023 16003 raft_consensus.cc:1240] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:38.100286 19776 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:38.105264 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:38.113662 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 44784.7us
I20260709 13:06:38.066625 20070 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.028527 20129 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:38.122435 15861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 780) took 1727 ms. Trace:
I20260709 13:06:38.122820 15861 rpcz_store.cc:276] 0709 13:06:36.385804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.388923 (+  3119us) service_pool.cc:224] Handling call
0709 13:06:36.401156 (+ 12233us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:37.088950 (+687794us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:38.112882 (+1023932us) spinlock_profiling.cc:243] Waited 631 ms on lock 0x7b5400426dd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:38.113039 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1451284736}
I20260709 13:06:38.021567 20225 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:38.125259 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:38.151875 16003 raft_consensus.cc:1240] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:06:38.093991 20161 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.094072 20135 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.069734 20224 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:38.117892 20153 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.117962 20165 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:38.150918 20071 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.153724 20158 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:38.169556 20111 consensus_meta.cc:287] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.636s	user 0.007s	sys 0.003s
I20260709 13:06:38.162667 20224 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:38.173434 20222 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.131204 20070 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:38.179098 20211 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:38.120958 19869 consensus_queue.cc:1048] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 13:06:38.164747 20161 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:38.197413 20224 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:38.146663 20110 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:38.178162 20153 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:38.222949 20110 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:38.223380 20153 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:38.213689 20159 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:38.197062 20111 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:38.214017 16005 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:06:38.198264 20133 raft_consensus.cc:3055] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:38.228688 15930 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:38.232421 20133 raft_consensus.cc:740] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:38.237160 16008 raft_consensus.cc:1240] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 10 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:38.210914 20222 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:38.170311 20135 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:38.248901 15935 raft_consensus.cc:1275] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:06:38.196543 20071 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:38.298178 20225 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:38.298568 15854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2180) took 1809 ms. Trace:
I20260709 13:06:38.336525 15854 rpcz_store.cc:276] 0709 13:06:36.490110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.490542 (+   432us) service_pool.cc:224] Handling call
0709 13:06:36.491043 (+   501us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:37.587029 (+1095986us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:38.298265 (+711236us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b540008d754. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:38.298500 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2518565760}
W20260709 13:06:38.302389 20113 consensus_meta.cc:287] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.879s	user 0.009s	sys 0.000s
I20260709 13:06:38.304157 20119 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:38.304735 20234 log.cc:927] Time spent T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.184s	user 0.000s	sys 0.000s
W20260709 13:06:38.305531 20233 log.cc:927] Time spent T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
I20260709 13:06:38.243746 20230 raft_consensus.cc:3055] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:38.193411 20165 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:38.313534 20135 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:38.242661 20203 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:38.314301 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:38.352046 15936 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:38.318358 16569 raft_consensus.cc:1275] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 13. (term mismatch)
I20260709 13:06:38.318812 20222 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:38.244622 19917 consensus_meta.cc:287] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.700s	user 0.009s	sys 0.001s
I20260709 13:06:38.322781 15900 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:38.242127 20097 consensus_meta.cc:287] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.674s	user 0.008s	sys 0.001s
I20260709 13:06:38.354103 20097 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:38.354809 20165 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:38.355470 20203 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:38.356240 20097 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:38.323165 15930 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
W20260709 13:06:38.323915 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:38.358168 20203 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:38.332602 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:38.333148 16561 raft_consensus.cc:1275] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:06:38.359213 16567 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
I20260709 13:06:38.348637 20119 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:38.360299 20163 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:38.360760 20163 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:38.361302 20119 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:38.363327 20071 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:38.258502 20124 consensus_meta.cc:287] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.842s	user 0.010s	sys 0.000s
I20260709 13:06:38.333312 16006 raft_consensus.cc:1240] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:06:38.334609 15827 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:38.373020 15860 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:06:38.373546 15827 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:38.212224 20241 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.338604 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:38.377557 15900 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:38.341853 16562 raft_consensus.cc:1275] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:06:38.344476 16564 raft_consensus.cc:1275] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:06:38.343933 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:38.235270 20133 consensus_queue.cc:260] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:38.379882 19917 raft_consensus.cc:2804] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:38.380155 20133 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:38.380273 19917 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:38.380447 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:38.380705 19917 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:38.381510 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:38.347877 20230 raft_consensus.cc:740] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
W20260709 13:06:38.385298 20157 consensus_meta.cc:287] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.793s	user 0.010s	sys 0.000s
I20260709 13:06:38.385789 20230 consensus_queue.cc:260] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:38.386853 20230 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:38.338843 20243 consensus_queue.cc:1048] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:06:38.390712 15928 raft_consensus.cc:1275] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:06:38.390434 20163 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:38.391076 15928 pending_rounds.cc:85] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 3
I20260709 13:06:38.391330 15928 pending_rounds.cc:107] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:06:38.391563 15928 raft_consensus.cc:2889] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:38.392442 16002 raft_consensus.cc:2410] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:06:38.393711 15900 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:38.394622 15900 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:38.396574 15900 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:38.371183 15936 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
I20260709 13:06:38.364173 20129 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:38.375697 20161 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:38.375970 20010 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:38.376459 20070 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:38.344700 19776 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:38.342909 20156 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:38.352856 16569 pending_rounds.cc:85] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5: Aborting all ops after (but not including) 12
I20260709 13:06:38.397010 15827 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:38.415454 16569 pending_rounds.cc:107] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5: Aborting uncommitted NO_OP operation due to leader change: 13.13
W20260709 13:06:38.415601 16566 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45246 (request call id 912) took 1892 ms (client timeout 1643 ms). Trace:
I20260709 13:06:38.415829 16569 raft_consensus.cc:2889] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:06:38.416100 16566 rpcz_store.cc:269] 0709 13:06:36.513132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.513755 (+   623us) service_pool.cc:224] Handling call
0709 13:06:38.405299 (+1891544us) spinlock_profiling.cc:243] Waited 368 ms on lock 0x7b54000cce54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:38.405402 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":846368000}
I20260709 13:06:38.417323 15827 leader_election.cc:400] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 9. Current term is 10.
I20260709 13:06:38.417766 15827 leader_election.cc:403] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:06:38.402192 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:38.403985 15900 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:38.406680 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:38.378132 20119 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:38.420183 16006 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
I20260709 13:06:38.378715 20165 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:38.396550 20241 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:38.400388 20135 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:38.648648 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:38.448688 20211 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:38.408038 20198 raft_consensus.cc:2749] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:06:38.400218 20120 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:38.571905 20158 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:38.700059 20129 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.708431 20129 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:38.709702 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:38.710470 20129 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:38.710978 16562 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEARNER]: Advancing to term 12
I20260709 13:06:38.596258 19894 raft_consensus.cc:3055] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:38.712167 19894 raft_consensus.cc:740] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:38.712956 19894 consensus_queue.cc:260] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:38.714252 19894 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:38.714483 20211 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:38.704864 20120 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:38.715438 20120 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:38.710501 16563 raft_consensus.cc:2410] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 11 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:06:38.651696 20133 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:38.548049 20163 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:38.434588 20165 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:38.732467 16563 raft_consensus.cc:1275] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:06:38.542096 19995 raft_consensus.cc:3055] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:38.738622 19995 raft_consensus.cc:740] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:38.745589 20133 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:38.745688 19995 consensus_queue.cc:260] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:38.748126 20133 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:38.751181 15928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2127) took 1436 ms. Trace:
I20260709 13:06:38.753108 15928 rpcz_store.cc:276] 0709 13:06:37.315076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.315187 (+   111us) service_pool.cc:224] Handling call
0709 13:06:37.323696 (+  8509us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:38.391991 (+1068295us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:38.750949 (+358958us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54000997d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:38.751122 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2453130496}
I20260709 13:06:38.755158 20241 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:38.731762 20165 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:38.747630 19995 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:38.507988 20257 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 9. Current term is 10.
I20260709 13:06:38.568797 19974 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:38.595520 20010 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:38.511616 19917 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:38.719298 16562 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 LEARNER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 13:06:38.472821 20240 consensus_queue.cc:1059] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.257s
I20260709 13:06:38.761978 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:38.766489 16562 pending_rounds.cc:85] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting all ops after (but not including) 7
I20260709 13:06:38.768121 16562 pending_rounds.cc:107] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 13:06:38.768361 16562 raft_consensus.cc:2889] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:38.769687 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:38.767659 16563 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
I20260709 13:06:38.496511 19990 consensus_queue.cc:260] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:38.511657 20224 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:38.773874 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:38.774796 16563 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:38.776119 16562 raft_consensus.cc:1240] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:38.782909 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:38.783505 16562 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:38.791344 20135 raft_consensus.cc:2749] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:38.797416 20224 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:38.797164 20165 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:38.798409 20152 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:38.781507 19990 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:38.771435 19974 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:38.802487 16562 raft_consensus.cc:1240] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:06:38.806871 19974 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:38.807539 19917 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:38.808496 20120 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:38.810086 20120 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:38.819713 16565 raft_consensus.cc:1275] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:06:38.823638 19806 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:38.824131 19806 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:38.828229 20165 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:38.838267 19990 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.838843 19990 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:38.840158 19806 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:38.850744 19806 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:38.840973 19990 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:38.942132 20110 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:38.942744 19806 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.942906 20110 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:38.943460 19806 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:38.945858 19806 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:38.945960 20110 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:38.973434 20097 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:38.976202 20097 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:39.001432 20097 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:39.012810 15936 raft_consensus.cc:1275] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 13:06:39.018765 15936 pending_rounds.cc:85] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 5
I20260709 13:06:39.019224 15936 pending_rounds.cc:107] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:06:39.023944 15936 raft_consensus.cc:2889] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:39.000895 15862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 793) took 1464 ms. Trace:
I20260709 13:06:38.984180 20172 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:06:38.997867 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:39.025154 15862 rpcz_store.cc:276] 0709 13:06:37.536256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.587967 (+ 51711us) service_pool.cc:224] Handling call
0709 13:06:37.588269 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:37.588579 (+   310us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:37.588840 (+   261us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:38.585580 (+996740us) spinlock_profiling.cc:243] Waited 22.5 ms on lock 0x7b2c00000f48. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e8ca1a11 00007f73e8cb1372 00007f73ea3e3a01 00007f73ea3a843a 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2
0709 13:06:38.591531 (+  5951us) log.cc:844] Serialized 26 byte log entry
0709 13:06:38.973185 (+381654us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:38.973233 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:39.000526 (+ 27293us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:39.000581 (+    55us) raft_consensus.cc:1685] finished
0709 13:06:39.000594 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:39.000829 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":180,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":81,"spinlock_wait_cycles":51726592,"thread_start_us":381233,"threads_started":1,"wal-append.queue_time_us":110647}
I20260709 13:06:38.512504 16006 raft_consensus.cc:1275] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:06:39.028291 16006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 779) took 1248 ms. Trace:
I20260709 13:06:39.028723 16006 rpcz_store.cc:276] 0709 13:06:37.780376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:38.498542 (+718166us) service_pool.cc:224] Handling call
0709 13:06:38.512196 (+ 13654us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:39.028032 (+515836us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:39.028233 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:38.507055 15902 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 2.094s (SENT) [suppressed 56 similar messages]
I20260709 13:06:39.029770 16006 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94"
dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
mode: GRACEFUL
new_leader_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:43700
W20260709 13:06:39.030459 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 523387us
I20260709 13:06:39.030520 16006 raft_consensus.cc:606] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 LEADER]: Received request to transfer leadership to TS 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:38.447978 15935 raft_consensus.cc:1240] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:39.037667 20097 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:39.039449 20242 log.cc:927] Time spent T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.800s	user 0.005s	sys 0.000s
I20260709 13:06:39.039980 20097 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:38.601864 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:39.022902 16008 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.010s	sys 0.005s
W20260709 13:06:39.052006 16008 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.012s	sys 0.006s
I20260709 13:06:39.052947 16008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2005) took 1325 ms. Trace:
W20260709 13:06:38.448783 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 44860.7us
I20260709 13:06:39.045383 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:38.585057 16003 raft_consensus.cc:3055] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:39.053399 16008 rpcz_store.cc:276] 0709 13:06:37.727645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:38.238188 (+510543us) service_pool.cc:224] Handling call
0709 13:06:38.238580 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:38.337548 (+ 98968us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:38.337617 (+    69us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:38.338577 (+   960us) log.cc:844] Serialized 26 byte log entry
0709 13:06:39.052511 (+713934us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:06:39.052567 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:39.052659 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:39.052719 (+    60us) raft_consensus.cc:1685] finished
0709 13:06:39.052737 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:39.052879 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55636,"raft.queue_time_us":60481,"raft.run_cpu_time_us":48,"raft.run_wall_time_us":48,"thread_start_us":673360,"threads_started":1,"wal-append.queue_time_us":211102}
W20260709 13:06:39.058784 16004 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41662 (request call id 2139) took 1678 ms (client timeout 2094 ms). Trace:
I20260709 13:06:39.054039 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:39.065097 16564 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
I20260709 13:06:39.046475 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:39.049572 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:39.067020 16567 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
I20260709 13:06:39.067513 16566 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
I20260709 13:06:39.054692 16568 raft_consensus.cc:1275] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:06:39.056423 16561 raft_consensus.cc:2410] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 12 because replica has last-logged OpId of term: 11 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
W20260709 13:06:39.059266 16004 rpcz_store.cc:269] 0709 13:06:37.380734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.391159 (+ 10425us) service_pool.cc:224] Handling call
0709 13:06:39.058630 (+1667471us) spinlock_profiling.cc:243] Waited 611 ms on lock 0x7b54003ee254. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:39.058728 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1404676864}
I20260709 13:06:39.056531 16003 raft_consensus.cc:740] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:39.057333 15900 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:39.072296 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:39.074565 15900 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:39.073097 16568 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
W20260709 13:06:39.078377 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:39.079101 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:39.078994 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:39.079799 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:39.079952 16561 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
I20260709 13:06:39.080811 16535 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:39.082223 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:39.083719 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:39.084857 15902 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:39.086987 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:39.089095 16568 raft_consensus.cc:1275] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:06:39.090962 16535 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:39.086658 20110 raft_consensus.cc:2749] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:39.073187 19806 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.086694 20106 raft_consensus.cc:2749] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:39.093246 20222 raft_consensus.cc:2749] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:39.094937 16535 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:39.088112 16003 consensus_queue.cc:260] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:39.091468 15902 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:39.094565 20225 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:39.109280 16003 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:39.112345 16535 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:39.093075 20120 raft_consensus.cc:2804] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:39.098335 20124 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.108961 20198 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:39.098682 20135 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:39.098726 20010 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.107056 19990 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:39.115379 20120 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:39.125001 16003 raft_consensus.cc:1275] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:06:39.111413 20211 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:39.125669 16003 pending_rounds.cc:85] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting all ops after (but not including) 8
I20260709 13:06:39.126065 16003 pending_rounds.cc:107] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:06:39.126370 16003 raft_consensus.cc:2889] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:39.117304 20222 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:39.121287 20133 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:39.107993 19806 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:39.126691 20120 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:39.129482 20177 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 13:06:39.127730 19990 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:39.120750 20106 consensus_queue.cc:1048] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 13:06:39.122553 20124 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:39.149091 19990 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:39.149492 20124 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:39.150029 20124 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:39.150525 20124 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:39.126435 20135 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:39.159600 20135 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:39.123935 20010 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:39.118921 19776 consensus_queue.cc:1048] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.012s
I20260709 13:06:39.124068 20198 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:39.127336 20211 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:39.147361 19806 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:39.159373 20120 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:39.162122 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 52981.6us
I20260709 13:06:39.172952 15902 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:39.175093 20135 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:39.175516 20124 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:39.174487 15902 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:39.164954 20211 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:39.182536 15826 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.026s	sys 0.018s
W20260709 13:06:39.182866 15826 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.667s	user 0.026s	sys 0.018s
I20260709 13:06:39.163226 20010 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:39.164698 20198 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:39.162153 20222 raft_consensus.cc:697] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:39.184530 20076 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:39.185114 20076 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:38.505841 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 88677us
I20260709 13:06:39.186444 15827 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:39.187191 20076 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:39.187533 15827 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:39.188367 20227 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:39.188488 20167 raft_consensus.cc:2804] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:39.188829 20227 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:39.188887 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:39.189328 20227 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:39.189613 15934 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
I20260709 13:06:39.188973 20167 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:39.190106 15934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 2133) took 1781 ms. Trace:
I20260709 13:06:39.190315 20167 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:39.190501 15934 rpcz_store.cc:276] 0709 13:06:37.409114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:39.188616 (+1779502us) service_pool.cc:224] Handling call
0709 13:06:39.190046 (+  1430us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:39.191911 20076 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.193192 20076 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:39.194756 15930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 2142) took 1168 ms. Trace:
I20260709 13:06:39.195003 20244 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.195215 20184 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:39.195170 15930 rpcz_store.cc:276] 0709 13:06:38.026705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:38.102405 (+ 75700us) service_pool.cc:224] Handling call
0709 13:06:39.194597 (+1092192us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b540044c0d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:39.194694 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":288166144}
I20260709 13:06:39.195552 20244 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:39.195910 20184 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:39.196672 15860 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 2052) took 1827 ms (client timeout 2210 ms). Trace:
I20260709 13:06:39.197217 19995 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.197376 20187 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.197778 19995 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:06:39.197889 20187 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:39.200560 19995 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:39.197054 15860 rpcz_store.cc:269] 0709 13:06:37.369091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.529431 (+160340us) service_pool.cc:224] Handling call
0709 13:06:39.196513 (+1667082us) spinlock_profiling.cc:243] Waited 819 ms on lock 0x7b540006ed54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:39.196612 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1883811072}
I20260709 13:06:39.201942 15858 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:06:39.202252 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:39.206661 20187 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:39.209138 15827 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:39.209904 20222 consensus_queue.cc:237] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:39.212776 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:39.167375 20120 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:39.173730 20225 raft_consensus.cc:697] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:39.222952 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:38.472049 15929 raft_consensus.cc:1240] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:39.224100 20260 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:39.224642 20260 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:39.224488 20227 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:39.225001 20260 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:39.225515 15858 raft_consensus.cc:1240] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:39.223649 20225 consensus_queue.cc:237] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:39.216796 15859 raft_consensus.cc:1240] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:39.217854 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:39.215991 19660 raft_consensus.cc:3055] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:39.221421 20010 consensus_queue.cc:1048] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:39.178295 20135 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:39.228600 15929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 850) took 1371 ms. Trace:
I20260709 13:06:39.228708 15857 raft_consensus.cc:2393] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 9: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:39.216109 20277 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:39.229135 15929 rpcz_store.cc:276] 0709 13:06:37.857562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.982952 (+125390us) service_pool.cc:224] Handling call
0709 13:06:37.983332 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:39.224519 (+1241187us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:39.225617 (+  1098us) spinlock_profiling.cc:243] Waited 488 ms on lock 0x7b5400327654. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:39.225761 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1123448704}
I20260709 13:06:39.230458 20277 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:39.230860 20277 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:39.228691 20124 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.232282 20124 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:39.234315 20124 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:39.228782 19660 raft_consensus.cc:740] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:39.236047 19660 consensus_queue.cc:260] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:39.237344 19660 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:39.229907 20188 consensus_queue.cc:1048] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 13:06:39.239385 15928 raft_consensus.cc:1240] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:39.239748 20135 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:39.241284 15928 raft_consensus.cc:1240] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:39.242697 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60156 (request call id 767) took 1528 ms. Trace:
I20260709 13:06:39.243016 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:39.243646 15935 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:39.244729 15929 raft_consensus.cc:1240] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:39.245836 15936 raft_consensus.cc:1275] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:06:39.246979 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:39.247071 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:39.243124 16002 rpcz_store.cc:276] 0709 13:06:37.714477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:38.026252 (+311775us) service_pool.cc:224] Handling call
0709 13:06:39.242518 (+1216266us) spinlock_profiling.cc:243] Waited 347 ms on lock 0x7b54000aa5d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:39.242641 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":798953856}
I20260709 13:06:39.249095 15936 raft_consensus.cc:2410] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:06:39.250437 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:39.255050 15935 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
W20260709 13:06:39.275743 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 47402.9us
I20260709 13:06:39.276544 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:39.280339 15935 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:06:39.252202 19894 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:39.252271 19917 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:39.277516 15929 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 12
W20260709 13:06:39.262347 20226 log.cc:927] Time spent T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 1.153s	user 0.003s	sys 0.003s
I20260709 13:06:39.286557 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:39.282507 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:39.283871 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:39.284669 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:39.288800 16566 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:06:39.285285 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:39.289697 16568 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:39.289806 15857 raft_consensus.cc:1240] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:39.290447 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:39.291131 15857 raft_consensus.cc:1275] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:06:39.295001 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:39.287384 16565 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:06:39.288187 16569 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
I20260709 13:06:39.296387 15936 raft_consensus.cc:2393] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 13: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:39.297209 16561 raft_consensus.cc:1240] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:39.298993 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:39.299543 15857 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:39.227731 20187 consensus_queue.cc:1048] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 13:06:39.322862 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:39.324002 15860 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
I20260709 13:06:39.328961 16568 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
I20260709 13:06:39.331483 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:39.285998 19917 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:39.298264 20166 consensus_queue.cc:1048] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 13:06:39.334903 19917 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:39.336421 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:39.282467 20262 consensus_queue.cc:1048] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:06:39.345911 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:39.383744 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:39.387635 15900 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:39.389294 16564 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:06:39.369475 15929 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:06:39.344934 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:39.345221 15855 raft_consensus.cc:1275] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:06:39.387182 15827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:39.398083 20289 raft_consensus.cc:2804] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:06:39.384155 16011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 769) took 1669 ms. Trace:
I20260709 13:06:39.383440 15902 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:39.295871 19894 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:06:39.303591 20076 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:39.401958 15937 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 836) took 2026 ms. Trace:
I20260709 13:06:39.407929 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:39.404574 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:39.406865 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:39.407702 15937 rpcz_store.cc:276] 0709 13:06:37.375888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.376022 (+   134us) service_pool.cc:224] Handling call
0709 13:06:37.376299 (+   277us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:37.376600 (+   301us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:37.376626 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:37.385635 (+  9009us) log.cc:844] Serialized 26 byte log entry
0709 13:06:37.952731 (+567096us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:37.952787 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.952874 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:39.251452 (+1298578us) spinlock_profiling.cc:243] Waited 3.35 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:06:39.373194 (+121742us) raft_consensus.cc:1685] finished
0709 13:06:39.373237 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:39.401875 (+ 28638us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":413,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":15405,"spinlock_wait_cycles":7695232,"thread_start_us":490065,"threads_started":1,"wal-append.queue_time_us":581890}
I20260709 13:06:39.410416 16563 raft_consensus.cc:2410] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:06:39.399293 16011 rpcz_store.cc:276] 0709 13:06:37.715049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:38.107102 (+392053us) service_pool.cc:224] Handling call
0709 13:06:38.107396 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:38.112285 (+  4889us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:38.163876 (+ 51591us) log.cc:844] Serialized 14 byte log entry
0709 13:06:38.295084 (+131208us) log.cc:844] Serialized 14 byte log entry
0709 13:06:38.334667 (+ 39583us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:38.338665 (+  3998us) log.cc:844] Serialized 26 byte log entry
0709 13:06:38.338799 (+   134us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:38.338828 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:38.338887 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:39.334724 (+995837us) spinlock_profiling.cc:243] Waited 3.23 ms on lock 0x7b18004d8a80. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb3f3 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd 000000000038f354 000000000038f199 00007f73e729e83a
0709 13:06:39.383869 (+ 49145us) raft_consensus.cc:1685] finished
0709 13:06:39.383919 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:39.384081 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70099,"prepare.queue_time_us":84012,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":79,"raft.queue_time_us":119096,"raft.run_cpu_time_us":387,"raft.run_wall_time_us":34525,"spinlock_wait_cycles":35740672,"thread_start_us":70972,"threads_started":1,"wal-append.queue_time_us":174918}
I20260709 13:06:39.412276 15827 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:39.415186 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:39.415958 15856 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:39.417627 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:39.418469 15860 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:06:39.423381 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:39.322840 20244 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:39.428793 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:39.431989 16565 raft_consensus.cc:2393] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 9: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:39.432205 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:39.432734 16562 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:06:39.433446 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:06:39.434433 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:39.379060 20283 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:39.448215 20277 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:39.450922 20277 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:39.413280 20188 consensus_queue.cc:1059] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:06:39.474236 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:39.474947 15860 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:39.475962 15862 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
I20260709 13:06:39.477145 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:39.477911 15862 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
I20260709 13:06:39.478937 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:39.481007 15857 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 13.
W20260709 13:06:39.481871 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 91863us
I20260709 13:06:39.482831 15862 raft_consensus.cc:2368] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 9. Current term is 10.
W20260709 13:06:39.484169 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 100844us
I20260709 13:06:39.484274 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:39.485070 15856 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
I20260709 13:06:39.485168 15862 raft_consensus.cc:2368] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 11. Current term is 12.
I20260709 13:06:39.485957 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:39.486726 15856 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
I20260709 13:06:39.487942 20285 consensus_queue.cc:1048] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:39.489334 20290 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:39.489943 20290 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:39.490397 20290 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:39.443991 20283 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:39.656661 20283 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:39.744540 15933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2143) took 1717 ms. Trace:
I20260709 13:06:39.745146 15933 rpcz_store.cc:276] 0709 13:06:38.027290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:38.382268 (+354978us) service_pool.cc:224] Handling call
0709 13:06:38.382524 (+   256us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:38.398329 (+ 15805us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:38.398366 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:38.405313 (+  6947us) log.cc:844] Serialized 26 byte log entry
0709 13:06:38.567481 (+162168us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:38.567536 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:38.567716 (+   180us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:39.744270 (+1176554us) raft_consensus.cc:1685] finished
0709 13:06:39.744292 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:39.744463 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15432,"raft.queue_time_us":15864,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":6758,"thread_start_us":74925,"threads_started":1,"wal-append.queue_time_us":140063}
I20260709 13:06:39.781963 20283 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:39.817044 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 334340us
I20260709 13:06:39.933840 15900 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:39.935844 15900 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:39.485085 15902 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:39.961285 15862 raft_consensus.cc:1275] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:06:39.852770 15860 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
W20260709 13:06:39.900627 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 434693us
I20260709 13:06:39.911348 20292 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:06:40.025256 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 540241us
I20260709 13:06:40.026033 15902 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:39.950107 20277 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:39.971240 20283 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:40.032326 20277 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:40.024803 20292 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:40.046572 20277 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:40.048084 20292 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:40.048664 20292 raft_consensus.cc:2804] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:40.049027 20292 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:40.049371 20292 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:40.053020 20283 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:40.053603 20283 raft_consensus.cc:2804] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:40.053920 20283 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:40.054247 20283 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:40.054613 20277 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:40.056670 20292 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:40.057430 20277 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:40.058112 20277 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.058594 20277 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:40.059283 20292 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:40.061326 20290 consensus_meta.cc:287] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.570s	user 0.008s	sys 0.000s
I20260709 13:06:40.061785 20290 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:40.064002 20290 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:40.035570 20291 raft_consensus.cc:2749] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:40.077813 20283 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:40.081485 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 55528.3us
I20260709 13:06:40.091926 20295 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.092525 20295 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:40.094933 20295 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:40.082808 20277 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:40.097609 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:40.093138 20283 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:40.105767 16007 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41674 (request call id 1995) took 2964 ms (client timeout 1667 ms). Trace:
W20260709 13:06:40.106213 16007 rpcz_store.cc:269] 0709 13:06:37.141693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.151721 (+ 10028us) service_pool.cc:224] Handling call
0709 13:06:40.105565 (+2953844us) spinlock_profiling.cc:243] Waited 35.8 ms on lock 0x7b54003ed854. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:40.105665 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":82320640}
I20260709 13:06:40.127763 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:40.128520 15935 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:06:40.135229 15932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 2052) took 2158 ms. Trace:
I20260709 13:06:40.138443 15932 rpcz_store.cc:276] 0709 13:06:37.976691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:38.026082 (+ 49391us) service_pool.cc:224] Handling call
0709 13:06:38.036785 (+ 10703us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:38.094849 (+ 58064us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:38.094880 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:38.095540 (+   660us) log.cc:844] Serialized 26 byte log entry
0709 13:06:38.348754 (+253214us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:38.377908 (+ 29154us) log.cc:844] Serialized 14 byte log entry
0709 13:06:38.379220 (+  1312us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:38.379303 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:40.134887 (+1755584us) raft_consensus.cc:1685] finished
0709 13:06:40.134913 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:40.135107 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":22942,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":52014,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":15945,"thread_start_us":180758,"threads_started":1,"wal-append.queue_time_us":334629}
W20260709 13:06:40.148185 20284 log.cc:927] Time spent T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.752s	user 0.003s	sys 0.002s
I20260709 13:06:40.157480 20285 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:40.158100 20285 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:40.161243 20277 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:40.161799 20277 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:40.160327 20285 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:40.164872 20277 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:40.187567 20211 consensus_meta.cc:287] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 1.008s	user 0.015s	sys 0.002s
I20260709 13:06:40.188103 20211 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:40.189323 15930 raft_consensus.cc:1240] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:40.197985 20222 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.198467 20222 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:40.200878 20222 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:40.204665 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:40.208097 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:40.208997 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:40.209452 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:40.210270 16007 raft_consensus.cc:2393] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 11: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:40.210374 16004 raft_consensus.cc:1240] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:40.211179 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:40.211838 20007 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:40.212344 20007 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:40.214212 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:40.214847 16002 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
I20260709 13:06:40.209585 16003 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:40.216527 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:40.211444 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:40.218255 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:40.220240 16008 raft_consensus.cc:1240] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:40.221242 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:40.221750 16008 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:40.217531 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:40.228736 16007 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:06:40.214236 20007 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:40.226285 20070 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:40.244439 20070 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:40.246886 20070 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:40.255919 16008 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:06:40.256919 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:40.252858 16003 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
I20260709 13:06:40.219336 20211 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:40.258399 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:06:40.259851 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:40.259351 20277 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:06:40.265057 16568 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.839s	user 0.009s	sys 0.024s
W20260709 13:06:40.268795 16568 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.842s	user 0.010s	sys 0.025s
I20260709 13:06:40.268976 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:40.267972 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:40.269706 16003 raft_consensus.cc:2410] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:06:40.271286 20277 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:40.273694 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:40.274616 15861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 2193) took 1056 ms. Trace:
I20260709 13:06:40.274890 20277 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:40.274921 15861 rpcz_store.cc:276] 0709 13:06:39.218031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:40.273360 (+1055329us) service_pool.cc:224] Handling call
0709 13:06:40.274559 (+  1199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:40.278267 15853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 2051) took 2909 ms. Trace:
I20260709 13:06:40.278663 15853 rpcz_store.cc:276] 0709 13:06:37.368553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.557828 (+189275us) service_pool.cc:224] Handling call
0709 13:06:37.558163 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:37.560079 (+  1916us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:37.560123 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:39.036679 (+1476556us) spinlock_profiling.cc:243] Waited 51.5 ms on lock 0x7b2c00000f48. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e8ca1a11 00007f73e8cb1372 00007f73ea3e3a01 00007f73ea3a843a 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2
0709 13:06:39.037078 (+   399us) log.cc:844] Serialized 26 byte log entry
0709 13:06:39.107409 (+ 70331us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:39.118052 (+ 10643us) log.cc:844] Serialized 14 byte log entry
0709 13:06:39.118298 (+   246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:39.118385 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:40.083083 (+964698us) spinlock_profiling.cc:243] Waited 2.96 ms on lock 0x7b1800435520. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb3f3 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd 000000000038f354 000000000038f199 00007f73e729e83a
0709 13:06:40.275738 (+192655us) spinlock_profiling.cc:243] Waited 1.1 ms on lock 0x7b1800435520. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb3f3 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd 000000000038f354 000000000038f199 00007f73e729e83a
0709 13:06:40.278054 (+  2316us) raft_consensus.cc:1685] finished
0709 13:06:40.278076 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:40.278209 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":11330,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":993,"raft.queue_time_us":1729,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":163,"spinlock_wait_cycles":127728000,"thread_start_us":69946,"threads_started":1,"wal-append.queue_time_us":91683}
I20260709 13:06:40.280066 19728 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:40.281733 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:40.282444 16008 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:06:40.283231 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:40.285017 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:06:40.290858 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:40.291761 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:40.292145 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:40.292809 16008 raft_consensus.cc:2393] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 9: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:40.293484 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:40.294571 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:40.295171 16008 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:40.296452 16003 raft_consensus.cc:2393] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 10: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:40.297400 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:40.298184 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:40.298601 16003 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:06:40.299155 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:40.299614 16003 raft_consensus.cc:2393] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 11: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:40.311961 19776 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.312333 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:40.312505 19776 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:40.312958 16003 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:40.314571 19776 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:40.315027 19776 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:40.296756 20222 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.316188 20222 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:40.324749 16008 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
I20260709 13:06:40.326088 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:40.326669 16008 raft_consensus.cc:2376] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10. Re-sending same reply.
W20260709 13:06:40.327818 16009 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60156 (request call id 762) took 2615 ms (client timeout 3117 ms). Trace:
W20260709 13:06:40.328217 16009 rpcz_store.cc:269] 0709 13:06:37.712542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.712701 (+   159us) service_pool.cc:224] Handling call
0709 13:06:40.327609 (+2614908us) spinlock_profiling.cc:243] Waited 347 ms on lock 0x7b54001dc754. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:40.327761 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":796951936}
I20260709 13:06:40.327510 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:40.319419 20222 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:40.370489 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:40.371249 16008 raft_consensus.cc:2393] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 9: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:38.471769 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:39.437016 20076 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:40.375639 16010 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:40.332937 16003 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
I20260709 13:06:40.377858 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:40.378412 16003 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:06:40.379187 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:40.385077 16010 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
W20260709 13:06:40.386052 16010 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41674 (request call id 2012) took 2027 ms (client timeout 2099 ms). Trace:
W20260709 13:06:40.386478 16010 rpcz_store.cc:269] 0709 13:06:38.358497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:38.407635 (+ 49138us) service_pool.cc:224] Handling call
0709 13:06:40.385997 (+1978362us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:40.355746 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:40.387916 19894 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:40.388379 16009 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
I20260709 13:06:40.388623 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:40.389333 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:40.389815 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:40.390318 16009 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
I20260709 13:06:40.390941 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:39.462013 20135 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:40.385578 20076 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:40.349478 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:40.391561 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:40.391840 16008 raft_consensus.cc:2410] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:06:40.392072 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:40.400175 16009 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
I20260709 13:06:40.348724 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:40.379667 16003 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:40.315457 19776 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:40.401458 16007 raft_consensus.cc:2393] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 11: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:40.408337 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:40.409701 16004 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ac847593d6724522bda12a2207f91c31"
dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
mode: GRACEFUL
new_leader_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:43700
W20260709 13:06:40.396602 20289 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.979s	user 0.005s	sys 0.003s
W20260709 13:06:40.411074 20289 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.012s	user 0.005s	sys 0.003s
I20260709 13:06:40.410498 16008 raft_consensus.cc:1275] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:06:40.412016 20289 raft_consensus.cc:697] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:40.412874 20289 consensus_queue.cc:237] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:40.413296 20135 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:40.417536 19776 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:40.402232 20076 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:40.425782 20076 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:40.429095 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:40.460326 16566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 938) took 1054 ms. Trace:
I20260709 13:06:40.451654 20076 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:40.464469 19660 consensus_meta.cc:287] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 1.227s	user 0.015s	sys 0.002s
I20260709 13:06:40.461459 16003 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:06:40.462121 16566 rpcz_store.cc:276] 0709 13:06:39.405604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:39.405747 (+   143us) service_pool.cc:224] Handling call
0709 13:06:39.406106 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:39.426746 (+ 20640us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:39.426793 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:39.427267 (+   474us) log.cc:844] Serialized 26 byte log entry
0709 13:06:40.435666 (+1008399us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:40.435724 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:40.435793 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:40.460104 (+ 24311us) raft_consensus.cc:1685] finished
0709 13:06:40.460125 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:40.460268 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8667,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":10753,"thread_start_us":41758,"threads_started":1,"wal-append.queue_time_us":996818}
W20260709 13:06:40.471741 16032 debug-util.cc:398] Leaking SignalData structure 0x7b08000eb800 after lost signal to thread 14751
I20260709 13:06:40.474450 20158 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:40.475529 20158 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:40.478230 20158 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:40.486773 16561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 937) took 1082 ms. Trace:
I20260709 13:06:38.571887 16005 raft_consensus.cc:1275] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:06:40.511322 16005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 780) took 2699 ms. Trace:
I20260709 13:06:40.511686 16005 rpcz_store.cc:276] 0709 13:06:37.811781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:38.571167 (+759386us) service_pool.cc:224] Handling call
0709 13:06:38.571524 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:40.511057 (+1939533us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:40.511255 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:40.512529 15931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 847) took 2975 ms. Trace:
I20260709 13:06:40.512915 15931 rpcz_store.cc:276] 0709 13:06:37.537082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.537236 (+   154us) service_pool.cc:224] Handling call
0709 13:06:37.537657 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:37.814789 (+277132us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:38.017528 (+202739us) log.cc:844] Serialized 14 byte log entry
0709 13:06:38.127181 (+109653us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:38.127253 (+    72us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:38.127316 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:39.038959 (+911643us) spinlock_profiling.cc:243] Waited 3.85 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:06:40.512306 (+1473347us) spinlock_profiling.cc:243] Waited 277 ms on lock 0x7b540019c3d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:40.512357 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:40.512474 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3013,"prepare.queue_time_us":323822,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":3190,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":26283,"spinlock_wait_cycles":645686400,"thread_start_us":268261,"threads_started":2,"wal-append.queue_time_us":189901,"wal-append.run_cpu_time_us":15013,"wal-append.run_wall_time_us":1960808}
W20260709 13:06:40.517417 20288 log.cc:927] Time spent T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 1.120s	user 0.011s	sys 0.000s
W20260709 13:06:40.528115 20167 consensus_meta.cc:287] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 1.337s	user 0.007s	sys 0.015s
I20260709 13:06:40.532732 20167 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:40.534745 15854 raft_consensus.cc:1240] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:06:40.536916 20167 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:40.540540 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:40.541091 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:40.542882 19806 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:40.543107 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:40.543887 16567 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:40.543439 19806 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:40.546340 20227 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:40.543913 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:40.548283 16564 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:06:40.554175 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:40.554960 16568 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:40.568607 16568 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
I20260709 13:06:40.570539 16564 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
I20260709 13:06:40.570760 16567 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
I20260709 13:06:40.574471 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:40.575809 16565 raft_consensus.cc:2410] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 12 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:06:40.571020 19806 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:40.578830 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:40.579646 16565 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:40.586429 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:40.587270 16568 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
W20260709 13:06:40.591338 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 642127us
I20260709 13:06:40.598188 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:40.599001 16566 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:40.598423 15900 connection.cc:256] RPC call timeout handler was delayed by -1.1946972979999373s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1502s for a response
W20260709 13:06:40.626296 16032 debug-util.cc:398] Leaking SignalData structure 0x7b08000eb840 after lost signal to thread 15794
I20260709 13:06:40.629153 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:40.629452 20106 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.630012 20106 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:40.630470 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:40.631819 20188 raft_consensus.cc:3055] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:40.631592 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:40.644089 15858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2188) took 1432 ms. Trace:
W20260709 13:06:40.657407 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 123396us
I20260709 13:06:40.661149 16565 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:06:40.661139 15858 rpcz_store.cc:276] 0709 13:06:39.212110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:39.212254 (+   144us) service_pool.cc:224] Handling call
0709 13:06:39.222615 (+ 10361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:39.225848 (+  3233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:40.643845 (+1417997us) spinlock_profiling.cc:243] Waited 2.74 ms on lock 0x7b54002486d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:40.644029 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6309632}
W20260709 13:06:40.633322 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:40.664294 16566 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
I20260709 13:06:40.637724 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:40.666654 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:40.667415 16564 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:06:40.653029 16563 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:06:40.670845 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:40.673131 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:40.673920 15937 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:40.675923 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:40.656080 20260 consensus_meta.cc:287] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 1.431s	user 0.012s	sys 0.009s
I20260709 13:06:40.676561 15929 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 9.
I20260709 13:06:40.677346 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:40.677506 16566 raft_consensus.cc:1275] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:06:40.677872 15928 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:40.678228 16535 leader_election.cc:336] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 3.117s (SENT)
I20260709 13:06:40.632118 20106 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:40.646523 20287 log.cc:927] Time spent T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 1.249s	user 0.007s	sys 0.003s
I20260709 13:06:40.684016 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:40.689656 15929 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:40.689702 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:40.657194 20156 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.685508 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:40.685743 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:40.686372 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:40.696022 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:40.708527 15935 raft_consensus.cc:2368] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 11. Current term is 12.
I20260709 13:06:40.678287 15902 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:40.632249 20188 raft_consensus.cc:740] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:40.713598 16562 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:06:40.677139 20289 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.715199 15931 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 10: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:40.716738 15932 raft_consensus.cc:2393] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 10: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:40.640527 20110 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:40.715312 15902 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:40.719931 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 48498.6us
I20260709 13:06:40.720348 15902 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:40.717533 15929 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
I20260709 13:06:40.487406 16561 rpcz_store.cc:276] 0709 13:06:39.404440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:39.404571 (+   131us) service_pool.cc:224] Handling call
0709 13:06:39.404920 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:39.405180 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:06:39.436123 (+ 30943us) log.cc:844] Serialized 14 byte log entry
0709 13:06:40.449141 (+1013018us) spinlock_profiling.cc:243] Waited 1.79 ms on lock 0x7b180033a000. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f4823a 00007f73ea3b2a2a
0709 13:06:40.486396 (+ 37255us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:40.486454 (+    58us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:40.486515 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:40.486593 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:40.486716 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":23038,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":1567,"raft.queue_time_us":138,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":332,"spinlock_wait_cycles":4123904,"thread_start_us":37028,"threads_started":1}
I20260709 13:06:40.724445 20304 raft_consensus.cc:2804] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:40.727316 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:40.728650 20188 consensus_queue.cc:260] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:40.729302 15854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2185) took 1529 ms. Trace:
I20260709 13:06:40.729619 20188 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:40.729575 20304 raft_consensus.cc:697] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 9 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:40.730763 20133 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:40.731287 20133 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:40.730871 20304 consensus_queue.cc:237] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:40.731920 16570 raft_consensus.cc:1240] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:40.723165 20289 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:40.734948 20195 raft_consensus.cc:2804] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:40.735390 20195 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:40.735762 20195 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:40.721975 20106 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:40.736820 20106 raft_consensus.cc:697] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 12 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:40.737264 20289 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:40.737664 20106 consensus_queue.cc:237] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:40.740100 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:40.740757 15931 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
I20260709 13:06:40.724596 20222 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.744776 20222 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:40.745591 20187 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.746075 20187 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:40.746825 20222 leader_election.cc:290] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:40.746546 20195 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:40.729651 15854 rpcz_store.cc:276] 0709 13:06:39.200098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:39.202787 (+  2689us) service_pool.cc:224] Handling call
0709 13:06:39.203049 (+   262us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:40.535139 (+1332090us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:40.729075 (+193936us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b54004268d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:40.729246 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3062039808}
I20260709 13:06:40.748168 20187 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:40.748728 20187 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.748847 20195 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:40.749224 20187 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:40.751787 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:40.752311 16562 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:40.753051 19974 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:40.680922 20260 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:40.754240 19974 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:40.755474 20260 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:40.759284 16562 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
W20260709 13:06:40.759964 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 554327us
I20260709 13:06:40.761667 19974 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:40.763454 15928 raft_consensus.cc:2410] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:06:40.763396 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:40.769908 16562 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:40.771833 16568 raft_consensus.cc:1240] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:06:40.775328 16568 raft_consensus.cc:1240] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:40.777421 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:40.777971 15935 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:40.779762 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:40.780665 16568 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:40.784680 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:40.785255 16563 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:40.718958 20156 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:40.792972 15900 leader_election.cc:400] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 election: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 9. Current term is 10.
I20260709 13:06:40.796450 15900 leader_election.cc:403] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:06:40.796650 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:40.797215 15928 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:40.795331 16563 raft_consensus.cc:2410] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:06:40.795461 16568 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:06:40.770582 20135 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.759997 20195 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:40.805742 16562 raft_consensus.cc:2410] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 13:06:40.803997 16568 raft_consensus.cc:1240] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
W20260709 13:06:40.807325 16533 connection.cc:256] RPC call timeout handler was delayed by -1.0983387609999227s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15200000000000002s for a response
I20260709 13:06:40.809444 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:40.809865 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:40.802976 15900 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:40.812436 15900 leader_election.cc:400] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 11. Current term is 12.
I20260709 13:06:40.812809 15900 leader_election.cc:403] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:40.811033 16533 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:40.814945 15900 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:40.726681 20110 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:39.412532 20198 consensus_queue.cc:1059] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:06:40.823647 20110 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:40.827550 19996 raft_consensus.cc:2749] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 11. Current term is 12.
I20260709 13:06:40.833349 15854 raft_consensus.cc:1240] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:40.833938 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:40.834131 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:40.834492 20187 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:40.833205 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:40.835103 15859 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:06:40.835150 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:40.835795 15857 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:40.836700 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:40.837036 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:40.837692 15861 raft_consensus.cc:2393] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 8: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:40.837756 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:40.838290 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:40.838768 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:40.838886 15929 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:40.839159 15928 raft_consensus.cc:1240] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:40.839425 15933 raft_consensus.cc:2393] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 12: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:40.836740 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:40.839775 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:40.840026 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:40.840371 15853 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:06:40.840692 15932 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 10: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:40.840927 20156 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:40.841243 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:40.841297 19917 consensus_queue.cc:1048] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:06:40.841885 15934 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:40.842646 15902 leader_election.cc:336] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 1.772s (SENT)
I20260709 13:06:40.843060 15902 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:40.840946 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:40.844410 15936 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:06:40.844542 20097 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.844766 20158 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:40.845537 20222 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:40.845221 20097 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:40.846102 20222 raft_consensus.cc:697] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:40.848340 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:40.810091 20135 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:40.849123 15937 raft_consensus.cc:1240] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:40.809916 20195 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:40.834710 15858 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:40.806169 15748 maintenance_manager.cc:643] P d33cebb189b14a989836214fdbafb898: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 8.586s	user 0.213s	sys 0.075s Metrics: {"bytes_written":30285,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":242796,"dirs.run_cpu_time_us":531,"dirs.run_wall_time_us":34017,"drs_written":1,"lbm_read_time_us":22575,"lbm_reads_1-10_ms":2,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":1,"lbm_write_time_us":146058,"lbm_writes_1-10_ms":2,"lbm_writes_10-100_ms":6,"lbm_writes_lt_1ms":26,"peak_mem_usage":0,"reinsert_count":0,"rows_written":63,"thread_start_us":246280,"threads_started":2,"update_count":21,"wal-append.queue_time_us":34715}
I20260709 13:06:40.820937 15825 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:40.859666 15935 raft_consensus.cc:1275] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:06:40.846982 20222 consensus_queue.cc:237] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:40.870657 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 49780.8us
I20260709 13:06:40.806389 20260 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.804198 20289 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 9. Current term is 10.
I20260709 13:06:40.881585 15825 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:40.882766 15825 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:40.881644 15934 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:40.884716 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:40.887562 15825 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:06:40.886615 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 70502.6us
I20260709 13:06:40.867967 20158 raft_consensus.cc:697] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 10 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:40.906065 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:40.874221 20110 raft_consensus.cc:2749] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:40.907146 15853 raft_consensus.cc:2393] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 11: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:40.906329 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:40.907234 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:40.907960 15857 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 9.
W20260709 13:06:40.906461 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 63134.2us
I20260709 13:06:40.869352 20304 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:40.910995 15861 raft_consensus.cc:2393] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 10: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:40.809053 20232 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:40.880478 19917 raft_consensus.cc:2804] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:40.914726 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:40.900570 19974 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:40.921384 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:40.900202 20088 raft_consensus.cc:2804] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:06:40.915023 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 146915us
I20260709 13:06:40.886781 20187 consensus_queue.cc:1048] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 13:06:40.904031 20167 raft_consensus.cc:2804] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:40.930927 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:40.927655 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:40.927541 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:40.932512 15857 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:40.929412 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:40.936177 15860 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:40.979260 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:40.887915 20260 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:40.986091 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 171126us
I20260709 13:06:41.192382 15900 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:40.930400 19917 raft_consensus.cc:697] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 10 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:41.196346 19917 consensus_queue.cc:237] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:41.199877 20110 consensus_queue.cc:1048] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:41.195134 15857 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
I20260709 13:06:40.916798 20304 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:40.925302 20088 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:40.940176 19974 raft_consensus.cc:697] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:40.997320 15749 maintenance_manager.cc:419] P d33cebb189b14a989836214fdbafb898: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10606 bytes on disk
W20260709 13:06:41.000304 20317 log.cc:927] Time spent T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:06:40.921521 20232 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:41.077883 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:41.118606 16032 debug-util.cc:398] Leaking SignalData structure 0x7b08000a8840 after lost signal to thread 15886
W20260709 13:06:41.142577 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 239258us
I20260709 13:06:41.176493 15930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2146) took 1940 ms. Trace:
I20260709 13:06:41.178378 20133 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:41.012850 20158 consensus_queue.cc:237] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:41.181674 20322 consensus_queue.cc:1059] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:41.183017 20195 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:41.185416 20328 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.186174 15861 raft_consensus.cc:2410] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:06:41.189107 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:41.190995 20167 raft_consensus.cc:697] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 12 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:40.917212 20198 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.987762 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:41.202845 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:41.207003 20260 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:41.203641 20088 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:41.204020 20325 raft_consensus.cc:2804] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:41.204613 20304 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:41.213603 15930 rpcz_store.cc:276] 0709 13:06:39.236099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:39.236231 (+   132us) service_pool.cc:224] Handling call
0709 13:06:39.236525 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:40.189718 (+953193us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:41.116847 (+927129us) spinlock_profiling.cc:243] Waited 952 ms on lock 0x7b54002c3154. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:41.176427 (+ 59580us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2190731136}
W20260709 13:06:41.213399 16032 debug-util.cc:398] Leaking SignalData structure 0x7b0800061bc0 after lost signal to thread 15887
I20260709 13:06:41.226076 15857 raft_consensus.cc:1275] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:06:41.230719 15748 maintenance_manager.cc:643] P d33cebb189b14a989836214fdbafb898: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.019s	user 0.001s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":445,"lbm_reads_lt_1ms":4}
I20260709 13:06:41.232434 15827 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:41.234064 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:41.246658 15856 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
I20260709 13:06:41.235038 20135 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:41.224793 16533 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:41.237962 15854 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:06:41.250994 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:41.210245 20071 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.203132 15860 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
W20260709 13:06:41.243371 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:41.256843 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:41.248164 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:41.257718 15861 raft_consensus.cc:2376] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 13. Re-sending same reply.
I20260709 13:06:41.257871 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:41.252343 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:41.258980 15857 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:06:41.260936 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:41.261793 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:41.260215 20325 raft_consensus.cc:697] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 10 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:41.264900 20232 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:41.265002 20133 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:41.265877 20167 consensus_queue.cc:237] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:41.308406 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:41.309099 20162 raft_consensus.cc:2804] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:41.309538 20162 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:41.309690 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:41.309916 20162 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:41.310315 15859 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:41.311152 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:41.269129 16570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 940) took 3883 ms. Trace:
I20260709 13:06:41.247013 20198 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:41.256078 20187 consensus_queue.cc:1048] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:06:41.273480 20097 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:41.273828 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:41.315565 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:41.312537 16570 rpcz_store.cc:276] 0709 13:06:37.385368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:37.385549 (+   181us) service_pool.cc:224] Handling call
0709 13:06:37.389035 (+  3486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:40.732311 (+3343276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:41.268937 (+536626us) spinlock_profiling.cc:243] Waited 3.34 s on lock 0x7b54002364d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:41.269074 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7687393024}
I20260709 13:06:41.311990 15859 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:41.274763 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:41.275134 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:41.275837 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:41.276715 15929 raft_consensus.cc:1240] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:41.274823 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:41.271543 15934 raft_consensus.cc:1240] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 9 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:41.273845 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:41.278180 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:41.279852 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 55097.3us
I20260709 13:06:41.279968 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:41.325148 15860 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
I20260709 13:06:41.293363 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:41.326139 15857 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
W20260709 13:06:41.326256 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 71814.4us
I20260709 13:06:41.239733 20328 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:41.208416 19974 consensus_queue.cc:237] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:41.314666 16032 debug-util.cc:398] Leaking SignalData structure 0x7b0800388660 after lost signal to thread 15961
I20260709 13:06:41.324596 16533 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:41.328393 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:41.331511 15856 raft_consensus.cc:2393] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 10: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:41.283830 15854 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
W20260709 13:06:41.331712 16032 debug-util.cc:398] Leaking SignalData structure 0x7b080036a980 after lost signal to thread 16035
W20260709 13:06:41.332886 16032 debug-util.cc:398] Leaking SignalData structure 0x7b08002dc280 after lost signal to thread 16594
I20260709 13:06:41.287009 20196 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:41.334599 15855 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:06:41.335144 20325 consensus_queue.cc:237] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 6, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:41.337077 16533 leader_election.cc:400] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) with higher term. Message: Invalid argument: T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 11. Current term is 12.
I20260709 13:06:41.337492 16533 leader_election.cc:403] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:41.338560 20155 raft_consensus.cc:2804] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:41.338626 16533 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:41.339066 20155 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:41.339514 20155 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:41.339881 16533 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:41.327543 15827 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:41.344020 15827 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:41.345244 15861 raft_consensus.cc:1240] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:41.348639 15827 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:41.347467 16533 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:41.349794 20133 raft_consensus.cc:2804] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:41.353194 20133 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:41.353631 20133 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:41.360044 20327 raft_consensus.cc:2804] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:41.360337 20328 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:41.360908 20198 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:41.279177 20071 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:41.362330 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:41.362787 20071 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:41.362947 15935 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:06:41.350157 20075 consensus_queue.cc:1059] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 13:06:41.343456 20196 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.364166 20196 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:41.366351 20196 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:41.367252 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:41.367962 15935 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:06:41.368615 15827 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:41.372042 15902 leader_election.cc:336] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 2.094s (SENT)
I20260709 13:06:41.341681 20222 raft_consensus.cc:2749] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) with higher term. Message: Invalid argument: T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 11. Current term is 12.
I20260709 13:06:41.345134 20326 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:41.398204 15827 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:41.330446 19917 consensus_queue.cc:1059] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:06:41.427546 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:41.360710 20327 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:41.428269 15854 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 12.
I20260709 13:06:41.423640 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:41.367712 20097 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:41.369748 20227 raft_consensus.cc:2804] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:06:41.493306 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 95199.2us
I20260709 13:06:41.494385 15827 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:41.502293 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:41.502789 16565 raft_consensus.cc:1240] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:41.502970 16570 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:06:41.395779 20196 raft_consensus.cc:2749] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:41.372079 20328 consensus_queue.cc:1048] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.127s
I20260709 13:06:41.427914 20165 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:41.425071 19223 raft_consensus.cc:3055] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:41.500933 15902 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:41.520175 15856 raft_consensus.cc:1240] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:41.496178 20326 raft_consensus.cc:697] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:41.492955 20227 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:41.505842 20075 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:41.446420 20327 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:41.525260 16533 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:41.522578 19223 raft_consensus.cc:740] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:41.535180 19723 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:06:41.535272 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 89874.1us
I20260709 13:06:41.523353 15902 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:41.514760 20097 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:41.515614 20232 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.551941 20232 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:41.537453 20227 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:41.542781 19223 consensus_queue.cc:260] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:41.561494 20184 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:41.562367 15902 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:41.562876 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:41.563233 20341 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:41.563786 20341 raft_consensus.cc:697] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 10 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:41.564659 20341 consensus_queue.cc:237] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:41.568003 15902 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:41.543825 20222 consensus_queue.cc:1048] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:41.561663 19223 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:41.577576 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 83316.8us
I20260709 13:06:41.578421 15827 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:41.521849 20165 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:41.537632 20075 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:41.583220 16533 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:41.533149 20337 raft_consensus.cc:2804] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:41.607983 15900 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:41.608394 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:41.610302 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:41.551379 20162 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:41.580796 20198 raft_consensus.cc:2804] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:41.618963 15900 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:41.643297 15900 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:41.643558 15827 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:41.644639 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:41.645124 15827 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:41.623724 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:41.624135 15854 raft_consensus.cc:1275] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:06:41.646752 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:41.618844 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:41.624680 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:41.586027 20135 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:41.606019 20075 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:41.647991 15857 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
I20260709 13:06:41.613641 20329 raft_consensus.cc:2804] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:41.588385 20165 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:41.614498 20097 raft_consensus.cc:2749] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:41.648746 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:41.669363 20333 log.cc:927] Time spent T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.326s	user 0.002s	sys 0.000s
I20260709 13:06:41.642720 20198 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:41.605793 20195 consensus_queue.cc:1048] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:06:41.651641 20262 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:41.654596 20167 raft_consensus.cc:2749] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:06:41.661114 20329 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:41.687109 15935 raft_consensus.cc:1275] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 6 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 13:06:41.699932 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 118437us
I20260709 13:06:41.700712 15902 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:41.704748 15902 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:41.707427 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 64234us
I20260709 13:06:41.709298 15900 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:41.651310 19917 consensus_queue.cc:1059] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:06:41.645968 20088 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:41.680065 20198 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:41.684592 20106 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:41.711654 20135 raft_consensus.cc:697] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:41.685114 20232 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:41.752779 20341 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:41.753499 20360 raft_consensus.cc:2804] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:41.757164 20356 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:06:41.757411 20340 log.cc:927] Time spent T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 13:06:41.655622 20324 consensus_queue.cc:1059] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.055s
W20260709 13:06:41.758877 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 137914us
I20260709 13:06:41.709384 20337 raft_consensus.cc:697] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:41.658089 20326 consensus_queue.cc:237] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:41.715008 20321 raft_consensus.cc:2749] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:41.702142 20329 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:41.758584 20262 raft_consensus.cc:697] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 12 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:41.803027 15853 raft_consensus.cc:2393] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 10: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:41.806143 16533 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:41.818859 20341 raft_consensus.cc:697] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:06:41.819854 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 115169us
I20260709 13:06:41.797590 20106 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:41.822247 15902 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:41.819463 20338 log.cc:927] Time spent T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 13:06:41.809163 20356 raft_consensus.cc:697] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 13 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:41.874770 20135 consensus_queue.cc:237] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:41.806871 20360 raft_consensus.cc:697] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:41.870452 20106 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:41.809506 20354 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:41.878536 20356 consensus_queue.cc:237] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:06:41.874042 20321 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:41.918380 16569 raft_consensus.cc:1275] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:06:41.880103 20343 log.cc:927] Time spent T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
W20260709 13:06:41.886659 20155 consensus_meta.cc:287] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.547s	user 0.004s	sys 0.004s
I20260709 13:06:41.885637 19723 raft_consensus.cc:3055] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:41.888258 15861 raft_consensus.cc:3055] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:41.921496 15861 raft_consensus.cc:740] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:41.889860 15931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 2061) took 1054 ms. Trace:
W20260709 13:06:41.891964 20319 log.cc:927] Time spent T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:06:41.922109 15931 rpcz_store.cc:276] 0709 13:06:40.835221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:40.852596 (+ 17375us) service_pool.cc:224] Handling call
0709 13:06:41.889681 (+1037085us) spinlock_profiling.cc:243] Waited 489 ms on lock 0x7b54003b8b54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:41.889805 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1123776256}
I20260709 13:06:41.921164 19806 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:41.931773 19806 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:41.918395 20321 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:41.922276 20133 consensus_meta.cc:287] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.568s	user 0.009s	sys 0.003s
I20260709 13:06:41.918921 20162 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:41.923237 19723 raft_consensus.cc:740] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:41.933497 15861 consensus_queue.cc:260] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:41.932197 19806 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:06:41.953733 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 147661us
W20260709 13:06:41.955945 20352 log.cc:927] Time spent T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 13:06:41.941007 20321 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:41.957279 20344 log.cc:927] Time spent T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.270s	user 0.001s	sys 0.001s
W20260709 13:06:41.959219 20351 log.cc:927] Time spent T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:06:41.962539 20227 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:41.892189 20337 consensus_queue.cc:237] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:41.963989 20088 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:41.921118 20262 consensus_queue.cc:237] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:41.965644 20327 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:41.966665 19723 consensus_queue.cc:260] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:41.954546 20341 consensus_queue.cc:237] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:42.120564 20327 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:42.121373 16570 raft_consensus.cc:2393] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 9: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:41.971742 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:41.942178 20133 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:41.948699 20304 consensus_meta.cc:287] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.689s	user 0.009s	sys 0.000s
I20260709 13:06:41.922960 20155 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:41.973944 15861 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:41.982825 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 273575us
I20260709 13:06:41.985452 20075 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:41.932890 20362 consensus_queue.cc:1059] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 13:06:42.108247 20106 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:42.134455 20106 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:42.137379 20133 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:42.137602 20198 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:42.127236 20304 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:42.140411 20304 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:41.973748 20360 consensus_queue.cc:237] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:42.141889 15861 raft_consensus.cc:1275] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 13:06:42.142418 19917 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:06:42.135408 20075 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:42.094172 20369 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:42.124850 16569 raft_consensus.cc:2393] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 10: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
W20260709 13:06:42.104110 20342 log.cc:927] Time spent T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.490s	user 0.000s	sys 0.002s
I20260709 13:06:42.126121 15900 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:42.130342 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:42.096550 20165 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:42.151878 20227 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:42.152535 20165 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:42.112844 19806 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:42.112218 19996 raft_consensus.cc:3055] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:06:42.149394 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:42.153226 20362 raft_consensus.cc:3055] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:42.153460 15930 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:42.153704 20362 raft_consensus.cc:740] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:42.149475 15932 raft_consensus.cc:1275] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:06:42.156096 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:42.156891 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:42.160470 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:42.160984 15900 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:42.161711 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:42.161947 15900 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:42.161051 15861 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:06:42.163584 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:42.163954 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:42.164075 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:42.168732 15855 raft_consensus.cc:2368] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 11. Current term is 12.
I20260709 13:06:42.169153 15931 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
I20260709 13:06:42.164415 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:42.171876 15930 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
I20260709 13:06:42.172786 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:42.173653 15855 raft_consensus.cc:2410] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:06:42.173837 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:42.175614 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:42.175836 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:42.177127 15855 raft_consensus.cc:2393] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 10: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:42.145578 20198 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:42.151273 20369 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:42.181502 15900 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:42.153259 19996 raft_consensus.cc:740] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:42.164670 20135 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:42.180529 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:42.183434 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:42.124600 20329 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:42.177922 20362 consensus_queue.cc:260] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:42.183020 20304 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:42.187072 19996 consensus_queue.cc:260] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:42.190958 20321 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:42.192153 19996 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:42.194167 20321 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:42.166363 20155 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:42.189605 20362 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:42.189913 20304 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:42.190027 20364 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:42.190834 20369 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:42.194590 20135 raft_consensus.cc:697] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 10 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:42.185192 20198 consensus_queue.cc:1048] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:06:42.196906 20361 consensus_queue.cc:1048] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:42.198110 20322 consensus_queue.cc:1048] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:42.201308 19806 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:42.180481 20356 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:42.247491 20356 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:42.247879 20356 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:42.248181 20068 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:42.248919 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:42.250432 15936 raft_consensus.cc:1275] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:06:42.215700 15857 raft_consensus.cc:1275] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:06:42.251073 20359 log.cc:927] Time spent T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.434s	user 0.001s	sys 0.000s
I20260709 13:06:42.215458 19813 raft_consensus.cc:3055] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:06:42.251997 16032 debug-util.cc:398] Leaking SignalData structure 0x7b08003ad820 after lost signal to thread 20152
I20260709 13:06:42.217141 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:06:42.217976 20358 log.cc:927] Time spent T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.300s	user 0.000s	sys 0.002s
W20260709 13:06:42.252358 16032 debug-util.cc:398] Leaking SignalData structure 0x7b0800347820 after lost signal to thread 20157
I20260709 13:06:42.252503 15854 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:42.252661 16032 debug-util.cc:398] Leaking SignalData structure 0x7b08004302c0 after lost signal to thread 20159
W20260709 13:06:42.252979 16032 debug-util.cc:398] Leaking SignalData structure 0x7b0800401e60 after lost signal to thread 20161
I20260709 13:06:42.220252 15860 raft_consensus.cc:1240] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:42.225103 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:42.217823 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:42.253285 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:42.229560 15934 raft_consensus.cc:1240] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 7. Current term is 8. Ops: []
W20260709 13:06:42.253232 16032 debug-util.cc:398] Leaking SignalData structure 0x7b08003b5d80 after lost signal to thread 20163
W20260709 13:06:42.254876 16032 debug-util.cc:398] Leaking SignalData structure 0x7b08003121a0 after lost signal to thread 20172
W20260709 13:06:42.255261 16032 debug-util.cc:398] Leaking SignalData structure 0x7b080037a7e0 after lost signal to thread 20203
W20260709 13:06:42.255741 16032 debug-util.cc:398] Leaking SignalData structure 0x7b08003b2240 after lost signal to thread 20237
I20260709 13:06:42.255833 15860 raft_consensus.cc:1275] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 10. (term mismatch)
W20260709 13:06:42.256052 16032 debug-util.cc:398] Leaking SignalData structure 0x7b0800409280 after lost signal to thread 20246
I20260709 13:06:42.256213 15860 pending_rounds.cc:85] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8: Aborting all ops after (but not including) 9
I20260709 13:06:42.231094 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:42.255344 15861 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:42.256485 15860 pending_rounds.cc:107] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 13:06:42.241966 16565 raft_consensus.cc:2368] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 9. Current term is 10.
I20260709 13:06:42.248694 20068 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:42.253592 15855 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:42.258013 15857 raft_consensus.cc:1275] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:06:42.226524 15859 raft_consensus.cc:1240] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:42.227443 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:06:42.226179 15853 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58214 (request call id 806) took 1305 ms (client timeout 1682 ms). Trace:
I20260709 13:06:42.253684 15862 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:42.261848 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:42.232838 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:42.256891 15929 raft_consensus.cc:2393] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 9: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:42.257184 15860 raft_consensus.cc:2889] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:06:42.262163 15853 rpcz_store.cc:269] 0709 13:06:40.920997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:40.921155 (+   158us) service_pool.cc:224] Handling call
0709 13:06:42.226004 (+1304849us) spinlock_profiling.cc:243] Waited 877 ms on lock 0x7b5400217354. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:42.226126 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2017577344}
I20260709 13:06:42.239934 20364 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:42.245046 20198 consensus_queue.cc:1243] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer 31882606e87447fb9a36bbf625129fe8 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:06:42.244321 20135 consensus_queue.cc:237] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:42.257673 20163 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:42.257994 20161 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:42.252013 19813 raft_consensus.cc:740] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:42.258535 20157 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:42.241103 20304 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:42.362347 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:42.264137 15928 raft_consensus.cc:2393] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 10: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:42.347754 20163 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:42.335490 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:42.342202 15862 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
I20260709 13:06:42.357179 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:42.260403 20068 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:42.348398 15855 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 13.
I20260709 13:06:42.363629 20329 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:42.371649 16570 raft_consensus.cc:1240] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:42.267894 20364 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:42.355414 15854 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:06:42.385133 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:42.385818 16566 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:42.385793 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 62595.6us
I20260709 13:06:42.389909 16564 raft_consensus.cc:1275] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:06:42.406422 15900 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:42.417515 15900 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:42.417861 16569 raft_consensus.cc:1275] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:06:42.374274 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:42.372285 20157 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:42.434317 16004 raft_consensus.cc:606] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Received request to transfer leadership to TS 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:42.434702 16004 raft_consensus.cc:614] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:42.455384 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2017) took 2226 ms. Trace:
W20260709 13:06:42.457809 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:42.458001 15974 connection.cc:256] RPC call timeout handler was delayed by -2.4065208319999556s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22250000000000003s for a response
W20260709 13:06:42.458596 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:42.459082 15974 connection.cc:256] RPC call timeout handler was delayed by -1.9858996270000944s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1769s for a response
W20260709 13:06:42.459549 15974 connection.cc:256] RPC call timeout handler was delayed by -1.0896117270000332s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16970000000000002s for a response
I20260709 13:06:42.460152 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:42.433380 20341 consensus_queue.cc:1048] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:06:42.461037 16003 raft_consensus.cc:2393] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 12: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:42.461431 20357 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:42.462061 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:42.461926 20357 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:42.462612 16010 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:42.366312 20369 consensus_queue.cc:1048] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.075s
I20260709 13:06:42.366153 20155 consensus_queue.cc:1059] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.091s
I20260709 13:06:42.464048 20070 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:42.464550 20070 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:42.347529 20361 consensus_queue.cc:1048] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.074s
I20260709 13:06:42.466405 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:42.456225 20354 raft_consensus.cc:2804] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:42.457217 16007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2019) took 2230 ms. Trace:
I20260709 13:06:42.460240 15974 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:42.468083 16007 rpcz_store.cc:276] 0709 13:06:40.226981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:40.407045 (+180064us) service_pool.cc:224] Handling call
0709 13:06:40.407305 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:42.456773 (+2049468us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:06:42.456813 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:42.456863 (+    50us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:42.456939 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:42.457019 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:42.457151 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2049146,"raft.queue_time_us":2056287,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":86}
I20260709 13:06:42.435076 20321 consensus_queue.cc:1048] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:06:42.509060 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:42.520955 16005 raft_consensus.cc:2376] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10. Re-sending same reply.
I20260709 13:06:42.455448 20195 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:42.362095 20161 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:42.467648 20354 raft_consensus.cc:697] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 9 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:42.531229 20195 raft_consensus.cc:697] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 13 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:06:42.536245 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:42.615381 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 193970us
I20260709 13:06:42.456266 16002 rpcz_store.cc:276] 0709 13:06:40.207443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:40.215476 (+  8033us) service_pool.cc:224] Handling call
0709 13:06:40.215849 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:40.216118 (+   269us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:40.266677 (+ 50559us) log.cc:844] Serialized 14 byte log entry
0709 13:06:42.433898 (+2167221us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:42.433948 (+    50us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:42.434004 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:42.434072 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:42.434176 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2142426,"prepare.queue_time_us":43991,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":2165045,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":85,"thread_start_us":35886,"threads_started":2,"wal-append.queue_time_us":33406,"wal-append.run_cpu_time_us":1032,"wal-append.run_wall_time_us":1001059}
W20260709 13:06:42.623056 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 274915us
W20260709 13:06:42.623430 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:42.623793 15900 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:06:42.624351 15827 leader_election.cc:336] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 1.667s (SENT)
I20260709 13:06:42.470849 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:42.625628 16008 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:42.627046 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:42.456256 16561 raft_consensus.cc:2393] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 9: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:42.634503 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:42.650580 15861 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
I20260709 13:06:42.654026 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:42.663964 15854 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:42.643536 20198 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:42.657066 20157 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:42.668061 20198 raft_consensus.cc:697] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:42.669260 20161 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:42.669000 20198 consensus_queue.cc:237] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:42.623960 20354 consensus_queue.cc:237] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:42.652463 19813 consensus_queue.cc:260] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:42.674659 19813 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:42.674856 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:42.670518 20357 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:42.671486 15854 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
I20260709 13:06:42.677167 20155 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:42.652776 20124 consensus_queue.cc:1048] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:06:42.677695 20155 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:42.654521 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:42.659118 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 100967us
I20260709 13:06:42.659879 20158 consensus_queue.cc:1048] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:42.680198 20155 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:42.682535 16004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:43700 (request call id 1) took 2414 ms. Trace:
I20260709 13:06:42.684795 16004 rpcz_store.cc:276] 0709 13:06:40.267758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:40.409584 (+141826us) service_pool.cc:224] Handling call
0709 13:06:42.682339 (+2272755us) spinlock_profiling.cc:243] Waited 2.02 s on lock 0x7b5400382cd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a2853 00007f73ef37060a 00007f73e88f67d4 00007f73e88f6762 00007f73e88f6691 00007f73e88f660c 00007f73e88f58c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:42.682449 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4654682880}
I20260709 13:06:42.688033 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:42.691156 15930 raft_consensus.cc:1275] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:06:42.692672 20357 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:42.707154 15861 raft_consensus.cc:1240] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 10. Current term is 11. Ops: []
W20260709 13:06:42.715845 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 2.49198e+06us
I20260709 13:06:42.716619 19728 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:42.717159 19728 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:42.717658 15976 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:42.718427 20071 consensus_queue.cc:1048] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:42.718972 15976 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:42.719229 19728 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:42.719914 20246 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:42.720422 20246 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:42.715901 20179 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:42.720921 20246 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:42.721117 20179 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:42.723320 20179 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:42.716023 20068 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:42.723881 20203 consensus_queue.cc:1059] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:42.724027 20068 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:42.725026 20203 raft_consensus.cc:2804] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:42.725466 20203 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:42.725854 20203 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:42.726404 20068 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:42.726991 20068 raft_consensus.cc:3055] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:42.727336 20068 raft_consensus.cc:740] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:42.719725 19728 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:42.728216 19728 consensus_queue.cc:1243] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer eb1e0493af1e46dbb6dec7e25525bab5 log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:06:42.728535 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 268365us
I20260709 13:06:42.729647 20068 consensus_queue.cc:260] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:42.717659 20237 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:42.732668 20068 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:42.731459 20152 consensus_queue.cc:1059] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 6.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:42.730676 20179 raft_consensus.cc:2764] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:06:42.731611 19728 consensus_queue.cc:1059] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:42.733651 15974 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:42.737383 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:42.743752 16004 raft_consensus.cc:2376] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10. Re-sending same reply.
I20260709 13:06:42.749503 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:42.762247 15861 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:42.750329 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 110270us
I20260709 13:06:42.732817 20237 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:42.770586 15858 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.064s	sys 0.117s
W20260709 13:06:42.785528 15858 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.064s	sys 0.117s
I20260709 13:06:42.792516 15861 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:06:42.807564 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:42.808282 16004 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:06:42.814023 15825 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:42.815189 16008 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
W20260709 13:06:42.742511 20262 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.095s	sys 0.109s
W20260709 13:06:42.821125 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 89852.3us
W20260709 13:06:42.743186 20360 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.022s	sys 0.030s
I20260709 13:06:42.818204 16533 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:42.820407 20262 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.096s	sys 0.109s
W20260709 13:06:42.869243 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 51121.5us
I20260709 13:06:42.817725 20183 raft_consensus.cc:3055] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:42.875958 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:42.816910 20172 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:42.817435 20088 raft_consensus.cc:2749] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:42.858256 15825 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:42.876629 16004 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
I20260709 13:06:42.742573 20177 consensus_queue.cc:1048] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.124s
W20260709 13:06:42.889536 16566 consensus_meta.cc:287] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.503s	user 0.005s	sys 0.009s
W20260709 13:06:42.870299 20360 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.023s	sys 0.030s
I20260709 13:06:42.770517 20156 consensus_queue.cc:1048] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.100s
I20260709 13:06:42.891554 15825 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:42.878670 16533 leader_election.cc:400] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 11. Current term is 12.
I20260709 13:06:42.935863 16566 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
W20260709 13:06:42.936756 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 58243.3us
W20260709 13:06:42.936965 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 45524.7us
I20260709 13:06:42.937589 16533 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:42.864642 20070 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:42.938298 15825 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:42.898993 19996 consensus_meta.cc:287] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.706s	user 0.009s	sys 0.009s
I20260709 13:06:42.940009 19996 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:42.940457 20088 raft_consensus.cc:2749] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:42.940591 19996 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:42.941413 20088 consensus_queue.cc:1059] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:06:42.942381 20088 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:42.869906 20163 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:42.866047 20341 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:42.922842 16010 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:06:42.817179 20289 consensus_queue.cc:1048] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 13:06:42.943439 15825 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:42.944221 20341 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:42.881094 20329 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:42.945295 20329 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:42.945695 20349 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:42.946038 15935 raft_consensus.cc:1275] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:06:42.946229 20349 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:42.945734 20329 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:42.946554 20163 consensus_queue.cc:1048] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:06:42.946779 20349 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:42.949508 19996 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:42.951448 16535 leader_election.cc:336] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 1.747s (SENT)
I20260709 13:06:42.946354 20341 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:42.947703 15825 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:42.956143 19917 raft_consensus.cc:2764] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:06:42.956965 20329 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:42.959547 20329 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:42.963913 20007 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:42.964362 20007 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:42.968027 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:42.876401 20183 raft_consensus.cc:740] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:42.968794 16004 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:06:42.969794 20183 consensus_queue.cc:260] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:42.971905 20382 log.cc:927] Time spent T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:06:42.893158 20172 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:42.907401 19660 raft_consensus.cc:2749] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:42.978921 16533 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:42.865201 20357 consensus_queue.cc:1059] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.111s
I20260709 13:06:42.988471 16533 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:42.986816 15825 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:42.990854 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:42.819821 20195 consensus_queue.cc:237] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:42.993714 16004 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
W20260709 13:06:42.983096 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 250291us
I20260709 13:06:42.976303 20183 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:42.981690 19776 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:43.002297 16533 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:42.992406 20369 raft_consensus.cc:2749] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:42.997046 15825 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:43.024418 16533 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:43.026916 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:43.027663 16010 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:43.001564 19660 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:06:43.018491 20356 consensus_meta.cc:287] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.759s	user 0.017s	sys 0.000s
W20260709 13:06:43.070276 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 45950.4us
I20260709 13:06:43.071959 20368 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:43.080760 16533 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:06:43.081600 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 58363.1us
I20260709 13:06:43.071856 19776 consensus_queue.cc:1059] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:06:43.075433 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 254976us
I20260709 13:06:43.005771 20357 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:42.994930 20347 consensus_queue.cc:1048] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:06:43.092293 16533 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:43.095402 15937 raft_consensus.cc:1275] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 13:06:43.136972 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:43.137529 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:43.139022 16566 raft_consensus.cc:1275] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:06:43.140246 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:43.140452 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:43.141515 16003 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 10: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:43.134465 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:43.020478 19806 raft_consensus.cc:2749] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:43.067807 19728 consensus_queue.cc:1243] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer 8430db80a39d4868bcf7641c16de35ea log is divergent from this leader: its last log entry 11.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:06:43.170679 16567 raft_consensus.cc:1275] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:06:43.126199 16010 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:06:43.148149 16564 raft_consensus.cc:1275] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:06:43.173755 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:43.147893 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:43.176242 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:06:43.161315 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:43.161063 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:06:43.169149 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 76943.4us
I20260709 13:06:43.062505 20088 raft_consensus.cc:2804] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:43.176687 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:43.148023 16005 raft_consensus.cc:1275] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:06:43.176846 16008 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:06:43.076948 19660 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:43.134431 20177 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.244395 16533 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:43.135154 20354 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:06:43.246546 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:43.250208 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:43.089911 20188 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.147307 20159 raft_consensus.cc:2749] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:43.246196 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:43.263839 16008 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:43.275661 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:43.017218 20152 consensus_queue.cc:1059] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.188s
I20260709 13:06:43.165903 20368 raft_consensus.cc:2749] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:43.288072 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:43.284034 16533 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:43.290865 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:43.291105 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:43.293268 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:43.293756 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:43.297489 16008 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
W20260709 13:06:43.299072 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 379884us
I20260709 13:06:43.302352 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:43.303560 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:43.304850 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:43.305744 16010 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 13.
I20260709 13:06:43.307740 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:43.308749 16010 raft_consensus.cc:2393] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 11: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:43.316776 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:43.093397 20356 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:43.322250 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:43.255270 19660 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:43.323088 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:43.258069 19728 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.178190 20260 consensus_queue.cc:1048] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.124s
I20260709 13:06:43.283648 20237 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:43.304592 20152 consensus_queue.cc:1243] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer eb1e0493af1e46dbb6dec7e25525bab5 log is divergent from this leader: its last log entry 9.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:06:43.301717 20088 raft_consensus.cc:697] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 10 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:43.285012 20177 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:43.281571 20329 raft_consensus.cc:2749] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:06:43.284462 20400 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.348088 20368 raft_consensus.cc:2804] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:43.345104 20322 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:43.357697 20349 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:43.296986 20188 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:43.292896 20354 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:43.343739 20357 consensus_queue.cc:1059] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:06:43.377120 20400 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:43.388119 20188 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:43.388478 20400 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:43.390285 20354 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:43.388113 20393 log.cc:927] Time spent T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
W20260709 13:06:43.379803 20379 log.cc:927] Time spent T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.384s	user 0.001s	sys 0.000s
W20260709 13:06:43.397393 20385 log.cc:927] Time spent T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
W20260709 13:06:43.402977 20246 consensus_meta.cc:287] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.681s	user 0.005s	sys 0.002s
I20260709 13:06:43.381765 20322 raft_consensus.cc:697] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:43.380069 20368 raft_consensus.cc:697] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:43.412300 20422 raft_consensus.cc:3055] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:43.412902 20422 raft_consensus.cc:740] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:43.415688 20422 consensus_queue.cc:260] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:43.412366 20368 consensus_queue.cc:237] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:43.420296 20380 log.cc:927] Time spent T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
W20260709 13:06:43.154397 15902 leader_election.cc:336] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 1.502s (SENT)
I20260709 13:06:43.306185 15974 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:06:43.332409 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 310003us
I20260709 13:06:43.563057 20177 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:43.473713 15903 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.008s	sys 0.020s
W20260709 13:06:43.463048 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:43.573010 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:43.486255 15928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 882) took 1050 ms. Trace:
I20260709 13:06:43.641525 20422 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:43.641945 15928 rpcz_store.cc:276] 0709 13:06:42.435561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:42.435700 (+   139us) service_pool.cc:224] Handling call
0709 13:06:42.436025 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:42.436334 (+   309us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:42.436360 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:42.436804 (+   444us) log.cc:844] Serialized 26 byte log entry
0709 13:06:42.735405 (+298601us) spinlock_profiling.cc:243] Waited 4.52 ms on lock 0x7b1800470620. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f480d1 00007f73ea363a0f
0709 13:06:42.998302 (+262897us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:42.998358 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:42.998448 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:43.485294 (+486846us) raft_consensus.cc:1685] finished
0709 13:06:43.485357 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:43.485961 (+   604us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":20429,"raft.run_cpu_time_us":70,"raft.run_wall_time_us":70,"spinlock_wait_cycles":10402048,"thread_start_us":262489,"threads_started":1,"wal-append.queue_time_us":948437}
I20260709 13:06:43.582518 20071 raft_consensus.cc:3055] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:43.496631 16006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 2152) took 3289 ms. Trace:
W20260709 13:06:43.644874 15903 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.766s	user 0.008s	sys 0.032s
I20260709 13:06:43.645118 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:06:43.646274 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:43.649827 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:43.651088 16535 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 4.048s (SENT) [suppressed 39 similar messages]
I20260709 13:06:43.560696 20088 consensus_queue.cc:237] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:43.653225 19974 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.570225 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:43.654400 19974 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:43.655572 16008 raft_consensus.cc:2410] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:06:43.656706 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:43.658033 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:43.470458 15860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 2113) took 1157 ms. Trace:
I20260709 13:06:43.342630 19728 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:43.659914 15860 rpcz_store.cc:276] 0709 13:06:42.312902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:42.313240 (+   338us) service_pool.cc:224] Handling call
0709 13:06:42.313781 (+   541us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:42.314316 (+   535us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:42.314357 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:42.314964 (+   607us) log.cc:844] Serialized 26 byte log entry
0709 13:06:42.786245 (+471281us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:42.786300 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:42.786381 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:43.469576 (+683195us) raft_consensus.cc:1685] finished
0709 13:06:43.469653 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:43.470161 (+   508us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":946,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":5466,"thread_start_us":414395,"threads_started":1,"wal-append.queue_time_us":678306}
I20260709 13:06:43.663056 19728 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:43.663972 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 353643us
W20260709 13:06:43.664469 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 358319us
I20260709 13:06:43.665162 20007 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:43.666121 20300 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:43.665978 20007 consensus_queue.cc:1059] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:43.666677 20300 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:43.666961 15974 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:43.667179 20300 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:06:43.670979 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:43.644412 20071 raft_consensus.cc:740] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:43.667217 20007 raft_consensus.cc:3055] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:06:43.560323 20304 consensus_meta.cc:287] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 1.171s	user 0.009s	sys 0.027s
I20260709 13:06:43.647133 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:43.672446 16010 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:43.421761 20364 consensus_meta.cc:287] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.998s	user 0.010s	sys 0.009s
I20260709 13:06:43.573173 20349 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:43.421870 20362 consensus_meta.cc:287] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 1.180s	user 0.015s	sys 0.004s
I20260709 13:06:43.685921 19806 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.686797 19806 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:43.689816 20071 consensus_queue.cc:260] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:43.675127 20007 raft_consensus.cc:740] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:43.692440 20421 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.693516 20421 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:43.694427 19723 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:43.697973 20409 log.cc:927] Time spent T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.306s	user 0.000s	sys 0.001s
I20260709 13:06:43.405061 20246 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:43.700313 20198 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:06:43.701395 20414 log.cc:927] Time spent T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.296s	user 0.000s	sys 0.001s
I20260709 13:06:43.701205 20198 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:43.702740 20246 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:43.704449 16007 raft_consensus.cc:2393] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 12: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:43.704793 20357 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:43.705672 20357 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:43.693923 20007 consensus_queue.cc:260] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:43.693516 20163 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:43.707386 20007 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:43.572022 20188 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.720901 20357 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:06:43.723505 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:43.576200 20361 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.736162 20172 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:43.737027 20304 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:43.644924 16006 rpcz_store.cc:276] 0709 13:06:40.206717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:40.217330 (+ 10613us) service_pool.cc:224] Handling call
0709 13:06:40.225835 (+  8505us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:42.361621 (+2135786us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:42.361652 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:42.714540 (+352888us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b2c000e5768. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e8ca1a11 00007f73e8cb1372 00007f73ea3e3a01 00007f73ea3a843a 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2
0709 13:06:42.715062 (+   522us) log.cc:844] Serialized 52 byte log entry
0709 13:06:43.063741 (+348679us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:43.144079 (+ 80338us) log.cc:844] Serialized 14 byte log entry
0709 13:06:43.144263 (+   184us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:43.144345 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:43.495892 (+351547us) raft_consensus.cc:1685] finished
0709 13:06:43.496098 (+   206us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:43.496430 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2466630,"prepare.queue_time_us":82770,"prepare.run_cpu_time_us":2563,"prepare.run_wall_time_us":39665,"raft.queue_time_us":2347277,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":19285,"spinlock_wait_cycles":610913152,"thread_start_us":348285,"threads_started":1,"wal-append.queue_time_us":666104}
W20260709 13:06:43.739059 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:43.707530 20163 raft_consensus.cc:697] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:43.739432 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:06:43.743177 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:43.745162 15931 raft_consensus.cc:1240] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:43.569307 20322 consensus_queue.cc:237] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:43.750628 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:43.764290 20188 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:06:43.767556 20188 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:43.768484 20407 log.cc:927] Time spent T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.387s	user 0.000s	sys 0.004s
I20260709 13:06:43.769421 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:43.749989 20198 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:43.762247 19974 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:43.779295 15932 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.015s	sys 0.004s
I20260709 13:06:43.783344 16005 raft_consensus.cc:2376] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8. Re-sending same reply.
I20260709 13:06:43.775344 20357 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.794730 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:06:43.795684 15932 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.866s	user 0.016s	sys 0.004s
I20260709 13:06:43.782346 20304 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:43.783119 20361 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:43.817055 16011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2016) took 3555 ms. Trace:
I20260709 13:06:43.825481 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:43.840211 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:43.826336 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:43.816656 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:43.848326 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 70771.6us
I20260709 13:06:43.874964 16010 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
W20260709 13:06:43.866511 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 215494us
I20260709 13:06:43.824047 20071 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:43.841091 19776 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.018s	sys 0.017s
W20260709 13:06:43.835992 20157 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.840s	user 0.012s	sys 0.029s
W20260709 13:06:43.877126 20410 log.cc:927] Time spent T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.426s	user 0.000s	sys 0.003s
I20260709 13:06:43.893896 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:43.864913 16011 rpcz_store.cc:276] 0709 13:06:40.207163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:40.213068 (+  5905us) service_pool.cc:224] Handling call
0709 13:06:40.213370 (+   302us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:40.266642 (+ 53272us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:40.266663 (+    21us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:40.295851 (+ 29188us) log.cc:844] Serialized 52 byte log entry
0709 13:06:40.356604 (+ 60753us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:40.369436 (+ 12832us) log.cc:844] Serialized 14 byte log entry
0709 13:06:42.779914 (+2410478us) log.cc:844] Serialized 14 byte log entry
0709 13:06:43.650274 (+870360us) spinlock_profiling.cc:243] Waited 3.47 ms on lock 0x7b18003b1a00. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb3f3 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd 000000000038f354 000000000038f199 00007f73e729e83a
0709 13:06:43.726291 (+ 76017us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:43.732304 (+  6013us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:43.749104 (+ 16800us) raft_consensus.cc:1685] finished
0709 13:06:43.749134 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:43.762504 (+ 13370us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2353781,"prepare.queue_time_us":453462,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":74,"raft.queue_time_us":4336648,"raft.run_cpu_time_us":348,"raft.run_wall_time_us":351,"spinlock_wait_cycles":7970560,"thread_start_us":1126690,"threads_started":5,"wal-append.queue_time_us":712968,"wal-append.run_cpu_time_us":4282,"wal-append.run_wall_time_us":1096213}
I20260709 13:06:43.898412 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:43.899179 15860 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:43.899955 15929 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:43.904273 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:43.905494 20361 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:43.906337 16005 raft_consensus.cc:2376] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10. Re-sending same reply.
I20260709 13:06:43.836476 20421 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:43.907647 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:43.909276 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:43.909327 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:43.909775 15974 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:43.910117 16005 raft_consensus.cc:2376] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10. Re-sending same reply.
W20260709 13:06:43.890074 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 728858us
I20260709 13:06:43.831203 20179 raft_consensus.cc:3055] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:43.835907 20364 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:43.892329 19776 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.019s	sys 0.018s
W20260709 13:06:43.892591 20068 consensus_meta.cc:287] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 1.150s	user 0.019s	sys 0.003s
W20260709 13:06:44.023015 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:43.892727 20157 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.960s	user 0.013s	sys 0.029s
I20260709 13:06:44.027865 16003 raft_consensus.cc:2410] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:06:43.847143 19806 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:43.910486 16007 raft_consensus.cc:2393] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 10: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:44.030067 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:44.030825 16007 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
I20260709 13:06:43.944623 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:43.917834 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:43.910989 20431 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 13:06:44.032280 16002 raft_consensus.cc:2376] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10. Re-sending same reply.
I20260709 13:06:43.925611 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:44.033735 16004 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:06:43.948060 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:43.929622 20155 raft_consensus.cc:3055] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:44.035781 20155 raft_consensus.cc:740] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:06:44.036338 20429 log.cc:927] Time spent T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.385s	user 0.004s	sys 0.000s
I20260709 13:06:43.957455 16008 raft_consensus.cc:2368] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 11. Current term is 12.
W20260709 13:06:43.916566 20394 log.cc:927] Time spent T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.549s	user 0.005s	sys 0.002s
I20260709 13:06:43.942656 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:43.931710 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:43.841148 20163 consensus_queue.cc:237] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:43.895726 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:06:43.950377 20384 log.cc:927] Time spent T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.995s	user 0.001s	sys 0.002s
I20260709 13:06:43.947865 16570 raft_consensus.cc:1217] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 LEARNER]: Deduplicated request from leader. Original: 6.5->[7.6-12.8]   Dedup: 7.6->[10.7-12.8]
I20260709 13:06:44.045209 16570 pending_rounds.cc:85] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting all ops after (but not including) 6
I20260709 13:06:43.914580 20299 raft_consensus.cc:2749] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:06:43.924471 15931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 874) took 1711 ms. Trace:
W20260709 13:06:43.953939 20390 log.cc:927] Time spent T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.602s	user 0.001s	sys 0.003s
W20260709 13:06:43.936097 19660 consensus_meta.cc:287] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.589s	user 0.010s	sys 0.006s
W20260709 13:06:43.910969 20203 consensus_meta.cc:287] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 1.184s	user 0.010s	sys 0.010s
I20260709 13:06:43.946374 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:43.933199 15932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 884) took 1293 ms. Trace:
I20260709 13:06:43.946863 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:43.929275 15861 raft_consensus.cc:1275] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:06:43.938234 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:43.946374 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:43.895359 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:43.835973 19974 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:43.932272 15853 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:06:43.830273 20357 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:43.934584 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:43.929550 15854 pending_rounds.cc:85] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8: Aborting all ops after (but not including) 7
I20260709 13:06:43.845850 20300 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:43.919251 20183 consensus_meta.cc:287] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.913s	user 0.016s	sys 0.000s
I20260709 13:06:43.945437 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:44.049075 15854 pending_rounds.cc:107] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:06:43.916988 15974 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:06:43.982271 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 90751.6us
W20260709 13:06:43.986873 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 194792us
W20260709 13:06:43.993361 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:43.993758 15860 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
I20260709 13:06:43.999320 20356 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:43.920040 19813 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.021132 20364 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:44.055612 19660 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:44.057658 15902 leader_election.cc:336] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 1.513s (SENT)
I20260709 13:06:44.057387 19813 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:44.057957 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:44.058152 15902 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:44.058759 15856 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:06:44.059245 19660 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:44.055810 20203 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:44.060021 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:44.060869 16011 raft_consensus.cc:2368] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 9. Current term is 10.
I20260709 13:06:44.061440 20357 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:44.061741 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:44.062620 16011 raft_consensus.cc:2368] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 11. Current term is 12.
I20260709 13:06:44.062314 19974 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:44.054273 15854 raft_consensus.cc:2889] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:44.064007 20356 raft_consensus.cc:2749] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:44.024896 15929 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:06:44.064764 19974 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:44.041258 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:44.065691 16007 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:06:44.066555 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:44.067107 16007 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:44.068077 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 151251us
I20260709 13:06:44.068164 16562 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239"
dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
mode: GRACEFUL
new_leader_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:42622
I20260709 13:06:44.069037 16562 raft_consensus.cc:606] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEADER]: Received request to transfer leadership to TS 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:44.069375 15862 raft_consensus.cc:1275] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:06:44.069716 15974 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:44.070277 15928 raft_consensus.cc:1240] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:44.062328 16566 raft_consensus.cc:2410] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:06:44.050302 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:44.046103 15931 rpcz_store.cc:276] 0709 13:06:42.212725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:42.218200 (+  5475us) service_pool.cc:224] Handling call
0709 13:06:42.218654 (+   454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:43.745653 (+1526999us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:43.924186 (+178533us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b540034a154. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:43.924412 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3497958144}
I20260709 13:06:44.014777 20179 raft_consensus.cc:740] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:44.045557 16570 pending_rounds.cc:107] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:06:44.063439 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:44.061882 20203 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:44.038816 16006 raft_consensus.cc:2410] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:06:44.066633 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:44.054952 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:44.072938 16011 raft_consensus.cc:2393] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 13: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:44.037034 20155 consensus_queue.cc:260] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:44.070400 20007 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:44.086182 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:44.076133 15930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 889) took 1380 ms. Trace:
I20260709 13:06:44.086948 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:44.072579 16570 raft_consensus.cc:2889] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:44.077003 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:44.077378 20155 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:44.079576 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:44.079875 20299 raft_consensus.cc:2749] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:44.080662 16567 raft_consensus.cc:1240] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:06:44.079631 20349 consensus_queue.cc:1048] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:44.082378 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:44.083144 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:44.083904 15859 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:44.083494 20179 consensus_queue.cc:260] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:06:44.084015 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:44.083065 15825 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:44.085810 16009 raft_consensus.cc:2393] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 9: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
W20260709 13:06:44.088449 15902 leader_election.cc:336] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 1.553s (SENT)
I20260709 13:06:44.077016 15974 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:44.100675 16005 raft_consensus.cc:2393] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 11: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:44.102015 16568 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a"
dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
mode: GRACEFUL
new_leader_uuid: "8430db80a39d4868bcf7641c16de35ea"
 from {username='slave'} at 127.0.0.1:42622
I20260709 13:06:44.095379 16561 raft_consensus.cc:1217] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Deduplicated request from leader. Original: 4.4->[6.5-9.6]   Dedup: 6.5->[9.6-9.6]
I20260709 13:06:44.104171 16561 raft_consensus.cc:1240] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 9. Current term is 10. Ops: [6.5-9.6]
I20260709 13:06:44.101653 16533 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:44.101828 16006 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
I20260709 13:06:44.106813 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:44.107478 15860 raft_consensus.cc:1275] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:06:44.047390 15932 rpcz_store.cc:276] 0709 13:06:42.639779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:42.639943 (+   164us) service_pool.cc:224] Handling call
0709 13:06:42.640265 (+   322us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:42.657335 (+ 17070us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:42.657365 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:42.682798 (+ 25433us) log.cc:844] Serialized 52 byte log entry
0709 13:06:42.871871 (+189073us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:43.881553 (+1009682us) log.cc:844] Serialized 14 byte log entry
0709 13:06:43.932812 (+ 51259us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:43.932906 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:43.932959 (+    53us) raft_consensus.cc:1685] finished
0709 13:06:43.932970 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:43.933134 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56270,"prepare.queue_time_us":451489,"prepare.run_cpu_time_us":335,"prepare.run_wall_time_us":2892,"raft.queue_time_us":60568,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":186,"spinlock_wait_cycles":1832704,"thread_start_us":1055303,"threads_started":2,"wal-append.queue_time_us":451776}
I20260709 13:06:44.109356 16535 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:44.110217 19776 consensus_queue.cc:1048] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:44.112207 19776 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:44.112226 20188 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:44.112985 19776 raft_consensus.cc:2749] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:44.113401 20188 raft_consensus.cc:697] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 12 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:44.114508 16535 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:44.115175 20188 consensus_queue.cc:237] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:44.115830 20203 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:44.116279 16535 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:44.116683 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:44.116559 20203 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:44.107659 16006 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 12.
I20260709 13:06:44.117551 15861 raft_consensus.cc:2410] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:06:44.117744 16535 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:44.118654 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:44.118750 20203 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:44.119426 16008 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
I20260709 13:06:44.120891 16008 raft_consensus.cc:1240] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:06:44.122263 16008 raft_consensus.cc:1240] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 7. Current term is 8. Ops: []
I20260709 13:06:44.126111 16003 raft_consensus.cc:1240] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:44.127367 16003 raft_consensus.cc:1240] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:44.130424 16565 raft_consensus.cc:1240] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 11. Current term is 13. Ops: []
I20260709 13:06:44.131361 20179 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:44.067130 20300 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:44.136036 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:44.138085 15827 leader_election.cc:400] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 9. Current term is 10.
I20260709 13:06:44.138406 15827 leader_election.cc:403] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:06:44.138736 16009 raft_consensus.cc:3055] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:44.139127 16009 raft_consensus.cc:740] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:44.139379 20183 consensus_queue.cc:1059] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:06:44.140173 20300 raft_consensus.cc:2749] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:44.140690 20183 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 9. Current term is 10.
W20260709 13:06:44.142248 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 82557.7us
W20260709 13:06:44.142364 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 65502.1us
I20260709 13:06:44.115935 20357 raft_consensus.cc:2749] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:44.123594 16008 raft_consensus.cc:1275] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:06:44.143924 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:44.099437 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:44.118873 15861 raft_consensus.cc:1240] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:44.102638 16568 raft_consensus.cc:606] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEADER]: Received request to transfer leadership to TS 8430db80a39d4868bcf7641c16de35ea
W20260709 13:06:44.152396 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:44.149385 16007 raft_consensus.cc:2410] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:06:44.153939 15929 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:44.154671 20368 raft_consensus.cc:2749] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:44.155980 20133 raft_consensus.cc:2749] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:44.155879 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:44.145022 16533 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:44.165284 15929 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:06:44.166555 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:44.087200 15930 rpcz_store.cc:276] 0709 13:06:42.695598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:42.696700 (+  1102us) service_pool.cc:224] Handling call
0709 13:06:42.697303 (+   603us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:42.698840 (+  1537us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:42.698874 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:42.699411 (+   537us) log.cc:844] Serialized 26 byte log entry
0709 13:06:43.029288 (+329877us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:43.029329 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:43.029396 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:43.915662 (+886266us) spinlock_profiling.cc:243] Waited 925 us on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:06:44.071601 (+155939us) raft_consensus.cc:1685] finished
0709 13:06:44.071622 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:44.075920 (+  4298us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":799,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":74,"spinlock_wait_cycles":2128128,"thread_start_us":328014,"threads_started":1,"wal-append.queue_time_us":633925}
I20260709 13:06:44.166918 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:44.174578 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:44.175787 16010 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
I20260709 13:06:44.157060 20183 raft_consensus.cc:3055] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:44.174441 16005 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
I20260709 13:06:44.177215 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:44.177836 16566 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
W20260709 13:06:44.185153 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:44.186447 16005 raft_consensus.cc:1275] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:06:44.191736 15859 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 13.
I20260709 13:06:44.193332 15859 raft_consensus.cc:1275] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:06:44.202508 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:44.208674 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:44.209340 16567 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:44.186002 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:44.212744 16564 raft_consensus.cc:1240] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 9. Current term is 10. Ops: []
W20260709 13:06:44.215539 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 97879.1us
I20260709 13:06:44.217763 16005 raft_consensus.cc:1240] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:44.217407 16009 consensus_queue.cc:260] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:44.217399 16007 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:44.219205 16003 raft_consensus.cc:1240] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:44.218864 20369 raft_consensus.cc:3055] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:44.225584 20369 raft_consensus.cc:740] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:44.225687 15974 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:44.226735 20369 consensus_queue.cc:260] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:44.225595 16562 raft_consensus.cc:2410] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:06:44.225987 16009 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:44.178483 20183 raft_consensus.cc:740] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:44.232117 16008 raft_consensus.cc:1275] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:06:44.187335 20357 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:44.244426 16002 raft_consensus.cc:1275] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:06:44.228664 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:44.248077 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:44.250373 16566 raft_consensus.cc:2410] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:06:44.254154 15856 raft_consensus.cc:1240] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 12 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 11. Current term is 12. Ops: []
W20260709 13:06:44.253351 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:44.254068 15853 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:44.243634 20007 raft_consensus.cc:2749] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:44.261653 15860 raft_consensus.cc:1275] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:06:44.263797 20357 raft_consensus.cc:697] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:44.268994 20357 consensus_queue.cc:237] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:44.270951 15757 catalog_manager.cc:5697] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 6 to 11. New cstate: current_term: 11 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:44.275573 15757 catalog_manager.cc:5697] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 3 to 8, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 8 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:06:44.277529 15757 catalog_manager.cc:5697] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 4 to 9. New cstate: current_term: 9 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:44.215516 20300 consensus_queue.cc:1048] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:06:44.280680 16009 raft_consensus.cc:1275] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:06:44.281471 16009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 810) took 1103 ms. Trace:
I20260709 13:06:44.281903 16009 rpcz_store.cc:276] 0709 13:06:43.177808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:44.138257 (+960449us) service_pool.cc:224] Handling call
0709 13:06:44.138581 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:44.281140 (+142559us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:44.281363 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:44.287299 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:44.288228 20133 consensus_queue.cc:1059] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:44.288967 16010 raft_consensus.cc:3055] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:06:44.289271 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:44.289063 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:44.289556 16010 raft_consensus.cc:740] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:44.290589 16010 consensus_queue.cc:260] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:44.290863 19776 consensus_queue.cc:1048] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:44.279539 15757 catalog_manager.cc:5697] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:44.295162 16535 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:44.287151 16004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 799) took 1122 ms. Trace:
I20260709 13:06:44.301257 20369 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:44.304332 20183 consensus_queue.cc:260] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:44.305964 20183 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:44.301270 20133 consensus_queue.cc:1059] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 13:06:44.314970 16005 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:06:44.315793 20133 raft_consensus.cc:3055] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:44.316155 20133 raft_consensus.cc:740] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:44.317150 20133 consensus_queue.cc:260] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:44.296255 16004 rpcz_store.cc:276] 0709 13:06:43.164961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:44.136987 (+972026us) service_pool.cc:224] Handling call
0709 13:06:44.176303 (+ 39316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:44.213507 (+ 37204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:44.213532 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:44.219327 (+  5795us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:44.232807 (+ 13480us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:44.232863 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:44.286976 (+ 54113us) spinlock_profiling.cc:243] Waited 39 ms on lock 0x7b540025a8d0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:44.287090 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":89600384}
I20260709 13:06:44.318666 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:06:44.328500 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 72307.7us
I20260709 13:06:44.329186 16564 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:44.331885 16011 raft_consensus.cc:2410] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 10 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 9.
I20260709 13:06:44.337694 16567 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
W20260709 13:06:44.339545 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:44.342806 20437 raft_consensus.cc:3055] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:44.344200 16533 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:44.344480 15853 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:06:44.334331 15825 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:06:44.343225 20440 log.cc:927] Time spent T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:06:44.249922 20203 consensus_queue.cc:1059] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.078s
I20260709 13:06:44.353935 15929 raft_consensus.cc:1275] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:06:44.355773 20368 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:44.356097 19974 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:44.356590 20155 raft_consensus.cc:2749] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:44.356683 19974 raft_consensus.cc:697] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:44.348318 19723 consensus_queue.cc:1059] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 13:06:44.358274 19723 consensus_queue.cc:1243] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Peer 31882606e87447fb9a36bbf625129fe8 log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:06:44.360165 15757 catalog_manager.cc:5697] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 5 to 7, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 7 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:06:44.350090 16010 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:44.343288 20437 raft_consensus.cc:740] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:44.351408 20007 consensus_queue.cc:1059] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:06:44.357839 19974 consensus_queue.cc:237] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:44.350603 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:44.363055 16564 raft_consensus.cc:2410] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:06:44.351187 16007 raft_consensus.cc:1275] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:06:44.369310 19660 consensus_queue.cc:1048] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:06:44.373123 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:44.374716 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:44.376745 16567 raft_consensus.cc:2410] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:06:44.376197 20437 consensus_queue.cc:260] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:44.377976 20437 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:44.259198 20431 consensus_queue.cc:1059] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.091s
I20260709 13:06:44.384990 19813 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:44.386461 20133 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:44.387595 15974 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:44.389304 16565 raft_consensus.cc:1240] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 8. Current term is 9. Ops: []
W20260709 13:06:44.419241 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 276188us
I20260709 13:06:44.404177 20007 raft_consensus.cc:3055] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:44.409260 20431 consensus_queue.cc:1243] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer 31882606e87447fb9a36bbf625129fe8 log is divergent from this leader: its last log entry 10.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:06:44.412035 20299 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:44.413985 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 154578us
W20260709 13:06:44.446689 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:44.450264 20299 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:44.452818 20299 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:44.479525 16006 raft_consensus.cc:1275] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 9 index: 10. (term mismatch)
W20260709 13:06:44.480458 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:44.453430 20299 raft_consensus.cc:2764] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:06:44.480652 16006 pending_rounds.cc:85] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting all ops after (but not including) 9
I20260709 13:06:44.485047 16006 pending_rounds.cc:107] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 8.10
I20260709 13:06:44.485404 16006 raft_consensus.cc:2889] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:44.443140 20007 raft_consensus.cc:740] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:44.483978 20349 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.489703 16005 raft_consensus.cc:1275] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:06:44.496730 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:44.498831 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:44.499603 16567 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
I20260709 13:06:44.510414 15900 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:44.512912 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:06:44.513090 20349 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:44.520248 16006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 787) took 1374 ms. Trace:
I20260709 13:06:44.520810 16006 rpcz_store.cc:276] 0709 13:06:43.157205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:44.117881 (+960676us) service_pool.cc:224] Handling call
0709 13:06:44.448312 (+330431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:44.493543 (+ 45231us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:44.520015 (+ 26472us) spinlock_profiling.cc:243] Waited 330 ms on lock 0x7b540010efd0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:44.520179 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":759003392}
W20260709 13:06:44.531934 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 140843us
W20260709 13:06:44.535933 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 52898.2us
W20260709 13:06:44.549047 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 161582us
I20260709 13:06:44.551980 16535 leader_election.cc:304] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:44.555035 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:44.556591 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:44.558171 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:44.562937 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:44.563022 15974 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:44.563776 16011 raft_consensus.cc:1240] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:44.563712 16569 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:06:44.569052 15855 raft_consensus.cc:1275] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:06:44.571610 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:44.574924 16568 raft_consensus.cc:1240] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:44.570329 19723 raft_consensus.cc:2749] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:44.580470 20260 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.581068 20260 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:44.581548 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:44.582101 16005 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:44.584157 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:44.584940 16006 raft_consensus.cc:2410] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:06:44.579285 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:44.583509 16535 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:44.573987 20349 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:44.572194 20183 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.593021 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:44.595379 16005 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:06:44.583501 20260 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:44.605865 15976 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:44.611442 15976 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:44.612378 20300 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:44.612738 20300 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:44.613109 20300 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:44.614755 20179 consensus_queue.cc:1059] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:44.615483 20179 consensus_queue.cc:1243] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer eb1e0493af1e46dbb6dec7e25525bab5 log is divergent from this leader: its last log entry 13.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:06:44.615960 20179 consensus_queue.cc:1059] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:44.616737 20179 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:44.617141 20179 raft_consensus.cc:697] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:44.618014 20179 consensus_queue.cc:237] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:44.618688 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:44.627642 15857 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
W20260709 13:06:44.629199 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 70311us
I20260709 13:06:44.604552 20431 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:44.608888 19723 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:06:44.637765 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 67562.1us
I20260709 13:06:44.593763 20341 consensus_queue.cc:1059] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.136s
I20260709 13:06:44.644157 15900 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:06:44.645364 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 61936.9us
I20260709 13:06:44.649511 15974 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:44.625286 20183 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:44.622150 20007 consensus_queue.cc:260] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:44.666675 20300 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:44.671865 20446 log.cc:927] Time spent T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.196s	user 0.002s	sys 0.000s
W20260709 13:06:44.672226 20449 log.cc:927] Time spent T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 13:06:44.640208 20360 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:44.667095 20299 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:44.626998 20356 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.022s
W20260709 13:06:44.699206 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 117366us
I20260709 13:06:44.681160 20071 raft_consensus.cc:3055] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:44.703358 16564 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:06:44.680847 20347 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.721146 16565 raft_consensus.cc:1240] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:44.704569 20071 raft_consensus.cc:740] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:44.724839 20071 consensus_queue.cc:260] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:44.725318 20341 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:44.725721 20341 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:44.726082 20341 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:44.726142 20071 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:44.728135 20007 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:44.702826 20299 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:06:44.701619 15827 connection.cc:256] RPC call timeout handler was delayed by -1.3588626340000474s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2284s for a response
W20260709 13:06:44.734215 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 54067.1us
I20260709 13:06:44.705214 20260 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.711290 19723 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.745684 16535 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
W20260709 13:06:44.746677 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 104861us
I20260709 13:06:44.736873 20299 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:44.723592 20068 raft_consensus.cc:3055] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:44.738941 20203 consensus_queue.cc:1059] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 13:06:44.749119 20347 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:44.754545 20068 raft_consensus.cc:740] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:06:44.750416 20453 log.cc:927] Time spent T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 13:06:44.752642 20133 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:44.757413 20133 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:44.757798 20133 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:06:44.753614 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:44.760382 20183 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:44.761927 20347 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:44.763880 20458 log.cc:927] Time spent T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
W20260709 13:06:44.765223 20460 log.cc:927] Time spent T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 13:06:44.766671 20454 log.cc:927] Time spent T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
W20260709 13:06:44.766768 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 104208us
I20260709 13:06:44.768481 20356 consensus_queue.cc:1048] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:44.771560 20364 consensus_queue.cc:1048] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:44.772238 19723 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:44.774380 19723 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:44.776270 20456 log.cc:927] Time spent T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 13:06:44.777096 20450 log.cc:927] Time spent T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 13:06:44.778398 20198 raft_consensus.cc:3055] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:44.778848 20198 raft_consensus.cc:740] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:44.779088 15902 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:44.779362 15827 connection.cc:256] RPC call timeout handler was delayed by -1.0329347010000447s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1706s for a response
W20260709 13:06:44.784636 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 135205us
I20260709 13:06:44.785696 15974 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:44.741871 20260 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:44.783162 19776 consensus_queue.cc:1048] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:06:44.807032 19723 consensus_queue.cc:1048] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:44.802966 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:44.807631 19723 consensus_queue.cc:1243] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer 31882606e87447fb9a36bbf625129fe8 log is divergent from this leader: its last log entry 10.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:06:44.810377 16010 raft_consensus.cc:1275] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:06:44.810771 16010 pending_rounds.cc:85] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting all ops after (but not including) 5
I20260709 13:06:44.811060 16010 pending_rounds.cc:107] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 10.6
W20260709 13:06:44.810667 20462 log.cc:927] Time spent T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.075s	user 0.000s	sys 0.002s
I20260709 13:06:44.811368 16010 raft_consensus.cc:2889] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:44.811643 20363 consensus_queue.cc:1048] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:06:44.812033 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:44.813705 15929 raft_consensus.cc:1275] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:06:44.815140 20260 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:44.815469 20357 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:44.810702 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:44.816303 20300 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:44.782704 20360 raft_consensus.cc:697] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 12 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:44.818706 16533 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:44.786429 20133 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:44.836464 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 57468.7us
I20260709 13:06:44.837471 15827 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:44.838685 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:44.839854 15900 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:44.850324 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:44.852473 16005 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
I20260709 13:06:44.853772 20188 raft_consensus.cc:2804] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:44.853967 20418 raft_consensus.cc:993] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5: : Instructing follower 31882606e87447fb9a36bbf625129fe8 to start an election
I20260709 13:06:44.854247 20188 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:44.854563 20418 raft_consensus.cc:1081] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEADER]: Signalling peer 31882606e87447fb9a36bbf625129fe8 to start an election
I20260709 13:06:44.854740 20188 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:44.860586 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:44.803607 20179 raft_consensus.cc:2804] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:06:44.860805 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 56445.4us
I20260709 13:06:44.867094 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:44.867650 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:44.868018 20300 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.868618 20300 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:44.869732 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:44.870497 16005 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 10: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:44.870620 20300 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:44.840098 20347 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.874635 16010 raft_consensus.cc:1275] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:06:44.875015 16010 pending_rounds.cc:85] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting all ops after (but not including) 9
I20260709 13:06:44.875237 16010 pending_rounds.cc:107] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:06:44.875474 16010 raft_consensus.cc:2889] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:44.833841 20431 raft_consensus.cc:2749] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:44.781831 20198 consensus_queue.cc:260] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:06:44.880471 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:44.827131 19974 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.882263 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:44.883455 16007 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:44.851328 20007 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:44.803866 20466 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:44.855782 19723 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:44.883225 16535 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:44.889127 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:44.841531 19776 consensus_queue.cc:1048] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:06:44.886695 16003 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:44.889822 16564 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 8.
I20260709 13:06:44.889680 16004 raft_consensus.cc:1275] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:06:44.893993 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 108437us
I20260709 13:06:44.895283 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:44.898023 16011 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
I20260709 13:06:44.817180 20364 consensus_queue.cc:1048] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 13:06:44.901300 15976 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:44.908521 15827 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:44.912254 15854 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:58214
I20260709 13:06:44.912380 20337 consensus_queue.cc:1048] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 13:06:44.913025 15854 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:44.913393 15854 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:44.915441 15856 pending_rounds.cc:85] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8: Aborting all ops after (but not including) 10
I20260709 13:06:44.915827 15856 pending_rounds.cc:107] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8: Aborting uncommitted NO_OP operation due to leader change: 10.11
I20260709 13:06:44.916100 15856 raft_consensus.cc:2889] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:44.916767 15902 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:44.889530 20179 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:44.890970 20469 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:44.897724 19723 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:44.894358 20357 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:44.902433 20369 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:44.947607 15855 raft_consensus.cc:1275] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:06:44.945281 20088 raft_consensus.cc:2804] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:44.948330 20088 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:44.948802 20088 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:44.949716 20464 consensus_queue.cc:1048] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 13:06:44.951099 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:44.951753 15936 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:44.952378 20133 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:44.898929 19974 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:44.953502 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:06:44.953549 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:44.886108 20347 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:44.929282 20179 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:44.945560 19723 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:44.947757 20470 log.cc:927] Time spent T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:06:44.930593 20163 raft_consensus.cc:2804] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:44.941085 20356 raft_consensus.cc:2749] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:44.946332 20198 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:44.932103 20437 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.969519 20469 raft_consensus.cc:697] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 13 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:06:44.988651 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:44.954888 20369 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:44.993059 20369 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:44.996093 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 79460.1us
I20260709 13:06:44.888832 20360 consensus_queue.cc:237] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:44.943092 20007 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:44.986225 20163 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:45.018492 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 120010us
I20260709 13:06:44.971746 20133 consensus_queue.cc:1048] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:06:45.021160 16007 raft_consensus.cc:1275] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 13:06:45.024598 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 46638.6us
I20260709 13:06:44.988334 20476 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:06:45.029021 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 136973us
I20260709 13:06:45.026564 16535 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:45.027537 20476 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:45.028414 16533 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:45.034718 16006 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:45.020110 20163 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:45.048539 16006 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
I20260709 13:06:44.980984 20357 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:45.024374 20368 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:45.064255 19813 raft_consensus.cc:2804] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:45.075047 20347 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:45.075567 19813 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:45.076058 19813 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:45.077795 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:45.078711 15935 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:06:45.080317 20341 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:45.018590 20437 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:45.088382 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 53966.3us
I20260709 13:06:45.092458 15854 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:45.096163 15857 raft_consensus.cc:1240] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:45.096341 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:45.098040 15854 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:45.104664 15902 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:45.082590 20088 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:45.091218 20481 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:45.111414 20088 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:45.112040 19974 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:45.101627 20469 consensus_queue.cc:237] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:45.112833 15827 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:45.115078 15827 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:45.116413 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:45.116721 20437 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:45.087277 20368 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:45.118224 20431 consensus_queue.cc:1059] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:45.118988 20368 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:45.106642 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:45.120009 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:45.121572 20459 log.cc:927] Time spent T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.444s	user 0.002s	sys 0.000s
I20260709 13:06:45.122241 20473 raft_consensus.cc:2749] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:45.122391 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:45.122710 15757 catalog_manager.cc:5697] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:06:45.123665 15902 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:45.123977 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:45.124513 20473 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:45.124883 20473 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:45.125180 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:45.125403 19974 raft_consensus.cc:2804] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:45.125850 19974 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:45.126034 15902 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:45.125229 20473 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:45.125154 15757 catalog_manager.cc:5697] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:06:45.127853 20341 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:45.126302 19974 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:45.136627 16568 raft_consensus.cc:1275] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:06:45.116773 20357 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:45.138545 20068 consensus_queue.cc:260] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:45.139845 20068 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:45.140242 20007 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:45.147001 15902 leader_election.cc:400] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 election: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 9. Current term is 10.
I20260709 13:06:45.134582 20361 raft_consensus.cc:2764] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
I20260709 13:06:45.119010 20431 consensus_queue.cc:1243] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer eb1e0493af1e46dbb6dec7e25525bab5 log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:06:45.150056 20071 raft_consensus.cc:3055] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:45.150405 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:45.151578 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:45.151916 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:45.152297 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:45.152675 15854 raft_consensus.cc:2393] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 9: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:45.153007 15853 raft_consensus.cc:2393] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 11: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:45.150449 20071 raft_consensus.cc:740] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:45.153976 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:45.154800 20071 consensus_queue.cc:260] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:45.155586 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:45.156136 20071 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:45.155254 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:45.157320 15902 leader_election.cc:400] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 11. Current term is 12.
I20260709 13:06:45.158032 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:45.158315 15902 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:45.158756 15858 raft_consensus.cc:2393] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 10: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:45.167176 16533 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:45.169585 20361 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:45.169763 20437 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:45.170029 20361 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:45.170511 20361 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:45.171085 16533 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:45.171473 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:45.171941 20437 raft_consensus.cc:2749] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:45.172230 16568 raft_consensus.cc:2393] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 11: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:45.168331 15825 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:45.173142 20349 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:06:45.173436 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 153845us
I20260709 13:06:45.173568 20349 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:45.173941 20349 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:45.174167 16535 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:45.173141 20431 consensus_queue.cc:1048] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:06:45.175943 20418 raft_consensus.cc:2749] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:45.182857 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:45.184499 20476 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:45.185097 20361 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:45.196178 20368 consensus_queue.cc:1048] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:45.187165 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:45.187578 16533 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:45.198832 19776 raft_consensus.cc:2749] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:45.199254 20476 consensus_queue.cc:1048] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:06:45.178467 20088 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:45.135309 20188 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:45.201155 20088 raft_consensus.cc:697] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
W20260709 13:06:45.201517 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:45.202332 20088 consensus_queue.cc:237] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:45.187767 20155 consensus_queue.cc:1048] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:06:45.192216 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 77255.3us
W20260709 13:06:45.194211 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:45.194689 15902 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:45.199577 20361 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:45.190325 19813 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:45.123359 20299 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:45.208698 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:45.216126 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:45.155236 20473 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:45.222183 15858 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
I20260709 13:06:45.224351 15860 raft_consensus.cc:1240] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:45.224623 16535 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:45.231118 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:45.235034 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:45.194912 19723 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:45.246464 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:45.247144 15862 raft_consensus.cc:2393] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 13: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:45.254694 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:45.259027 20299 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:45.259652 19776 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.260182 19776 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:45.269101 19813 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:45.275218 20473 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:45.277875 15854 raft_consensus.cc:1240] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 9. Current term is 10. Ops: [9.9-9.9]
W20260709 13:06:45.280115 15744 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 83a1b79cdcf046c78453695c798a14a5 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:06:45.280529 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:06:45.281239 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 45872.5us
W20260709 13:06:45.282188 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 87603us
I20260709 13:06:45.283598 15902 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:45.286813 15902 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:45.290208 20188 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:45.296159 15857 raft_consensus.cc:1240] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 12. Current term is 13. Ops: [7.6-12.8]
I20260709 13:06:45.297123 20349 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:45.230310 20431 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:45.298791 20431 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:45.299883 15858 raft_consensus.cc:1240] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:45.300894 20431 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:45.230388 20356 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:45.302780 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:45.318869 16564 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 10.
I20260709 13:06:45.320513 15976 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:45.282208 15756 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58434 (request call id 343) took 20731 ms (client timeout 15000 ms). Trace:
I20260709 13:06:45.329293 15853 raft_consensus.cc:1217] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Deduplicated request from leader. Original: 4.4->[6.5-9.6]   Dedup: 6.5->[9.6-9.6]
I20260709 13:06:45.330109 15853 raft_consensus.cc:1240] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 9. Current term is 10. Ops: [6.5-9.6]
I20260709 13:06:45.253984 15974 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:45.246160 20357 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:06:45.334729 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 47987.1us
I20260709 13:06:45.335549 15902 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
W20260709 13:06:45.342662 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:45.346766 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:45.356370 15856 raft_consensus.cc:1217] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Deduplicated request from leader. Original: 9.10->[11.11-11.11]   Dedup: 11.11->[]
I20260709 13:06:45.356964 15856 raft_consensus.cc:1240] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 11. Current term is 13. Ops: [11.11-11.11]
W20260709 13:06:45.364357 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:45.337528 20364 raft_consensus.cc:2804] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:45.332795 20361 raft_consensus.cc:2804] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:45.370740 20474 consensus_queue.cc:1048] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:06:45.365403 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:45.381461 15930 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:45.327422 20198 raft_consensus.cc:2804] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:45.383180 20198 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:06:45.375056 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 150493us
I20260709 13:06:45.337531 20431 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:45.257370 20368 consensus_queue.cc:1059] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.102s
I20260709 13:06:45.389569 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:45.346130 20357 raft_consensus.cc:697] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:45.366333 20299 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:45.399228 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:06:45.400429 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 142120us
I20260709 13:06:45.405025 16006 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:45.346503 20341 consensus_queue.cc:1059] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 13:06:45.371698 20364 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:45.383621 20198 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:45.394804 20473 consensus_queue.cc:1048] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:06:45.365830 20304 consensus_queue.cc:1059] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:06:45.377590 20361 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:06:45.409937 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 128002us
I20260709 13:06:45.352511 20363 consensus_queue.cc:1048] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
W20260709 13:06:45.327555 15756 rpcz_store.cc:269] 0709 13:06:24.551213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.564001 (+ 12788us) service_pool.cc:224] Handling call
0709 13:06:24.722861 (+158860us) spinlock_profiling.cc:243] Waited 22.8 ms on lock 0x7b5400030890. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e72840b2 00007f73f01fda89 00007f73f0195da6 00007f73f0167088 00007f73f026537e 00007f73ee7262a4 00007f73ee726232 00007f73ee726161 00007f73ee7260dc
0709 13:06:45.255031 (+20532170us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:35.983990 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:35.984589 (+   599us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:35.984636 (+    47us) write_op.cc:435] Acquired schema lock
0709 13:06:35.984655 (+    19us) tablet.cc:662] Decoding operations
0709 13:06:35.986327 (+  1672us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:35.986403 (+    76us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:35.986431 (+    28us) tablet.cc:685] Acquiring locks for 21 operations
0709 13:06:36.146399 (+159968us) tablet.cc:701] Row locks acquired
0709 13:06:36.146459 (+    60us) write_op.cc:260] PREPARE: finished
0709 13:06:36.146695 (+   236us) write_op.cc:270] Start()
0709 13:06:36.146853 (+   158us) write_op.cc:276] Timestamp: P: 1783602396146607 usec, L: 0
0709 13:06:36.146886 (+    33us) op_driver.cc:348] REPLICATION: starting
0709 13:06:36.155686 (+  8800us) log.cc:844] Serialized 8231 byte log entry
0709 13:06:36.490002 (+334316us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e729e0ac 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b
0709 13:06:37.204165 (+714163us) op_driver.cc:464] REPLICATION: finished
0709 13:06:37.345756 (+141591us) write_op.cc:301] APPLY: starting
0709 13:06:37.359038 (+ 13282us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:37.397092 (+ 38054us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:37.397135 (+    43us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:37.441155 (+ 44020us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:37.441586 (+   431us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=42
0709 13:06:37.441644 (+    58us) write_op.cc:312] APPLY: finished
0709 13:06:37.442196 (+   552us) log.cc:844] Serialized 145 byte log entry
0709 13:06:37.442426 (+   230us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:37.442644 (+   218us) write_op.cc:454] Released schema lock
0709 13:06:37.442727 (+    83us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:37.443223 (+   496us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:37.443265 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":570581,"reported_tablets":46,"spinlock_wait_cycles":52486400,"child_traces":[["op",{"apply.queue_time_us":121802,"apply.run_cpu_time_us":6383,"apply.run_wall_time_us":107036,"num_ops":21,"prepare.queue_time_us":370482,"prepare.run_cpu_time_us":285490,"prepare.run_wall_time_us":1043350,"raft.queue_time_us":189336,"raft.run_cpu_time_us":23568,"raft.run_wall_time_us":107404,"replication_time_us":1032019,"spinlock_wait_cycles":26921216,"thread_start_us":1087740,"threads_started":4,"wal-append.queue_time_us":793363,"wal-append.run_cpu_time_us":48823,"wal-append.run_wall_time_us":1523125}]]}
I20260709 13:06:45.327152 20179 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:45.392503 20431 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:45.437840 20431 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:45.423645 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:45.440560 15929 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:45.441071 16005 raft_consensus.cc:1240] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 9. Current term is 10. Ops: []
W20260709 13:06:45.409014 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 155137us
I20260709 13:06:45.434901 15902 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:45.445513 16006 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
I20260709 13:06:45.449617 19660 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:45.401429 20474 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:06:45.421432 20364 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:45.410182 20418 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:45.454950 15974 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:45.440557 20361 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:45.456101 20437 raft_consensus.cc:3055] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:45.480917 15827 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:45.481081 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 70272.5us
I20260709 13:06:45.456547 20476 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.456238 20360 raft_consensus.cc:2749] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:45.457890 20474 consensus_queue.cc:1243] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer 85e997ba5003439781ec4f1f8db0c3ee log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:06:45.455133 20349 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:45.483311 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 77880.8us
I20260709 13:06:45.475451 20300 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:45.480146 20437 raft_consensus.cc:740] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:45.495651 16533 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:45.486162 19660 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:45.504686 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:45.474725 20369 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:45.507373 15974 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:45.509320 15974 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:45.495101 19723 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:45.512598 15929 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
I20260709 13:06:45.516630 15930 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 9.
I20260709 13:06:45.516886 20499 raft_consensus.cc:2749] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:45.519542 20071 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:45.520254 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 61384.7us
I20260709 13:06:45.523115 19776 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:45.499366 19974 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:45.493782 20494 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:45.524718 20494 raft_consensus.cc:697] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 9 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:45.525362 19660 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:45.525700 20494 consensus_queue.cc:237] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:45.518401 20300 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:45.523643 20347 consensus_queue.cc:1048] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:45.526221 19974 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:45.528870 20488 log.cc:927] Time spent T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
W20260709 13:06:45.530786 20492 log.cc:927] Time spent T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:06:45.530797 15860 raft_consensus.cc:1240] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 10. Current term is 11. Ops: []
W20260709 13:06:45.531916 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:45.537429 15974 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:45.557819 15974 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:45.560513 20356 consensus_queue.cc:1048] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:45.486851 20476 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:45.559656 15974 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:45.478363 20418 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:45.481740 20357 consensus_queue.cc:237] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:45.574656 20007 raft_consensus.cc:2749] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:45.580770 20504 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:45.580199 15827 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:45.581277 20007 consensus_queue.cc:1059] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:45.581382 20504 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:45.582517 15827 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:45.583489 20347 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:45.504395 20437 consensus_queue.cc:260] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:45.561173 19974 consensus_queue.cc:1048] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 13:06:45.563230 20299 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:45.566716 20068 raft_consensus.cc:2749] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:06:45.584179 20347 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:45.608278 20068 raft_consensus.cc:3055] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:45.583642 20071 raft_consensus.cc:2749] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:45.583729 20504 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:45.590615 20299 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:45.611826 20179 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:45.615562 20418 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:45.616645 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:45.616917 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:45.635423 20369 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:45.618397 16535 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:45.638293 20418 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:45.638818 20418 raft_consensus.cc:697] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:45.641072 20473 raft_consensus.cc:3055] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:45.641490 20473 raft_consensus.cc:740] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:45.641376 20418 consensus_queue.cc:237] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:45.645190 20163 consensus_meta.cc:287] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.573s	user 0.008s	sys 0.001s
I20260709 13:06:45.645641 20163 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:45.619336 20437 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:45.576756 20467 consensus_queue.cc:1059] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 13:06:45.587653 19776 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.587918 20499 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.670212 20499 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:45.585109 20188 consensus_queue.cc:1048] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:06:45.620792 20476 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:45.632052 20474 consensus_queue.cc:1048] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:06:45.634064 20299 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:45.617972 16010 raft_consensus.cc:1275] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:06:45.647768 16008 raft_consensus.cc:2393] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 9: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:45.651158 15900 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:45.652062 20163 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:45.628576 20068 raft_consensus.cc:740] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:06:45.673324 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 97724.4us
I20260709 13:06:45.682482 20068 consensus_queue.cc:260] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:45.683583 20068 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:45.625523 20360 consensus_queue.cc:1048] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:06:45.694098 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:45.694401 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:45.694847 16567 raft_consensus.cc:2393] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 9: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:45.695055 16570 raft_consensus.cc:2393] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 12: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:45.695829 16565 raft_consensus.cc:1275] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:06:45.696111 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:06:45.696898 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:45.697847 20304 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:45.698329 20304 raft_consensus.cc:697] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:45.698925 20364 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:45.691493 20361 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:45.700531 20469 raft_consensus.cc:3055] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:45.700915 20469 raft_consensus.cc:740] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:45.696168 16562 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 LEARNER]: Advancing to term 13
I20260709 13:06:45.701679 20469 consensus_queue.cc:260] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:45.702953 20469 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:45.704382 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:45.707124 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:45.707934 15856 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 10: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:45.708870 20431 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:45.710597 16568 raft_consensus.cc:1240] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:45.711037 15825 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:45.711216 20431 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:45.682531 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:45.713555 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:45.717823 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:45.719043 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:45.719502 15853 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:06:45.724448 15853 raft_consensus.cc:1240] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:45.725835 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:45.726107 15933 raft_consensus.cc:1240] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:45.727694 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:45.728264 16562 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEARNER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:06:45.727824 16564 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
I20260709 13:06:45.729115 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:45.729534 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:45.729878 15929 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 10: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:45.730095 15855 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:45.732336 15853 raft_consensus.cc:1240] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:45.680868 20179 raft_consensus.cc:3055] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:45.738036 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:45.730556 20088 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:45.739306 20088 raft_consensus.cc:697] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 10 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:45.739965 20473 consensus_queue.cc:260] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:45.740232 20088 consensus_queue.cc:237] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:45.741256 20473 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:45.755402 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:45.721123 20499 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:45.699151 20304 consensus_queue.cc:237] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:45.680302 19776 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:45.735129 20179 raft_consensus.cc:740] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:45.737139 20198 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:45.774827 20007 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.800693 15929 raft_consensus.cc:1240] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:45.810269 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:45.811149 15860 raft_consensus.cc:2368] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 12. Current term is 13.
I20260709 13:06:45.812909 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:45.813930 16562 raft_consensus.cc:2393] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 9: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:45.820358 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:45.821305 16564 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:45.841986 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:45.842661 16009 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:06:45.844485 15974 leader_election.cc:400] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 12. Current term is 13.
I20260709 13:06:45.844919 15974 leader_election.cc:403] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:06:45.848934 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:45.855090 16003 raft_consensus.cc:1275] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:06:45.848425 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:45.858877 15853 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 12.
I20260709 13:06:45.865828 16008 raft_consensus.cc:1275] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:06:45.868216 16566 raft_consensus.cc:1275] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:06:45.873346 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:45.875386 16005 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:45.872998 16564 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
I20260709 13:06:45.877276 16568 raft_consensus.cc:1240] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:45.876173 16533 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:45.879280 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:45.880537 16004 raft_consensus.cc:1275] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:06:45.886996 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:45.888562 16008 raft_consensus.cc:2376] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10. Re-sending same reply.
I20260709 13:06:45.889648 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:45.890331 16008 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:45.894070 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:45.894934 16564 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:06:45.897259 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:45.897724 16563 raft_consensus.cc:1275] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:06:45.848837 20260 consensus_queue.cc:1059] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:06:45.899660 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:06:45.900050 20507 log.cc:927] Time spent T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 13:06:45.900902 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:45.901077 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:45.901495 16002 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
I20260709 13:06:45.902324 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:45.902443 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:45.902951 16003 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:45.903326 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:45.905586 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:45.839274 20509 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:45.905926 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:45.906195 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:45.906370 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:45.906435 20155 raft_consensus.cc:2804] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:45.808465 20179 consensus_queue.cc:260] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:45.908023 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:45.909775 20364 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:45.910634 16565 raft_consensus.cc:1275] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:06:45.911144 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:45.912266 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:45.946655 16007 raft_consensus.cc:2410] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 11 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:06:45.845837 20357 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.950107 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:45.950549 15933 raft_consensus.cc:1240] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:45.919950 16002 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:45.920928 16005 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:06:45.951218 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:45.880936 16004 pending_rounds.cc:85] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee: Aborting all ops after (but not including) 9
I20260709 13:06:45.952024 16004 pending_rounds.cc:107] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 13:06:45.952394 16004 raft_consensus.cc:2889] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:45.929100 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:45.808252 20007 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:45.861011 20431 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:45.931821 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:45.932451 15935 raft_consensus.cc:1240] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:45.933911 15930 raft_consensus.cc:1240] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:45.935160 15931 raft_consensus.cc:2410] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:06:45.937561 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:45.959189 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:45.959573 16568 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
I20260709 13:06:45.959913 16005 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:45.938977 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:45.960575 16564 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:06:45.941836 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:45.941968 19776 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:45.942678 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:45.962110 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:45.867214 20513 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.947191 15934 raft_consensus.cc:2393] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 11: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:45.839658 20369 consensus_queue.cc:1059] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.086s
I20260709 13:06:45.848495 20368 consensus_queue.cc:1048] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.091s
I20260709 13:06:45.949671 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:45.969290 16007 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:06:45.955819 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 75174.7us
I20260709 13:06:45.956274 15902 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:45.957413 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:45.938395 20260 raft_consensus.cc:3055] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:45.963573 15974 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:45.944324 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 263502us
I20260709 13:06:46.011405 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:45.964890 16003 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
I20260709 13:06:45.962850 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:46.010637 16566 raft_consensus.cc:1217] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Deduplicated request from leader. Original: 10.10->[11.11-11.11]   Dedup: 11.11->[]
I20260709 13:06:45.966043 15928 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 10: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:45.967217 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:45.968185 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:46.013677 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:46.014276 16004 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
I20260709 13:06:46.015053 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:46.016070 15976 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:46.017431 15976 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:46.019879 20516 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:45.972429 20260 raft_consensus.cc:740] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:46.021297 20299 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:46.021755 20260 consensus_queue.cc:260] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:46.022945 20260 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:46.023232 16006 raft_consensus.cc:2393] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 12: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:45.930507 20155 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:46.023880 20155 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:46.025405 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:46.025974 16006 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:45.966866 20509 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:46.028108 20198 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:45.969031 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:46.030043 20509 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:46.030622 16011 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 8.
I20260709 13:06:46.031595 20361 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:46.031937 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:46.032235 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:46.033077 16011 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
I20260709 13:06:46.034032 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:46.035184 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:45.972393 20513 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:06:46.041718 20513 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:46.042256 20490 consensus_queue.cc:1059] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:46.043713 16004 raft_consensus.cc:2410] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:06:45.956962 20515 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.044476 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:46.044409 20515 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:46.046545 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:46.047194 15928 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
I20260709 13:06:46.009168 20088 consensus_queue.cc:1059] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:06:46.007407 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:45.914795 19723 consensus_queue.cc:1059] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:06:45.928980 20349 consensus_queue.cc:1059] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:06:46.009331 16570 raft_consensus.cc:1240] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 12. Current term is 13. Ops: []
W20260709 13:06:46.010526 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 54319.1us
I20260709 13:06:46.032384 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:46.063351 16568 raft_consensus.cc:1240] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:06:46.073499 15933 raft_consensus.cc:2410] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:06:46.006855 20179 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:46.085757 19776 raft_consensus.cc:3055] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:46.048182 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:46.048887 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:46.030644 15936 raft_consensus.cc:2393] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 9: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:46.052067 16567 raft_consensus.cc:1275] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:06:46.049681 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:46.088934 16003 raft_consensus.cc:2410] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:06:46.054514 15937 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
I20260709 13:06:46.060397 16566 raft_consensus.cc:1240] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 11. Current term is 12. Ops: [11.11-11.11]
W20260709 13:06:46.071121 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 53745.1us
I20260709 13:06:46.075119 15902 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:46.093204 16008 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:06:46.093477 16005 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
W20260709 13:06:46.076225 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 105141us
W20260709 13:06:46.077665 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 64970us
I20260709 13:06:46.022945 20007 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:46.081938 20300 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:46.082104 20068 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:46.053440 20133 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.125923 16007 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:06:46.082710 20509 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:46.127381 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:46.052392 19723 raft_consensus.cc:3055] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:46.047998 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:46.084623 19660 consensus_queue.cc:1059] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:46.138284 15936 raft_consensus.cc:1217] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 13 LEADER]: Deduplicated request from leader. Original: 6.5->[7.6-12.8]   Dedup: 7.6->[10.7-12.8]
I20260709 13:06:46.045745 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:46.089146 16011 raft_consensus.cc:2368] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 11. Current term is 12.
I20260709 13:06:46.093755 16535 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
W20260709 13:06:46.140239 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 46562.6us
I20260709 13:06:45.972961 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:46.094276 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:06:46.099756 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:45.973186 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:46.142552 15929 raft_consensus.cc:2393] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 9: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:46.023401 20299 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:46.147930 15931 raft_consensus.cc:1240] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:46.105336 15934 raft_consensus.cc:2410] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:06:46.106791 16533 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:46.115866 16002 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
I20260709 13:06:46.149766 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:46.117650 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:46.150118 15936 raft_consensus.cc:1240] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 13 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 12. Current term is 13. Ops: [7.6-12.8]
I20260709 13:06:46.150694 16003 raft_consensus.cc:2393] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 12: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:46.118381 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:46.147287 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:46.083869 15933 raft_consensus.cc:1217] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Deduplicated request from leader. Original: 6.5->[10.6-10.6]   Dedup: 10.6->[]
W20260709 13:06:46.121793 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:46.152415 15933 raft_consensus.cc:1240] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:06:46.153682 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:46.154883 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 198571us
I20260709 13:06:46.134941 16006 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 12.
I20260709 13:06:45.962011 20357 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:46.116683 15855 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 12.
I20260709 13:06:46.155725 15974 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:46.151918 16002 raft_consensus.cc:2393] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 13: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:46.128535 16007 raft_consensus.cc:2410] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:06:46.132040 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:46.157905 16567 raft_consensus.cc:2393] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 12: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:46.158021 16004 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 10: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:46.159034 16535 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:46.155443 16008 raft_consensus.cc:1275] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:06:46.160353 16008 pending_rounds.cc:85] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee: Aborting all ops after (but not including) 9
I20260709 13:06:46.105100 19776 raft_consensus.cc:740] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:46.160660 16008 pending_rounds.cc:107] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:06:46.122862 20467 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.138612 20437 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.165952 15932 raft_consensus.cc:1275] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 13:06:46.172981 15932 pending_rounds.cc:85] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 10
I20260709 13:06:46.173228 15932 pending_rounds.cc:107] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 13:06:46.173452 15932 raft_consensus.cc:2889] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:46.166213 16533 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:46.156126 20360 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:46.139012 20513 raft_consensus.cc:2749] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:46.134526 19723 raft_consensus.cc:740] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:46.171001 15931 raft_consensus.cc:1240] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 9. Current term is 10. Ops: []
W20260709 13:06:46.175258 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 100377us
I20260709 13:06:46.177942 15902 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:46.175858 16562 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
I20260709 13:06:46.158246 20418 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:46.155331 20509 raft_consensus.cc:697] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:46.169304 16002 raft_consensus.cc:1240] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:06:46.158053 20163 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.158576 20481 raft_consensus.cc:2804] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:46.161176 20071 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.167026 20504 raft_consensus.cc:3055] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:46.199905 16011 pending_rounds.cc:85] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee: Aborting all ops after (but not including) 8
I20260709 13:06:46.215988 16011 pending_rounds.cc:107] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:06:46.216259 16011 raft_consensus.cc:2889] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:46.162882 16008 raft_consensus.cc:2889] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:46.171463 20490 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.139781 20133 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:46.228327 16008 raft_consensus.cc:1240] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 11. Current term is 12. Ops: [10.9-11.10]
I20260709 13:06:46.183434 20418 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:46.171528 20499 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:06:46.225209 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:46.233414 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:46.149040 20068 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:46.212239 16006 raft_consensus.cc:1275] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:06:46.229458 16008 raft_consensus.cc:1240] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:46.216588 20469 consensus_queue.cc:1048] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 13:06:46.181200 20007 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:46.230890 16561 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:06:46.180536 20437 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:46.230813 20490 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:46.239238 16005 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:46.244081 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:46.256464 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:46.212397 20474 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:46.260476 20521 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:46.261044 20521 raft_consensus.cc:697] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:46.215878 20504 raft_consensus.cc:740] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:46.259443 16005 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:06:46.202018 20515 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:46.264199 20521 consensus_queue.cc:237] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:46.267733 20502 raft_consensus.cc:2749] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:46.220976 20071 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:06:46.185663 20198 consensus_queue.cc:1048] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 13:06:46.273487 20071 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:46.211866 20361 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:06:46.274202 20361 consensus_queue.cc:1243] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer eb1e0493af1e46dbb6dec7e25525bab5 log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:06:46.244469 20418 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:46.269292 20504 consensus_queue.cc:260] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:46.262245 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 96104.3us
I20260709 13:06:46.291141 16533 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:46.201318 20163 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:46.293355 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 134422us
I20260709 13:06:46.224660 20364 consensus_queue.cc:1059] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 13:06:46.227412 20473 consensus_queue.cc:1059] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 13:06:46.175418 20467 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:46.242298 20505 consensus_queue.cc:1048] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 13:06:46.322424 16008 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:46.258009 20499 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:46.296627 20474 raft_consensus.cc:697] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 9 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:46.315807 20509 consensus_queue.cc:237] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:46.307763 19723 consensus_queue.cc:260] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:46.327652 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 172007us
I20260709 13:06:46.458168 15930 raft_consensus.cc:1217] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 10 LEADER]: Deduplicated request from leader. Original: 6.5->[9.6-9.6]   Dedup: 9.6->[]
I20260709 13:06:46.314792 19813 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:46.372422 19660 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:46.430699 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:46.430020 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:46.483749 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:46.426877 16007 raft_consensus.cc:1240] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:06:46.495601 16007 pending_rounds.cc:85] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee: Aborting all ops after (but not including) 5
I20260709 13:06:46.496039 16007 pending_rounds.cc:107] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:06:46.496402 16007 raft_consensus.cc:2889] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:46.430087 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:46.498767 15859 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:46.350906 20494 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:46.321897 20364 consensus_queue.cc:1243] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer eb1e0493af1e46dbb6dec7e25525bab5 log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:06:46.486472 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:46.372224 20464 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:46.475836 20469 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.486442 15974 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:46.483192 15930 raft_consensus.cc:1240] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 10 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:06:46.432181 20515 raft_consensus.cc:3055] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:06:46.533830 15744 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 83a1b79cdcf046c78453695c798a14a5 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:06:46.371724 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:46.317998 20504 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:46.459621 16008 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 13.
I20260709 13:06:46.564308 16004 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
W20260709 13:06:46.568421 20260 consensus_meta.cc:287] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.538s	user 0.012s	sys 0.000s
I20260709 13:06:46.537057 20502 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.536684 19660 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:46.602520 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:46.561136 20515 raft_consensus.cc:740] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:46.612039 16004 pending_rounds.cc:85] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting all ops after (but not including) 6
I20260709 13:06:46.617308 16004 pending_rounds.cc:107] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:06:46.617672 16004 raft_consensus.cc:2889] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:46.618057 16004 pending_rounds.cc:107] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:06:46.557397 20494 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:46.587605 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 104469us
I20260709 13:06:46.524739 20481 raft_consensus.cc:697] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:46.618212 20474 consensus_queue.cc:237] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:46.618371 16004 raft_consensus.cc:2889] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:46.592720 20184 raft_consensus.cc:3055] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:46.563663 20464 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:46.583149 20071 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:46.565735 20469 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
W20260709 13:06:46.619905 20155 consensus_meta.cc:287] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.596s	user 0.009s	sys 0.001s
I20260709 13:06:46.615942 20188 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:46.590418 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:46.612831 20431 consensus_meta.cc:287] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.649s	user 0.010s	sys 0.000s
I20260709 13:06:46.615950 20467 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:46.616652 20260 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:46.647300 15859 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
I20260709 13:06:46.663411 15974 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:46.670521 15854 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:06:46.683660 20505 consensus_queue.cc:1048] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:46.685539 20505 consensus_queue.cc:1048] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:46.631695 20163 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:46.649273 20133 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:46.691998 15902 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:46.699193 20260 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:46.703445 20260 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:46.704242 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:46.659606 20184 raft_consensus.cc:740] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:46.676275 20188 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:46.706792 20541 log.cc:927] Time spent T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:06:46.707832 15827 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:46.708866 19660 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:46.672310 20515 consensus_queue.cc:260] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:46.710413 20515 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:46.710922 20516 consensus_meta.cc:287] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.690s	user 0.005s	sys 0.004s
I20260709 13:06:46.711357 20516 raft_consensus.cc:2749] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:46.713748 15827 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:46.698371 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:46.648823 20502 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:46.722743 19660 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:46.723248 19660 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:46.723641 19660 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:46.724416 20502 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:46.717136 20481 consensus_queue.cc:237] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:46.728559 20188 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:46.734537 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:46.735337 15860 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:06:46.736442 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:06:46.737205 20538 log.cc:927] Time spent T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:06:46.697178 20299 raft_consensus.cc:2749] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:46.689970 20431 raft_consensus.cc:2749] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 12. Current term is 13.
I20260709 13:06:46.697988 20499 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:46.755471 15936 raft_consensus.cc:1240] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 8. Current term is 9. Ops: []
W20260709 13:06:46.761837 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 98501.7us
I20260709 13:06:46.762631 15974 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:46.666869 20155 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:46.770341 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:46.773972 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:46.779851 20437 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:46.717701 20490 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:46.782145 15859 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 10: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:46.788084 15974 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:46.697731 20534 consensus_queue.cc:1059] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.304s
I20260709 13:06:46.793259 20163 raft_consensus.cc:2749] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:06:46.780570 20525 log.cc:927] Time spent T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.292s	user 0.001s	sys 0.000s
I20260709 13:06:46.798763 15974 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:46.763396 20548 raft_consensus.cc:2749] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:06:46.828864 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 137010us
I20260709 13:06:46.790428 20299 raft_consensus.cc:2749] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:46.748051 20546 consensus_queue.cc:1059] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
W20260709 13:06:46.845449 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:46.854931 15855 raft_consensus.cc:1275] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:06:46.825075 20467 raft_consensus.cc:2749] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:06:46.861083 16533 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.009s	sys 0.000s
I20260709 13:06:46.792208 20260 consensus_queue.cc:1048] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:06:46.859941 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:46.830999 15902 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:06:46.861860 16533 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.009s	sys 0.000s
W20260709 13:06:46.857903 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 59221.2us
I20260709 13:06:46.868505 15974 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:46.779960 20184 consensus_queue.cc:260] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:46.866286 20494 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:46.786931 20535 consensus_queue.cc:1048] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 13:06:46.870728 15902 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
W20260709 13:06:46.873900 20418 consensus_meta.cc:287] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.598s	user 0.007s	sys 0.002s
I20260709 13:06:46.873976 19660 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:46.877151 20347 raft_consensus.cc:3055] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:46.878353 15859 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:06:46.837361 20474 consensus_queue.cc:1048] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:06:46.870183 20548 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:46.878973 20546 raft_consensus.cc:3055] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:46.894151 15974 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:46.896077 15974 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:46.877871 19776 consensus_queue.cc:260] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:46.888438 20418 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:46.916446 20155 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:46.916405 20418 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:46.901408 20469 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
I20260709 13:06:46.900233 15902 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:46.918754 19776 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:46.919135 15902 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:46.919950 20516 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:46.889722 20347 raft_consensus.cc:740] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:46.920539 20516 raft_consensus.cc:697] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:46.921465 20516 consensus_queue.cc:237] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:46.900965 15861 raft_consensus.cc:1240] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:46.888044 20556 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:46.923130 20556 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:46.923832 20540 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:46.924517 20540 raft_consensus.cc:697] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 12 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:46.925593 20540 consensus_queue.cc:237] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:46.926268 20556 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:46.893471 20546 raft_consensus.cc:740] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:46.927959 20341 raft_consensus.cc:2804] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:46.928478 20341 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:46.929163 20341 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:46.931002 20548 raft_consensus.cc:697] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 9 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:46.928377 20546 consensus_queue.cc:260] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:46.897536 20467 raft_consensus.cc:2749] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:46.932132 20469 consensus_queue.cc:1059] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:46.899155 15859 raft_consensus.cc:1240] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:46.932376 20546 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:46.932333 20548 consensus_queue.cc:237] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:46.934092 15859 raft_consensus.cc:1240] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:46.921170 20347 consensus_queue.cc:260] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:46.882551 20184 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:06:46.947752 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 176204us
I20260709 13:06:46.948505 15974 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:46.948784 15827 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:46.951164 16570 pending_rounds.cc:85] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting all ops after (but not including) 6
I20260709 13:06:46.951539 16570 pending_rounds.cc:107] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:06:46.951539 15974 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:46.951958 16570 raft_consensus.cc:2889] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:46.952282 16570 pending_rounds.cc:107] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:06:46.952531 16570 raft_consensus.cc:2889] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:06:46.900871 20179 consensus_meta.cc:287] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.816s	user 0.011s	sys 0.000s
I20260709 13:06:46.970263 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:46.971120 15937 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 9.
I20260709 13:06:46.972894 15974 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:46.975106 15929 raft_consensus.cc:3055] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:46.976613 16009 raft_consensus.cc:1275] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:06:46.939790 20299 raft_consensus.cc:2749] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:06:46.975570 15929 raft_consensus.cc:740] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:06:46.995354 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:46.997396 16533 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:46.998687 16533 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:46.957610 20163 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:46.944134 20363 consensus_queue.cc:1048] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:06:46.966253 20467 raft_consensus.cc:2749] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:47.004923 16533 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:46.975934 20347 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:46.974457 20071 raft_consensus.cc:2749] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:06:47.044176 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 88646.8us
I20260709 13:06:47.051430 15827 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:47.054553 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 49714.1us
I20260709 13:06:47.002405 20509 raft_consensus.cc:2749] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:47.014932 20490 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:47.055593 16533 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:47.063519 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 93369.8us
I20260709 13:06:47.063616 15974 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:47.015252 20502 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:47.006801 20499 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:47.076663 15974 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:47.057090 20505 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.087106 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:47.015617 20163 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:47.090804 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:47.097415 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:47.088454 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:47.111754 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:47.107158 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:47.119059 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:47.119813 15937 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:06:47.120703 20575 raft_consensus.cc:2804] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:47.072453 20502 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:47.072834 20155 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.121345 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:47.124908 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:47.118176 15902 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:06:47.122663 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:47.131325 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:47.086299 20071 raft_consensus.cc:2749] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:06:47.067519 20467 consensus_queue.cc:1048] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 13:06:47.102205 20299 raft_consensus.cc:2764] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:06:47.108044 20530 raft_consensus.cc:2749] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:06:47.102502 20179 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:47.104929 20068 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:47.105647 20490 raft_consensus.cc:697] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:47.116053 20418 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:47.129037 20502 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:47.121235 20500 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:47.116979 20509 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:47.084075 20499 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:47.138302 20564 log.cc:927] Time spent T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:06:47.124243 20575 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:47.116436 20505 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:47.150151 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:47.153826 15928 raft_consensus.cc:2393] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 13: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:47.142834 20503 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:47.146612 20071 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:47.138622 20155 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:47.188125 16005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41662 (request call id 2240) took 1150 ms. Trace:
I20260709 13:06:47.191591 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:47.176869 15974 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:47.188714 16005 rpcz_store.cc:276] 0709 13:06:46.014696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.106722 (+ 92026us) service_pool.cc:224] Handling call
0709 13:06:47.164606 (+1057884us) spinlock_profiling.cc:243] Waited 58.2 ms on lock 0x7b54003ed854. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:47.164714 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":133843328}
I20260709 13:06:47.155459 20503 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:47.192675 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:47.147848 20179 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:47.155308 20575 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:47.178231 20360 consensus_meta.cc:287] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.994s	user 0.006s	sys 0.007s
I20260709 13:06:47.198375 20503 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:47.188230 20068 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.153038 20500 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:47.149395 20582 consensus_queue.cc:1059] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:06:47.187811 20299 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.203774 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:47.155058 20509 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:47.207652 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:47.206687 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:47.210456 16005 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
I20260709 13:06:47.212191 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:47.176755 20071 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:47.201968 20360 raft_consensus.cc:2749] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:06:47.209765 20504 consensus_meta.cc:287] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.638s	user 0.010s	sys 0.002s
I20260709 13:06:47.159927 20490 consensus_queue.cc:237] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:47.205159 20553 raft_consensus.cc:2804] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:47.207691 20467 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.207477 20068 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:47.223811 20071 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:47.211457 20299 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:47.224987 20179 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:47.225198 20505 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:47.225422 20585 raft_consensus.cc:2804] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:47.226267 20499 leader_election.cc:290] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:47.227535 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:47.228320 20577 raft_consensus.cc:3055] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:47.219149 20506 consensus_queue.cc:1059] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.105s
I20260709 13:06:47.228440 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:47.229221 20571 log.cc:927] Time spent T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:06:47.234807 15860 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:06:47.229828 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:47.306216 16005 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
W20260709 13:06:47.230140 20569 log.cc:927] Time spent T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 13:06:47.242492 20573 log.cc:927] Time spent T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
W20260709 13:06:47.243652 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:47.245435 20068 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:47.247570 20568 log.cc:927] Time spent T 00000000000000000000000000000000 P d33cebb189b14a989836214fdbafb898: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
W20260709 13:06:47.248533 20574 log.cc:927] Time spent T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 13:06:47.312455 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:47.248976 20572 log.cc:927] Time spent T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
W20260709 13:06:47.246954 20566 log.cc:927] Time spent T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 13:06:47.256527 15928 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:06:47.230475 20565 log.cc:927] Time spent T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 13:06:47.217604 20582 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:47.314855 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:47.265226 20585 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:47.316015 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:06:47.259824 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:47.260214 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:47.259548 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:47.232456 20553 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:47.278509 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:47.318482 15853 raft_consensus.cc:2393] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 12: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:47.318821 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:47.276676 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 226149us
I20260709 13:06:47.258620 20188 raft_consensus.cc:2804] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:47.241490 20357 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:47.259057 20071 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.313292 15855 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 10.
I20260709 13:06:47.317987 20585 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:47.343655 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:47.320722 15827 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:47.257484 20467 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:47.327070 15928 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
I20260709 13:06:47.334731 20502 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:47.335965 20553 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:47.336524 20509 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:47.337133 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:47.337687 20155 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:47.338698 20500 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:47.339381 20341 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:47.352068 20467 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:47.352766 20582 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:47.353155 20582 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:47.353492 20582 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:47.353564 20341 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:47.341192 20299 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:47.355391 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:47.351248 15827 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:47.356915 20588 raft_consensus.cc:2749] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:47.358029 15827 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:47.359172 20567 log.cc:927] Time spent T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.274s	user 0.000s	sys 0.000s
I20260709 13:06:47.360078 16004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 2237) took 1346 ms. Trace:
I20260709 13:06:47.360469 16004 rpcz_store.cc:276] 0709 13:06:46.013816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.574832 (+561016us) service_pool.cc:224] Handling call
0709 13:06:46.606114 (+ 31282us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:06:46.660695 (+ 54581us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:46.660915 (+   220us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:06:46.671395 (+ 10480us) log.cc:844] Serialized 78 byte log entry
0709 13:06:46.846645 (+175250us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:46.846688 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:46.846756 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:47.359840 (+513084us) raft_consensus.cc:1685] finished
0709 13:06:47.359865 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:47.360020 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":173,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":88,"thread_start_us":114478,"threads_started":1,"wal-append.queue_time_us":292770}
I20260709 13:06:47.361590 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:47.362557 15934 raft_consensus.cc:2410] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:06:47.348744 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:47.369781 15861 raft_consensus.cc:2410] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:06:47.343173 20071 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:47.374554 20071 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:47.376236 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:47.377592 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:47.343263 20503 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:47.378229 16004 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 9.
I20260709 13:06:47.379068 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:06:47.380445 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:47.380861 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:47.340986 20575 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:47.277267 20577 raft_consensus.cc:740] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:47.278214 20163 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:47.347815 16562 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
I20260709 13:06:47.350236 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:47.384469 15855 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:47.379583 16004 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:47.382782 15827 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:47.386896 16535 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:47.389858 20500 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:47.390265 20500 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:47.390609 20500 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:47.394570 20502 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:47.395499 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:47.395387 20585 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:06:47.398614 15929 consensus_queue.cc:260] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:47.384208 20588 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:47.399925 15929 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:47.400101 20588 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:47.400631 20588 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:47.404572 16004 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 12.
I20260709 13:06:47.407007 15855 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
I20260709 13:06:47.395496 20577 consensus_queue.cc:260] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:47.387945 20503 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:47.416622 20555 log.cc:927] Time spent T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.359s	user 0.002s	sys 0.000s
I20260709 13:06:47.405838 16535 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:47.425782 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:47.423610 15929 raft_consensus.cc:1275] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:06:47.426683 16004 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:47.428354 15858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 2139) took 1052 ms. Trace:
I20260709 13:06:47.428815 15858 rpcz_store.cc:276] 0709 13:06:46.376267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.376401 (+   134us) service_pool.cc:224] Handling call
0709 13:06:46.430255 (+ 53854us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:46.553230 (+122975us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:06:46.553261 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:46.553861 (+   600us) log.cc:844] Serialized 26 byte log entry
0709 13:06:46.715452 (+161591us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:46.715502 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:46.715585 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:47.415930 (+700345us) spinlock_profiling.cc:243] Waited 2.4 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:06:47.428040 (+ 12110us) raft_consensus.cc:1685] finished
0709 13:06:47.428064 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:47.428293 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7203,"raft.queue_time_us":111879,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":23270,"spinlock_wait_cycles":5516672,"thread_start_us":200147,"threads_started":2,"wal-append.queue_time_us":421705}
I20260709 13:06:47.423951 20071 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.432047 20071 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:47.432557 20473 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
I20260709 13:06:47.434201 20071 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:47.435389 20418 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:47.436007 20418 raft_consensus.cc:697] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:06:47.437034 20523 log.cc:927] Time spent T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 1.225s	user 0.000s	sys 0.009s
I20260709 13:06:47.437073 20418 consensus_queue.cc:237] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:47.417739 19974 raft_consensus.cc:2804] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:47.436044 16007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 845) took 1282 ms. Trace:
I20260709 13:06:47.443012 15900 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:47.445362 20341 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:47.446020 20341 consensus_queue.cc:1048] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:47.443091 16007 rpcz_store.cc:276] 0709 13:06:46.153311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.937695 (+784384us) service_pool.cc:224] Handling call
0709 13:06:46.958205 (+ 20510us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:47.130552 (+172347us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:47.130597 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:47.130652 (+    55us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:47.130704 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:47.435810 (+305106us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b54001dbd54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:47.435859 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:47.435988 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":395750784}
I20260709 13:06:47.447068 20474 consensus_queue.cc:1059] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:47.470435 16566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45244 (request call id 1148) took 1071 ms. Trace:
I20260709 13:06:47.470760 15827 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:47.471069 16566 rpcz_store.cc:276] 0709 13:06:46.388087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.389006 (+   919us) service_pool.cc:224] Handling call
0709 13:06:47.459464 (+1070458us) spinlock_profiling.cc:243] Waited 29.7 ms on lock 0x7b5400101728. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f0185924 00007f73ef36e227 00007f73ef36dab8 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:47.459873 (+   409us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":68335360}
I20260709 13:06:47.476032 15827 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:47.477841 16004 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
I20260709 13:06:47.447405 20341 consensus_queue.cc:1059] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.011s
W20260709 13:06:47.478827 20544 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.000s	sys 0.005s
I20260709 13:06:47.481119 16535 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:47.496173 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:47.488696 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:47.388477 20553 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:47.498451 16004 raft_consensus.cc:2393] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 13: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:47.498344 16007 raft_consensus.cc:2376] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11. Re-sending same reply.
I20260709 13:06:47.471133 20585 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:47.500257 15827 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:47.512938 15827 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:47.475113 20577 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:47.514103 15827 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:47.479355 19974 raft_consensus.cc:697] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:47.479267 20260 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:06:47.505744 20594 log.cc:927] Time spent T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:06:47.517956 16567 raft_consensus.cc:2393] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 11: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:47.504971 20490 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:47.519068 15827 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:47.499830 20544 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.004s	sys 0.009s
I20260709 13:06:47.519780 16008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41662 (request call id 2241) took 1272 ms. Trace:
I20260709 13:06:47.520493 16006 raft_consensus.cc:1240] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:06:47.474494 20494 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.465585 20500 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:47.500494 20184 raft_consensus.cc:2749] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:47.500975 20575 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:47.525183 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:47.524778 16008 rpcz_store.cc:276] 0709 13:06:46.247874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.256238 (+  8364us) service_pool.cc:224] Handling call
0709 13:06:47.519581 (+1263343us) spinlock_profiling.cc:243] Waited 65.1 ms on lock 0x7b54003831d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:47.519724 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":149659520}
I20260709 13:06:47.528772 16007 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:06:47.529114 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:47.517493 20347 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:47.529711 16008 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 9.
I20260709 13:06:47.532318 16008 raft_consensus.cc:1275] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:06:47.532867 16008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 850) took 1084 ms. Trace:
I20260709 13:06:47.533159 16008 rpcz_store.cc:276] 0709 13:06:46.448920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:47.530668 (+1081748us) service_pool.cc:224] Handling call
0709 13:06:47.530915 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:47.532654 (+  1739us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:47.532821 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:47.570390 16007 raft_consensus.cc:1240] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:47.571346 16007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 849) took 1122 ms. Trace:
I20260709 13:06:47.571786 16007 rpcz_store.cc:276] 0709 13:06:46.448630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:47.529384 (+1080754us) service_pool.cc:224] Handling call
0709 13:06:47.570149 (+ 40765us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:47.571008 (+   859us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:47.571278 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:47.540011 20260 raft_consensus.cc:2749] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:47.530495 20490 raft_consensus.cc:697] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:06:47.584235 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 65224.8us
I20260709 13:06:47.588555 15827 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:47.538141 20599 raft_consensus.cc:2749] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:47.538760 20590 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:47.587199 20490 consensus_queue.cc:237] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:47.589790 20500 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:47.585572 20582 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:47.602906 15827 leader_election.cc:400] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 11. Current term is 12.
I20260709 13:06:47.604260 15827 leader_election.cc:403] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:06:47.534422 20494 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:47.601922 20544 raft_consensus.cc:2804] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:47.601051 20590 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:47.610488 15827 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:47.534900 19974 consensus_queue.cc:237] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:47.614880 20544 raft_consensus.cc:697] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 9 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:47.615847 20347 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:47.616470 20184 consensus_queue.cc:1048] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:47.620081 20544 consensus_queue.cc:237] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:47.627051 20347 raft_consensus.cc:2749] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:47.610064 20599 raft_consensus.cc:2749] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 11. Current term is 12.
W20260709 13:06:47.633697 15827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:47.612788 20590 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:47.654326 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:47.654281 15860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 2149) took 1080 ms. Trace:
I20260709 13:06:47.655045 15860 rpcz_store.cc:276] 0709 13:06:46.573601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.736158 (+162557us) service_pool.cc:224] Handling call
0709 13:06:47.654049 (+917891us) spinlock_profiling.cc:243] Waited 486 ms on lock 0x7b540032da54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:47.654170 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1117756672}
I20260709 13:06:47.662101 15936 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2234) took 1199 ms. Trace:
I20260709 13:06:47.662514 15936 rpcz_store.cc:276] 0709 13:06:46.462491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.463948 (+  1457us) service_pool.cc:224] Handling call
0709 13:06:46.464214 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:46.764685 (+300471us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:47.661876 (+897191us) spinlock_profiling.cc:243] Waited 291 ms on lock 0x7b54002c1d54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:47.662047 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":669367040}
I20260709 13:06:47.669001 16006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2071) took 1719 ms. Trace:
I20260709 13:06:47.669425 16006 rpcz_store.cc:276] 0709 13:06:45.949655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.234488 (+284833us) service_pool.cc:224] Handling call
0709 13:06:47.520264 (+1285776us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:47.524909 (+  4645us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:47.668796 (+143887us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54003831d0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:47.668916 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2899616512}
I20260709 13:06:47.670674 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:47.678903 15854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 2145) took 1204 ms. Trace:
W20260709 13:06:47.691107 20528 log.cc:927] Time spent T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 1.101s	user 0.004s	sys 0.000s
I20260709 13:06:47.701138 16011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 844) took 1685 ms. Trace:
I20260709 13:06:47.701915 16011 rpcz_store.cc:276] 0709 13:06:46.015794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.781360 (+765566us) service_pool.cc:224] Handling call
0709 13:06:46.807152 (+ 25792us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:46.832150 (+ 24998us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:46.832193 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:46.832232 (+    39us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:46.832270 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:47.700690 (+868420us) spinlock_profiling.cc:243] Waited 24.8 ms on lock 0x7b54001f7cd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:47.700840 (+   150us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:47.701054 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":57024768}
I20260709 13:06:47.718304 20505 raft_consensus.cc:3055] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:47.718999 20505 raft_consensus.cc:740] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:47.722174 16005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 846) took 1568 ms. Trace:
W20260709 13:06:47.732949 20536 log.cc:927] Time spent T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 1.113s	user 0.005s	sys 0.000s
I20260709 13:06:47.736989 19723 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:47.740285 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:47.679471 15854 rpcz_store.cc:276] 0709 13:06:46.474145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.474311 (+   166us) service_pool.cc:224] Handling call
0709 13:06:47.678642 (+1204331us) spinlock_profiling.cc:243] Waited 78.1 ms on lock 0x7b54000b5c54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:47.678791 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":179731200}
I20260709 13:06:47.759465 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:47.760358 16004 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:47.768694 16562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45244 (request call id 1146) took 1397 ms. Trace:
I20260709 13:06:47.769304 16562 rpcz_store.cc:276] 0709 13:06:46.385572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.426527 (+ 40955us) service_pool.cc:224] Handling call
0709 13:06:47.768440 (+1341913us) spinlock_profiling.cc:243] Waited 838 ms on lock 0x7b5400503f54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:47.768609 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1927185792}
W20260709 13:06:47.776727 20529 log.cc:927] Time spent T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 1.293s	user 0.002s	sys 0.000s
I20260709 13:06:47.783969 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:47.784401 15976 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:47.785233 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:47.785800 16007 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:06:47.786167 20163 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:47.786600 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:47.784724 20431 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.787191 16007 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:06:47.788029 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:47.787076 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:47.787977 20431 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:47.789465 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:47.790040 15855 raft_consensus.cc:2376] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11. Re-sending same reply.
I20260709 13:06:47.791375 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:47.792834 20431 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:47.795075 20505 consensus_queue.cc:260] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:47.800434 20198 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.832134 20198 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:47.834883 20198 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:47.806262 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:47.808883 16569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 1144) took 1761 ms. Trace:
I20260709 13:06:47.836169 16569 rpcz_store.cc:276] 0709 13:06:46.047084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.048985 (+  1901us) service_pool.cc:224] Handling call
0709 13:06:46.049305 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:46.049740 (+   435us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:46.049774 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:46.070312 (+ 20538us) log.cc:844] Serialized 26 byte log entry
0709 13:06:46.310483 (+240171us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:46.310535 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:46.310617 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:47.690341 (+1379724us) spinlock_profiling.cc:243] Waited 34.2 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:06:47.808461 (+118120us) raft_consensus.cc:1685] finished
0709 13:06:47.808492 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:47.808778 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":604,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"spinlock_wait_cycles":78632192,"thread_start_us":162383,"threads_started":1,"wal-append.queue_time_us":375941}
I20260709 13:06:47.671397 16006 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:47.845325 15935 raft_consensus.cc:1240] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 12 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:47.848563 15935 raft_consensus.cc:1275] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:06:47.850430 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:47.722791 16005 rpcz_store.cc:276] 0709 13:06:46.153582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:47.316887 (+1163305us) service_pool.cc:224] Handling call
0709 13:06:47.339417 (+ 22530us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:47.375468 (+ 36051us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:47.375500 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:47.376035 (+   535us) log.cc:844] Serialized 26 byte log entry
0709 13:06:47.402808 (+ 26773us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:47.481979 (+ 79171us) log.cc:844] Serialized 14 byte log entry
0709 13:06:47.482360 (+   381us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:47.721557 (+239197us) spinlock_profiling.cc:243] Waited 35.6 ms on lock 0x7b54002e1dd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:47.721625 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:47.721699 (+    74us) raft_consensus.cc:1685] finished
0709 13:06:47.721833 (+   134us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:47.722087 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":123306,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":10079,"raft.queue_time_us":7585,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":12083,"spinlock_wait_cycles":81919232,"thread_start_us":89160,"threads_started":2,"wal-append.queue_time_us":64732}
I20260709 13:06:47.863467 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:47.817938 16011 raft_consensus.cc:1240] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:06:47.788311 15861 raft_consensus.cc:1275] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:06:47.828086 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:47.829900 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:47.868521 15862 raft_consensus.cc:2376] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9. Re-sending same reply.
I20260709 13:06:47.875162 20578 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:47.876014 20578 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:06:47.887444 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:47.892469 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:47.899154 15937 raft_consensus.cc:1240] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:47.900031 20431 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:47.900776 20431 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:47.903525 20431 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:47.906862 20431 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.907523 20431 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:47.911290 20431 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:47.919837 16535 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:47.926339 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:47.932598 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:47.935206 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:47.939473 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:47.941161 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:47.942932 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:47.943603 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:47.933620 16005 raft_consensus.cc:1275] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:06:47.943665 16564 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:06:47.934800 20437 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:47.932096 20499 raft_consensus.cc:2804] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:47.649753 20588 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:47.948861 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:47.945385 16011 raft_consensus.cc:1275] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:06:47.954320 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:47.955124 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:47.957648 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:47.987596 16566 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:06:47.947417 16568 raft_consensus.cc:2393] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 10: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:47.991896 16563 raft_consensus.cc:1240] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:47.992245 15853 raft_consensus.cc:2393] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 11: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:47.992322 16568 raft_consensus.cc:1240] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:47.993173 20588 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:47.945863 20599 consensus_queue.cc:1048] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:06:47.960000 15976 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:47.965797 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:47.921360 20553 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:47.967903 20437 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:47.972478 20582 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:47.932232 20357 consensus_queue.cc:1048] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:06:47.974843 20494 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:47.955822 20604 log.cc:927] Time spent T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.337s	user 0.000s	sys 0.001s
I20260709 13:06:47.983764 16010 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 13.
I20260709 13:06:47.995484 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:47.938788 20469 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:47.972752 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:48.002765 16564 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:06:47.965075 20499 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:47.998349 20431 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:48.019255 16004 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
I20260709 13:06:48.006319 20499 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:48.020941 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:48.021591 15935 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
I20260709 13:06:48.021713 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:48.022482 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:48.023293 15929 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:48.022444 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:48.024822 15936 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 6.
I20260709 13:06:48.009076 20431 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.025736 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:48.026609 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:48.031240 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:48.033409 16011 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:48.033517 16006 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:06:48.034730 15935 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 10.
I20260709 13:06:48.035174 16006 raft_consensus.cc:1275] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:06:48.039532 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:48.040498 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:48.006438 20469 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:48.045241 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:48.023960 20516 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:48.047660 20516 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:48.033147 20431 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:48.069638 15929 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
I20260709 13:06:48.114570 16011 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
I20260709 13:06:48.121536 16011 raft_consensus.cc:1240] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:06:48.076589 20590 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:48.106866 20577 consensus_meta.cc:287] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.590s	user 0.009s	sys 0.000s
I20260709 13:06:48.105702 20516 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:48.142148 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 203145us
I20260709 13:06:48.124058 20163 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:48.105788 20437 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:48.162245 16535 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:48.200367 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:48.206048 20608 log.cc:927] Time spent T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:06:48.206704 20590 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:48.177486 20418 consensus_queue.cc:1048] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:06:48.196549 20155 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:48.195847 20469 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:48.178892 20163 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:48.194800 16535 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:48.216759 20605 log.cc:927] Time spent T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
I20260709 13:06:48.198613 20437 consensus_queue.cc:1048] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
W20260709 13:06:48.218446 20610 log.cc:927] Time spent T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.199s	user 0.002s	sys 0.000s
I20260709 13:06:48.218916 20155 raft_consensus.cc:697] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:06:48.230585 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 189232us
W20260709 13:06:48.243709 20613 log.cc:927] Time spent T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:06:48.245110 16007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 2249) took 1102 ms. Trace:
I20260709 13:06:48.245481 16007 rpcz_store.cc:276] 0709 13:06:47.142625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:47.789904 (+647279us) service_pool.cc:224] Handling call
0709 13:06:47.790170 (+   266us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:47.790588 (+   418us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:47.790625 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:47.791043 (+   418us) log.cc:844] Serialized 26 byte log entry
0709 13:06:47.999427 (+208384us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:47.999463 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:47.999522 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:48.244864 (+245342us) raft_consensus.cc:1685] finished
0709 13:06:48.244886 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:48.245047 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":100,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":102,"thread_start_us":207424,"threads_started":1,"wal-append.queue_time_us":332074}
I20260709 13:06:48.231849 19723 raft_consensus.cc:2804] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:48.253939 16561 raft_consensus.cc:1275] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:06:48.255880 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:48.253219 20618 log.cc:927] Time spent T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.130s	user 0.002s	sys 0.000s
I20260709 13:06:48.253782 19723 raft_consensus.cc:697] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:48.277189 20585 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.310050 15858 raft_consensus.cc:1240] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 12. Current term is 13. Ops: [12.11-12.11]
W20260709 13:06:48.325266 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:48.333787 20611 log.cc:927] Time spent T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:06:48.335054 16008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 2243) took 1388 ms. Trace:
I20260709 13:06:48.335465 16008 rpcz_store.cc:276] 0709 13:06:46.947089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:47.533838 (+586749us) service_pool.cc:224] Handling call
0709 13:06:47.534097 (+   259us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:47.575225 (+ 41128us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:47.575271 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:47.578802 (+  3531us) log.cc:844] Serialized 26 byte log entry
0709 13:06:48.028054 (+449252us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:48.028097 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:48.028162 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:48.334834 (+306672us) raft_consensus.cc:1685] finished
0709 13:06:48.334856 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:48.334995 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":35147,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":3076,"thread_start_us":426192,"threads_started":1,"wal-append.queue_time_us":579500}
I20260709 13:06:48.337529 16008 raft_consensus.cc:1240] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:48.339270 16008 raft_consensus.cc:1240] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 11. Current term is 12. Ops: []
W20260709 13:06:48.340030 20612 log.cc:927] Time spent T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 13:06:48.341104 16008 raft_consensus.cc:1275] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:06:48.341689 16008 pending_rounds.cc:85] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting all ops after (but not including) 5
W20260709 13:06:48.341980 20617 log.cc:927] Time spent T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:06:48.342250 16008 pending_rounds.cc:107] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:06:48.343349 16008 raft_consensus.cc:2889] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:06:48.372563 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 107322us
I20260709 13:06:48.390009 16008 raft_consensus.cc:1240] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:48.284288 20155 consensus_queue.cc:237] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:48.409699 16009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 2218) took 2565 ms. Trace:
I20260709 13:06:48.410137 16009 rpcz_store.cc:276] 0709 13:06:45.844030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.844365 (+   335us) service_pool.cc:224] Handling call
0709 13:06:45.844724 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:46.977089 (+1132365us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:48.409472 (+1432383us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54000ed8d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:48.409644 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2602071808}
I20260709 13:06:48.412763 16009 raft_consensus.cc:1275] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:06:48.413952 16009 raft_consensus.cc:1275] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:06:48.421973 16009 raft_consensus.cc:1275] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:06:48.424827 16009 raft_consensus.cc:1240] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:06:48.426916 15930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 2079) took 2504 ms. Trace:
I20260709 13:06:48.427491 15930 rpcz_store.cc:276] 0709 13:06:45.907563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.062086 (+154523us) service_pool.cc:224] Handling call
0709 13:06:46.062526 (+   440us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:46.559613 (+497087us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:48.411876 (+1852263us) spinlock_profiling.cc:243] Waited 395 ms on lock 0x7b54003b7c54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:48.412030 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":909498624}
I20260709 13:06:48.351372 20585 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:48.390916 20625 consensus_queue.cc:1048] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 13:06:48.460891 20590 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.461539 20590 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:48.461947 20585 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:48.463202 20577 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.463446 15974 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:06:48.449990 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:48.450171 20622 log.cc:927] Time spent T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.101s	user 0.001s	sys 0.001s
W20260709 13:06:48.450657 20616 log.cc:927] Time spent T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.191s	user 0.000s	sys 0.000s
W20260709 13:06:48.452244 20623 log.cc:927] Time spent T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 13:06:48.472672 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:48.463721 20577 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:48.480881 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:48.481781 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:48.482141 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:48.482647 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:48.474982 15974 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:48.496327 15974 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:48.490912 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:48.408306 19723 consensus_queue.cc:237] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:48.498569 15974 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:06:48.508116 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 231478us
I20260709 13:06:48.484107 20585 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:48.503958 15974 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:48.504222 20625 raft_consensus.cc:2749] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:48.517557 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:48.514212 15902 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:48.494678 20418 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:48.506167 20627 raft_consensus.cc:2764] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:06:48.510237 20631 raft_consensus.cc:2749] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:48.463658 20590 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:48.519304 15902 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:48.521359 15974 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:48.514714 20437 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.516081 19660 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:48.532986 15902 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:48.524452 20469 raft_consensus.cc:2804] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:48.525065 20638 raft_consensus.cc:2749] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:06:48.547091 20499 consensus_meta.cc:287] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.515s	user 0.006s	sys 0.003s
I20260709 13:06:48.527288 20418 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:48.550472 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:48.535841 20503 raft_consensus.cc:2764] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:06:48.553449 15861 raft_consensus.cc:2410] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:06:48.538017 20585 raft_consensus.cc:2749] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:48.557977 15902 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:48.548694 20634 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:48.548978 20469 raft_consensus.cc:697] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:48.569123 15974 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:48.572212 20474 raft_consensus.cc:3055] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:48.572616 20474 raft_consensus.cc:740] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:48.573016 15902 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:48.573400 20474 consensus_queue.cc:260] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:48.574041 20540 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:48.574462 20540 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.575248 20540 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:48.582895 20615 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.583542 20615 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:48.585920 20615 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:48.586777 20357 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.586400 15930 consensus_queue.cc:237] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:48.587297 20357 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:48.589411 20357 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:48.590270 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:48.590459 20641 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:48.539381 20437 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:48.593236 16533 leader_election.cc:304] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:48.595153 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:48.598991 16533 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:48.564339 20632 raft_consensus.cc:2804] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:48.596341 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:48.574429 20474 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:48.603998 16533 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:48.554006 20431 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:48.603084 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:48.583412 20619 log.cc:927] Time spent T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.188s	user 0.000s	sys 0.000s
I20260709 13:06:48.566120 20634 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:48.602419 20368 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:48.631793 20634 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:48.600142 20615 raft_consensus.cc:2749] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:48.607753 20490 raft_consensus.cc:2804] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:48.626379 20590 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.636312 20615 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:48.636494 20490 raft_consensus.cc:697] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:48.636838 20590 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:48.636940 20615 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:48.637575 20490 consensus_queue.cc:237] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:48.590500 20357 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:48.640182 20615 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:48.639986 20357 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:48.619357 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:48.555132 20499 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:48.595999 15855 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 10.
I20260709 13:06:48.642237 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:48.643807 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:48.649505 16011 raft_consensus.cc:1240] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 12. Current term is 14. Ops: [12.10-12.10]
W20260709 13:06:48.652340 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:48.624464 20632 raft_consensus.cc:697] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 13 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:48.654379 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:48.646601 20188 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:06:48.654929 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 82084.6us
I20260709 13:06:48.656544 15902 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:48.661558 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:48.653095 20577 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:48.650256 20590 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:48.674590 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:48.680514 15902 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:48.657160 20188 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:48.685784 15974 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:48.687238 16568 raft_consensus.cc:1275] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:06:48.691134 20642 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:48.691602 20642 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:48.691982 20642 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:48.679873 20633 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:48.684350 20363 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:48.688766 15974 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:48.697458 15974 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:48.698562 15974 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:48.699262 15974 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:48.664168 20469 consensus_queue.cc:237] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:48.702142 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:48.701825 15974 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:48.703934 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:48.704586 15855 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 12.
I20260709 13:06:48.705392 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:48.705879 15855 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:06:48.711150 20521 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.131s	user 0.001s	sys 0.016s
W20260709 13:06:48.711649 20521 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.133s	user 0.001s	sys 0.016s
I20260709 13:06:48.701488 20631 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:48.720270 20431 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:48.720489 20631 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.721065 20631 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:48.722077 20357 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:48.711956 20585 raft_consensus.cc:2749] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:48.667063 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:48.712173 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:48.694550 20418 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:48.730872 15858 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
I20260709 13:06:48.716523 15974 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:48.711177 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:48.719276 20649 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:48.719954 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:48.734443 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 124403us
I20260709 13:06:48.731597 20357 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:48.738701 20642 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:48.738747 20357 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:48.739794 20585 raft_consensus.cc:2804] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:48.701370 20625 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:06:48.701498 20641 raft_consensus.cc:2749] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:48.720445 20499 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:48.701673 20363 raft_consensus.cc:697] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 12 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:48.745776 20642 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:48.746224 20363 consensus_queue.cc:237] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:48.742797 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:48.741287 20585 raft_consensus.cc:697] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 13 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:48.749933 15858 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:48.737407 16533 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:48.762188 20499 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:48.762836 20578 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:48.698206 20632 consensus_queue.cc:237] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:48.741554 20437 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:48.766079 15853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58214 (request call id 888) took 1371 ms. Trace:
I20260709 13:06:48.741990 20625 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:48.766520 15853 rpcz_store.cc:276] 0709 13:06:47.394889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:47.395241 (+   352us) service_pool.cc:224] Handling call
0709 13:06:48.765837 (+1370596us) spinlock_profiling.cc:243] Waited 596 ms on lock 0x7b54003b6354. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:48.766008 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1370435328}
I20260709 13:06:48.768514 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:48.772192 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:48.773808 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:48.774416 15853 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:48.775357 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 181586us
I20260709 13:06:48.776279 15855 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
W20260709 13:06:48.776804 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:48.778404 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:48.779052 20368 raft_consensus.cc:2749] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:48.777520 15855 raft_consensus.cc:1240] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:48.781855 15930 raft_consensus.cc:1275] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:06:48.782967 16533 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:48.783638 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:48.784452 15902 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 2.279s (SENT) [suppressed 13 similar messages]
I20260709 13:06:48.784745 20368 raft_consensus.cc:2749] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:48.786901 16533 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:48.789410 20368 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:48.789108 20469 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:48.790441 20469 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:48.792543 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:48.794916 15930 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 10.
I20260709 13:06:48.792860 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:48.795850 15929 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:48.795957 20368 consensus_queue.cc:1048] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:48.799588 20368 consensus_queue.cc:1048] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:48.801155 20356 consensus_queue.cc:1048] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:48.802736 15929 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:06:48.805478 15929 raft_consensus.cc:1240] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:48.806941 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:48.773618 20625 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:48.820367 16533 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:48.824966 15974 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:48.825914 20641 raft_consensus.cc:2804] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:06:48.826187 20651 log.cc:927] Time spent T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 13:06:48.826423 20641 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.826856 20641 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:48.827904 20634 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:48.828464 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:48.829785 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:48.830489 20634 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:48.830890 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:48.838014 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:48.841346 15853 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
I20260709 13:06:48.836759 15974 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:48.847577 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:48.848320 15853 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
I20260709 13:06:48.850167 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:48.850293 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:48.850718 15929 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:48.851008 15853 raft_consensus.cc:2393] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 13: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:48.851809 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:48.852874 15853 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 11: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:48.853554 15936 pending_rounds.cc:85] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 9
I20260709 13:06:48.853973 15936 pending_rounds.cc:107] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:06:48.854287 15936 raft_consensus.cc:2889] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:48.820623 20490 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:48.850657 16533 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:48.837833 20368 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:48.861291 16533 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:48.849191 20642 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:48.848287 20634 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.852115 20356 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.928768 20503 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.930662 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:48.940443 15853 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 8.
I20260709 13:06:48.847462 20437 consensus_queue.cc:1059] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.079s
I20260709 13:06:48.851742 20188 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:48.904989 20521 raft_consensus.cc:2804] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:48.904757 20431 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:48.905614 20368 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:48.954679 16008 raft_consensus.cc:1275] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:06:48.957777 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:48.958626 15853 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:06:48.849218 20585 consensus_queue.cc:237] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:06:48.943955 20642 raft_consensus.cc:697] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:48.904458 20490 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:48.955883 20646 log.cc:927] Time spent T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.244s	user 0.000s	sys 0.001s
I20260709 13:06:48.978153 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:48.908417 20631 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:48.946339 20521 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.960863 20368 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:48.942456 20503 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:48.960577 20638 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.008564 20638 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:48.953642 20649 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.010528 20649 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:48.991431 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:48.941411 20634 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:49.015373 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:49.016582 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:06:48.987692 20645 log.cc:927] Time spent T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.226s	user 0.000s	sys 0.001s
I20260709 13:06:48.960522 20620 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:48.987401 20469 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:48.943732 20356 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:48.980331 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:49.029028 15853 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
I20260709 13:06:49.024413 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:48.992507 20521 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:49.033953 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:49.030973 16004 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:49.031793 20642 consensus_queue.cc:237] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:48.960865 20431 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:49.054785 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:49.032843 20649 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:49.022241 20620 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:49.081673 20356 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:49.081509 20659 log.cc:927] Time spent T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 13:06:49.080569 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:49.079821 20357 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:49.092475 20474 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:49.103013 16008 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:06:49.106154 15853 raft_consensus.cc:1275] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:06:49.115410 20188 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:49.115728 20490 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:49.116748 16564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45246 (request call id 991) took 1165 ms. Trace:
I20260709 13:06:49.117197 16564 rpcz_store.cc:276] 0709 13:06:47.951717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:47.965473 (+ 13756us) service_pool.cc:224] Handling call
0709 13:06:49.116377 (+1150904us) spinlock_profiling.cc:243] Waited 7.13 ms on lock 0x7b5400550ad4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:49.116680 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16391424}
I20260709 13:06:49.123445 20431 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:49.124055 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:49.113497 20474 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:49.123100 20503 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:49.125223 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:49.126354 20631 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:49.125317 20638 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:49.130859 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:49.131695 15853 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:06:49.123467 20188 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.132954 20188 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:49.133808 20540 consensus_meta.cc:287] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.558s	user 0.009s	sys 0.003s
I20260709 13:06:49.134197 20540 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:49.134908 20188 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:49.135147 15931 raft_consensus.cc:1240] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:49.138761 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:49.145247 16004 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
W20260709 13:06:49.171452 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:49.240317 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:49.244087 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:49.244912 16004 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 13.
I20260709 13:06:49.245986 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:49.247074 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:49.248406 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:49.249054 15853 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
I20260709 13:06:49.249146 15855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 2166) took 1341 ms. Trace:
I20260709 13:06:49.250033 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:49.250878 15853 raft_consensus.cc:2393] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 11: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:49.027655 15929 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:06:49.255808 15853 raft_consensus.cc:1275] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:06:49.259248 15853 raft_consensus.cc:3055] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:49.263772 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:49.260291 15853 raft_consensus.cc:740] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
W20260709 13:06:49.263186 20363 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.029s	sys 0.043s
I20260709 13:06:49.246368 20649 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:49.274612 16565 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.021s	sys 0.057s
W20260709 13:06:49.277657 16565 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.021s	sys 0.057s
I20260709 13:06:49.249692 15855 rpcz_store.cc:276] 0709 13:06:47.908140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.783623 (+875483us) service_pool.cc:224] Handling call
0709 13:06:48.787789 (+  4166us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:48.788115 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:48.788162 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:48.788785 (+   623us) log.cc:844] Serialized 26 byte log entry
0709 13:06:49.248689 (+459904us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:49.248748 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:49.248854 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:49.248913 (+    59us) raft_consensus.cc:1685] finished
0709 13:06:49.248931 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:49.249078 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":459386,"threads_started":1,"wal-append.queue_time_us":171980}
I20260709 13:06:49.280418 15855 raft_consensus.cc:1275] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:06:49.272537 15853 consensus_queue.cc:260] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:49.276696 20363 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.030s	sys 0.043s
I20260709 13:06:49.284027 15853 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:49.277079 20585 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:49.277244 20634 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:49.260504 20641 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:49.297464 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 95023.2us
I20260709 13:06:49.296590 16006 raft_consensus.cc:1240] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:49.293885 15933 raft_consensus.cc:1240] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 12 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:49.288964 20540 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:49.291812 20464 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:49.359189 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:49.282528 20649 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:49.363869 15853 raft_consensus.cc:1275] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 13:06:49.372164 15853 pending_rounds.cc:85] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8: Aborting all ops after (but not including) 10
I20260709 13:06:49.372514 15853 pending_rounds.cc:107] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 13:06:49.372761 15853 raft_consensus.cc:2889] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:49.351619 20188 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.390339 20188 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:49.392581 20188 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:49.389735 15900 leader_election.cc:336] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.665s (SENT)
W20260709 13:06:49.394827 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:49.404038 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:49.404958 15853 raft_consensus.cc:2410] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 13 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:06:49.391508 20649 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:49.408402 20668 log.cc:927] Time spent T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:06:49.409750 16567 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45232 (request call id 994) took 2307 ms (client timeout 2112 ms). Trace:
W20260709 13:06:49.410156 16567 rpcz_store.cc:269] 0709 13:06:47.102594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:47.228182 (+125588us) service_pool.cc:224] Handling call
0709 13:06:49.409529 (+2181347us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b5400545e54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:49.409651 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":484951424}
I20260709 13:06:49.411175 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:49.413386 20641 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:49.414070 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:49.414655 15862 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:49.416862 20521 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:49.419217 16011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2074) took 1637 ms. Trace:
I20260709 13:06:49.419561 16011 rpcz_store.cc:276] 0709 13:06:47.781394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.127360 (+345966us) service_pool.cc:224] Handling call
0709 13:06:48.127666 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:48.649949 (+522283us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:49.418991 (+769042us) spinlock_profiling.cc:243] Waited 522 ms on lock 0x7b54001dc754. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:49.419159 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1199458176}
W20260709 13:06:49.422559 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:49.423202 15856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 2138) took 3048 ms. Trace:
I20260709 13:06:49.423607 15856 rpcz_store.cc:276] 0709 13:06:46.375015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.375139 (+   124us) service_pool.cc:224] Handling call
0709 13:06:46.375458 (+   319us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:46.394321 (+ 18863us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:46.428442 (+ 34121us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:46.472740 (+ 44298us) log.cc:844] Serialized 52 byte log entry
0709 13:06:46.660972 (+188232us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:46.861401 (+200429us) log.cc:844] Serialized 14 byte log entry
0709 13:06:47.050025 (+188624us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:47.050116 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:47.397652 (+347536us) spinlock_profiling.cc:243] Waited 4.88 ms on lock 0x7b180014a400. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb522 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd 000000000038f354 000000000038f199 00007f73e729e83a
0709 13:06:49.421217 (+2023565us) spinlock_profiling.cc:243] Waited 28 ms on lock 0x7b180014a400. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb3f3 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd 000000000038f354 000000000038f199 00007f73e729e83a
0709 13:06:49.422973 (+  1756us) raft_consensus.cc:1685] finished
0709 13:06:49.422991 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:49.423127 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28874,"prepare.queue_time_us":207278,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":28323,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":42135,"spinlock_wait_cycles":75545088,"thread_start_us":514406,"threads_started":3,"wal-append.queue_time_us":423248}
I20260709 13:06:49.426028 16570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 992) took 1469 ms. Trace:
I20260709 13:06:49.299793 20585 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:49.432514 15862 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
I20260709 13:06:49.426434 16570 rpcz_store.cc:276] 0709 13:06:47.956369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:47.985583 (+ 29214us) service_pool.cc:224] Handling call
0709 13:06:47.985931 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:47.986277 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:47.986307 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:47.986759 (+   452us) log.cc:844] Serialized 26 byte log entry
0709 13:06:48.191409 (+204650us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:48.280114 (+ 88705us) log.cc:844] Serialized 14 byte log entry
0709 13:06:48.280380 (+   266us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:48.280541 (+   161us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:49.425227 (+1144686us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b24002f66d8. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71419c4 00007f73e713facb 00007f73e72fadc9 00007f73e72faa55 00007f73e72f9fb0 00007f73e9f70488 00007f73e9f70be8 00007f73e9f48398 00007f73e9f4376a 00007f73e9f42bb6
0709 13:06:49.425742 (+   515us) raft_consensus.cc:1685] finished
0709 13:06:49.425853 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:49.425976 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":214882,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":8587,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":24030,"spinlock_wait_cycles":24054144,"thread_start_us":285104,"threads_started":2,"wal-append.queue_time_us":252902}
I20260709 13:06:49.438311 20368 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:49.444314 16568 raft_consensus.cc:1240] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:49.439706 20585 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:49.445736 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:49.449537 20624 log.cc:927] Time spent T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 1.169s	user 0.015s	sys 0.000s
I20260709 13:06:49.452440 16563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 987) took 1519 ms. Trace:
I20260709 13:06:49.452756 16563 rpcz_store.cc:276] 0709 13:06:47.932756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:47.933528 (+   772us) service_pool.cc:224] Handling call
0709 13:06:47.991629 (+ 58101us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:47.992259 (+   630us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:49.452230 (+1459971us) spinlock_profiling.cc:243] Waited 44.8 ms on lock 0x7b5400102550. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:49.452387 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":103052672}
I20260709 13:06:49.458186 16003 raft_consensus.cc:1275] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
W20260709 13:06:49.458968 16010 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41662 (request call id 2219) took 3612 ms (client timeout 2279 ms). Trace:
W20260709 13:06:49.459386 16010 rpcz_store.cc:269] 0709 13:06:45.845495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.845607 (+   112us) service_pool.cc:224] Handling call
0709 13:06:49.457634 (+3612027us) spinlock_profiling.cc:243] Waited 1.78 s on lock 0x7b54001f72d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:49.457752 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4083786112}
W20260709 13:06:49.454938 20625 consensus_meta.cc:287] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.621s	user 0.002s	sys 0.012s
I20260709 13:06:49.468003 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:49.469197 20163 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:49.470899 16003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 2239) took 3456 ms. Trace:
I20260709 13:06:49.471437 16003 rpcz_store.cc:276] 0709 13:06:46.014415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.722579 (+708164us) service_pool.cc:224] Handling call
0709 13:06:49.457851 (+2735272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:49.458589 (+   738us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:49.470643 (+ 12054us) spinlock_profiling.cc:243] Waited 2.73 s on lock 0x7b54001f72d0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:49.470824 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6289762688}
I20260709 13:06:49.486577 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:49.487497 15860 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
I20260709 13:06:49.488695 20620 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:49.480744 20505 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:49.483891 20521 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:49.470117 20625 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:49.498374 20614 log.cc:927] Time spent T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.750s	user 0.007s	sys 0.005s
I20260709 13:06:49.501215 20625 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:49.501741 15931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2249) took 1021 ms. Trace:
I20260709 13:06:49.502194 15931 rpcz_store.cc:276] 0709 13:06:48.480832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.580141 (+ 99309us) service_pool.cc:224] Handling call
0709 13:06:48.580512 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:49.135592 (+555080us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:49.501502 (+365910us) spinlock_profiling.cc:243] Waited 554 ms on lock 0x7b5400328a54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:49.501685 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1275117312}
I20260709 13:06:49.505100 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 2247) took 2547 ms. Trace:
I20260709 13:06:49.504066 15974 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:49.505470 16002 rpcz_store.cc:276] 0709 13:06:46.957389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:47.673747 (+716358us) service_pool.cc:224] Handling call
0709 13:06:47.674150 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:47.674637 (+   487us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:47.674678 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:47.675235 (+   557us) log.cc:844] Serialized 26 byte log entry
0709 13:06:48.151280 (+476045us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:48.187840 (+ 36560us) log.cc:844] Serialized 14 byte log entry
0709 13:06:48.245972 (+ 58132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:48.246349 (+   377us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:48.747709 (+501360us) spinlock_profiling.cc:243] Waited 1.16 ms on lock 0x7b18000247c0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb3f3 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd 000000000038f354 000000000038f199 00007f73e729e83a
0709 13:06:49.497786 (+750077us) spinlock_profiling.cc:243] Waited 1.52 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:06:49.499750 (+  1964us) raft_consensus.cc:1685] finished
0709 13:06:49.499781 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:49.505046 (+  5265us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":29490,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":61817,"raft.run_cpu_time_us":313,"raft.run_wall_time_us":2887,"spinlock_wait_cycles":6160512,"thread_start_us":531725,"threads_started":2,"wal-append.queue_time_us":572819}
I20260709 13:06:49.508720 15974 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:49.509624 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:49.513377 15974 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:49.515539 20585 raft_consensus.cc:2749] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:49.515584 20163 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:49.515823 15974 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:49.516235 20163 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.516446 20585 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:49.516681 20163 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:49.516726 20641 consensus_queue.cc:1048] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:49.517203 20585 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:49.517608 20585 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.517949 20585 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:49.517035 15974 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:49.518707 20641 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:49.519161 20641 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:49.521104 15827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:49.521242 15974 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:49.522743 15827 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:49.519565 20641 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:49.528878 20625 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:49.538663 20626 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:49.545210 20626 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:49.546087 20649 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.546582 20649 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:49.548455 20649 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:49.550473 20481 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:49.550352 20163 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:49.552067 20418 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.552613 20418 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:49.554203 20437 raft_consensus.cc:3055] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:49.554692 20437 raft_consensus.cc:740] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:06:49.558229 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:49.562211 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:49.562870 16561 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 10.
I20260709 13:06:49.564380 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:49.565412 16561 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:49.569170 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:49.573444 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:49.555409 20418 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:49.532869 20368 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:06:49.577920 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:49.581008 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:49.583628 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:49.585028 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:49.510018 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:49.604293 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:49.605072 16567 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 10.
I20260709 13:06:49.605540 15976 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:49.606098 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:49.607087 20669 raft_consensus.cc:2804] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:49.607568 20669 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:49.607869 20626 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:49.609477 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:49.610634 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:49.611249 16565 raft_consensus.cc:2368] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 11. Current term is 12.
I20260709 13:06:49.614590 20676 consensus_queue.cc:1059] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:06:49.616880 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:49.617415 15976 leader_election.cc:400] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 11. Current term is 12.
I20260709 13:06:49.617697 15857 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 10.
I20260709 13:06:49.607959 20669 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:49.617758 15976 leader_election.cc:403] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:06:49.625824 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 103138us
I20260709 13:06:49.584758 15936 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:06:49.633965 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:49.537684 20625 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:49.607950 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:49.635857 16566 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:49.543579 20474 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:49.640058 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:49.640920 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:49.639403 15936 raft_consensus.cc:1240] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:06:49.645280 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:49.638875 16561 raft_consensus.cc:2410] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:06:49.637643 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:49.576319 20506 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:49.648689 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:49.646188 15976 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:49.650170 16005 raft_consensus.cc:1275] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:06:49.654521 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:49.655243 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:49.656016 16010 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 11: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:49.661043 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:49.661734 15933 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 12.
I20260709 13:06:49.665678 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:49.668347 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:49.632164 20626 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.668030 20642 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:49.638782 20625 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:49.678086 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:49.678424 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:49.637509 20464 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:49.683199 20632 consensus_queue.cc:1048] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:49.683360 20634 consensus_queue.cc:1048] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:49.684401 20363 consensus_queue.cc:1048] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:06:49.639647 20631 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:06:49.692127 15931 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:06:49.678166 20515 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:49.693264 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:49.555546 20437 consensus_queue.cc:260] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:49.701766 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 65088us
I20260709 13:06:49.720103 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:49.720497 16533 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:49.720736 16568 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
I20260709 13:06:49.687848 20676 raft_consensus.cc:3055] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:49.653632 20506 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:49.723279 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:49.733775 16533 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:49.701155 20626 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:49.724304 20676 raft_consensus.cc:740] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:49.700248 20474 consensus_queue.cc:1048] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:06:49.736897 16533 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:49.717267 20515 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:49.706367 20464 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:49.717046 20615 consensus_queue.cc:1048] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 13:06:49.733381 20437 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:49.762606 16569 raft_consensus.cc:1275] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:06:49.763747 16562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 1170) took 1033 ms. Trace:
I20260709 13:06:49.751863 20163 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:06:49.766106 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 80573us
W20260709 13:06:49.767237 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 48643.8us
I20260709 13:06:49.768754 15902 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:49.770040 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 48648.3us
I20260709 13:06:49.737771 20521 consensus_queue.cc:1059] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:06:49.766013 16562 rpcz_store.cc:276] 0709 13:06:48.730601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.730746 (+   145us) service_pool.cc:224] Handling call
0709 13:06:48.731056 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:48.731300 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:06:48.837565 (+106265us) log.cc:844] Serialized 14 byte log entry
0709 13:06:49.760959 (+923394us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:49.761009 (+    50us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:49.761718 (+   709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:49.762192 (+   474us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:49.763661 (+  1469us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":320569,"prepare.queue_time_us":26384,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":320618,"raft.run_cpu_time_us":55,"raft.run_wall_time_us":56,"thread_start_us":696374,"threads_started":3,"wal-append.queue_time_us":475672}
I20260709 13:06:49.772751 15902 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:49.768343 15825 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:49.781117 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:49.781716 15860 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:49.764914 20369 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:49.786862 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:49.787504 15857 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:49.782964 20678 consensus_queue.cc:1048] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:06:49.774152 20634 raft_consensus.cc:2749] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:49.787380 20368 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:49.803249 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:49.816620 15862 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 11: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
W20260709 13:06:49.807020 20660 log.cc:927] Time spent T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.784s	user 0.004s	sys 0.007s
I20260709 13:06:49.807089 20626 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:49.813374 15860 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 12.
I20260709 13:06:49.796434 20680 raft_consensus.cc:3055] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:49.815528 16566 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
I20260709 13:06:49.818626 20506 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:49.874570 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 60160.6us
I20260709 13:06:49.822156 20532 consensus_queue.cc:1048] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:49.805294 20515 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:49.872148 20680 raft_consensus.cc:740] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:49.886933 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:49.871867 20687 raft_consensus.cc:2804] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:49.888028 20687 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.887753 20680 consensus_queue.cc:260] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:49.888401 20687 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:49.837598 15976 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:49.889436 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 51932.7us
I20260709 13:06:49.823946 20632 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:49.884369 20464 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:49.767917 20676 consensus_queue.cc:260] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:49.884595 20503 consensus_queue.cc:1048] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:49.917730 20676 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:49.890564 15976 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:49.881994 20631 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 11. Current term is 12.
W20260709 13:06:49.918536 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:49.919087 20631 raft_consensus.cc:2804] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:49.891714 15902 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:49.919620 20631 raft_consensus.cc:697] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:49.920616 15902 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:49.921262 20631 consensus_queue.cc:237] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:49.924376 20670 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:49.924847 20670 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:49.926826 20670 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:49.894670 20626 raft_consensus.cc:2749] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:49.895001 20474 consensus_queue.cc:1048] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:49.893354 20632 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:49.928472 20632 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:49.929304 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:49.930508 20634 raft_consensus.cc:2749] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:49.931201 20634 raft_consensus.cc:2749] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:06:49.938649 15902 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:49.900765 20481 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:49.939854 15933 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:46118
I20260709 13:06:49.890897 20188 consensus_queue.cc:1059] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 13:06:49.941332 20678 raft_consensus.cc:3055] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:06:49.941733 20678 raft_consensus.cc:740] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:49.898196 20669 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:49.907595 20540 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.942606 20678 consensus_queue.cc:260] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:49.934154 20585 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:49.919190 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:49.953830 15900 leader_election.cc:336] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 2.204s (SENT)
I20260709 13:06:49.893507 20680 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:49.956634 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:49.957144 20368 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:49.940471 15933 raft_consensus.cc:606] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 13 LEADER]: Received request to transfer leadership to TS 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:49.942770 20540 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:49.907888 20687 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:49.944698 20641 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:49.945228 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:49.960506 16566 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:49.960565 15860 raft_consensus.cc:1275] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:06:49.909050 20369 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:49.947280 16002 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 12.
I20260709 13:06:49.947952 20556 raft_consensus.cc:2749] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:49.948379 15857 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
I20260709 13:06:49.949290 20678 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:49.950519 16567 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:06:49.957217 15862 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:49.957890 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:49.991003 15854 raft_consensus.cc:2410] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:06:49.958520 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:49.959054 15902 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:49.892226 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 236611us
W20260709 13:06:49.958453 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 221599us
I20260709 13:06:49.939344 20503 raft_consensus.cc:1064] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:06:49.991406 16569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 1176) took 1207 ms. Trace:
I20260709 13:06:50.005182 16569 rpcz_store.cc:276] 0709 13:06:48.784303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.799146 (+ 14843us) service_pool.cc:224] Handling call
0709 13:06:49.762303 (+963157us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:49.763049 (+   746us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:49.976677 (+213628us) spinlock_profiling.cc:243] Waited 963 ms on lock 0x7b54002b5f50. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:49.991351 (+ 14674us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2214396288}
I20260709 13:06:49.945746 20669 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:49.906806 20625 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:50.003602 15858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 2164) took 2172 ms. Trace:
I20260709 13:06:50.022452 15858 rpcz_store.cc:276] 0709 13:06:47.831276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.804258 (+972982us) service_pool.cc:224] Handling call
0709 13:06:48.821434 (+ 17176us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:48.821701 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:48.823979 (+  2278us) log.cc:844] Serialized 14 byte log entry
0709 13:06:49.463988 (+640009us) spinlock_profiling.cc:243] Waited 8.45 ms on lock 0x7b1800236920. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f4823a 00007f73ea3b2a2a
0709 13:06:49.710552 (+246564us) log.cc:844] Serialized 14 byte log entry
0709 13:06:49.963142 (+252590us) log.cc:844] Serialized 14 byte log entry
0709 13:06:49.976097 (+ 12955us) log.cc:844] Serialized 14 byte log entry
0709 13:06:49.976281 (+   184us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:49.976332 (+    51us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:49.976386 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:50.003396 (+ 27010us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:50.003535 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28296,"prepare.queue_time_us":211920,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":1653,"raft.queue_time_us":131092,"raft.run_cpu_time_us":502,"raft.run_wall_time_us":92553,"spinlock_wait_cycles":28069888,"thread_start_us":478532,"threads_started":4,"wal-append.queue_time_us":755570}
W20260709 13:06:50.004451 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 45500.2us
I20260709 13:06:50.007581 16568 raft_consensus.cc:1275] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:06:50.014837 15862 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 10.
I20260709 13:06:50.017493 16533 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:50.018213 15930 raft_consensus.cc:1275] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:06:49.968422 20368 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:49.963780 20649 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.024806 15902 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:49.964035 20626 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:50.027639 16566 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
I20260709 13:06:50.029237 15902 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:50.040196 15976 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:50.041882 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:50.030249 16533 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:50.043061 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:50.043262 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:50.028539 16003 raft_consensus.cc:1240] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:50.027976 15974 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:50.049510 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:50.049822 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:50.051133 15936 raft_consensus.cc:1275] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:06:50.053757 15757 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58966 (request call id 244) took 26824 ms (client timeout 15000 ms). Trace:
I20260709 13:06:50.056741 20687 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:50.058357 20155 raft_consensus.cc:2804] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:49.976353 20163 consensus_queue.cc:1048] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 13:06:50.036810 20506 raft_consensus.cc:2804] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:50.058931 20155 raft_consensus.cc:697] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:50.059446 20506 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.059921 20506 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:50.060057 20626 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:50.060128 20155 consensus_queue.cc:237] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:50.062335 20626 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:50.063030 20669 raft_consensus.cc:2804] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:06:50.063305 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 44444.5us
I20260709 13:06:50.063727 20631 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:50.064131 20641 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:50.064246 20631 raft_consensus.cc:697] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 10 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:50.065284 20585 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:50.065152 20631 consensus_queue.cc:237] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:50.020529 20505 raft_consensus.cc:2749] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:50.029114 20437 consensus_queue.cc:1048] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:06:50.036556 20357 raft_consensus.cc:2804] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:50.069160 20357 raft_consensus.cc:697] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:50.070065 20357 consensus_queue.cc:237] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:50.071099 20585 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:49.972886 20198 consensus_queue.cc:1059] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.073s
I20260709 13:06:50.071555 20670 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:06:50.071614 20585 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:50.072321 20198 raft_consensus.cc:3055] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:50.072715 20198 raft_consensus.cc:740] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:50.073676 20585 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:50.073455 20198 consensus_queue.cc:260] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:50.074937 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:50.075003 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:50.068158 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:50.075562 15854 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:50.075644 15862 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:50.076001 15931 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:50.076216 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:50.080977 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:50.083020 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:50.089238 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:50.089905 15856 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:50.090504 15854 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:06:50.091781 15974 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:50.093158 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:50.093940 15862 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:06:50.096098 15974 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:50.096678 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:50.097460 15932 raft_consensus.cc:1275] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:06:50.098671 16533 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:50.099473 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:50.099745 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:50.100054 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:50.058857 20625 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:50.076444 20474 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:50.082329 20198 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:50.167065 15931 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
I20260709 13:06:50.168290 15856 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
W20260709 13:06:50.105074 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 306289us
I20260709 13:06:50.103590 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:50.109232 15933 raft_consensus.cc:1240] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:50.109480 15862 raft_consensus.cc:1240] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:50.109890 15936 raft_consensus.cc:1275] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:06:50.109920 15860 raft_consensus.cc:1240] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 13. Current term is 14. Ops: []
W20260709 13:06:50.110947 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.111191 20163 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:50.108345 15854 raft_consensus.cc:1275] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:06:50.113385 15859 raft_consensus.cc:3055] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:50.172366 15859 raft_consensus.cc:740] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:50.088969 15760 catalog_manager.cc:5697] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 4 to 9. New cstate: current_term: 9 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:50.172529 15974 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:50.110020 16533 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:50.115933 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:50.175354 15930 raft_consensus.cc:2393] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 12: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:50.124735 15928 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:46118
W20260709 13:06:50.175410 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 65455.6us
I20260709 13:06:50.176043 15928 raft_consensus.cc:606] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Received request to transfer leadership to TS eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:50.176537 15928 raft_consensus.cc:614] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:50.084127 20669 raft_consensus.cc:697] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 11 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:50.029067 20481 consensus_queue.cc:1048] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.084s
I20260709 13:06:50.047566 20649 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:50.031320 20499 consensus_queue.cc:1048] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.072s
I20260709 13:06:50.135852 20679 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:50.173398 15859 consensus_queue.cc:260] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:50.101769 20418 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:50.102429 20641 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:50.182927 16003 raft_consensus.cc:1275] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:06:50.209242 16566 raft_consensus.cc:1275] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:06:50.209600 16566 pending_rounds.cc:85] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting all ops after (but not including) 8
I20260709 13:06:50.209820 16566 pending_rounds.cc:107] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:06:50.210047 16566 raft_consensus.cc:2889] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:50.191303 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:50.102975 20626 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:50.196647 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:50.215302 16011 raft_consensus.cc:2393] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 7: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:50.215826 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:50.216414 16010 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
I20260709 13:06:50.208484 20679 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:50.217902 20679 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:50.219753 20641 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:50.221884 20641 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:50.193615 20521 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:50.200778 20688 consensus_queue.cc:1048] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.012s
I20260709 13:06:50.223517 20669 consensus_queue.cc:237] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:50.226899 20540 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:50.227927 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:50.228605 15930 raft_consensus.cc:2393] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 12: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:50.228976 20418 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:50.229595 20418 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:50.231328 20695 log.cc:927] Time spent T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.107s	user 0.002s	sys 0.000s
I20260709 13:06:50.199074 16004 raft_consensus.cc:1275] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:06:50.201850 15859 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:50.222786 20521 raft_consensus.cc:697] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:50.172294 16535 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:50.200600 20625 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:06:50.242856 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 70669.7us
W20260709 13:06:50.246789 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 251328us
I20260709 13:06:50.200707 20670 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:50.263828 16535 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:50.271554 15859 raft_consensus.cc:1275] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:06:50.272202 15859 pending_rounds.cc:85] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8: Aborting all ops after (but not including) 8
I20260709 13:06:50.272523 15859 pending_rounds.cc:107] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:06:50.272856 15859 raft_consensus.cc:2889] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:06:50.054185 15757 rpcz_store.cc:269] 0709 13:06:23.229840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.257233 (+ 27393us) service_pool.cc:224] Handling call
0709 13:06:50.053659 (+26796426us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:45.436848 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:45.437600 (+   752us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:45.437650 (+    50us) write_op.cc:435] Acquired schema lock
0709 13:06:45.437666 (+    16us) tablet.cc:662] Decoding operations
0709 13:06:45.438572 (+   906us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:45.438678 (+   106us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:45.438706 (+    28us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:06:45.439455 (+   749us) tablet.cc:701] Row locks acquired
0709 13:06:45.439504 (+    49us) write_op.cc:260] PREPARE: finished
0709 13:06:45.446526 (+  7022us) write_op.cc:270] Start()
0709 13:06:45.471090 (+ 24564us) write_op.cc:276] Timestamp: P: 1783602405439604 usec, L: 0
0709 13:06:45.478828 (+  7738us) op_driver.cc:348] REPLICATION: starting
0709 13:06:45.504173 (+ 25345us) log.cc:844] Serialized 2876 byte log entry
0709 13:06:45.802377 (+298204us) op_driver.cc:464] REPLICATION: finished
0709 13:06:46.041568 (+239191us) write_op.cc:301] APPLY: starting
0709 13:06:46.041687 (+   119us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:46.238055 (+196368us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:46.238182 (+   127us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:46.874908 (+636726us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:46.875264 (+   356us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=14,key_file_lookups=14,delta_file_lookups=14,mrs_lookups=0
0709 13:06:46.875316 (+    52us) write_op.cc:312] APPLY: finished
0709 13:06:46.875702 (+   386us) log.cc:844] Serialized 73 byte log entry
0709 13:06:46.999999 (+124297us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:47.000174 (+   175us) write_op.cc:454] Released schema lock
0709 13:06:47.019001 (+ 18827us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:47.102130 (+ 83129us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:47.107812 (+  5682us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1630953,"reported_tablets":42,"child_traces":[["op",{"apply.queue_time_us":238845,"apply.run_cpu_time_us":32541,"apply.run_wall_time_us":1024852,"cfile_cache_hit":26,"cfile_cache_hit_bytes":27235,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6222,"cfile_init":2,"lbm_read_time_us":44891,"lbm_reads_10-100_ms":2,"lbm_reads_lt_1ms":10,"num_ops":7,"prepare.queue_time_us":20671,"prepare.run_cpu_time_us":14971,"prepare.run_wall_time_us":223134,"raft.queue_time_us":131232,"raft.run_cpu_time_us":27418,"raft.run_wall_time_us":167485,"replication_time_us":323358,"thread_start_us":506621,"threads_started":5,"wal-append.queue_time_us":333597,"wal-append.run_cpu_time_us":29590,"wal-append.run_wall_time_us":2378874}]]}
I20260709 13:06:50.283612 20369 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:50.200662 20615 consensus_queue.cc:1059] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 13:06:50.259541 20437 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:50.260627 20626 raft_consensus.cc:697] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:50.275727 15760 catalog_manager.cc:5697] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 4 to 9, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 9 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:50.238571 20680 consensus_queue.cc:1059] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.052s
I20260709 13:06:50.263697 20648 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:50.319058 20648 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:50.321468 20648 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:50.321398 16535 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:50.201112 20368 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:50.323735 20368 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:50.324608 20368 raft_consensus.cc:2749] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:50.317991 15760 catalog_manager.cc:5697] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 4 to 7, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 7 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:50.214874 20632 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:50.329921 20632 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:50.327005 16569 raft_consensus.cc:1240] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 9. Current term is 10. Ops: []
W20260709 13:06:50.333393 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 75514.1us
I20260709 13:06:50.315430 16004 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc"
dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
mode: GRACEFUL
new_leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:43700
I20260709 13:06:50.334275 20686 consensus_queue.cc:1059] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:50.334898 15902 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:50.335029 16004 raft_consensus.cc:606] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Received request to transfer leadership to TS eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:50.335543 20686 raft_consensus.cc:3055] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:50.335556 16004 raft_consensus.cc:614] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:50.336835 20505 consensus_queue.cc:1048] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:50.337163 20634 consensus_queue.cc:1048] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:50.335994 20632 raft_consensus.cc:2804] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:50.339221 15932 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:46118
I20260709 13:06:50.339888 15932 raft_consensus.cc:606] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Received request to transfer leadership to TS 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:50.340178 15932 raft_consensus.cc:614] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:50.340834 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:50.341547 15930 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:50.344499 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.346724 15760 catalog_manager.cc:5697] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 3 to 7, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 7 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:06:50.276970 20625 raft_consensus.cc:697] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:06:50.364450 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 102141us
I20260709 13:06:50.364828 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:50.336076 20686 raft_consensus.cc:740] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:06:50.366048 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 158646us
I20260709 13:06:50.372561 16008 raft_consensus.cc:1240] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 10 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:06:50.373703 20707 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.374228 20707 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:50.368384 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 174686us
I20260709 13:06:50.313324 20642 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:50.376658 20707 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:50.377363 20707 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:50.377198 20642 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:50.378017 20707 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:50.361799 20696 log.cc:927] Time spent T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:06:50.317420 20163 raft_consensus.cc:697] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:50.381387 15860 raft_consensus.cc:1240] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:50.338979 20632 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:50.357719 20505 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.274029 20670 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:50.334973 20540 consensus_queue.cc:1048] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:06:50.390072 15928 raft_consensus.cc:1240] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:50.393036 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:50.399744 15930 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:06:50.400398 15860 raft_consensus.cc:1240] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:50.404335 15932 raft_consensus.cc:1240] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 13. Current term is 14. Ops: [13.13-13.13]
I20260709 13:06:50.411796 15860 raft_consensus.cc:1240] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:50.414666 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:50.415931 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:50.415956 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:50.416812 16004 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:50.360795 20556 consensus_queue.cc:1048] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:06:50.360960 20634 consensus_queue.cc:1059] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.032s
W20260709 13:06:50.435422 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 61485.2us
I20260709 13:06:50.418015 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:50.419958 15858 raft_consensus.cc:1240] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:50.421207 16566 raft_consensus.cc:1240] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:06:50.437477 16002 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
I20260709 13:06:50.422151 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:50.438692 16002 raft_consensus.cc:1275] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:06:50.426677 16008 raft_consensus.cc:1275] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:06:50.441599 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.429091 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:50.429411 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:50.442310 15859 raft_consensus.cc:2393] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 11: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:50.442417 15860 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:06:50.400545 20632 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:50.445822 15825 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:50.433041 16004 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 14.
I20260709 13:06:50.436959 15974 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:50.457614 16002 raft_consensus.cc:1240] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 13. Current term is 14. Ops: [12.10-13.11]
I20260709 13:06:50.380240 20677 consensus_queue.cc:1059] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:06:50.447914 15858 raft_consensus.cc:1240] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:50.383827 20688 consensus_queue.cc:1059] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.052s
I20260709 13:06:50.424389 20615 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.457793 16004 raft_consensus.cc:1275] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:06:50.466856 16004 raft_consensus.cc:1275] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:06:50.453534 15859 raft_consensus.cc:1240] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:50.387555 20155 consensus_queue.cc:1048] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:06:50.473690 15860 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:06:50.404734 20505 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:50.496039 15859 raft_consensus.cc:1240] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:50.380033 20503 consensus_queue.cc:237] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:50.443042 20687 consensus_queue.cc:1048] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
W20260709 13:06:50.478025 20676 consensus_meta.cc:287] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.554s	user 0.006s	sys 0.007s
I20260709 13:06:50.478386 20615 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:50.398943 20521 consensus_queue.cc:237] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:50.578110 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:50.506712 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:50.579169 15859 raft_consensus.cc:2376] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11. Re-sending same reply.
I20260709 13:06:50.449749 20490 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:50.508090 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 71205.7us
I20260709 13:06:50.476855 20620 raft_consensus.cc:3055] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:50.514459 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:50.516959 15853 raft_consensus.cc:1240] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 9. Current term is 10. Ops: []
W20260709 13:06:50.518182 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.593231 20703 consensus_queue.cc:1048] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:06:50.594230 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.595288 20676 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:06:50.596095 20719 log.cc:927] Time spent T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.203s	user 0.001s	sys 0.000s
I20260709 13:06:50.496717 20464 raft_consensus.cc:3055] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:50.469228 20437 consensus_queue.cc:1048] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:06:50.464298 20481 consensus_queue.cc:1059] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 8, Time since last communication: 0.041s
W20260709 13:06:50.507750 20697 log.cc:927] Time spent T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.312s	user 0.001s	sys 0.002s
I20260709 13:06:50.498549 20685 raft_consensus.cc:2804] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:50.443141 20686 consensus_queue.cc:260] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:50.613350 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 102934us
W20260709 13:06:50.564441 20506 consensus_meta.cc:287] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.504s	user 0.007s	sys 0.002s
I20260709 13:06:50.457778 20625 consensus_queue.cc:237] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:50.457406 20626 consensus_queue.cc:237] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:50.502437 15760 catalog_manager.cc:5697] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
I20260709 13:06:50.577582 15857 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:06:50.465062 20649 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:50.581045 15935 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
I20260709 13:06:50.587875 15974 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:50.585361 20642 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:50.610998 20710 log.cc:927] Time spent T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:06:50.520673 15858 raft_consensus.cc:1240] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 10. Current term is 11. Ops: []
W20260709 13:06:50.598242 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 172858us
I20260709 13:06:50.640287 15974 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:50.569089 20707 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:50.600639 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:50.644093 16566 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
I20260709 13:06:50.484215 20163 consensus_queue.cc:237] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:50.603096 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:50.645331 15853 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
I20260709 13:06:50.646104 15974 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:50.646256 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:50.603618 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.611756 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:06:50.594233 20698 log.cc:927] Time spent T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.253s	user 0.001s	sys 0.001s
I20260709 13:06:50.512970 20688 raft_consensus.cc:3055] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:50.652148 20688 raft_consensus.cc:740] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:50.653789 15861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2258) took 1130 ms. Trace:
I20260709 13:06:50.654182 15861 rpcz_store.cc:276] 0709 13:06:49.523441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:49.523557 (+   116us) service_pool.cc:224] Handling call
0709 13:06:49.524356 (+   799us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:49.651156 (+126800us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:49.651204 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:49.955871 (+304667us) spinlock_profiling.cc:243] Waited 6.98 ms on lock 0x7b2c00000f48. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e8ca1a11 00007f73e8cb1372 00007f73ea3e3a01 00007f73ea3a843a 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2
0709 13:06:49.956224 (+   353us) log.cc:844] Serialized 26 byte log entry
0709 13:06:50.097532 (+141308us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:50.097585 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:50.097674 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:50.653476 (+555802us) raft_consensus.cc:1685] finished
0709 13:06:50.653498 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:50.653728 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6494,"raft.run_cpu_time_us":66,"raft.run_wall_time_us":66,"spinlock_wait_cycles":16054016,"thread_start_us":262345,"threads_started":2,"wal-append.queue_time_us":277671}
I20260709 13:06:50.615010 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:50.656785 15859 raft_consensus.cc:1275] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:06:50.657141 20615 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:50.592070 20620 raft_consensus.cc:740] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:50.595443 20679 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:50.622565 15855 raft_consensus.cc:1275] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:06:50.658625 16569 raft_consensus.cc:1240] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:50.659006 20688 consensus_queue.cc:260] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:50.599702 20155 consensus_queue.cc:1048] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:06:50.645251 20642 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.660315 20688 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:50.660391 20642 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:50.634985 20731 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:50.642654 20721 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:50.662937 20731 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:50.663635 20505 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:50.644788 20631 consensus_queue.cc:1048] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:06:50.581593 20532 consensus_queue.cc:1048] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:06:50.665036 20481 consensus_queue.cc:1048] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:06:50.638577 20506 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:50.595814 20676 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:50.667172 20506 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:50.667363 20676 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:50.591269 20490 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:50.658062 20620 consensus_queue.cc:260] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:50.669217 20490 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:50.670984 20731 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:50.671900 20642 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:50.673239 20722 log.cc:927] Time spent T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:06:50.676420 20679 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:50.638443 20685 raft_consensus.cc:697] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:50.681447 20685 consensus_queue.cc:237] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:50.684733 20418 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:50.686781 20418 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:50.687917 16570 raft_consensus.cc:1240] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:50.646291 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:50.648137 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:50.648371 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:50.599253 20702 log.cc:927] Time spent T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.211s	user 0.000s	sys 0.001s
I20260709 13:06:50.698930 15902 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:50.651763 20670 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:50.609594 20705 log.cc:927] Time spent T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 13:06:50.602120 20464 raft_consensus.cc:740] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
W20260709 13:06:50.700752 15827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.689121 15974 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:50.710474 15974 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:50.710558 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:50.711342 15760 catalog_manager.cc:5697] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:06:50.712345 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:50.714483 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:50.715263 16561 raft_consensus.cc:2410] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:06:50.716296 15902 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:50.679565 20720 log.cc:927] Time spent T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.176s	user 0.000s	sys 0.000s
I20260709 13:06:50.717355 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:50.718549 15827 connection.cc:256] RPC call timeout handler was delayed by -1.5694312389999823s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18180000000000002s for a response
I20260709 13:06:50.720286 15974 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:50.688191 20686 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:50.720745 15827 connection.cc:256] RPC call timeout handler was delayed by -1.5503860350000878s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16160000000000002s for a response
I20260709 13:06:50.722062 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:06:50.725431 15827 connection.cc:256] RPC call timeout handler was delayed by -1.5019312389999868s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18930000000000002s for a response
W20260709 13:06:50.725817 15827 connection.cc:256] RPC call timeout handler was delayed by -1.4821312390000685s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1915s for a response
W20260709 13:06:50.726269 15827 connection.cc:256] RPC call timeout handler was delayed by -1.4182312389999652s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1986s for a response
I20260709 13:06:50.727190 15827 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:50.727219 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:50.728259 16562 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:50.729009 15827 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:50.730849 20677 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:50.724109 20556 consensus_queue.cc:1059] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 13:06:50.689067 20707 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:50.746641 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:50.722553 15974 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:50.733593 16564 raft_consensus.cc:1275] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:06:50.696197 20649 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:06:50.727085 15853 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
I20260709 13:06:50.750116 16533 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:50.750507 15974 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:50.718271 20676 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:50.724337 20642 raft_consensus.cc:2749] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:50.761683 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:50.724164 20678 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:50.736304 20721 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:50.746830 20506 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:50.775193 15827 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:50.745594 20188 raft_consensus.cc:2749] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:50.745589 20729 raft_consensus.cc:2804] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:50.785778 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:50.786906 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 230961us
I20260709 13:06:50.799357 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:50.761354 20726 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:50.801643 16562 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:06:50.782404 20641 raft_consensus.cc:2749] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:50.783625 20642 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:50.791563 20729 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.808326 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:50.820182 16561 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:50.809957 16563 raft_consensus.cc:1240] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 12. Current term is 13. Ops: []
W20260709 13:06:50.810000 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.768029 20676 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:50.807435 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:50.770117 20678 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:50.768596 20707 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:06:50.793686 20721 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:50.770498 20464 consensus_queue.cc:260] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 8, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:50.937904 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 322908us
I20260709 13:06:50.978670 16561 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
W20260709 13:06:50.979766 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.930392 16005 raft_consensus.cc:1240] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:06:50.822508 20729 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:50.985932 15936 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.009s	sys 0.022s
I20260709 13:06:50.798954 20716 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:06:50.976579 20632 consensus_meta.cc:287] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.515s	user 0.011s	sys 0.004s
W20260709 13:06:50.819025 20669 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.000s	sys 0.017s
W20260709 13:06:50.988787 15936 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.010s	sys 0.022s
W20260709 13:06:50.986634 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.992905 15928 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:46118
I20260709 13:06:50.995191 15928 raft_consensus.cc:606] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 12 LEADER]: Received request to transfer leadership to TS eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:50.995824 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.978237 16564 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
W20260709 13:06:50.996307 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:50.994527 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 205742us
W20260709 13:06:50.997819 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.812167 20687 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:06:51.002418 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 224808us
W20260709 13:06:50.998688 20669 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.772s	user 0.010s	sys 0.058s
W20260709 13:06:51.024289 20725 log.cc:927] Time spent T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:06:50.998016 15827 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:50.847890 20726 raft_consensus.cc:697] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:51.025894 20726 consensus_queue.cc:237] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:50.987287 20670 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:06:51.029726 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.939213 20625 raft_consensus.cc:2749] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:50.912417 20728 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:50.811183 20727 consensus_queue.cc:1059] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.208s
I20260709 13:06:50.994261 20506 raft_consensus.cc:697] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 14 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:06:51.004115 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:51.048307 16567 raft_consensus.cc:1275] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:06:51.060981 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:51.036199 20649 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.003261 20632 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:51.039141 15827 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:51.001950 20716 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:51.114077 15933 raft_consensus.cc:1240] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:51.114857 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:50.993881 20721 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:51.118564 16003 raft_consensus.cc:1240] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 13. Current term is 14. Ops: []
W20260709 13:06:51.121222 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 82153.2us
I20260709 13:06:51.126943 15827 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:51.133489 15928 raft_consensus.cc:1240] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 9. Current term is 10. Ops: []
W20260709 13:06:51.167783 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 417310us
I20260709 13:06:51.169795 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:51.050865 20728 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:51.133406 20188 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:06:51.133404 20631 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:06:51.189806 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 75993us
I20260709 13:06:51.107918 20649 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:51.204242 16565 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:06:51.208069 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:51.187207 20687 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:06:51.212533 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 167151us
I20260709 13:06:51.190726 20556 consensus_queue.cc:1243] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer 85e997ba5003439781ec4f1f8db0c3ee log is divergent from this leader: its last log entry 10.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:06:51.215813 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 81586.8us
I20260709 13:06:51.219417 15827 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:51.220854 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:51.221624 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:51.222548 15853 raft_consensus.cc:2393] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 9: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:51.221621 15854 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 11: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:51.226365 15827 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:51.178465 20506 consensus_queue.cc:237] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:51.253335 15858 raft_consensus.cc:1275] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:06:51.251039 16005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 2275) took 1063 ms. Trace:
I20260709 13:06:51.198938 20631 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:51.273707 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 47422us
I20260709 13:06:51.271873 15930 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:46118
I20260709 13:06:51.280076 15930 raft_consensus.cc:606] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 10 LEADER]: Received request to transfer leadership to TS eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:51.268445 20632 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:51.291554 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:06:51.286629 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:51.236706 20678 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:51.295164 20752 raft_consensus.cc:2764] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:06:51.303189 20649 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:51.303861 20734 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.295270 20753 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.304500 20734 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:51.294597 20704 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.305531 20704 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:51.306154 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:51.306742 20734 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:51.307251 20734 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:06:51.294811 20742 log.cc:927] Time spent T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:06:51.295110 20767 raft_consensus.cc:3055] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:51.343146 20767 raft_consensus.cc:740] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:51.344242 20767 consensus_queue.cc:260] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:51.345289 16003 raft_consensus.cc:1240] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:51.345777 20767 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:51.346977 16003 raft_consensus.cc:1275] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:06:51.295367 20721 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:51.295578 20743 log.cc:927] Time spent T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:06:51.295117 20754 consensus_queue.cc:1048] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:06:51.285843 20749 consensus_queue.cc:1048] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.097s
I20260709 13:06:51.298380 20748 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.350581 20748 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:51.351051 16006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 2274) took 1166 ms. Trace:
I20260709 13:06:51.352706 20748 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:51.299104 20716 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:51.278988 20687 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:51.353600 20687 raft_consensus.cc:697] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:51.303097 15974 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:51.304565 20753 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:06:51.269465 20728 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:51.307610 20704 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:51.307284 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:51.293937 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:51.236068 20188 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:51.267047 20515 consensus_queue.cc:1059] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.028s
W20260709 13:06:51.325904 20642 consensus_meta.cc:287] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.503s	user 0.008s	sys 0.000s
I20260709 13:06:51.293910 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:51.333824 16005 rpcz_store.cc:276] 0709 13:06:50.187405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:50.187580 (+   175us) service_pool.cc:224] Handling call
0709 13:06:50.194945 (+  7365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:50.983787 (+788842us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:51.250738 (+266951us) spinlock_profiling.cc:243] Waited 675 ms on lock 0x7b54002e1b54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:51.250971 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1552658176}
I20260709 13:06:51.367211 16008 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:51.337432 20686 consensus_meta.cc:287] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.605s	user 0.004s	sys 0.005s
W20260709 13:06:51.337779 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 110691us
I20260709 13:06:51.342440 20734 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:51.354873 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 51818.7us
I20260709 13:06:51.357174 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:51.351500 16006 rpcz_store.cc:276] 0709 13:06:50.185067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:50.185242 (+   175us) service_pool.cc:224] Handling call
0709 13:06:50.185604 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:50.186090 (+   486us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:50.186129 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:50.186656 (+   527us) log.cc:844] Serialized 26 byte log entry
0709 13:06:50.375005 (+188349us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:50.375347 (+   342us) log.cc:844] Serialized 14 byte log entry
0709 13:06:51.350529 (+975182us) spinlock_profiling.cc:243] Waited 1.66 ms on lock 0x7b1800417880. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e9f504bc 00007f73e729e48b 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72b8455 00007f73ea3a0060 00007f73ea3b2716 00007f73ea3c3287
0709 13:06:51.350660 (+   131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:51.350731 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:51.350776 (+    45us) raft_consensus.cc:1685] finished
0709 13:06:51.350871 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:51.350994 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":5924,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":23538,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":29990,"spinlock_wait_cycles":3827968,"thread_start_us":163244,"threads_started":2,"wal-append.queue_time_us":238888}
I20260709 13:06:51.379966 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:51.381038 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:51.343335 20676 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:51.381912 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:51.372134 15860 raft_consensus.cc:1275] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
W20260709 13:06:51.375716 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:51.385933 20753 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
I20260709 13:06:51.390429 15827 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:51.391583 15827 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:51.391898 20748 consensus_queue.cc:1059] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:51.393261 15827 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:51.408336 20556 raft_consensus.cc:3055] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:51.393769 20777 consensus_queue.cc:1059] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:51.408982 15827 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:51.409360 20777 raft_consensus.cc:3055] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:06:51.367090 20690 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.966s	user 0.011s	sys 0.006s
I20260709 13:06:51.409844 20777 raft_consensus.cc:740] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:06:51.410344 20690 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.026s	user 0.011s	sys 0.006s
I20260709 13:06:51.409425 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:51.410885 20777 consensus_queue.cc:260] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:51.413187 20777 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:51.413574 15935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53636 (request call id 960) took 1021 ms. Trace:
I20260709 13:06:51.413952 15935 rpcz_store.cc:276] 0709 13:06:50.392680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:50.392819 (+   139us) service_pool.cc:224] Handling call
0709 13:06:51.413423 (+1020604us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b54002c3154. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:51.413524 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":408228096}
I20260709 13:06:51.414119 20729 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:51.415232 20648 raft_consensus.cc:3055] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:51.395704 20704 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.415930 20704 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:51.418103 20704 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:51.396097 20632 consensus_queue.cc:1059] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:51.418973 20632 raft_consensus.cc:3055] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:51.419317 20632 raft_consensus.cc:740] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:51.420269 20729 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:51.420182 20632 consensus_queue.cc:260] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:51.421352 20632 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:51.422849 20648 raft_consensus.cc:740] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:51.399222 20744 consensus_queue.cc:1059] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:06:51.423800 20648 consensus_queue.cc:260] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:51.401343 20620 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:51.402058 15900 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:51.424757 20648 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:51.425722 15900 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:51.402891 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:06:51.426620 20769 log.cc:927] Time spent T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.128s	user 0.000s	sys 0.002s
I20260709 13:06:51.362651 20641 raft_consensus.cc:2749] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:51.427119 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:51.364813 20687 consensus_queue.cc:237] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:51.428109 20737 log.cc:927] Time spent T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.130s	user 0.000s	sys 0.003s
I20260709 13:06:51.428488 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:51.429611 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:06:51.430004 20755 log.cc:927] Time spent T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.168s	user 0.000s	sys 0.002s
I20260709 13:06:51.406162 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:51.393867 20699 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:51.408380 20768 raft_consensus.cc:2749] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:51.409768 20188 raft_consensus.cc:2749] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:51.412138 20690 raft_consensus.cc:3055] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:51.394539 20678 raft_consensus.cc:2749] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:51.394990 20752 consensus_queue.cc:1048] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:06:51.375765 20634 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.000s	sys 0.009s
W20260709 13:06:51.404217 20730 log.cc:927] Time spent T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.611s	user 0.003s	sys 0.000s
I20260709 13:06:51.430896 16009 raft_consensus.cc:2393] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 11: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:51.450176 16006 raft_consensus.cc:1240] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 11 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:51.430619 20556 raft_consensus.cc:740] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:06:51.451869 20764 log.cc:927] Time spent T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.185s	user 0.004s	sys 0.001s
I20260709 13:06:51.452502 20556 consensus_queue.cc:260] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:51.453891 20556 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:51.439206 20198 raft_consensus.cc:2749] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:06:51.458544 20738 log.cc:927] Time spent T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.195s	user 0.000s	sys 0.004s
I20260709 13:06:51.376720 20642 raft_consensus.cc:2749] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:51.432337 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:51.432986 15827 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:51.433770 15900 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:51.434748 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:51.407290 20734 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:51.437858 20744 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:51.438567 16007 raft_consensus.cc:1240] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:51.438367 20503 consensus_queue.cc:1048] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:51.438943 16005 raft_consensus.cc:1240] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:06:51.477900 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:51.444398 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:51.447023 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 888) took 1022 ms. Trace:
I20260709 13:06:51.449331 16010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 2289) took 1030 ms. Trace:
I20260709 13:06:51.479662 16002 rpcz_store.cc:276] 0709 13:06:50.424218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:50.439326 (+ 15108us) service_pool.cc:224] Handling call
0709 13:06:50.457283 (+ 17957us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:50.458207 (+   924us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:51.446833 (+988626us) spinlock_profiling.cc:243] Waited 17.6 ms on lock 0x7b5400031450. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:51.446971 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40579328}
W20260709 13:06:51.477933 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 45050.3us
I20260709 13:06:51.481900 15827 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:51.485383 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:51.485889 15932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 958) took 1097 ms. Trace:
I20260709 13:06:51.486477 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:51.486835 16004 raft_consensus.cc:1275] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:06:51.486261 15932 rpcz_store.cc:276] 0709 13:06:50.388312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:50.388499 (+   187us) service_pool.cc:224] Handling call
0709 13:06:50.404111 (+ 15612us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:50.418889 (+ 14778us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:51.479926 (+1061037us) spinlock_profiling.cc:243] Waited 15.3 ms on lock 0x7b5400197b50. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:51.485830 (+  5904us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35145984}
W20260709 13:06:51.450645 20634 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.000s	sys 0.009s
I20260709 13:06:51.475972 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:51.488224 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:51.492784 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:51.486033 16002 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
I20260709 13:06:51.496193 15861 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:51.496788 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:51.443804 20690 raft_consensus.cc:740] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:06:51.464020 20685 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.002s	sys 0.001s
I20260709 13:06:51.489005 20707 consensus_queue.cc:1048] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 13:06:51.512457 20188 consensus_queue.cc:1059] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:51.512696 16011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 2284) took 1312 ms. Trace:
I20260709 13:06:51.486138 20699 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:51.513240 20188 consensus_queue.cc:1243] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer 85e997ba5003439781ec4f1f8db0c3ee log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:06:51.496433 20682 raft_consensus.cc:3055] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:51.513165 16011 rpcz_store.cc:276] 0709 13:06:50.200127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:50.216849 (+ 16722us) service_pool.cc:224] Handling call
0709 13:06:50.217182 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:50.217470 (+   288us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:06:50.448430 (+230960us) log.cc:844] Serialized 14 byte log entry
0709 13:06:50.697226 (+248796us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:50.714710 (+ 17484us) log.cc:844] Serialized 26 byte log entry
0709 13:06:50.714840 (+   130us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:50.714869 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:50.714925 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:51.403611 (+688686us) spinlock_profiling.cc:243] Waited 2.49 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:06:51.512418 (+108807us) raft_consensus.cc:1685] finished
0709 13:06:51.512525 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:51.512639 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":119965,"prepare.queue_time_us":78585,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":70663,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":24758,"spinlock_wait_cycles":5720192,"thread_start_us":312918,"threads_started":2,"wal-append.queue_time_us":297499}
I20260709 13:06:51.514899 20682 raft_consensus.cc:740] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:06:51.515450 20760 log.cc:927] Time spent T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.243s	user 0.002s	sys 0.000s
I20260709 13:06:51.515823 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:51.516571 16011 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:06:51.485620 20768 raft_consensus.cc:2749] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:51.504845 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:06:51.501086 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:51.517695 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:51.479786 16010 rpcz_store.cc:276] 0709 13:06:50.419123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:50.419400 (+   277us) service_pool.cc:224] Handling call
0709 13:06:50.449655 (+ 30255us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:50.802212 (+352557us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:50.802321 (+   109us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:50.846932 (+ 44611us) log.cc:844] Serialized 52 byte log entry
0709 13:06:51.058582 (+211650us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:51.114878 (+ 56296us) log.cc:844] Serialized 14 byte log entry
0709 13:06:51.125510 (+ 10632us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:51.125608 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:51.429447 (+303839us) spinlock_profiling.cc:243] Waited 49.5 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:06:51.439803 (+ 10356us) raft_consensus.cc:1685] finished
0709 13:06:51.449122 (+  9319us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:51.449279 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14224,"prepare.queue_time_us":56774,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":69504,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":41455,"spinlock_wait_cycles":113882496,"thread_start_us":415108,"threads_started":2,"wal-append.queue_time_us":294750}
W20260709 13:06:51.520485 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 303668us
I20260709 13:06:51.467646 16008 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
W20260709 13:06:51.531626 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 97917.5us
I20260709 13:06:51.534976 15861 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
W20260709 13:06:51.517107 20685 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.003s	sys 0.001s
I20260709 13:06:51.536530 15902 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:51.540159 16535 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:51.544116 15860 raft_consensus.cc:1275] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:06:51.539498 15901 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.015s	sys 0.012s
W20260709 13:06:51.548988 15901 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.015s	sys 0.012s
I20260709 13:06:51.549364 15859 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:06:51.548573 15932 raft_consensus.cc:1275] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:06:51.549652 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:51.550027 16535 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:51.550580 15935 raft_consensus.cc:1240] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 14 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:51.553331 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:51.558086 16535 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:51.558477 16006 raft_consensus.cc:1275] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:06:51.499393 20774 consensus_queue.cc:1048] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.083s
I20260709 13:06:51.550154 16567 raft_consensus.cc:1275] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:06:51.553175 16533 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:51.559031 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:51.561995 20369 raft_consensus.cc:2764] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:06:51.516719 20758 consensus_queue.cc:1059] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.074s
I20260709 13:06:51.562296 15902 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:51.540758 20682 consensus_queue.cc:260] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:51.563282 20682 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:51.565114 20773 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:51.565554 20773 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.565869 20773 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:51.506990 20690 consensus_queue.cc:260] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:51.570242 20690 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:51.571234 20753 raft_consensus.cc:993] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea: : Instructing follower eb1e0493af1e46dbb6dec7e25525bab5 to start an election
I20260709 13:06:51.571882 20464 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 14
W20260709 13:06:51.573328 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:51.573819 20703 consensus_queue.cc:1175] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "85e997ba5003439781ec4f1f8db0c3ee" responder_term: 14 status { last_received { term: 9 index: 9 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 13. Current term is 14. Ops: [12.10-13.11]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:06:51.575452 20693 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:51.575917 20693 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.576247 20693 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:51.575934 20753 raft_consensus.cc:1081] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 12 LEADER]: Signalling peer eb1e0493af1e46dbb6dec7e25525bab5 to start an election
I20260709 13:06:51.578271 20631 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:51.535866 20620 raft_consensus.cc:2804] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:51.579593 20620 raft_consensus.cc:697] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 9 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:51.580494 20620 consensus_queue.cc:237] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:51.555385 20437 raft_consensus.cc:2804] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:51.585523 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:51.582628 20773 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:51.562155 20198 raft_consensus.cc:2804] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:06:51.608335 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:51.579603 20745 log.cc:927] Time spent T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.314s	user 0.000s	sys 0.002s
I20260709 13:06:51.611588 15935 raft_consensus.cc:1275] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:06:51.611778 15900 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:51.590981 20437 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:51.599910 20703 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:06:51.583549 20499 consensus_queue.cc:1175] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8430db80a39d4868bcf7641c16de35ea" responder_term: 12 status { last_received { term: 11 index: 10 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 11. Current term is 12. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:06:51.617658 20687 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.559525 20503 consensus_queue.cc:1059] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 13:06:51.583360 20726 consensus_queue.cc:1059] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:06:51.636154 20437 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:51.627159 20716 consensus_queue.cc:1048] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:06:51.627506 20758 consensus_queue.cc:1048] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 13:06:51.684336 15827 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:51.700320 15827 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:51.692837 20625 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:51.699154 20677 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:06:51.752605 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:51.761478 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 61249.9us
I20260709 13:06:51.701768 20188 consensus_queue.cc:1059] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
W20260709 13:06:51.758322 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 146676us
I20260709 13:06:51.716970 20515 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:51.666759 20687 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:51.753948 20704 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:51.780524 20704 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:51.782766 20704 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:51.795840 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:51.796507 15860 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
I20260709 13:06:51.715313 20752 consensus_queue.cc:1048] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:06:51.797309 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:51.717760 20768 consensus_queue.cc:1059] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.057s
W20260709 13:06:51.815197 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 179220us
I20260709 13:06:51.751176 20198 raft_consensus.cc:697] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 10 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:51.844811 20198 consensus_queue.cc:237] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:51.761076 20749 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:51.848490 20749 raft_consensus.cc:697] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:51.850288 20749 consensus_queue.cc:237] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:51.853355 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 88299.4us
I20260709 13:06:51.764432 20625 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:51.898567 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:51.899168 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:51.899780 15930 raft_consensus.cc:1275] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:06:51.900290 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:51.901046 15932 raft_consensus.cc:1813] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Handling vote request from an unknown peer 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:51.901784 15932 raft_consensus.cc:2410] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:06:51.903895 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:51.904212 15930 raft_consensus.cc:3055] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:51.904541 15930 raft_consensus.cc:740] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:51.899744 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:51.906769 15933 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:06:51.908568 15935 raft_consensus.cc:2393] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 10: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:51.913199 15859 raft_consensus.cc:1275] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:06:51.905406 15930 consensus_queue.cc:260] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:51.886679 20796 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.078s
W20260709 13:06:51.943650 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 47071.5us
W20260709 13:06:51.928001 20787 log.cc:927] Time spent T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 13:06:51.943776 20797 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:51.976430 20797 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:51.977981 20625 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:51.978670 20437 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:51.979941 16567 raft_consensus.cc:1240] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:51.983945 20798 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:51.984786 20798 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:51.924929 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 50412.2us
I20260709 13:06:51.967383 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:51.965548 15928 raft_consensus.cc:1275] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 13:06:51.992033 15928 pending_rounds.cc:85] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 7
I20260709 13:06:51.992376 15928 pending_rounds.cc:107] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 13:06:51.992704 15928 raft_consensus.cc:2889] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:51.983243 20474 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.987412 20798 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:52.008174 15974 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:52.048202 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:52.049098 15856 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
I20260709 13:06:52.052399 20767 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.052635 20626 raft_consensus.cc:2804] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:51.963008 20648 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:51.945634 20729 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:51.931078 20748 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:52.000281 20775 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:52.080186 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 72133us
I20260709 13:06:52.007367 20634 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.053217 20626 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:51.986199 20693 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:52.083467 15974 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:52.052969 20767 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:52.107336 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:52.123124 16569 raft_consensus.cc:1240] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:52.083931 20648 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:52.078596 20797 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:52.128410 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:52.045243 20474 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:52.044579 20723 consensus_queue.cc:1048] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.099s
W20260709 13:06:52.084872 20620 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.004s	sys 0.021s
I20260709 13:06:52.087643 20729 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:52.088922 20626 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:52.438284 15932 raft_consensus.cc:1275] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 13:06:52.584887 15932 pending_rounds.cc:85] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 7
I20260709 13:06:52.585387 15932 pending_rounds.cc:107] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:06:52.585772 15932 raft_consensus.cc:2889] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:52.433221 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:06:52.558456 15828 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.014s	sys 0.039s
W20260709 13:06:52.591048 15828 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.014s	sys 0.039s
I20260709 13:06:52.516832 20721 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:06:52.533069 20744 consensus_meta.cc:287] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 1.038s	user 0.011s	sys 0.005s
W20260709 13:06:52.489055 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:52.525665 20767 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:52.250706 20792 log.cc:927] Time spent T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.172s	user 0.000s	sys 0.000s
W20260709 13:06:52.229511 20805 log.cc:927] Time spent T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.183s	user 0.000s	sys 0.000s
I20260709 13:06:52.581737 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:52.111558 20631 raft_consensus.cc:2749] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:06:52.230273 20806 log.cc:927] Time spent T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:06:52.591734 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:52.642970 16010 raft_consensus.cc:1240] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 11. Current term is 12. Ops: []
W20260709 13:06:52.642913 15857 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 2269) took 2557 ms (client timeout 1798 ms). Trace:
I20260709 13:06:52.090764 20775 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:52.582379 15855 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.005s	sys 0.005s
I20260709 13:06:52.648799 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:52.646381 15857 rpcz_store.cc:269] 0709 13:06:50.075893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:50.076023 (+   130us) service_pool.cc:224] Handling call
0709 13:06:52.633221 (+2557198us) spinlock_profiling.cc:243] Waited 501 ms on lock 0x7b540008cd54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:52.633345 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1151385472}
I20260709 13:06:52.272190 16570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1025) took 1763 ms. Trace:
I20260709 13:06:52.649819 16570 rpcz_store.cc:276] 0709 13:06:50.508372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:50.508512 (+   140us) service_pool.cc:224] Handling call
0709 13:06:50.509439 (+   927us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:50.696531 (+187092us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:52.271964 (+1575433us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b5400746154. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:52.272130 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":409993088}
I20260709 13:06:52.252959 15858 raft_consensus.cc:3055] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:52.651010 15858 raft_consensus.cc:740] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
W20260709 13:06:52.635262 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 551825us
I20260709 13:06:52.636821 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:52.524763 20648 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:52.515746 20641 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:52.639197 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:52.202787 20810 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:52.654896 16007 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
I20260709 13:06:52.332415 20437 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:52.404693 20188 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:52.088025 20748 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:52.584478 20777 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:52.100545 20634 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:52.630215 15827 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.724s	user 0.001s	sys 0.009s
W20260709 13:06:52.662108 15827 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.002s	sys 0.009s
I20260709 13:06:52.251621 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:52.651970 15853 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:06:52.663767 16009 raft_consensus.cc:1240] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 10 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 9. Current term is 10. Ops: [9.5-9.5]
W20260709 13:06:52.648933 15855 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.759s	user 0.005s	sys 0.006s
I20260709 13:06:52.664548 15853 raft_consensus.cc:1240] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 13. Current term is 14. Ops: [13.13-13.13]
I20260709 13:06:52.665824 15855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 2195) took 1120 ms. Trace:
W20260709 13:06:52.648377 20556 consensus_meta.cc:287] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 1.194s	user 0.019s	sys 0.002s
I20260709 13:06:52.173204 20758 consensus_queue.cc:1059] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.470s
I20260709 13:06:52.666493 20677 raft_consensus.cc:3055] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:06:52.666878 20677 raft_consensus.cc:740] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:52.632117 20753 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.667450 20753 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:52.667611 20677 consensus_queue.cc:260] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:52.668609 20729 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:52.669602 20753 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:52.657970 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:52.671270 15859 raft_consensus.cc:1275] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:06:52.632925 20721 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:52.640919 20821 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:52.652081 15974 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:52.674655 20821 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:52.675346 15974 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:52.676793 20821 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:52.658655 16011 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:06:52.661077 15860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 2200) took 1113 ms. Trace:
I20260709 13:06:52.660830 20810 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:52.674927 20826 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.638227 20744 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:06:52.678596 15860 rpcz_store.cc:276] 0709 13:06:51.547387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.798111 (+250724us) service_pool.cc:224] Handling call
0709 13:06:51.799464 (+  1353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:51.799799 (+   335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:51.799831 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:51.800210 (+   379us) log.cc:844] Serialized 26 byte log entry
0709 13:06:51.907280 (+107070us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:51.907318 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:51.907395 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:52.660887 (+753492us) raft_consensus.cc:1685] finished
0709 13:06:52.660907 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:52.661022 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4621,"raft.queue_time_us":2827,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":12284,"thread_start_us":106082,"threads_started":1,"wal-append.queue_time_us":250866}
I20260709 13:06:52.684600 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:52.685185 15860 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
W20260709 13:06:52.681814 15976 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.003s	sys 0.022s
W20260709 13:06:52.686018 15976 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.004s	sys 0.023s
I20260709 13:06:52.687203 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:52.657119 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:52.694801 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:52.695377 15857 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
I20260709 13:06:52.690538 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:52.688144 15855 rpcz_store.cc:276] 0709 13:06:51.545434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.545562 (+   128us) service_pool.cc:224] Handling call
0709 13:06:51.570691 (+ 25129us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:51.679423 (+108732us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:51.679450 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:51.679955 (+   505us) log.cc:844] Serialized 26 byte log entry
0709 13:06:51.882849 (+202894us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:51.890109 (+  7260us) log.cc:844] Serialized 14 byte log entry
0709 13:06:52.664363 (+774254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:52.665508 (+  1145us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b54004263d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:52.665559 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:52.665601 (+    42us) raft_consensus.cc:1685] finished
0709 13:06:52.665671 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:52.665779 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39145,"prepare.queue_time_us":8469,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":44,"raft.queue_time_us":130175,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":12331,"spinlock_wait_cycles":249284992,"thread_start_us":976457,"threads_started":2,"wal-append.queue_time_us":276925}
I20260709 13:06:52.654748 15858 consensus_queue.cc:260] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:52.682190 20631 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:52.699211 15901 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.716s	user 0.048s	sys 0.077s
I20260709 13:06:52.684484 20723 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:06:52.703317 15901 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.048s	sys 0.078s
I20260709 13:06:52.658763 20641 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:52.715857 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:52.704939 15858 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:52.663144 20777 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:52.693744 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 564253us
I20260709 13:06:52.712452 15931 raft_consensus.cc:1240] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:06:52.723789 15931 raft_consensus.cc:1240] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:52.707285 15974 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:52.704739 15827 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:52.720048 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:52.713181 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:52.694862 20749 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:52.747104 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:52.703579 20642 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:52.661650 20188 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:52.746217 15860 raft_consensus.cc:1275] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:06:52.702841 20734 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:52.711266 20649 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:52.752094 20824 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:52.763365 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 58739.2us
I20260709 13:06:52.766186 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:52.686990 20826 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:52.789170 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 81972.4us
I20260709 13:06:52.757799 20723 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:52.797295 16008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 901) took 1325 ms. Trace:
I20260709 13:06:52.761528 20814 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.797857 16008 rpcz_store.cc:276] 0709 13:06:51.471466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.533732 (+ 62266us) service_pool.cc:224] Handling call
0709 13:06:51.534005 (+   273us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:51.539838 (+  5833us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:51.539868 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:51.556277 (+ 16409us) log.cc:844] Serialized 52 byte log entry
0709 13:06:51.716252 (+159975us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:51.755058 (+ 38806us) log.cc:844] Serialized 14 byte log entry
0709 13:06:51.755264 (+   206us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:51.755371 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:52.797037 (+1041666us) raft_consensus.cc:1685] finished
0709 13:06:52.797066 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:52.797234 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5088,"prepare.queue_time_us":39539,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":6890,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":3705,"spinlock_wait_cycles":8784000,"thread_start_us":94521,"threads_started":1,"wal-append.queue_time_us":356358}
I20260709 13:06:52.852329 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:52.853351 15853 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 13.
W20260709 13:06:52.860440 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 72959.8us
I20260709 13:06:52.754457 20749 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:52.861621 15900 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:52.854141 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:52.761245 20649 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:52.903623 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 138252us
I20260709 13:06:52.903369 15853 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 13.
I20260709 13:06:52.926028 15900 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:52.927006 20620 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:52.927505 20620 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.928437 20620 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:52.758160 20642 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:52.759562 20734 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:06:52.769593 20824 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:52.968858 15825 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:52.863850 20772 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.853770 20817 raft_consensus.cc:2749] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:52.867265 20474 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:52.866160 20198 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:52.994030 20775 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:52.857677 20814 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:53.008230 20821 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:53.088994 20822 log.cc:927] Time spent T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.335s	user 0.000s	sys 0.001s
I20260709 13:06:52.821380 20723 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:53.033375 20198 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.052642 20556 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:53.120191 15853 raft_consensus.cc:1275] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:06:53.137003 15853 raft_consensus.cc:1240] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:06:53.139861 20830 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.139981 20620 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:53.140466 20830 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:53.142187 20620 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:53.142441 20830 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:53.122740 20833 raft_consensus.cc:2749] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:53.122057 20821 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:53.144148 20634 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:53.144395 20834 log.cc:927] Time spent T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
W20260709 13:06:53.145599 20831 log.cc:927] Time spent T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.260s	user 0.002s	sys 0.000s
W20260709 13:06:53.132256 20840 log.cc:927] Time spent T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.062s	user 0.001s	sys 0.001s
I20260709 13:06:53.148168 15933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 2145) took 1243 ms. Trace:
W20260709 13:06:53.148224 20829 log.cc:927] Time spent T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.180s	user 0.002s	sys 0.000s
I20260709 13:06:53.148671 15933 rpcz_store.cc:276] 0709 13:06:51.905053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.907559 (+  2506us) service_pool.cc:224] Handling call
0709 13:06:52.672095 (+764536us) spinlock_profiling.cc:243] Waited 5.64 ms on lock 0x7b5800003698. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef381fc0 00007f73ef36c929 00007f73ef36c3d2 00007f73e88e3bd4
0709 13:06:52.672242 (+   147us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:52.672592 (+   350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:52.672616 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:52.673114 (+   498us) log.cc:844] Serialized 26 byte log entry
0709 13:06:52.761811 (+ 88697us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:52.761864 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:52.761978 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:53.147854 (+385876us) raft_consensus.cc:1685] finished
0709 13:06:53.147873 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:53.148088 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":368,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":77,"spinlock_wait_cycles":12972800,"thread_start_us":88271,"threads_started":1,"wal-append.queue_time_us":231333}
I20260709 13:06:53.150007 20641 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:53.132097 20198 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:53.152881 20649 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:53.153402 20649 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.153838 20649 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:53.008834 20772 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:53.158179 16007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 2299) took 1320 ms. Trace:
I20260709 13:06:53.161410 16007 rpcz_store.cc:276] 0709 13:06:51.837896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:52.656800 (+818904us) service_pool.cc:224] Handling call
0709 13:06:52.657121 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:52.657349 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:06:52.790837 (+133488us) log.cc:844] Serialized 14 byte log entry
0709 13:06:53.157855 (+367018us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:53.157907 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:53.157947 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:53.158005 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:53.158103 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":344566,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"thread_start_us":499823,"threads_started":3}
I20260709 13:06:53.168754 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:53.149269 20642 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:53.190052 20642 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.190562 20642 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:53.191619 20626 consensus_meta.cc:287] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.593s	user 0.005s	sys 0.005s
I20260709 13:06:53.192114 20626 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:53.194249 20626 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:53.117863 20825 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:53.200116 20854 consensus_queue.cc:1059] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:06:53.201709 20819 log.cc:927] Time spent T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.392s	user 0.002s	sys 0.000s
I20260709 13:06:53.203056 16004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 909) took 1665 ms. Trace:
I20260709 13:06:53.203496 16004 rpcz_store.cc:276] 0709 13:06:51.537372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.662009 (+124637us) service_pool.cc:224] Handling call
0709 13:06:51.662363 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:51.689076 (+ 26713us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:51.689119 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:51.713109 (+ 23990us) log.cc:844] Serialized 26 byte log entry
0709 13:06:52.082006 (+368897us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:52.082046 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:52.082119 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:53.202851 (+1120732us) raft_consensus.cc:1685] finished
0709 13:06:53.202872 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:53.203001 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10633,"raft.queue_time_us":26886,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":7061,"thread_start_us":327328,"threads_started":1,"wal-append.queue_time_us":967292}
I20260709 13:06:53.205621 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:06:53.209703 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:53.116998 20824 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:53.462989 15856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 914) took 1362 ms. Trace:
I20260709 13:06:53.463248 16007 raft_consensus.cc:1240] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 11 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:53.157312 20649 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:53.463429 15856 rpcz_store.cc:276] 0709 13:06:52.100317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:52.100500 (+   183us) service_pool.cc:224] Handling call
0709 13:06:52.100845 (+   345us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:52.211837 (+110992us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:52.211884 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:52.213178 (+  1294us) log.cc:844] Serialized 52 byte log entry
0709 13:06:52.755949 (+542771us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:52.755989 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:52.756054 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:53.462770 (+706716us) raft_consensus.cc:1685] finished
0709 13:06:53.462791 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:53.462937 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43665,"raft.queue_time_us":162891,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":79,"spinlock_wait_cycles":1303296,"thread_start_us":479837,"threads_started":1,"wal-append.queue_time_us":548166}
I20260709 13:06:53.464862 16007 raft_consensus.cc:1275] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:06:53.334429 15858 consensus_meta.cc:287] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.615s	user 0.005s	sys 0.001s
I20260709 13:06:53.465298 20642 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:53.466140 20699 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 13:06:53.465868 16007 raft_consensus.cc:1240] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:06:53.348564 20849 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:53.158502 20556 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:53.467494 20734 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:53.423383 20825 raft_consensus.cc:2749] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:06:53.466234 15858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2278) took 2178 ms. Trace:
W20260709 13:06:53.432847 15974 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.001s	sys 0.017s
I20260709 13:06:53.469007 15858 rpcz_store.cc:276] 0709 13:06:51.287665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.287907 (+   242us) service_pool.cc:224] Handling call
0709 13:06:52.252581 (+964674us) spinlock_profiling.cc:243] Waited 4.74 ms on lock 0x7b5800005798. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef381fc0 00007f73ef36c929 00007f73ef36c3d2 00007f73e88e3bd4
0709 13:06:52.252817 (+   236us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:53.465838 (+1213021us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:53.465869 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:53.465905 (+    36us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:53.465949 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:53.466044 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:53.466172 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3489,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":89,"spinlock_wait_cycles":10891136}
W20260709 13:06:53.469939 15974 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.002s	sys 0.018s
I20260709 13:06:53.209877 20626 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.225627 16009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 2303) took 1386 ms. Trace:
I20260709 13:06:53.470863 20626 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:53.348577 20772 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:53.430431 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:53.351358 20832 log.cc:927] Time spent T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:06:53.314306 16011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 2301) took 1475 ms. Trace:
W20260709 13:06:53.350348 20631 consensus_meta.cc:287] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.638s	user 0.007s	sys 0.003s
I20260709 13:06:53.307814 20837 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:53.462913 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 494129us
W20260709 13:06:53.470536 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 680552us
W20260709 13:06:53.642711 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:53.473148 20626 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:53.473845 16011 rpcz_store.cc:276] 0709 13:06:51.838569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:52.677651 (+839082us) service_pool.cc:224] Handling call
0709 13:06:52.677898 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:52.678078 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:06:52.858676 (+180598us) log.cc:844] Serialized 14 byte log entry
0709 13:06:53.313838 (+455162us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:53.313914 (+    76us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:53.313985 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:53.314073 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:53.314230 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":373252,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":149774,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":250,"thread_start_us":635009,"threads_started":2}
I20260709 13:06:53.348366 20850 raft_consensus.cc:3055] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:53.471179 16009 rpcz_store.cc:276] 0709 13:06:51.839599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:52.713369 (+873770us) service_pool.cc:224] Handling call
0709 13:06:52.713809 (+   440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:52.714099 (+   290us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:06:52.717279 (+  3180us) log.cc:844] Serialized 14 byte log entry
0709 13:06:53.225169 (+507890us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:53.225248 (+    79us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:53.225308 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:53.225397 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:53.225554 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":327637,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":47195,"thread_start_us":506148,"threads_started":2,"wal-append.queue_time_us":403575}
I20260709 13:06:53.486929 20849 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:53.486531 20814 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:53.662336 15974 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:53.352337 20723 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:53.235165 20847 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:53.369480 20198 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:53.326220 20830 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.218611 20777 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:53.470525 20556 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:53.642010 20631 raft_consensus.cc:2749] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:53.679504 15974 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:53.724287 15825 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:53.647419 20850 raft_consensus.cc:740] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:06:53.737867 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 58461.5us
I20260709 13:06:53.740401 15974 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:53.758313 15825 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:53.715492 20847 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:53.809798 15977 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.009s	sys 0.033s
W20260709 13:06:53.810395 15977 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.010s	sys 0.033s
I20260709 13:06:53.647657 20772 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.798377 15936 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 957) took 3624 ms. Trace:
I20260709 13:06:53.727162 20777 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:53.735472 20814 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.839555 15936 rpcz_store.cc:276] 0709 13:06:50.173898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:50.174210 (+   312us) service_pool.cc:224] Handling call
0709 13:06:50.174489 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:50.195493 (+ 21004us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:06:50.239844 (+ 44351us) log.cc:844] Serialized 14 byte log entry
0709 13:06:50.994858 (+755014us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:50.994909 (+    51us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:50.994954 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:53.798098 (+2803144us) spinlock_profiling.cc:243] Waited 20.8 ms on lock 0x7b540019c3d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:53.798154 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:53.798314 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24730,"prepare.queue_time_us":97489,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":24557,"raft.queue_time_us":23425,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":11376,"spinlock_wait_cycles":47875456,"thread_start_us":737180,"threads_started":2,"wal-append.queue_time_us":270915,"wal-append.run_cpu_time_us":1097,"wal-append.run_wall_time_us":1050333}
W20260709 13:06:53.793546 15976 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.000s	sys 0.043s
I20260709 13:06:53.713171 20842 consensus_queue.cc:1059] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:06:53.714344 20626 consensus_queue.cc:1048] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:06:53.645115 20837 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:53.769395 15974 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:06:53.863263 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 93953.7us
I20260709 13:06:53.722426 20830 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:53.797442 20847 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:53.868605 15974 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:53.869781 15974 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
W20260709 13:06:53.852237 15976 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.000s	sys 0.044s
I20260709 13:06:53.801184 20198 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:53.774650 20841 raft_consensus.cc:2749] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:53.735211 20853 consensus_queue.cc:1059] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.115s
W20260709 13:06:53.872555 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 714505us
I20260709 13:06:53.874548 15974 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:53.856276 20649 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:53.879536 16007 raft_consensus.cc:1240] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
W20260709 13:06:53.886497 16008 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.845s	user 0.035s	sys 0.069s
I20260709 13:06:53.851610 20772 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:53.886837 16008 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.881s	user 0.035s	sys 0.069s
I20260709 13:06:53.902835 16008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 2304) took 2062 ms. Trace:
I20260709 13:06:53.840884 20723 consensus_queue.cc:1048] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:06:53.862407 20862 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:53.903335 16008 rpcz_store.cc:276] 0709 13:06:51.840195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:52.806428 (+966233us) service_pool.cc:224] Handling call
0709 13:06:52.806742 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:52.806945 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:52.940453 (+133508us) log.cc:844] Serialized 14 byte log entry
0709 13:06:53.902351 (+961898us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:53.902428 (+    77us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:53.902496 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:53.902588 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:53.902763 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45903,"prepare.queue_time_us":283308,"prepare.run_cpu_time_us":876,"prepare.run_wall_time_us":26954,"raft.queue_time_us":183976,"raft.run_cpu_time_us":60,"raft.run_wall_time_us":60,"thread_start_us":1094622,"threads_started":3,"wal-append.queue_time_us":255844}
I20260709 13:06:53.871372 20825 raft_consensus.cc:2804] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:53.877182 20631 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:53.878592 20847 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:53.857981 20777 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:53.860460 20814 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:53.873483 20734 consensus_queue.cc:1048] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:06:53.924408 20855 raft_consensus.cc:2804] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:53.913067 20631 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:53.912035 20825 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:53.914776 20847 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:53.888682 20649 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:53.931979 20631 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:53.952078 20863 consensus_queue.cc:1048] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 13:06:53.953553 20866 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:53.954095 20866 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:53.933902 20847 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:53.931612 20855 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:53.912685 20862 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:53.933113 20825 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:54.334921 20830 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:54.202943 20772 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:54.335613 20847 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:54.338090 20847 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:53.937057 20852 raft_consensus.cc:2749] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:53.981010 20849 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:53.935664 20860 raft_consensus.cc:2749] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:53.980129 20814 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:53.956248 20866 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:53.978343 20777 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:53.984124 20837 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:54.305966 20855 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:53.960880 20857 consensus_queue.cc:1048] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.575s
I20260709 13:06:53.960307 20859 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:54.343240 20862 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:54.383826 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:54.387508 15825 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.000s	sys 0.008s
W20260709 13:06:54.388080 15825 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.000s	sys 0.008s
W20260709 13:06:54.388583 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 626173us
W20260709 13:06:54.408723 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:54.410704 15853 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.196s	user 0.009s	sys 0.027s
W20260709 13:06:54.411425 15853 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.197s	user 0.009s	sys 0.027s
I20260709 13:06:54.413375 15853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 2211) took 1647 ms. Trace:
I20260709 13:06:54.413846 15853 rpcz_store.cc:276] 0709 13:06:52.766314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.158797 (+392483us) service_pool.cc:224] Handling call
0709 13:06:53.159457 (+   660us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:53.162946 (+  3489us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:53.162986 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:53.214453 (+ 51467us) log.cc:844] Serialized 556 byte log entry
0709 13:06:54.412434 (+1197981us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:54.412561 (+   127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:54.412760 (+   199us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:54.412873 (+   113us) raft_consensus.cc:1685] finished
0709 13:06:54.412906 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:54.413230 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":1197140,"threads_started":1,"wal-append.queue_time_us":783627}
W20260709 13:06:54.432832 15900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.696s	user 0.014s	sys 0.019s
W20260709 13:06:54.433465 15900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.015s	sys 0.019s
W20260709 13:06:54.442775 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:54.453584 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:54.453296 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:54.455374 15932 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
I20260709 13:06:54.453289 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:54.469061 15928 raft_consensus.cc:1275] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:06:54.469565 15928 pending_rounds.cc:85] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 8
I20260709 13:06:54.469872 15928 pending_rounds.cc:107] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 10.9
I20260709 13:06:54.470407 15928 raft_consensus.cc:2917] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Aborting config change with OpId 10.9: Aborted: Op aborted by new leader
I20260709 13:06:54.471354 15928 raft_consensus.cc:2889] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:54.472185 15928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:58226 (request call id 8) took 6042 ms. Trace:
I20260709 13:06:54.472594 15928 rpcz_store.cc:276] 0709 13:06:48.429514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.431079 (+  1565us) service_pool.cc:224] Handling call
0709 13:06:48.585476 (+154397us) spinlock_profiling.cc:243] Waited 20.8 ms on lock 0x7b580069b9d0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3040aa 00007f73ea386447 00007f73ea3a370a 00007f73ea3a476a 00007f73ea3a391b 00007f73ea3af03c 00007f73ea3ae6b3 00007f73ea3a69fa 00007f73ef36e4c9 00007f73e88eac54
0709 13:06:48.647210 (+ 61734us) log.cc:844] Serialized 494 byte log entry
0709 13:06:54.472018 (+5824808us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":250,"mutex_wait_us":8926,"raft.queue_time_us":4235,"raft.run_cpu_time_us":1407,"raft.run_wall_time_us":13802,"spinlock_wait_cycles":47904000}
W20260709 13:06:54.474406 15974 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.000s	sys 0.011s
W20260709 13:06:54.474906 15974 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.000s	sys 0.011s
W20260709 13:06:54.475473 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 594008us
I20260709 13:06:54.481320 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:54.482486 15853 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 10.
I20260709 13:06:54.484375 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:54.486167 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:54.489274 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:54.490826 15976 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.000s	sys 0.012s
W20260709 13:06:54.491434 15976 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.000s	sys 0.015s
I20260709 13:06:54.493243 15932 raft_consensus.cc:1240] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:54.495514 15933 raft_consensus.cc:2410] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:06:54.511109 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:54.514321 16536 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.001s	sys 0.000s
W20260709 13:06:54.514819 16536 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.002s	sys 0.000s
W20260709 13:06:54.516538 20631 consensus_meta.cc:287] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.565s	user 0.008s	sys 0.000s
I20260709 13:06:54.517256 20631 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:54.517733 20649 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:54.517946 20505 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:54.518854 20505 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:54.522240 20505 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:54.522814 20505 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.522941 20626 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:06:54.714691 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:54.523572 20505 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:54.526436 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:54.741313 20872 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.741245 20852 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:54.525990 20723 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:54.742465 20872 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:54.989842 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:54.991575 16569 raft_consensus.cc:1240] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:54.991776 20814 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:54.992702 20814 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:54.994719 16007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2111) took 2307 ms. Trace:
I20260709 13:06:54.995908 20852 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:54.998558 20852 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:55.001184 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:55.002568 15854 raft_consensus.cc:2410] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 13:06:55.002648 15744 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 83a1b79cdcf046c78453695c798a14a5 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) due to error UNKNOWN_ERROR; will retry: Aborted: Op aborted by new leader
I20260709 13:06:55.003825 20631 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:54.995121 16007 rpcz_store.cc:276] 0709 13:06:52.687113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.532780 (+845667us) service_pool.cc:224] Handling call
0709 13:06:53.533210 (+   430us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:53.884276 (+351066us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:54.994097 (+1109821us) spinlock_profiling.cc:243] Waited 346 ms on lock 0x7b540010d6d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:54.994499 (+   402us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":795890432}
I20260709 13:06:55.014686 15855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 2202) took 2340 ms. Trace:
I20260709 13:06:55.016242 20868 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:55.017638 20868 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:55.017973 20505 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:55.018885 16570 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 10.
I20260709 13:06:55.019835 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.020774 16564 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
W20260709 13:06:55.003211 15854 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 2277) took 3739 ms (client timeout 1647 ms). Trace:
W20260709 13:06:55.004719 20865 log.cc:927] Time spent T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.267s	user 0.002s	sys 0.000s
I20260709 13:06:55.009137 20540 raft_consensus.cc:3055] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:55.041608 20540 raft_consensus.cc:740] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:55.009835 20626 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:55.002513 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:55.008546 20736 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.017333 15855 rpcz_store.cc:276] 0709 13:06:52.674022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:52.751686 (+ 77664us) service_pool.cc:224] Handling call
0709 13:06:52.752491 (+   805us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:52.763459 (+ 10968us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:52.763498 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:52.766899 (+  3401us) log.cc:844] Serialized 26 byte log entry
0709 13:06:55.013544 (+2246645us) spinlock_profiling.cc:243] Waited 19.4 ms on lock 0x7b18000a7500. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e9f504bc 00007f73e729e48b 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f480d1
0709 13:06:55.013860 (+   316us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:55.013932 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:55.014035 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:55.014077 (+    42us) raft_consensus.cc:1685] finished
0709 13:06:55.014092 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:55.014448 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10528,"raft.queue_time_us":34033,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":8897,"spinlock_wait_cycles":44704512,"thread_start_us":45541,"threads_started":1,"wal-append.queue_time_us":185934,"wal-append.run_cpu_time_us":3034,"wal-append.run_wall_time_us":1179584}
I20260709 13:06:55.029335 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.030081 16563 raft_consensus.cc:1240] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:55.030910 16004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 913) took 3105 ms. Trace:
I20260709 13:06:55.031335 20851 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:06:55.033385 15900 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.783s (SENT) [suppressed 8 similar messages]
W20260709 13:06:55.036998 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:55.037449 20723 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:55.040051 16010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 899) took 3572 ms. Trace:
I20260709 13:06:55.040757 20872 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:55.021519 15854 rpcz_store.cc:269] 0709 13:06:51.263914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.272627 (+  8713us) service_pool.cc:224] Handling call
0709 13:06:55.000709 (+3728082us) spinlock_profiling.cc:243] Waited 4.81 ms on lock 0x7b5800005798. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:06:55.003113 (+  2404us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11054848}
I20260709 13:06:54.995500 20814 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:55.048583 16010 rpcz_store.cc:276] 0709 13:06:51.467625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.529316 (+ 61691us) service_pool.cc:224] Handling call
0709 13:06:51.552096 (+ 22780us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:52.646265 (+1094169us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:55.039409 (+2393144us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b5400305a54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:55.039829 (+   420us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2508165376}
I20260709 13:06:55.059772 15853 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:06:55.060165 20877 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:55.061112 20877 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:55.062150 20626 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:55.063025 20736 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:55.063766 20877 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:55.066572 20736 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:55.046881 16004 rpcz_store.cc:276] 0709 13:06:51.925286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.259728 (+1334442us) service_pool.cc:224] Handling call
0709 13:06:53.463152 (+203424us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:53.468060 (+  4908us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:53.468089 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:53.468675 (+   586us) log.cc:844] Serialized 52 byte log entry
0709 13:06:53.926039 (+457364us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:54.713840 (+787801us) log.cc:844] Serialized 14 byte log entry
0709 13:06:55.030235 (+316395us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:55.030380 (+   145us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:55.030470 (+    90us) raft_consensus.cc:1685] finished
0709 13:06:55.030487 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:55.030713 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":301035,"prepare.queue_time_us":787652,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":97,"raft.queue_time_us":310595,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":271,"thread_start_us":659330,"threads_started":2,"wal-append.queue_time_us":488571}
I20260709 13:06:55.074539 20723 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:55.075802 20842 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:55.076843 20842 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:55.052541 20852 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:55.080494 20851 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:55.081924 20852 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:55.043193 20540 consensus_queue.cc:260] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:55.090621 20540 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:55.083487 20851 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:55.123529 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:55.124913 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:55.129472 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:55.130847 20490 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:55.131582 20490 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:55.132242 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.133944 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.134608 20490 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:55.135078 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.134655 16570 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
I20260709 13:06:55.136627 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:55.145845 20785 log.cc:927] Time spent T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 3.473s	user 0.012s	sys 0.033s
I20260709 13:06:55.147042 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:06:55.147822 15977 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.022s	sys 0.017s
W20260709 13:06:55.148191 15977 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.022s	sys 0.017s
I20260709 13:06:55.148252 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.145926 16570 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEARNER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:06:55.149622 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.150732 16568 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840"
dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:45244
I20260709 13:06:55.151535 20726 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.152385 20726 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:55.154968 20726 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:55.156069 20749 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:55.158758 20852 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:55.160746 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.163296 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.164049 16564 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
I20260709 13:06:55.165616 16564 raft_consensus.cc:1275] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 13. (term mismatch)
I20260709 13:06:55.165944 16564 pending_rounds.cc:85] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting all ops after (but not including) 12
I20260709 13:06:55.166198 16564 pending_rounds.cc:107] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting uncommitted NO_OP operation due to leader change: 13.13
I20260709 13:06:55.166415 16564 raft_consensus.cc:2889] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:55.137437 16569 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
W20260709 13:06:55.169556 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 136203us
I20260709 13:06:55.162405 16566 raft_consensus.cc:1240] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:55.167431 20748 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:55.173897 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.123253 20842 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:55.173312 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.185674 16569 raft_consensus.cc:1240] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:55.186810 20881 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:55.186825 20884 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:55.187352 20868 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:55.187712 20881 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:06:55.188180 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:55.187867 20884 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:55.189246 15937 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 2147) took 3281 ms (client timeout 1738 ms). Trace:
W20260709 13:06:55.189666 15937 rpcz_store.cc:269] 0709 13:06:51.908077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.908282 (+   205us) service_pool.cc:224] Handling call
0709 13:06:55.187848 (+3279566us) spinlock_profiling.cc:243] Waited 5.88 ms on lock 0x7b5800003698. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:06:55.189136 (+  1288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13524224}
I20260709 13:06:55.190256 20884 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:55.190840 20884 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:55.195551 20825 consensus_meta.cc:287] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.860s	user 0.008s	sys 0.008s
I20260709 13:06:55.195684 20851 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:55.195956 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:55.196221 20825 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:55.196686 15853 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
I20260709 13:06:55.196530 20851 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:55.198045 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:55.199245 20851 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:55.201550 20816 log.cc:927] Time spent T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 2.970s	user 0.000s	sys 0.047s
I20260709 13:06:55.199307 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:55.155555 20726 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:55.196364 15936 raft_consensus.cc:1275] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:06:55.214186 16563 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:06:55.199245 20825 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:55.190263 20881 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:55.225201 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:55.216676 16533 connection.cc:256] RPC call timeout handler was delayed by -2.7106801280000354s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1723s for a response
W20260709 13:06:55.226706 16533 connection.cc:256] RPC call timeout handler was delayed by -2.6980801280000151s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17370000000000002s for a response
W20260709 13:06:55.227092 16533 connection.cc:256] RPC call timeout handler was delayed by -2.662980128000072s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1776s for a response
I20260709 13:06:55.227980 15976 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:55.228909 20682 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.229018 20723 consensus_queue.cc:1059] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:06:55.231137 20786 log.cc:927] Time spent T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 3.578s	user 0.013s	sys 0.029s
I20260709 13:06:55.231055 20734 consensus_queue.cc:1059] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:55.229764 20726 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:55.232990 16568 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:55.191501 20884 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:55.344020 16568 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:06:55.323320 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 235958us
I20260709 13:06:55.343523 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:55.229597 20877 raft_consensus.cc:2804] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:06:55.260511 20855 consensus_meta.cc:287] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.915s	user 0.006s	sys 0.014s
I20260709 13:06:55.233245 20703 raft_consensus.cc:3055] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:55.346603 20703 raft_consensus.cc:740] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:55.262502 15928 raft_consensus.cc:1240] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:06:55.276196 20814 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:55.348567 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:55.349435 15928 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:06:55.349800 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:55.350558 15974 connection.cc:256] RPC call timeout handler was delayed by -1.3323545859999513s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1738s for a response
I20260709 13:06:55.278421 20821 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:55.228037 16533 connection.cc:256] RPC call timeout handler was delayed by -2.1399130700000342s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16690000000000002s for a response
W20260709 13:06:55.352607 16533 connection.cc:256] RPC call timeout handler was delayed by -2.1399130700000342s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16690000000000002s for a response
W20260709 13:06:55.353052 16533 connection.cc:256] RPC call timeout handler was delayed by -1.5235801280000487s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3042s for a response
W20260709 13:06:55.353490 16533 connection.cc:256] RPC call timeout handler was delayed by -1.3164130700000669s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2584s for a response
W20260709 13:06:55.354053 16533 connection.cc:256] RPC call timeout handler was delayed by -1.3164130700000669s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2584s for a response
I20260709 13:06:55.345579 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:55.348416 20814 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:55.323869 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:55.278110 16565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1040) took 3448 ms. Trace:
I20260709 13:06:55.264740 16566 raft_consensus.cc:1275] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:06:55.345537 20877 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:55.379148 20877 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:55.229625 20682 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:55.381922 20682 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:55.367367 15976 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:55.261926 20723 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:55.384253 20723 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:55.386340 15976 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:55.345561 15937 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:06:55.387458 20734 raft_consensus.cc:2804] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:55.387912 20734 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:55.388175 20825 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:55.388373 20734 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:55.388720 20825 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:55.389174 20825 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:55.346341 20855 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:55.393440 20855 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:55.372262 20693 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:55.376260 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:55.395233 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:55.395814 15854 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:55.378473 20884 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:55.276176 20833 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:55.386518 20723 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:55.386989 20703 consensus_queue.cc:260] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:55.228300 20868 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.424592 20868 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:55.364444 15853 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:06:55.265014 20882 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.406488 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:55.410118 15974 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:06:55.399513 20815 log.cc:927] Time spent T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 2.594s	user 0.036s	sys 0.048s
I20260709 13:06:55.420102 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:55.424975 20855 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:55.397827 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:55.373186 20676 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 3.742s	user 0.030s	sys 0.038s
W20260709 13:06:55.434906 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:55.427529 16568 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:55.441737 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:55.437988 15930 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:55.451105 20868 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:55.440537 20736 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.452550 16570 raft_consensus.cc:1240] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:55.426416 20726 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:55.439324 15860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 2206) took 2762 ms. Trace:
W20260709 13:06:55.437750 15935 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 2148) took 3529 ms (client timeout 2215 ms). Trace:
W20260709 13:06:55.455370 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 45648.4us
I20260709 13:06:55.452222 20736 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:55.440989 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:55.436110 20676 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 3.818s	user 0.030s	sys 0.039s
I20260709 13:06:55.451128 16533 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:55.461892 20858 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:06:55.457923 15935 rpcz_store.cc:269] 0709 13:06:51.908359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.909317 (+   958us) service_pool.cc:224] Handling call
0709 13:06:55.419686 (+3510369us) spinlock_profiling.cc:243] Waited 6.33 ms on lock 0x7b5800003698. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:06:55.437674 (+ 17988us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14568832}
I20260709 13:06:55.441552 20877 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:55.464063 16003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2104) took 3907 ms. Trace:
I20260709 13:06:55.464232 15856 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
I20260709 13:06:55.464658 20855 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:55.467345 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2105) took 3667 ms. Trace:
I20260709 13:06:55.467455 15853 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
I20260709 13:06:55.468137 20723 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:55.469060 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:55.442010 20882 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:55.441953 20752 raft_consensus.cc:3055] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:55.371344 16565 rpcz_store.cc:276] 0709 13:06:51.829978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.830113 (+   135us) service_pool.cc:224] Handling call
0709 13:06:51.830591 (+   478us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:51.920759 (+ 90168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:51.920803 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:51.921315 (+   512us) log.cc:844] Serialized 26 byte log entry
0709 13:06:51.952107 (+ 30792us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:51.952161 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:51.952248 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:55.200609 (+3248361us) spinlock_profiling.cc:243] Waited 3.28 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:06:55.216400 (+ 15791us) raft_consensus.cc:1685] finished
0709 13:06:55.216432 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:55.277921 (+ 61489us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11441,"raft.queue_time_us":59408,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":14668,"spinlock_wait_cycles":7536896,"thread_start_us":120108,"threads_started":2,"wal-append.queue_time_us":166188}
I20260709 13:06:55.456774 15860 rpcz_store.cc:276] 0709 13:06:52.676569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:52.757503 (+ 80934us) service_pool.cc:224] Handling call
0709 13:06:52.759376 (+  1873us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:52.952961 (+193585us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:52.953006 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:52.953065 (+    59us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:52.953108 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:55.438827 (+2485719us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b540008b1d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:55.438899 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:55.439128 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":444585856}
W20260709 13:06:55.480080 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 53957.4us
I20260709 13:06:55.452519 20814 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:55.473912 16003 rpcz_store.cc:276] 0709 13:06:51.557033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.687134 (+130101us) service_pool.cc:224] Handling call
0709 13:06:51.687413 (+   279us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:51.697272 (+  9859us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:51.697302 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:51.697729 (+   427us) log.cc:844] Serialized 26 byte log entry
0709 13:06:55.418399 (+3720670us) spinlock_profiling.cc:243] Waited 1.53 ms on lock 0x7b1800362320. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f480d1 00007f73ea363a0f
0709 13:06:55.461400 (+ 43001us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:55.461466 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:55.461588 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:55.463613 (+  2025us) raft_consensus.cc:1685] finished
0709 13:06:55.463642 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:55.463912 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9463,"raft.queue_time_us":9636,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":16793,"spinlock_wait_cycles":3516800,"thread_start_us":34215,"threads_started":1,"wal-append.queue_time_us":3763861}
I20260709 13:06:55.484668 20626 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.484723 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:55.485435 20626 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:55.486626 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:55.487353 15856 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:55.482250 20749 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:55.489295 20749 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:55.489905 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.491460 15931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 2292) took 1042 ms. Trace:
I20260709 13:06:55.491839 15931 rpcz_store.cc:276] 0709 13:06:54.449365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.452390 (+  3025us) service_pool.cc:224] Handling call
0709 13:06:55.440585 (+988195us) spinlock_profiling.cc:243] Waited 2.66 ms on lock 0x7b5800003698. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:06:55.491380 (+ 50795us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6110080}
I20260709 13:06:55.493943 20877 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:55.481389 20752 raft_consensus.cc:740] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:55.495848 20752 consensus_queue.cc:260] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:55.496925 20752 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:55.501178 20882 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:55.480755 20723 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:55.502640 20723 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:55.506237 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:55.508749 20855 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:55.511190 15856 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
I20260709 13:06:55.440081 20734 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:55.524284 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.517838 15974 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:55.518651 15825 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:55.474757 16002 rpcz_store.cc:276] 0709 13:06:51.800240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.800393 (+   153us) service_pool.cc:224] Handling call
0709 13:06:51.800789 (+   396us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:51.801287 (+   498us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:51.801328 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:51.801988 (+   660us) log.cc:844] Serialized 52 byte log entry
0709 13:06:52.084338 (+282350us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:52.084381 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:52.084453 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:55.398932 (+3314479us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b240024a0e8. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71419c4 00007f73e713facb 00007f73e72fadc9 00007f73e72faa55 00007f73e72fa09d 00007f73e9f2c10e 00007f73e9f4388c 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221
0709 13:06:55.466856 (+ 67924us) raft_consensus.cc:1685] finished
0709 13:06:55.466877 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:55.467149 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":111,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":17659,"spinlock_wait_cycles":32174336,"thread_start_us":222294,"threads_started":1,"wal-append.queue_time_us":920388}
I20260709 13:06:55.528815 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:06:55.531572 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:55.543527 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.545506 16566 raft_consensus.cc:2393] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 13: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
W20260709 13:06:55.549309 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:55.550876 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:55.544819 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 63226.1us
I20260709 13:06:55.551738 16561 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
I20260709 13:06:55.531302 20499 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:06:55.571156 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:55.542135 20626 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:55.532822 20735 log.cc:927] Time spent T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 4.432s	user 0.006s	sys 0.025s
I20260709 13:06:55.576156 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.578451 15930 raft_consensus.cc:1275] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 13:06:55.578742 16565 raft_consensus.cc:1275] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:06:55.578994 15930 pending_rounds.cc:85] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 4
I20260709 13:06:55.579342 15930 pending_rounds.cc:107] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:06:55.579625 15930 raft_consensus.cc:2889] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:55.531265 20825 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:55.573503 20768 raft_consensus.cc:2804] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:55.592816 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.521279 20833 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:55.610033 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:55.612033 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.605692 20626 raft_consensus.cc:2804] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:55.615496 15861 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:55.593487 20768 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:55.615768 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 186493us
I20260709 13:06:55.619506 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.623489 16002 raft_consensus.cc:1240] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:55.617120 20626 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:55.618237 20768 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:55.628219 16561 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 9.
W20260709 13:06:55.630674 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 118112us
I20260709 13:06:55.631232 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:55.627476 20626 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:06:55.636324 20894 log.cc:927] Time spent T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 13:06:55.640156 20750 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 13:06:55.640610 15930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 2141) took 3736 ms. Trace:
I20260709 13:06:55.640995 15930 rpcz_store.cc:276] 0709 13:06:51.903750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.903872 (+   122us) service_pool.cc:224] Handling call
0709 13:06:51.904101 (+   229us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:55.415000 (+3510899us) spinlock_profiling.cc:243] Waited 1.85 ms on lock 0x7b580064f1d0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3040aa 00007f73ea386447 00007f73ea39efe7 00007f73ea3a07de 00007f73ea3aab49 00007f73ea3ab416 00007f73ea3a7d8f 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4
0709 13:06:55.629524 (+214524us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:55.640531 (+ 11007us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4257024}
I20260709 13:06:55.642232 16005 raft_consensus.cc:1240] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:55.646159 15862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 912) took 5216 ms. Trace:
I20260709 13:06:55.649536 15974 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:55.652635 15854 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:06:55.664126 20825 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:55.665287 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:55.666262 16004 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:06:55.671948 20775 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.672605 20775 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:55.676383 20825 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:55.676492 20677 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:06:55.681052 20833 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:55.646512 15862 rpcz_store.cc:276] 0709 13:06:50.429326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:50.429825 (+   499us) service_pool.cc:224] Handling call
0709 13:06:50.430096 (+   271us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:50.430434 (+   338us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:06:50.460131 (+ 29697us) log.cc:844] Serialized 14 byte log entry
0709 13:06:50.984538 (+524407us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:50.991980 (+  7442us) log.cc:844] Serialized 52 byte log entry
0709 13:06:50.992311 (+   331us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:50.992381 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:50.992509 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:55.531966 (+4539457us) spinlock_profiling.cc:243] Waited 6.05 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:06:55.645618 (+113652us) raft_consensus.cc:1685] finished
0709 13:06:55.645744 (+   126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:55.645951 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4131,"prepare.queue_time_us":45850,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":34,"raft.queue_time_us":3524,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":26220,"spinlock_wait_cycles":13920128,"thread_start_us":470953,"threads_started":2,"wal-append.queue_time_us":514031}
I20260709 13:06:55.686983 20188 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:55.688015 16003 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
I20260709 13:06:55.688122 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.688831 16570 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:06:55.683527 20676 consensus_queue.cc:1059] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 13:06:55.679440 20721 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:55.690990 16568 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:55.694712 20689 consensus_queue.cc:1048] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 13:06:55.671044 20734 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:55.695744 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 69789.1us
I20260709 13:06:55.697451 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:55.698520 15929 raft_consensus.cc:2410] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 11 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:06:55.698940 15929 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 2142) took 3794 ms (client timeout 5004 ms). Trace:
I20260709 13:06:55.700978 15857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 2201) took 3027 ms. Trace:
I20260709 13:06:55.701356 15857 rpcz_store.cc:276] 0709 13:06:52.673731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:52.697318 (+ 23587us) service_pool.cc:224] Handling call
0709 13:06:52.697566 (+   248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:52.763765 (+ 66199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:52.763794 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:52.763830 (+    36us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:52.763863 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:55.700579 (+2936716us) spinlock_profiling.cc:243] Waited 66 ms on lock 0x7b540032d554. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:55.700642 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:55.700818 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":151752448}
I20260709 13:06:55.695104 20676 raft_consensus.cc:3055] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:55.703091 20676 raft_consensus.cc:740] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:55.703380 20736 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:55.704139 20676 consensus_queue.cc:260] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:55.705451 20676 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:55.711305 20850 consensus_queue.cc:260] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:55.712801 20850 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:55.715040 20723 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:55.715991 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 66513.2us
I20260709 13:06:55.717334 20723 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:55.717338 15974 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:55.719740 15974 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:55.720135 20854 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:55.720423 16004 raft_consensus.cc:1275] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:06:55.720616 20855 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:55.720747 20854 raft_consensus.cc:697] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:55.721045 20855 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:55.721513 20855 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:55.723335 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:55.724131 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:55.724920 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:55.725613 16002 raft_consensus.cc:1240] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:55.722019 20854 consensus_queue.cc:237] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:55.726799 20626 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:55.724931 16005 raft_consensus.cc:2393] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 13: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
W20260709 13:06:55.699321 15929 rpcz_store.cc:269] 0709 13:06:51.904106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.904541 (+   435us) service_pool.cc:224] Handling call
0709 13:06:55.697088 (+3792547us) spinlock_profiling.cc:243] Waited 6.83 ms on lock 0x7b5800003698. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:06:55.698881 (+  1793us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15699840}
I20260709 13:06:55.728318 20670 raft_consensus.cc:3055] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:55.728755 20670 raft_consensus.cc:740] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:55.729406 15825 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:55.729832 20670 consensus_queue.cc:260] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:55.731124 20670 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:55.732448 20729 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.733134 20689 raft_consensus.cc:2804] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:55.733085 20729 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:55.733690 20689 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:55.734184 20689 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:55.733175 20817 consensus_queue.cc:1059] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:55.735328 20726 consensus_queue.cc:1048] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:55.737478 16533 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:55.738428 20883 raft_consensus.cc:2749] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:06:55.740409 20481 consensus_queue.cc:1048] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:55.683537 20825 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:55.744067 20825 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:55.748246 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:55.749842 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:55.749989 16006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 907) took 4251 ms. Trace:
I20260709 13:06:55.751245 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:55.750339 16006 rpcz_store.cc:276] 0709 13:06:51.498451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.617327 (+118876us) service_pool.cc:224] Handling call
0709 13:06:52.252377 (+635050us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:52.252761 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:52.252793 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:52.574666 (+321873us) log.cc:844] Serialized 52 byte log entry
0709 13:06:52.755138 (+180472us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:52.996329 (+241191us) log.cc:844] Serialized 14 byte log entry
0709 13:06:53.426478 (+430149us) log.cc:844] Serialized 14 byte log entry
0709 13:06:53.426775 (+   297us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:53.426873 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:53.426918 (+    45us) raft_consensus.cc:1685] finished
0709 13:06:53.426935 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:55.749691 (+2322756us) spinlock_profiling.cc:243] Waited 633 ms on lock 0x7b540037b9d0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:55.749845 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":323812,"prepare.run_cpu_time_us":1102,"prepare.run_wall_time_us":11674,"raft.queue_time_us":236067,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":199,"spinlock_wait_cycles":1454949632,"thread_start_us":707375,"threads_started":3,"wal-append.queue_time_us":173909,"wal-append.run_cpu_time_us":12720,"wal-append.run_wall_time_us":2148441}
I20260709 13:06:55.753291 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:55.753983 16008 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 13.
I20260709 13:06:55.754068 20729 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:55.755194 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:55.755810 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:55.756471 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:55.754276 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:55.756997 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:55.756484 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:55.758914 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:55.761089 20773 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:52.547180 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:55.762393 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.758991 16008 raft_consensus.cc:2393] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 10: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:55.769336 16570 raft_consensus.cc:2410] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:06:55.763152 16567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 1010) took 4177 ms. Trace:
I20260709 13:06:55.763494 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:55.770246 16565 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:55.757969 16010 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:55.767549 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.763144 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:55.771525 16566 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:55.778923 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:55.784138 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.785445 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.785799 15858 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 7.
I20260709 13:06:55.786444 20842 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.786543 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.786927 20842 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:55.788383 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:55.788779 20784 log.cc:927] Time spent T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 4.132s	user 0.049s	sys 0.096s
I20260709 13:06:55.789240 15930 raft_consensus.cc:2410] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:06:55.790220 16004 raft_consensus.cc:1240] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:55.791015 20853 raft_consensus.cc:3055] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:55.791486 20853 raft_consensus.cc:740] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:55.796653 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.797472 16562 raft_consensus.cc:2393] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 15: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:55.798295 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.799299 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.769842 16567 rpcz_store.cc:276] 0709 13:06:51.585767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.586027 (+   260us) service_pool.cc:224] Handling call
0709 13:06:51.586333 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:51.980339 (+394006us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:55.762704 (+3782365us) spinlock_profiling.cc:243] Waited 393 ms on lock 0x7b540059c9d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:55.762974 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":904752384}
I20260709 13:06:55.800271 16561 raft_consensus.cc:1240] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:55.800982 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.801496 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.802091 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.802846 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.803233 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.787237 16568 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:06:55.804040 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:55.805311 16568 raft_consensus.cc:1275] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:06:55.806613 20843 log.cc:927] Time spent T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 1.921s	user 0.007s	sys 0.008s
I20260709 13:06:55.759069 20826 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:55.814522 20687 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:55.815989 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:55.816957 20626 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:55.817709 20775 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:55.819882 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:55.820348 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 43785.4us
W20260709 13:06:55.820726 15934 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 2143) took 3916 ms (client timeout 2339 ms). Trace:
W20260709 13:06:55.821079 15934 rpcz_store.cc:269] 0709 13:06:51.904440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.904614 (+   174us) service_pool.cc:224] Handling call
0709 13:06:55.819584 (+3914970us) spinlock_profiling.cc:243] Waited 6.45 ms on lock 0x7b5800003698. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:06:55.820656 (+  1072us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14828928}
I20260709 13:06:55.822306 20749 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:55.822935 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:55.823240 16005 raft_consensus.cc:2393] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 13: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:55.729452 15974 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:06:55.829377 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 99977us
I20260709 13:06:55.830075 15861 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:06:55.831030 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:55.832048 16007 raft_consensus.cc:2393] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 14: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:55.832216 15858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 2294) took 1350 ms. Trace:
W20260709 13:06:55.828526 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 84716us
I20260709 13:06:55.847543 16565 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:06:55.832617 15858 rpcz_store.cc:276] 0709 13:06:54.481500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.485743 (+  4243us) service_pool.cc:224] Handling call
0709 13:06:55.832032 (+1346289us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54002d14d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:55.832140 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2986964864}
I20260709 13:06:55.851969 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:55.853310 16006 raft_consensus.cc:2393] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 11: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:55.854859 16008 raft_consensus.cc:1240] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:55.795240 20825 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:55.859872 16563 raft_consensus.cc:1240] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:55.819332 20768 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:55.877044 20768 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:55.840868 20626 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:55.877629 20626 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:55.868517 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:55.878047 20626 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:55.878572 16005 raft_consensus.cc:2393] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 12: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:55.879733 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:55.880442 16005 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:55.866536 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:55.875247 16566 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
I20260709 13:06:55.890473 16007 raft_consensus.cc:2393] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 12: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:55.890867 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:55.893507 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.893549 16533 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:55.896492 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:55.898120 16002 raft_consensus.cc:1240] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:55.901299 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.901767 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.899880 15974 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:55.873759 20842 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:55.897320 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.907245 16567 raft_consensus.cc:2410] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:06:55.905218 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.908427 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.909192 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:06:55.911896 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:55.914924 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.902458 16562 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:06:55.916855 20808 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:55.917435 20808 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:55.917881 15902 connection.cc:256] RPC call timeout handler was delayed by -2.1653855449999355s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.166s for a response
W20260709 13:06:55.918393 15902 connection.cc:256] RPC call timeout handler was delayed by -1.9259855450000032s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1926s for a response
W20260709 13:06:55.918720 15902 connection.cc:256] RPC call timeout handler was delayed by -1.9259855450000032s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1926s for a response
W20260709 13:06:55.919029 15902 connection.cc:256] RPC call timeout handler was delayed by -1.9205855449999945s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1932s for a response
W20260709 13:06:55.919373 15902 connection.cc:256] RPC call timeout handler was delayed by -1.8719855449999159s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1986s for a response
I20260709 13:06:55.904711 20768 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:55.909454 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:55.919924 20648 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:55.919982 20556 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:55.921422 20825 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:55.921556 16009 raft_consensus.cc:1240] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:55.924329 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:55.924214 20620 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.942523 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:55.944157 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:55.944172 15862 raft_consensus.cc:1240] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:55.919628 20808 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:55.924674 20626 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:55.932003 16568 raft_consensus.cc:2393] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 12: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:55.932873 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:55.933166 15902 connection.cc:256] RPC call timeout handler was delayed by -1.8719855449999159s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1986s for a response
W20260709 13:06:55.934012 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:55.934408 16005 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
I20260709 13:06:55.944654 15855 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:06:55.935526 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: true dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:55.935513 16007 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
I20260709 13:06:55.909978 16006 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 12.
I20260709 13:06:55.937494 20768 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:55.939640 20556 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:55.940809 15976 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:55.941658 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:55.926236 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:55.960172 16008 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:06:55.938280 20723 raft_consensus.cc:2804] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:55.968768 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:55.959947 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:55.974057 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:55.969684 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:55.976926 16011 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:55.978200 15976 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:55.953222 20648 raft_consensus.cc:697] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:55.984053 15858 raft_consensus.cc:1275] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:06:55.984838 15855 raft_consensus.cc:1240] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:55.993829 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:55.995221 20879 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.995646 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:55.996435 15861 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:55.997481 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:55.999835 15857 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
I20260709 13:06:56.001425 16533 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:56.001744 20772 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:06:56.005076 20883 consensus_queue.cc:1048] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:56.006732 20689 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:06:56.005266 20868 raft_consensus.cc:2749] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:55.980715 20825 raft_consensus.cc:2804] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:56.017542 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:56.022806 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:55.983819 20490 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.023475 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:55.983107 20812 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.035977 15860 raft_consensus.cc:2410] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:06:56.036098 20812 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:56.036970 20490 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:56.038750 20812 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:55.989079 20723 raft_consensus.cc:697] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 14 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:56.039255 20490 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:56.023581 15854 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 7.
I20260709 13:06:56.024353 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:56.041404 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:56.053089 15931 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:56.040103 20723 consensus_queue.cc:237] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:56.032335 20648 consensus_queue.cc:237] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:56.041373 20825 raft_consensus.cc:697] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:56.023840 20642 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:56.030790 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:56.030387 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:56.043324 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:56.027709 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:56.044873 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:55.989933 20855 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:56.069988 20855 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:56.032817 20768 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:56.046476 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:56.072106 15853 raft_consensus.cc:2393] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 11: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:56.072425 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:56.073221 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:56.026998 20626 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:56.059041 15935 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:06:56.048532 20689 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:06:56.040795 20901 log.cc:927] Time spent T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.085s	user 0.002s	sys 0.000s
I20260709 13:06:56.056180 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:56.066447 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:56.048048 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:56.059927 15858 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
W20260709 13:06:56.081562 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 53856.4us
I20260709 13:06:55.988507 20620 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:56.025554 20556 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:56.087760 16011 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 12.
I20260709 13:06:56.055298 20490 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.995781 20879 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:56.148525 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.276265 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:56.092250 15825 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:56.092787 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:56.084015 15861 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 14.
I20260709 13:06:56.128850 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:56.285521 15929 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:06:56.287868 20620 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:56.129843 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:56.289316 15935 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:56.127508 20642 raft_consensus.cc:697] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:56.297300 20642 consensus_queue.cc:237] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:56.249688 15931 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:06:56.220512 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:56.309293 16564 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:06:56.312371 15935 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:06:56.087906 15857 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
W20260709 13:06:56.102911 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.093701 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:56.369622 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 277447us
W20260709 13:06:56.370523 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:56.370985 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.184033 15902 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:56.376305 20704 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:56.087810 20858 consensus_queue.cc:1048] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.289s
I20260709 13:06:56.377027 20704 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:56.378862 20556 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:56.379333 20556 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:56.379921 20556 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:56.090535 20768 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:56.382524 20490 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:56.388284 15856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2305) took 1193 ms. Trace:
I20260709 13:06:56.388790 15856 rpcz_store.cc:276] 0709 13:06:55.195075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.515028 (+319953us) service_pool.cc:224] Handling call
0709 13:06:55.524011 (+  8983us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:55.584742 (+ 60731us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:55.584777 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:55.608961 (+ 24184us) log.cc:844] Serialized 26 byte log entry
0709 13:06:55.767005 (+158044us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:55.767042 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:55.767122 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:56.040114 (+272992us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:06:56.387943 (+347829us) raft_consensus.cc:1685] finished
0709 13:06:56.387964 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:56.388200 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2434,"raft.queue_time_us":4613,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":25101,"spinlock_wait_cycles":29189376,"thread_start_us":136360,"threads_started":1,"wal-append.queue_time_us":277205}
W20260709 13:06:56.391067 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 296488us
I20260709 13:06:56.276846 20825 consensus_queue.cc:237] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:56.252518 20758 consensus_queue.cc:1048] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.305s
I20260709 13:06:56.395915 20677 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:56.396332 20677 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:56.396679 20677 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:56.398357 20775 consensus_queue.cc:1048] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:06:56.400089 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.401036 20703 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:56.401228 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:56.401882 16004 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:06:56.402825 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:56.403447 16009 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:06:56.404275 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:56.405397 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:56.405997 16003 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:06:56.406867 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:56.408226 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:56.410130 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:56.410869 16011 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
I20260709 13:06:56.412529 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:56.413221 16011 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 7.
I20260709 13:06:56.381341 20768 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:56.426815 16564 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:06:56.430088 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:56.430228 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:56.432926 16007 raft_consensus.cc:2393] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 12: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:56.438059 16006 raft_consensus.cc:2393] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 12: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:56.379361 20704 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:56.460711 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:56.461781 15861 raft_consensus.cc:2393] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 15: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:56.472177 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:56.472942 16006 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
W20260709 13:06:56.466639 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 432386us
W20260709 13:06:56.473578 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 360800us
I20260709 13:06:56.476838 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:56.479628 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:56.486049 20803 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.486672 20810 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:56.486915 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 485546us
I20260709 13:06:56.488953 16533 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:56.489684 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:56.491043 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:56.491899 15856 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 10.
I20260709 13:06:56.495265 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:56.496428 15857 raft_consensus.cc:2410] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:06:56.526124 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:06:56.528560 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 477671us
I20260709 13:06:56.530241 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:56.530910 15859 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:56.533432 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:56.534219 15857 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:56.537961 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 46247.8us
I20260709 13:06:56.536064 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:56.486640 20803 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:56.555008 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:56.555440 20803 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:56.555646 15854 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:56.530323 20850 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.018s	sys 0.045s
W20260709 13:06:56.560741 16566 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.006s	sys 0.083s
W20260709 13:06:56.568665 16566 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.007s	sys 0.083s
W20260709 13:06:56.576115 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.580689 20490 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:06:56.581363 15827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:56.582814 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 140220us
I20260709 13:06:56.590173 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:56.613593 15861 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:56.619306 20810 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:56.619457 15827 connection.cc:256] RPC call timeout handler was delayed by -1.7066525729999285s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.166s for a response
W20260709 13:06:56.620141 15827 connection.cc:256] RPC call timeout handler was delayed by -1.6086582169999701s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16570000000000001s for a response
W20260709 13:06:56.620605 15827 connection.cc:256] RPC call timeout handler was delayed by -1.5170234559999471s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1569s for a response
W20260709 13:06:56.620891 15827 connection.cc:256] RPC call timeout handler was delayed by -1.4729234559999895s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16180000000000003s for a response
W20260709 13:06:56.621240 15827 connection.cc:256] RPC call timeout handler was delayed by -1.4729234559999895s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16180000000000003s for a response
W20260709 13:06:56.621603 15827 connection.cc:256] RPC call timeout handler was delayed by -1.4621234559999721s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.163s for a response
W20260709 13:06:56.621985 15827 connection.cc:256] RPC call timeout handler was delayed by -1.4621234559999721s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.163s for a response
W20260709 13:06:56.622363 15827 connection.cc:256] RPC call timeout handler was delayed by -1.4378234559999328s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16570000000000001s for a response
W20260709 13:06:56.622743 15827 connection.cc:256] RPC call timeout handler was delayed by -1.4351234559999284s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.166s for a response
W20260709 13:06:56.623027 15827 connection.cc:256] RPC call timeout handler was delayed by -1.434223455999927s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16610000000000003s for a response
W20260709 13:06:56.623376 15827 connection.cc:256] RPC call timeout handler was delayed by -1.434223455999927s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16610000000000003s for a response
W20260709 13:06:56.623893 15827 connection.cc:256] RPC call timeout handler was delayed by -1.4018234559999883s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16970000000000002s for a response
W20260709 13:06:56.582257 20850 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.018s	sys 0.045s
W20260709 13:06:56.624315 15827 connection.cc:256] RPC call timeout handler was delayed by -1.33252345599999s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1774s for a response
I20260709 13:06:56.558914 20855 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:06:56.627861 15827 connection.cc:256] RPC call timeout handler was delayed by -1.33252345599999s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1774s for a response
W20260709 13:06:56.632360 15827 connection.cc:256] RPC call timeout handler was delayed by -1.3082234559999506s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1801s for a response
W20260709 13:06:56.632800 15827 connection.cc:256] RPC call timeout handler was delayed by -1.2776234559999011s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1835s for a response
I20260709 13:06:56.632639 15902 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:56.633713 15827 connection.cc:256] RPC call timeout handler was delayed by -1.2749234560000104s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18380000000000002s for a response
W20260709 13:06:56.634193 15827 connection.cc:256] RPC call timeout handler was delayed by -1.2749234560000104s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18380000000000002s for a response
W20260709 13:06:56.634641 15827 connection.cc:256] RPC call timeout handler was delayed by -1.1750234559999626s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19490000000000002s for a response
W20260709 13:06:56.635061 15827 connection.cc:256] RPC call timeout handler was delayed by -1.0355234559999644s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21040000000000003s for a response
W20260709 13:06:56.635494 15827 connection.cc:256] RPC call timeout handler was delayed by -1.034623455999963s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21050000000000002s for a response
W20260709 13:06:56.635959 15827 connection.cc:256] RPC call timeout handler was delayed by -1.0229234559999441s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2118s for a response
W20260709 13:06:56.636382 15827 connection.cc:256] RPC call timeout handler was delayed by -1.0229234559999441s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2118s for a response
W20260709 13:06:56.636814 15827 connection.cc:256] RPC call timeout handler was delayed by -1.0139234559999295s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21280000000000002s for a response
W20260709 13:06:56.637248 15827 connection.cc:256] RPC call timeout handler was delayed by -1.0139234559999295s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21280000000000002s for a response
I20260709 13:06:56.582093 20775 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.646644 15827 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
W20260709 13:06:56.646978 20903 log.cc:927] Time spent T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:06:56.647174 20729 raft_consensus.cc:2804] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:56.647615 20729 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:56.647991 20729 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:56.658107 20729 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:56.660687 20729 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:56.661028 20853 consensus_queue.cc:260] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:56.663161 20853 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:56.664907 20620 consensus_queue.cc:1059] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:56.666046 20620 raft_consensus.cc:3055] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:56.666466 20620 raft_consensus.cc:740] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:56.672410 15825 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:56.624223 20490 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:56.582020 20556 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:56.679569 15825 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:56.694794 15861 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:06:56.692080 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:56.698787 15935 raft_consensus.cc:1240] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:06:56.698854 15825 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:56.697070 15860 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
I20260709 13:06:56.711840 15854 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
W20260709 13:06:56.707619 20723 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.008s	sys 0.025s
W20260709 13:06:56.737115 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.687693 20752 raft_consensus.cc:2804] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:56.735090 15900 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:56.729424 15902 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:56.762871 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 116312us
W20260709 13:06:56.766868 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.719587 20689 raft_consensus.cc:2804] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:56.725036 20808 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:56.674562 20775 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:56.677640 20855 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:56.767741 20723 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.008s	sys 0.026s
W20260709 13:06:56.783792 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 49108.8us
I20260709 13:06:56.733693 20750 consensus_queue.cc:1059] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 13:06:56.793466 20775 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:56.794095 20775 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:56.794469 20775 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:56.794870 20775 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:56.688180 20810 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:56.773262 20759 consensus_queue.cc:1048] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:06:56.797204 20759 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:56.797431 20810 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:56.799129 20841 raft_consensus.cc:2804] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:56.799758 20841 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:56.780432 20808 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:56.800217 20841 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:56.800508 20808 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:56.802400 20879 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:56.667565 20620 consensus_queue.cc:260] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:56.807928 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:56.812088 16568 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:06:56.813295 20620 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:56.815568 20556 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:56.808645 15857 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:06:56.797533 20759 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:56.810534 15974 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:56.822309 15976 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:56.781133 20768 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:56.840890 15974 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:56.782784 20729 consensus_queue.cc:1059] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 13:06:56.837407 20759 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:56.842936 20704 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:06:56.842988 20723 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:56.852721 20768 consensus_queue.cc:1048] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.011s
I20260709 13:06:56.852898 20670 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:56.866160 20670 raft_consensus.cc:697] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:56.867056 20670 consensus_queue.cc:237] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:56.859386 15974 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:56.859490 20704 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:56.860224 20723 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:56.872303 20704 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:06:56.872458 20723 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:56.873569 20677 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:56.873884 20626 raft_consensus.cc:2749] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:56.877614 16563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 1229) took 1109 ms. Trace:
I20260709 13:06:56.877700 15827 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:56.878057 16563 rpcz_store.cc:276] 0709 13:06:55.768193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.768301 (+   108us) service_pool.cc:224] Handling call
0709 13:06:55.848725 (+ 80424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:55.875267 (+ 26542us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.877390 (+1002123us) spinlock_profiling.cc:243] Waited 69.2 ms on lock 0x7b5400714b50. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:56.877535 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":159270272}
I20260709 13:06:56.843565 15859 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
I20260709 13:06:56.880169 15974 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:56.891573 15974 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:56.890959 20736 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:06:56.901368 15858 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.011s	sys 0.028s
I20260709 13:06:56.890650 20723 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:56.895164 20736 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:06:56.906451 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:56.901672 15858 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.011s	sys 0.028s
W20260709 13:06:56.901199 20648 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.025s	sys 0.010s
W20260709 13:06:56.932360 20648 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.025s	sys 0.010s
I20260709 13:06:56.907812 20736 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:56.940286 20626 raft_consensus.cc:2749] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:06:56.960552 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 472879us
W20260709 13:06:56.944211 20703 consensus_meta.cc:287] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.543s	user 0.003s	sys 0.013s
I20260709 13:06:56.980659 20704 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:56.968839 20916 log.cc:927] Time spent T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:06:56.968291 20775 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:56.983631 20704 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:56.987775 20677 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:56.987303 20736 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:56.984139 20723 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:56.989501 20868 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.011s	sys 0.028s
I20260709 13:06:56.985982 20556 raft_consensus.cc:3055] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:56.986568 20775 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:57.000764 20868 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.011s	sys 0.029s
I20260709 13:06:56.964356 20752 raft_consensus.cc:697] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:56.999302 20768 consensus_queue.cc:1059] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 13:06:57.003510 20677 raft_consensus.cc:2749] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:56.986171 20841 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:57.003860 20556 raft_consensus.cc:740] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:57.036743 20703 consensus_queue.cc:1048] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:56.999315 20731 consensus_queue.cc:1048] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:06:57.038497 20736 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:57.039448 20752 consensus_queue.cc:237] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:57.041474 20759 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:57.045251 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.047896 20841 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:57.050441 16535 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.016s	sys 0.004s
W20260709 13:06:57.088568 16535 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.017s	sys 0.005s
W20260709 13:06:57.089362 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 559516us
I20260709 13:06:57.093114 16566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1075) took 1102 ms. Trace:
I20260709 13:06:57.093604 16566 rpcz_store.cc:276] 0709 13:06:55.990595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.990718 (+   123us) service_pool.cc:224] Handling call
0709 13:06:55.996022 (+  5304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:55.996274 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:06:56.569203 (+572929us) log.cc:844] Serialized 14 byte log entry
0709 13:06:57.092708 (+523505us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:57.092775 (+    67us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:06:57.092837 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:57.092915 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:57.093050 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38259,"prepare.queue_time_us":20154,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"thread_start_us":955426,"threads_started":2}
I20260709 13:06:57.114385 20759 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:57.114763 20855 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:57.120306 20556 consensus_queue.cc:260] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:57.124416 15759 catalog_manager.cc:5697] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:57.102972 20768 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:57.132306 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 432471us
I20260709 13:06:57.152876 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:57.154014 15931 raft_consensus.cc:2393] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 8: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
W20260709 13:06:57.154753 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.144035 16535 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:57.142125 20841 raft_consensus.cc:3055] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:57.163447 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:57.163580 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:57.163797 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:57.164532 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:57.142011 20703 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.169401 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:06:57.171622 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.177425 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:57.178031 16562 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:06:57.181357 15744 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 83a1b79cdcf046c78453695c798a14a5 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 8430db80a39d4868bcf7641c16de35ea is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:06:57.182219 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:57.182806 15859 raft_consensus.cc:1240] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:57.182143 20912 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:57.190116 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 455057us
I20260709 13:06:57.167133 20841 raft_consensus.cc:740] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:57.147118 20768 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:06:57.199159 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 500905us
I20260709 13:06:57.199855 15857 raft_consensus.cc:1240] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
W20260709 13:06:57.201928 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 58025us
I20260709 13:06:57.204499 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:57.205317 15854 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:06:57.218576 15759 catalog_manager.cc:5697] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 6 to 8, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 8 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
I20260709 13:06:57.200565 20556 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:57.183104 20808 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:57.228912 15902 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:57.230129 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:57.230916 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:57.231030 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:57.180042 20703 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:57.227488 16535 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:57.236039 16535 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:57.239975 20929 consensus_queue.cc:1059] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:06:57.239326 16535 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:57.260197 16535 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:57.220669 20731 consensus_queue.cc:1059] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:06:57.226022 20689 raft_consensus.cc:697] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:57.243160 20648 raft_consensus.cc:2764] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:06:57.242172 20499 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:57.244910 20803 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:57.273980 15853 raft_consensus.cc:1240] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:57.275214 20773 raft_consensus.cc:3055] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:57.252319 20750 raft_consensus.cc:2749] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:57.279484 15902 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:57.263615 15858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 2239) took 1197 ms. Trace:
I20260709 13:06:57.276651 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:57.286031 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 408358us
I20260709 13:06:57.272792 20499 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.293067 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:57.293812 15862 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
I20260709 13:06:57.277953 15759 catalog_manager.cc:5697] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 9 to 11, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 11 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
I20260709 13:06:57.274570 20803 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.283068 15858 rpcz_store.cc:276] 0709 13:06:56.066233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.093263 (+ 27030us) service_pool.cc:224] Handling call
0709 13:06:56.126744 (+ 33481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:56.126961 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:06:56.237753 (+110792us) log.cc:844] Serialized 14 byte log entry
0709 13:06:57.263260 (+1025507us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:57.263318 (+    58us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:57.263372 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:57.263445 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:57.263558 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3278,"prepare.queue_time_us":56863,"prepare.run_cpu_time_us":386,"prepare.run_wall_time_us":552,"raft.queue_time_us":177677,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":11761,"thread_start_us":894198,"threads_started":2,"wal-append.queue_time_us":762812}
I20260709 13:06:57.300016 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:57.304494 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:57.305526 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 45442.1us
I20260709 13:06:57.263011 20883 consensus_queue.cc:1059] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:06:57.281384 20773 raft_consensus.cc:740] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:57.319468 15935 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 12.
I20260709 13:06:57.322114 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: true dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:06:57.324115 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 56666.9us
I20260709 13:06:57.324116 15827 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:57.285497 20855 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.275303 20797 consensus_queue.cc:1059] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:06:57.295760 20803 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:57.298187 20499 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:57.316777 20759 raft_consensus.cc:2749] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:57.334450 16006 raft_consensus.cc:1240] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 10. Current term is 11. Ops: []
W20260709 13:06:57.335592 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 445857us
W20260709 13:06:57.339367 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.271201 20841 consensus_queue.cc:260] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:57.339154 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:57.336956 15974 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:57.314616 20917 raft_consensus.cc:2804] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:57.342514 15974 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:57.344007 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:57.345561 15974 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:57.348290 15974 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:57.345561 15934 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
I20260709 13:06:57.352830 15902 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:57.352613 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:57.354514 15937 raft_consensus.cc:1240] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:57.355867 15974 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:57.359035 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.359400 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:57.360972 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.361014 15827 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:57.365265 15862 raft_consensus.cc:2410] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:06:57.368584 15936 raft_consensus.cc:2393] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 12: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:57.371912 15827 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:57.373333 15827 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:57.374504 15935 raft_consensus.cc:1240] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:06:57.375557 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:57.376223 15929 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:06:57.376508 15856 raft_consensus.cc:1240] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:57.325066 20935 consensus_queue.cc:1048] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 13:06:57.380388 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:57.385376 15900 leader_election.cc:336] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.778s (SENT)
I20260709 13:06:57.384346 15858 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
I20260709 13:06:57.354570 20670 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:06:57.386471 15900 leader_election.cc:336] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.783s (SENT)
I20260709 13:06:57.362361 20736 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:06:57.402242 15900 leader_election.cc:336] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.941s (SENT)
I20260709 13:06:57.360656 20723 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:57.405407 20939 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:57.407230 16533 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:57.407618 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:57.340269 20855 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:57.380725 20879 raft_consensus.cc:2804] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:57.439946 20773 consensus_queue.cc:260] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:57.441439 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:57.441813 20773 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:57.442217 15855 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:06:57.364224 20917 raft_consensus.cc:697] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:57.444620 20917 consensus_queue.cc:237] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:57.368371 20704 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:57.448037 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:57.449692 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:57.450631 15860 raft_consensus.cc:2393] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 8: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:06:57.455776 20923 log.cc:927] Time spent T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.173s	user 0.001s	sys 0.001s
I20260709 13:06:57.394135 20810 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:57.460263 20875 raft_consensus.cc:3055] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:57.460834 20875 raft_consensus.cc:740] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:57.462327 20772 raft_consensus.cc:2749] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:57.461918 20875 consensus_queue.cc:260] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:57.353251 20689 consensus_queue.cc:237] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:06:57.467375 15759 catalog_manager.cc:5697] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 11 to 14, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 14 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:06:57.404791 20736 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:57.435781 20670 raft_consensus.cc:697] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:57.438416 16533 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:57.484651 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 46316.1us
I20260709 13:06:57.439097 20927 raft_consensus.cc:2749] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:57.440152 20703 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:57.393255 20768 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:06:57.490777 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.418280 15856 raft_consensus.cc:1275] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 13:06:57.477867 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 100335us
I20260709 13:06:57.492914 15827 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:57.451556 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:57.481910 15759 catalog_manager.cc:5697] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:57.484557 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 154284us
I20260709 13:06:57.483587 20879 raft_consensus.cc:697] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:57.501181 15856 raft_consensus.cc:1275] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:06:57.392457 20868 consensus_queue.cc:1059] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.081s
I20260709 13:06:57.463573 20875 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:57.500478 20808 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:57.500712 15827 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:57.487452 16562 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
I20260709 13:06:57.484550 20736 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:57.468597 20723 raft_consensus.cc:697] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:57.522079 16533 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:57.491577 20726 raft_consensus.cc:3055] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:57.525380 15976 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:57.529003 15902 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:57.501183 20750 raft_consensus.cc:3055] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:57.491797 20731 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:57.487700 20704 raft_consensus.cc:697] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:57.541345 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:57.542142 15929 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:57.545838 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:57.548561 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 47950.9us
I20260709 13:06:57.535172 20752 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:57.537779 20868 raft_consensus.cc:2804] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:57.557979 15931 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:06:57.533129 20726 raft_consensus.cc:740] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:57.537432 20750 raft_consensus.cc:740] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:57.539976 20731 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:57.564461 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 153182us
I20260709 13:06:57.549072 20677 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:57.528398 20648 consensus_queue.cc:1059] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:06:57.550105 20929 raft_consensus.cc:3055] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:57.579064 15902 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:57.558131 20945 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:57.575644 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:57.561012 20868 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.588897 20750 consensus_queue.cc:260] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:57.590607 20750 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:57.592005 20948 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.592118 20726 consensus_queue.cc:260] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:57.592618 20948 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:57.593763 20703 raft_consensus.cc:3055] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:57.594172 20703 raft_consensus.cc:740] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:57.594889 20948 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:57.595232 20855 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:57.596107 20704 consensus_queue.cc:237] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:57.578630 20929 raft_consensus.cc:740] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:57.599665 15827 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:57.605049 16006 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 9.
I20260709 13:06:57.604100 15825 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:57.595032 20703 consensus_queue.cc:260] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:57.611393 15929 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:06:57.572621 20731 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:57.613322 15825 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:57.614944 20944 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:57.637246 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:57.643958 15853 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 10.
I20260709 13:06:57.645642 15759 catalog_manager.cc:5697] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 4 to 8, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 8 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:57.604723 20868 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:57.618777 20808 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:57.610401 20938 raft_consensus.cc:2749] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:06:57.658350 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 85802.3us
W20260709 13:06:57.664301 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.608493 20945 raft_consensus.cc:697] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:57.601330 20935 consensus_queue.cc:1048] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:06:57.642495 20772 raft_consensus.cc:2804] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:57.664736 20808 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.638180 20855 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:57.656927 20947 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.666920 20912 raft_consensus.cc:2749] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:57.663817 20703 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:57.694579 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 134865us
W20260709 13:06:57.683228 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 83630us
I20260709 13:06:57.607997 20670 consensus_queue.cc:237] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:57.608186 20879 consensus_queue.cc:237] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:57.635376 20723 consensus_queue.cc:237] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:57.685771 20808 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:57.703800 15827 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:57.706826 15861 raft_consensus.cc:1240] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 13. Current term is 14. Ops: []
W20260709 13:06:57.674302 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 129488us
I20260709 13:06:57.709678 15855 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
I20260709 13:06:57.710670 16533 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:57.692917 20944 raft_consensus.cc:697] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:57.667979 20929 consensus_queue.cc:260] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:57.735379 15827 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:57.691905 20797 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
W20260709 13:06:57.744515 20961 log.cc:927] Time spent T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 13:06:57.745842 20956 log.cc:927] Time spent T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:06:57.691051 20855 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:57.757076 16562 raft_consensus.cc:1275] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:06:57.788126 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 52069.6us
W20260709 13:06:57.781653 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:57.792253 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.764583 15827 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:57.783416 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:57.794432 15931 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
W20260709 13:06:57.750993 15900 leader_election.cc:336] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 2.243s (SENT)
I20260709 13:06:57.692216 20947 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:57.788776 15932 raft_consensus.cc:1275] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:06:57.798254 15827 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:57.795511 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 44564.9us
I20260709 13:06:57.772629 15930 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
W20260709 13:06:57.764547 20952 log.cc:927] Time spent T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
W20260709 13:06:57.768146 20955 log.cc:927] Time spent T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
W20260709 13:06:57.767611 20953 log.cc:927] Time spent T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
I20260709 13:06:57.765951 20752 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:57.765771 20677 raft_consensus.cc:2749] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:57.771827 20768 raft_consensus.cc:2749] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:57.751889 20920 consensus_queue.cc:1059] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 13:06:57.766985 20772 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:57.795699 20858 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.800631 20935 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:57.747638 20736 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:57.809185 20970 raft_consensus.cc:2749] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:57.838330 20772 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:57.848420 20937 consensus_queue.cc:1059] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:57.849602 20937 raft_consensus.cc:3055] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:57.849929 20937 raft_consensus.cc:740] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:57.850661 20937 consensus_queue.cc:260] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:57.851889 20949 log.cc:927] Time spent T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.248s	user 0.000s	sys 0.001s
I20260709 13:06:57.755878 20945 consensus_queue.cc:237] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:57.863581 20947 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:57.864351 20944 consensus_queue.cc:237] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:57.873441 20868 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:57.877244 20803 consensus_meta.cc:287] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.536s	user 0.004s	sys 0.005s
I20260709 13:06:57.877719 20803 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:57.882121 15825 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:57.882798 20736 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:06:57.888011 20556 consensus_meta.cc:287] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.649s	user 0.012s	sys 0.000s
I20260709 13:06:57.843559 20858 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:57.888752 20731 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:57.889811 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 161695us
I20260709 13:06:57.890618 20858 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:57.893863 20499 consensus_meta.cc:287] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.532s	user 0.011s	sys 0.000s
I20260709 13:06:57.895404 15902 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:06:57.896775 15902 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:57.897256 20936 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:57.897874 20936 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:57.898089 20938 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:57.898409 20936 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:57.898547 20938 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:57.899101 20938 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:57.899608 20750 consensus_queue.cc:1048] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:06:57.900225 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 102077us
I20260709 13:06:57.901037 15827 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:57.901499 20975 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:57.902232 15827 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:57.898922 15902 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:57.905503 20768 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:57.905956 20768 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:57.907910 20936 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:57.908860 20868 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:57.908715 20808 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:57.910063 20803 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:57.910316 20936 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:57.910813 20808 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:57.894254 20499 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:57.913934 20499 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:57.914623 20937 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:57.918942 20855 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:57.924901 20689 raft_consensus.cc:2804] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:57.925388 20689 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.925722 20689 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:57.927659 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:57.928483 16570 raft_consensus.cc:2393] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 14: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:57.928620 20929 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:57.929517 20841 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:57.929574 20752 consensus_queue.cc:1048] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:57.932157 20948 raft_consensus.cc:2749] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:57.936932 20773 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:06:57.938109 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.938840 20750 consensus_queue.cc:1059] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:57.940452 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:57.940639 20556 raft_consensus.cc:3055] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:57.941066 15936 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:57.941095 20556 raft_consensus.cc:740] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:57.942227 20556 consensus_queue.cc:260] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:57.906353 20768 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:57.943724 20556 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:57.945317 15934 raft_consensus.cc:1240] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:57.945473 15902 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:57.946302 20797 raft_consensus.cc:2749] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:57.946646 15902 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:57.948189 20750 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:57.902252 20975 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:57.948734 20810 consensus_queue.cc:1059] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:57.949329 20975 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:57.949725 20975 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:57.949796 20677 raft_consensus.cc:3055] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:57.939340 20689 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:57.950344 20677 raft_consensus.cc:740] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:57.951296 20677 consensus_queue.cc:260] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:57.952656 20677 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:57.952931 20689 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:57.890658 20731 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:06:57.832094 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 103569us
I20260709 13:06:57.955657 16533 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:57.956861 20875 raft_consensus.cc:2804] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:57.957221 20875 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.957531 20875 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:57.957643 16533 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:57.958963 20920 raft_consensus.cc:2749] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:57.959306 20731 consensus_queue.cc:1059] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:57.960261 20731 raft_consensus.cc:3055] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:57.960578 20731 raft_consensus.cc:740] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:57.960670 20726 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:57.961025 20934 raft_consensus.cc:2749] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:57.961441 20731 consensus_queue.cc:260] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:57.962700 20731 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:57.963882 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.964175 15902 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:06:57.964476 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.965019 20936 raft_consensus.cc:2749] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:57.965334 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:57.965961 15932 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
W20260709 13:06:57.966249 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.967156 15825 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:06:57.968310 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 64698.3us
I20260709 13:06:57.968899 20934 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:57.969090 15827 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:57.969336 20934 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.969976 20810 raft_consensus.cc:2764] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:06:57.969679 20934 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:57.971992 15759 catalog_manager.cc:5697] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:57.973569 15759 catalog_manager.cc:5697] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
I20260709 13:06:57.978152 15932 raft_consensus.cc:1240] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:57.989336 20855 consensus_queue.cc:1048] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:57.976927 20369 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:57.990480 20369 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:57.991415 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:57.992069 15930 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:57.967211 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 75638.3us
I20260709 13:06:57.959061 16533 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:57.995852 15974 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:57.989462 20689 consensus_queue.cc:1059] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:57.999161 20983 raft_consensus.cc:3055] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:57.999577 20983 raft_consensus.cc:740] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:58.000571 20858 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:57.992838 20369 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:57.996833 20920 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:58.000592 20983 consensus_queue.cc:260] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:58.002396 20983 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:58.003993 20927 raft_consensus.cc:3055] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:58.004436 20927 raft_consensus.cc:740] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:57.974876 15759 catalog_manager.cc:5697] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 7 to 10, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 10 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:06:58.005411 20927 consensus_queue.cc:260] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:58.006824 20917 consensus_queue.cc:1175] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "31882606e87447fb9a36bbf625129fe8" responder_term: 14 status { last_received { term: 12 index: 10 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 13. Current term is 14. Ops: [13.11-13.11]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:06:58.007931 16006 raft_consensus.cc:1275] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:06:58.006572 20927 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:58.011231 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:58.011813 16006 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 11
W20260709 13:06:58.013830 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.015616 15854 raft_consensus.cc:1275] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:06:58.017804 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:58.018337 15929 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:58.021484 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:58.006357 15759 catalog_manager.cc:5697] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:58.022984 15759 catalog_manager.cc:5697] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:58.024904 15759 catalog_manager.cc:5697] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 9 to 12, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 12 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:58.029592 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:58.030184 16007 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:58.036075 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:58.036722 16005 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:58.046434 16005 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:06:57.970415 15827 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:58.047492 20983 raft_consensus.cc:2749] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:06:58.048703 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 78355.2us
I20260709 13:06:58.048846 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:58.049801 15827 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:58.050587 20983 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:58.050959 20983 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:58.051332 20983 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:58.064633 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:58.065340 15854 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:58.072047 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 76389.8us
I20260709 13:06:58.072525 15854 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:06:58.073644 15974 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:58.074457 20855 consensus_queue.cc:1048] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:58.075992 20855 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:58.077193 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:58.078445 20936 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.079145 20936 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:58.081897 20936 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:58.083110 16007 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
I20260709 13:06:58.089481 20920 raft_consensus.cc:2749] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:06:57.982087 15760 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58426 (request call id 343) took 33420 ms (client timeout 15000 ms). Trace:
I20260709 13:06:57.979651 20934 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:58.027168 15759 catalog_manager.cc:5697] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 7 to 9, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 9 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:58.094415 15827 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:57.966198 15902 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:58.097096 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:58.111114 15929 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
I20260709 13:06:58.122957 16011 raft_consensus.cc:1240] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:58.124840 16535 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:58.129783 16006 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
I20260709 13:06:58.107290 15974 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:06:58.129271 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:58.133221 15932 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:58.134775 15827 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:58.134995 16533 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:58.135550 15759 catalog_manager.cc:5697] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 3 to 11, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 11 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:06:58.045058 20917 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.087s
I20260709 13:06:58.138015 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:58.143891 15854 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:06:58.141633 15827 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:58.146708 15759 catalog_manager.cc:5697] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 6 to 13, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 13 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:06:58.148138 15827 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:58.130136 15936 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:06:58.152271 20978 raft_consensus.cc:2749] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:58.154616 15930 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:06:57.868235 20772 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:58.156370 20853 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:58.157061 20853 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:58.157470 20853 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:58.157819 20772 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:58.161361 20912 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:58.162118 20912 raft_consensus.cc:697] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:58.162768 15825 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:58.163204 20930 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:58.164096 20979 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:58.164673 20979 raft_consensus.cc:697] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:58.165648 20979 consensus_queue.cc:237] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:58.156286 20948 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:58.146111 20752 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:58.147066 20945 raft_consensus.cc:2804] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:58.173067 20945 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:58.173578 20945 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:58.173734 20934 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:58.174500 20930 raft_consensus.cc:697] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:58.146699 20936 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:58.175391 20930 consensus_queue.cc:237] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:58.176306 20936 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:58.176467 20983 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:58.176967 20936 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:58.149478 16535 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:58.182710 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.187173 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:58.188196 16011 raft_consensus.cc:2393] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 11: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:58.193053 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:58.200920 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:58.204787 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:58.205516 15855 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:58.207186 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:58.199810 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:58.213805 16567 raft_consensus.cc:1275] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:06:58.214597 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:58.214485 15759 catalog_manager.cc:5697] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 6 to 8, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 8 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:58.218111 16003 raft_consensus.cc:1240] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:06:58.218257 15825 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:58.217875 15759 catalog_manager.cc:5697] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 6 to 10, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 10 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:58.219108 20944 raft_consensus.cc:2804] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:58.216197 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:58.220193 15928 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:58.220753 15853 raft_consensus.cc:2410] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:06:58.222348 15855 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:06:58.219694 20944 raft_consensus.cc:697] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:58.225219 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:58.225836 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:58.225973 15861 raft_consensus.cc:1240] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:58.227991 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:06:58.226085 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 76860.3us
W20260709 13:06:58.226595 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 46293.2us
I20260709 13:06:58.227536 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:58.232963 16006 raft_consensus.cc:2393] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 8: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:58.234103 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:58.229920 16533 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:58.235002 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:58.221621 15759 catalog_manager.cc:5697] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 4 to 10, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 10 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:58.234984 16569 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:06:58.225561 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:58.242017 16563 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:58.121973 20734 consensus_queue.cc:1048] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.092s
I20260709 13:06:58.250058 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:58.247655 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 281522us
I20260709 13:06:58.178597 20983 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:58.071645 20768 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:58.253077 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:58.252234 15902 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:58.259886 15931 raft_consensus.cc:1240] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:58.265161 15759 catalog_manager.cc:5697] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 7 to 10, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 10 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:58.260085 16533 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:57.987489 20875 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:58.272215 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:58.272977 15861 raft_consensus.cc:2393] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 14: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
W20260709 13:06:58.276981 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 47544.3us
I20260709 13:06:58.241609 20726 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:58.216832 20858 consensus_queue.cc:1059] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:06:58.281399 15759 catalog_manager.cc:5697] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 9 to 12, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 12 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:58.286530 15857 raft_consensus.cc:1240] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:58.284861 16533 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:58.138274 20938 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:58.292014 16566 raft_consensus.cc:2393] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 11: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:58.291217 15759 catalog_manager.cc:5697] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:06:58.163367 20912 consensus_queue.cc:237] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:58.267134 20868 raft_consensus.cc:2804] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:58.298247 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:58.299005 15853 raft_consensus.cc:2376] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 14. Re-sending same reply.
I20260709 13:06:58.271754 20982 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:58.304697 16009 raft_consensus.cc:1275] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:06:58.297210 15974 leader_election.cc:336] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 1.738s (SENT)
I20260709 13:06:58.308777 15858 raft_consensus.cc:1240] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:06:58.308669 15759 catalog_manager.cc:5697] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 4 to 11, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 11 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:06:58.311305 15759 catalog_manager.cc:5697] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 4 to 6, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 6 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
I20260709 13:06:58.312700 15759 catalog_manager.cc:5697] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:58.315661 15934 raft_consensus.cc:1240] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:58.314414 15759 catalog_manager.cc:5697] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:58.272327 20883 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.025s
W20260709 13:06:58.107600 15760 rpcz_store.cc:269] 0709 13:06:24.564663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.581097 (+ 16434us) service_pool.cc:224] Handling call
0709 13:06:57.981811 (+33400714us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:51.236229 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:51.299887 (+ 63658us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:51.299923 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:06:51.299942 (+    19us) tablet.cc:662] Decoding operations
0709 13:06:51.300746 (+   804us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:51.300815 (+    69us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:51.300847 (+    32us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:06:51.317110 (+ 16263us) tablet.cc:701] Row locks acquired
0709 13:06:51.317135 (+    25us) write_op.cc:260] PREPARE: finished
0709 13:06:51.317303 (+   168us) write_op.cc:270] Start()
0709 13:06:51.317428 (+   125us) write_op.cc:276] Timestamp: P: 1783602411317251 usec, L: 0
0709 13:06:51.317453 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:06:51.318179 (+   726us) log.cc:844] Serialized 2444 byte log entry
0709 13:06:51.701236 (+383057us) op_driver.cc:464] REPLICATION: finished
0709 13:06:51.891985 (+190749us) write_op.cc:301] APPLY: starting
0709 13:06:51.895937 (+  3952us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:51.958210 (+ 62273us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:51.958311 (+   101us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:52.404802 (+446491us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:52.416220 (+ 11418us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=12,key_file_lookups=12,delta_file_lookups=12,mrs_lookups=0
0709 13:06:52.416305 (+    85us) write_op.cc:312] APPLY: finished
0709 13:06:52.416679 (+   374us) log.cc:844] Serialized 65 byte log entry
0709 13:06:52.417969 (+  1290us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:52.418113 (+   144us) write_op.cc:454] Released schema lock
0709 13:06:52.418178 (+    65us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:52.423724 (+  5546us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:52.423765 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4702,"reported_tablets":46,"spinlock_wait_cycles":22784,"child_traces":[["op",{"apply.queue_time_us":183782,"apply.run_cpu_time_us":11560,"apply.run_wall_time_us":536565,"cfile_cache_hit":26,"cfile_cache_hit_bytes":29267,"num_ops":6,"prepare.queue_time_us":410034,"prepare.run_cpu_time_us":24311,"prepare.run_wall_time_us":318925,"raft.queue_time_us":121246,"raft.run_cpu_time_us":9983,"raft.run_wall_time_us":122157,"replication_time_us":381057,"thread_start_us":673196,"threads_started":4,"wal-append.queue_time_us":219097,"wal-append.run_cpu_time_us":28161,"wal-append.run_wall_time_us":1880553}]]}
I20260709 13:06:58.319231 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:58.331342 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:58.337229 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 52454.4us
I20260709 13:06:58.302027 20797 raft_consensus.cc:2749] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:06:58.355063 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.363595 20726 raft_consensus.cc:697] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:58.363876 20731 raft_consensus.cc:2804] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:58.364387 20731 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:58.364784 20731 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:58.364682 20726 consensus_queue.cc:237] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:58.367816 20868 raft_consensus.cc:697] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:58.368903 15759 catalog_manager.cc:5697] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 2 to 11, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 11 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:58.367966 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:58.300928 20858 consensus_queue.cc:1243] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer 31882606e87447fb9a36bbf625129fe8 log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 13:06:58.395531 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 143377us
I20260709 13:06:58.332348 20980 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:58.316231 20648 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:06:58.405189 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 67195.6us
W20260709 13:06:58.411365 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 114165us
W20260709 13:06:58.416407 15974 leader_election.cc:336] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 1.828s (SENT)
W20260709 13:06:58.371984 20969 log.cc:927] Time spent T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:06:58.415429 15854 raft_consensus.cc:1275] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:06:58.426708 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:58.420918 20980 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:58.433938 16004 raft_consensus.cc:3055] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:58.434366 16004 raft_consensus.cc:740] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:58.437546 15932 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:06:58.436597 15759 catalog_manager.cc:5697] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 6 to 8, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 8 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:06:58.440618 15853 raft_consensus.cc:2393] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 8: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:58.435357 20980 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:58.438750 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:58.458604 15861 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
I20260709 13:06:58.447875 15902 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:58.308205 20944 consensus_queue.cc:237] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:58.470398 15759 catalog_manager.cc:5697] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 9 to 12, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 12 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
I20260709 13:06:58.479712 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:58.482941 15759 catalog_manager.cc:5697] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:06:58.317088 20982 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:58.488365 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 71976.7us
I20260709 13:06:58.480480 20648 raft_consensus.cc:697] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:06:58.565829 20995 log.cc:927] Time spent T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:06:58.530776 20981 consensus_queue.cc:1048] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 13:06:58.584625 21000 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:58.584924 20773 consensus_meta.cc:287] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.647s	user 0.006s	sys 0.003s
I20260709 13:06:58.585397 20773 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:58.586241 15935 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:58.587911 15857 raft_consensus.cc:1240] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 10. Current term is 11. Ops: []
W20260709 13:06:58.577152 20986 log.cc:927] Time spent T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.161s	user 0.000s	sys 0.003s
I20260709 13:06:58.496285 20930 consensus_queue.cc:1059] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:06:58.588121 20938 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:58.578049 20648 consensus_queue.cc:237] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:58.579888 20990 log.cc:927] Time spent T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.170s	user 0.000s	sys 0.003s
I20260709 13:06:58.493462 20868 consensus_queue.cc:237] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:58.636525 15853 raft_consensus.cc:1240] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:58.637203 15857 raft_consensus.cc:1275] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:06:58.580710 15902 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:58.594519 16565 raft_consensus.cc:1240] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:58.602896 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:58.603490 16563 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:06:58.634163 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:58.633775 15928 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:06:58.635847 15935 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:06:58.637006 15759 catalog_manager.cc:5697] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 6 to 9, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 9 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:06:58.639951 15854 raft_consensus.cc:2376] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12. Re-sending same reply.
I20260709 13:06:58.652719 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:58.633056 20988 log.cc:927] Time spent T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.237s	user 0.005s	sys 0.000s
I20260709 13:06:58.666486 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:06:58.662042 15974 leader_election.cc:336] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 2.215s (SENT)
W20260709 13:06:58.637147 20677 consensus_meta.cc:287] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.684s	user 0.006s	sys 0.007s
I20260709 13:06:58.670364 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:58.672583 15932 raft_consensus.cc:1275] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 13:06:58.667702 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 87030.6us
W20260709 13:06:58.637010 20975 consensus_meta.cc:287] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.687s	user 0.006s	sys 0.006s
W20260709 13:06:58.639905 20937 consensus_meta.cc:287] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.725s	user 0.008s	sys 0.005s
I20260709 13:06:58.675542 15856 raft_consensus.cc:1275] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:06:58.680714 15974 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:58.678731 15759 catalog_manager.cc:5697] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 6 to 11. New cstate: current_term: 11 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
I20260709 13:06:58.682505 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:58.636693 20810 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.684340 15825 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:06:58.673327 16004 consensus_queue.cc:260] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:06:58.693012 15900 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:06:58.694829 15827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.696619 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:58.696962 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:58.697399 16563 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:58.705566 15759 catalog_manager.cc:5697] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
W20260709 13:06:58.675844 20985 log.cc:927] Time spent T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.394s	user 0.004s	sys 0.006s
W20260709 13:06:58.713955 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 51947.6us
W20260709 13:06:58.714846 15974 leader_election.cc:336] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 2.339s (SENT)
I20260709 13:06:58.707581 16004 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:58.587271 20936 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:58.675647 20938 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:58.722892 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:58.715329 15974 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:58.732867 16011 raft_consensus.cc:2393] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 11: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
W20260709 13:06:58.678018 20945 consensus_meta.cc:287] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.504s	user 0.012s	sys 0.000s
I20260709 13:06:58.730430 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:58.734820 15932 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:06:58.735375 15759 catalog_manager.cc:5697] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 10 to 11, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 11 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:58.733222 15900 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:58.724429 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:58.676740 20875 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:58.737130 16565 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
W20260709 13:06:58.744077 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 265363us
I20260709 13:06:58.746694 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:58.747359 16006 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:58.749787 16563 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:06:58.668648 20731 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:58.753494 15900 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:58.754292 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:58.722949 20855 raft_consensus.cc:2749] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:58.754683 15759 catalog_manager.cc:5697] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 10 to 13, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 13 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:58.685524 20975 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:58.734333 20772 consensus_queue.cc:1059] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:06:58.775363 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:58.776572 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:58.762678 15759 catalog_manager.cc:5697] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:58.759817 16009 raft_consensus.cc:3055] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:58.777415 16009 raft_consensus.cc:740] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:58.775228 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:58.755579 20937 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:58.780975 16006 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:06:58.758415 20841 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:58.705806 20810 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:58.781749 15900 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:58.757795 20736 raft_consensus.cc:2749] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:58.859328 15759 catalog_manager.cc:5697] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 6 to 11, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 11 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:58.864393 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 149575us
I20260709 13:06:58.865890 15974 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:58.746011 20945 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:58.882511 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:58.763291 20768 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:58.756083 20983 consensus_queue.cc:1048] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.116s
W20260709 13:06:58.889063 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 107358us
I20260709 13:06:58.916299 15900 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:58.891616 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:58.890237 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 479173us
I20260709 13:06:58.917750 15936 raft_consensus.cc:2393] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 8: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:58.887928 15935 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:06:58.882208 16009 consensus_queue.cc:260] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:58.919334 16009 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:58.756469 20875 consensus_queue.cc:1059] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 13:06:58.762440 21006 raft_consensus.cc:2804] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:58.931317 16008 raft_consensus.cc:2393] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 11: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:58.933122 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:58.760968 21009 consensus_queue.cc:1048] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.139s
I20260709 13:06:58.883601 20879 raft_consensus.cc:2749] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:58.887367 20750 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:06:58.922106 20556 consensus_meta.cc:287] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.978s	user 0.015s	sys 0.000s
I20260709 13:06:58.946269 20999 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:58.900468 20937 raft_consensus.cc:697] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 14 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:58.904556 20841 raft_consensus.cc:697] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:58.962538 20841 consensus_queue.cc:237] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:58.968117 21002 log.cc:927] Time spent T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.313s	user 0.000s	sys 0.001s
I20260709 13:06:58.968902 20937 consensus_queue.cc:237] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:58.977888 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:58.978152 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:58.978838 15930 raft_consensus.cc:2368] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 12. Current term is 13.
I20260709 13:06:58.980047 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:58.980768 15933 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:06:58.981727 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:58.982393 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:58.983486 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:58.983873 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.986232 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:58.987320 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:58.987886 15931 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:06:58.988072 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:58.988690 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:58.989462 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:58.997252 20810 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:58.911489 20855 consensus_queue.cc:1059] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.089s
I20260709 13:06:59.000516 20855 consensus_queue.cc:1243] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer 31882606e87447fb9a36bbf625129fe8 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:06:59.008685 15929 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:06:59.015067 15932 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
I20260709 13:06:58.883783 21007 consensus_queue.cc:1059] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 13:06:59.010465 15930 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:06:59.028343 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:59.008286 20853 consensus_meta.cc:287] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.850s	user 0.014s	sys 0.002s
I20260709 13:06:59.032671 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:59.033926 15936 raft_consensus.cc:2368] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 10. Current term is 11.
I20260709 13:06:59.036954 15937 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 12: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
W20260709 13:06:59.022089 20752 consensus_meta.cc:287] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.852s	user 0.017s	sys 0.001s
I20260709 13:06:58.945492 21006 raft_consensus.cc:697] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:59.020455 20723 raft_consensus.cc:3055] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:59.009133 21013 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:59.052114 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:59.052742 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 368457us
I20260709 13:06:59.008509 20670 consensus_queue.cc:1048] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
W20260709 13:06:59.060343 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 327605us
I20260709 13:06:59.065264 15827 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:59.031757 20976 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:59.072515 15827 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:59.043424 20982 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:59.049708 20752 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:59.048029 20723 raft_consensus.cc:740] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:06:59.086445 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:59.041699 20853 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:59.093298 15858 raft_consensus.cc:1240] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:59.094767 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:59.104305 20945 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:59.072206 21009 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.107209 21009 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:59.071856 20768 consensus_queue.cc:1048] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:06:59.120399 20723 consensus_queue.cc:260] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:59.121889 20723 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:59.124634 20752 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:59.129739 20853 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:59.130959 20853 raft_consensus.cc:2804] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:59.132462 15859 raft_consensus.cc:1240] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:59.051268 21007 consensus_queue.cc:1059] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:06:59.166960 16009 raft_consensus.cc:1275] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:06:59.080194 15935 raft_consensus.cc:2393] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 13: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:59.076154 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:59.060508 20946 consensus_queue.cc:1048] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:06:59.178774 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:59.249763 15929 raft_consensus.cc:2393] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 12: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:59.079277 20936 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:59.182667 16004 raft_consensus.cc:1275] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:06:59.247476 21000 consensus_meta.cc:287] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.661s	user 0.007s	sys 0.005s
I20260709 13:06:59.075627 20976 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:59.250535 21012 log.cc:927] Time spent T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:06:59.210049 21009 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:06:59.198499 21014 log.cc:927] Time spent T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.053s	user 0.001s	sys 0.001s
I20260709 13:06:59.197813 20975 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:59.220108 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:59.233776 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:59.216257 20731 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:59.079595 15860 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
I20260709 13:06:59.246587 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:59.131493 20853 raft_consensus.cc:697] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:59.251412 16004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2140) took 1054 ms. Trace:
I20260709 13:06:59.250363 20920 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.265380 16004 rpcz_store.cc:276] 0709 13:06:58.197135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.197263 (+   128us) service_pool.cc:224] Handling call
0709 13:06:58.197588 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:58.433839 (+236251us) spinlock_profiling.cc:243] Waited 191 ms on lock 0x7b5400165920. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea31fdc0 00007f73ea3aa1c0 00007f73ea3ab231 00007f73ea3a7d8f 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c
0709 13:06:58.672976 (+239137us) spinlock_profiling.cc:243] Waited 4.81 ms on lock 0x7b4c0026cd00. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e8674a20 00007f73e8685a29 00007f73e864b644 00007f73e8672f2d 00007f73e8672b93 00007f73ea3a318b 00007f73ea3a3cb5 00007f73ea39ef30 00007f73ea3a07de 00007f73ea3aab49 00007f73ea3ab416
0709 13:06:59.251184 (+578208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:59.251361 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":449625216}
W20260709 13:06:59.252385 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 235312us
W20260709 13:06:59.253266 20704 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
W20260709 13:06:59.267805 20938 consensus_meta.cc:287] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.524s	user 0.012s	sys 0.000s
I20260709 13:06:59.213125 21006 consensus_queue.cc:237] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:59.270819 16564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 1039) took 1078 ms. Trace:
I20260709 13:06:59.271185 16564 rpcz_store.cc:276] 0709 13:06:58.192530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.192658 (+   128us) service_pool.cc:224] Handling call
0709 13:06:58.192925 (+   267us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:58.264042 (+ 71117us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:58.264089 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:58.280597 (+ 16508us) log.cc:844] Serialized 26 byte log entry
0709 13:06:58.724258 (+443661us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:58.724299 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:58.724368 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:59.270613 (+546245us) raft_consensus.cc:1685] finished
0709 13:06:59.270631 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:59.270765 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41039,"raft.queue_time_us":54405,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":9826,"thread_start_us":391384,"threads_started":1,"wal-append.queue_time_us":781457}
I20260709 13:06:59.264569 15931 raft_consensus.cc:2393] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 13: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:59.253638 21000 raft_consensus.cc:2749] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:59.268285 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:59.268257 20938 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:06:59.286536 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 214073us
W20260709 13:06:59.290866 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:59.291148 15858 raft_consensus.cc:1275] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:06:59.291205 15827 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:59.294287 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:59.299075 15827 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:59.301704 15827 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:06:59.305709 15936 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:06:59.305729 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:59.310108 16564 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEARNER]: Advancing to term 14
I20260709 13:06:59.313058 15934 raft_consensus.cc:2410] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:06:59.314443 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:06:59.335049 21020 log.cc:927] Time spent T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:06:59.339967 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:59.341454 15931 raft_consensus.cc:1240] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:06:59.342288 15928 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:06:59.343559 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:59.345158 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:59.345454 21025 consensus_queue.cc:1048] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:59.346553 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:59.347184 20648 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.347714 20648 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:59.347985 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:59.348677 20976 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:59.349043 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:59.307981 21019 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:59.349932 20648 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:59.350143 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:59.350739 15928 raft_consensus.cc:2368] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 12. Current term is 13.
I20260709 13:06:59.351014 15930 raft_consensus.cc:2393] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 8: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:59.352152 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:59.352574 15935 raft_consensus.cc:1240] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:06:59.353302 15930 raft_consensus.cc:2393] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 11: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:59.354105 15855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2344) took 1089 ms. Trace:
I20260709 13:06:59.348701 15936 raft_consensus.cc:2393] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 13: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:59.351608 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:59.342381 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:59.354525 15855 rpcz_store.cc:276] 0709 13:06:58.265092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.265542 (+   450us) service_pool.cc:224] Handling call
0709 13:06:58.265852 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:58.266057 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:58.266080 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:58.266487 (+   407us) log.cc:844] Serialized 26 byte log entry
0709 13:06:58.702405 (+435918us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:58.702724 (+   319us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:58.703123 (+   399us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:59.353900 (+650777us) raft_consensus.cc:1685] finished
0709 13:06:59.353922 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:59.354050 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":401629,"threads_started":1,"wal-append.queue_time_us":776806}
I20260709 13:06:59.354027 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:59.273201 20920 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:59.357909 15929 raft_consensus.cc:1240] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:59.308969 20768 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:59.378650 15930 raft_consensus.cc:1275] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:06:59.383333 15936 raft_consensus.cc:1240] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:59.384298 15937 raft_consensus.cc:1275] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:06:59.312436 21018 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:06:59.388777 16564 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 LEARNER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:06:59.353287 20975 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.388854 15930 raft_consensus.cc:1275] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:06:59.360373 20810 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:59.394371 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:59.412671 15931 raft_consensus.cc:2393] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 11: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:59.395936 21028 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.387367 20853 consensus_queue.cc:237] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:59.445776 16564 raft_consensus.cc:1275] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:06:59.411727 20975 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:59.441923 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:59.447492 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:59.448226 15931 raft_consensus.cc:2376] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12. Re-sending same reply.
I20260709 13:06:59.452175 20920 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:59.446273 20677 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:59.453467 20677 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:59.454878 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:59.455451 15930 raft_consensus.cc:2376] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12. Re-sending same reply.
I20260709 13:06:59.455489 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:59.456746 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:59.457059 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:59.457734 15935 raft_consensus.cc:2393] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 14: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:59.457782 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:59.458385 15929 raft_consensus.cc:2376] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12. Re-sending same reply.
I20260709 13:06:59.458539 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:59.459227 15935 raft_consensus.cc:2393] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 13: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:59.459164 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:59.459975 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:59.460516 20677 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:59.460914 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:59.461272 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:59.462008 15931 raft_consensus.cc:2368] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 10. Current term is 11.
I20260709 13:06:59.443154 21028 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:59.463177 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:59.463896 15936 raft_consensus.cc:2376] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12. Re-sending same reply.
I20260709 13:06:59.456764 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:59.464571 21028 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:59.466519 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:59.453807 20752 raft_consensus.cc:2749] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:59.439988 21019 raft_consensus.cc:697] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:59.473345 15929 raft_consensus.cc:2393] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 13: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:59.476675 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:59.478497 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:59.481478 16563 raft_consensus.cc:2410] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:06:59.429262 20810 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:59.486392 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 67287.1us
I20260709 13:06:59.486583 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:59.488513 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:59.489507 15931 raft_consensus.cc:2393] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 13: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:59.488382 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:06:59.488722 15974 leader_election.cc:336] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 1.521s (SENT)
I20260709 13:06:59.490126 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:59.490338 15974 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:59.490839 15928 raft_consensus.cc:2393] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 8: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:59.496271 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:59.504566 15928 raft_consensus.cc:2393] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 13: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:59.496469 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:59.505906 15936 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 12: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:06:59.497983 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:59.507223 16565 raft_consensus.cc:2368] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 12. Current term is 13.
I20260709 13:06:59.498304 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:59.509105 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:59.511804 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:59.512063 16565 raft_consensus.cc:1240] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 LEARNER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:06:59.513175 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:59.513688 16565 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:06:59.516453 16563 raft_consensus.cc:1240] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:59.516635 16570 raft_consensus.cc:1240] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:59.518481 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:59.519165 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:59.520684 15933 raft_consensus.cc:3055] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:59.526319 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:59.529834 16565 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:06:59.521032 15933 raft_consensus.cc:740] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:59.533311 16565 raft_consensus.cc:1240] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:59.534596 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:59.534830 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:59.535859 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:59.532795 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:59.536841 15936 raft_consensus.cc:2393] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 11: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:06:59.538169 16568 raft_consensus.cc:1240] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:06:59.496942 20975 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:59.551515 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:59.552624 16569 raft_consensus.cc:1240] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:59.552742 21019 consensus_queue.cc:237] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:59.556464 21001 log.cc:927] Time spent T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.890s	user 0.010s	sys 0.014s
I20260709 13:06:59.557715 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2138) took 1363 ms. Trace:
I20260709 13:06:59.558044 16002 rpcz_store.cc:276] 0709 13:06:58.194643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.194842 (+   199us) service_pool.cc:224] Handling call
0709 13:06:58.227111 (+ 32269us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:58.245508 (+ 18397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:58.245537 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:58.270180 (+ 24643us) log.cc:844] Serialized 26 byte log entry
0709 13:06:58.363027 (+ 92847us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:58.370370 (+  7343us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:58.370446 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:59.489395 (+1118949us) spinlock_profiling.cc:243] Waited 3.01 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:06:59.557444 (+ 68049us) raft_consensus.cc:1685] finished
0709 13:06:59.557463 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:59.557655 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":20698,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":12160,"spinlock_wait_cycles":6918528,"thread_start_us":60889,"threads_started":1,"wal-append.queue_time_us":206161}
I20260709 13:06:59.561897 20810 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:59.562883 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:59.491410 21040 consensus_queue.cc:1048] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.095s
W20260709 13:06:59.586818 21005 log.cc:927] Time spent T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.999s	user 0.010s	sys 0.007s
I20260709 13:06:59.544322 21022 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:59.588199 21022 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:59.590341 21022 leader_election.cc:290] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:59.545692 15933 consensus_queue.cc:260] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:59.591418 15933 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:59.591704 20920 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:06:59.592321 20920 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:59.594563 20920 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:59.595060 20920 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.596256 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:59.601989 21040 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:59.604197 16568 raft_consensus.cc:1275] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:06:59.633914 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 145193us
W20260709 13:06:59.634954 15974 leader_election.cc:336] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 1.649s (SENT)
I20260709 13:06:59.595503 20920 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:59.637732 20920 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:59.654451 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:59.661044 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:59.666514 15902 leader_election.cc:400] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 12. Current term is 13.
I20260709 13:06:59.666922 15902 leader_election.cc:403] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
W20260709 13:06:59.668112 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:59.669554 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:59.670150 15856 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
W20260709 13:06:59.672039 15974 leader_election.cc:336] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 1.735s (SENT)
I20260709 13:06:59.672665 15974 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:59.602537 21040 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:59.674413 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:59.675421 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:59.677109 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:06:59.677212 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:59.679097 15929 raft_consensus.cc:1275] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:06:59.679292 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:59.681116 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:06:59.678218 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:59.688385 15933 raft_consensus.cc:1275] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:06:59.691100 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:59.666692 20920 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:59.666940 20810 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:59.671247 20752 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.698123 20752 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:59.678540 20797 raft_consensus.cc:2749] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 12. Current term is 13.
I20260709 13:06:59.679854 20944 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:59.697417 20768 raft_consensus.cc:2749] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:59.708065 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:59.708776 15853 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
I20260709 13:06:59.696219 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:59.701251 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:59.701331 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:06:59.701484 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 55715us
I20260709 13:06:59.705339 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:59.721932 15902 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:59.726802 15857 raft_consensus.cc:2368] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 10. Current term is 11.
I20260709 13:06:59.727604 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:59.728596 15856 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
I20260709 13:06:59.726477 20797 consensus_queue.cc:1059] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 13:06:59.729857 21006 consensus_queue.cc:1059] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:59.714809 20944 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:06:59.732734 21006 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:59.733178 20556 raft_consensus.cc:3055] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:59.733590 20556 raft_consensus.cc:740] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:59.733482 21006 consensus_queue.cc:1059] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:59.735759 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:59.739784 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:59.704265 21039 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:59.734783 20936 raft_consensus.cc:3055] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:59.743175 20936 raft_consensus.cc:740] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:59.743788 20937 raft_consensus.cc:3055] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:59.745594 20937 raft_consensus.cc:740] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:59.746482 20937 consensus_queue.cc:260] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:59.747578 21004 log.cc:927] Time spent T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 1.081s	user 0.010s	sys 0.000s
I20260709 13:06:59.712407 20934 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:06:59.748415 20934 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:59.748337 20936 consensus_queue.cc:260] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:59.748893 15859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 2246) took 1546 ms. Trace:
I20260709 13:06:59.749650 20936 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:59.749374 15859 rpcz_store.cc:276] 0709 13:06:58.202841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.202950 (+   109us) service_pool.cc:224] Handling call
0709 13:06:58.225459 (+ 22509us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:59.132979 (+907520us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:59.748632 (+615653us) spinlock_profiling.cc:243] Waited 907 ms on lock 0x7b540008e654. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:06:59.748829 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2085364608}
I20260709 13:06:59.750561 20934 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:06:59.751060 16008 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41674 (request call id 2137) took 1552 ms (client timeout 1632 ms). Trace:
W20260709 13:06:59.751427 16008 rpcz_store.cc:269] 0709 13:06:58.192714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.192846 (+   132us) service_pool.cc:224] Handling call
0709 13:06:59.744956 (+1552110us) spinlock_profiling.cc:243] Waited 737 ms on lock 0x7b54001b9754. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:59.745109 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1695453056}
I20260709 13:06:59.699147 20920 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:59.752327 20937 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:59.754132 20920 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:59.704862 20810 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:59.754879 15860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 2261) took 1118 ms. Trace:
I20260709 13:06:59.755380 15860 rpcz_store.cc:276] 0709 13:06:58.636584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.636701 (+   117us) service_pool.cc:224] Handling call
0709 13:06:59.754692 (+1117991us) spinlock_profiling.cc:243] Waited 9.15 ms on lock 0x7b5400217d54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:06:59.754814 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21055744}
I20260709 13:06:59.756678 20810 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:59.762581 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:59.769374 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:59.770555 16564 raft_consensus.cc:2393] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 11: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:06:59.770362 21000 consensus_queue.cc:1059] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:06:59.742726 21040 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:59.773869 21000 consensus_queue.cc:1243] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer eb1e0493af1e46dbb6dec7e25525bab5 log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:06:59.775302 21000 raft_consensus.cc:3055] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:59.775660 21000 raft_consensus.cc:740] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:06:59.776657 20968 raft_consensus.cc:3055] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:59.778759 20968 raft_consensus.cc:740] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:06:59.779657 20968 consensus_queue.cc:260] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:59.780061 21000 consensus_queue.cc:260] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:59.781380 21000 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:06:59.782325 20993 log.cc:927] Time spent T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 1.317s	user 0.000s	sys 0.013s
I20260709 13:06:59.783972 20752 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:59.790242 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:06:59.791635 16533 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:06:59.793177 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:59.795923 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:59.798131 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:59.805686 20920 raft_consensus.cc:2749] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:59.807789 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:59.808684 15933 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:06:59.806926 16533 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:59.812508 15937 raft_consensus.cc:1275] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:06:59.812772 16533 leader_election.cc:400] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 10. Current term is 11.
I20260709 13:06:59.813202 16533 leader_election.cc:403] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:59.818120 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:59.819002 15936 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
I20260709 13:06:59.825719 15825 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:06:59.829811 20875 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.785s	user 0.005s	sys 0.018s
W20260709 13:06:59.830309 20875 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.790s	user 0.006s	sys 0.018s
I20260709 13:06:59.820811 16533 leader_election.cc:304] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:59.807567 20917 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.804886 20850 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:06:59.825249 21040 consensus_queue.cc:1048] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:06:59.829387 20369 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:59.805254 20938 consensus_queue.cc:1059] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:06:59.829102 20920 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 10. Current term is 11.
I20260709 13:06:59.828583 21044 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:59.831537 20934 raft_consensus.cc:2749] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:06:59.847728 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 58718.3us
I20260709 13:06:59.855971 15827 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:06:59.837707 20981 raft_consensus.cc:2804] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:59.860623 15827 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:06:59.842581 20369 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:59.855891 20994 log.cc:927] Time spent T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 1.428s	user 0.000s	sys 0.015s
I20260709 13:06:59.838020 20917 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:06:59.865264 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:59.878129 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:59.868322 20810 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:59.868690 20369 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:59.869207 21058 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.897953 21058 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:59.869618 20934 consensus_queue.cc:1048] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:59.901288 21058 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:59.904053 16008 raft_consensus.cc:1275] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 13:06:59.904517 16008 pending_rounds.cc:85] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting all ops after (but not including) 7
W20260709 13:06:59.869927 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:59.904800 16008 pending_rounds.cc:107] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:06:59.905112 16008 raft_consensus.cc:2889] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:59.844780 20944 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:59.907989 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:59.908733 16011 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
I20260709 13:06:59.908957 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:59.862172 21038 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:06:59.887331 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:59.860241 20981 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:06:59.949534 20981 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:59.950443 15862 raft_consensus.cc:1240] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:59.950959 15855 raft_consensus.cc:1275] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:06:59.893898 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:59.952087 15859 raft_consensus.cc:1240] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:59.952294 16004 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:59.953670 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:59.954643 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:59.955417 15856 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:06:59.956281 15933 raft_consensus.cc:1275] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 13:06:59.956637 15933 pending_rounds.cc:85] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 11
I20260709 13:06:59.956832 15933 pending_rounds.cc:107] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:06:59.957031 15933 raft_consensus.cc:2889] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:59.895212 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:59.870999 20556 consensus_queue.cc:260] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:59.839425 20850 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:59.958683 16009 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:06:59.874089 20768 consensus_queue.cc:1048] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:59.911458 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:59.960949 16010 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:06:59.962369 20850 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:06:59.963344 16533 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:06:59.964480 21058 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.964946 21058 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:06:59.911885 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:06:59.966670 16002 raft_consensus.cc:2393] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 12: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:59.892822 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:59.967010 21058 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:06:59.967434 16006 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:06:59.974210 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:06:59.975113 15859 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
I20260709 13:06:59.975790 15929 raft_consensus.cc:1275] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:06:59.916343 16008 raft_consensus.cc:1275] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:06:59.919242 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:59.976830 16005 raft_consensus.cc:2410] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:06:59.920017 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:06:59.977309 16007 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:06:59.977439 16533 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:06:59.928754 21008 log.cc:927] Time spent T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 1.328s	user 0.000s	sys 0.023s
I20260709 13:06:59.932649 20982 raft_consensus.cc:3055] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:59.907424 20810 raft_consensus.cc:697] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:06:59.937875 16561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1093) took 1888 ms. Trace:
I20260709 13:06:59.946267 20917 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:06:59.947834 21013 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.948655 15935 raft_consensus.cc:1275] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:07:00.025425 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:06:59.987075 16004 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:07:00.026798 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:06:59.984838 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:00.029014 16008 raft_consensus.cc:2393] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 11: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:06:59.967538 21058 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:07:00.027468 20841 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.059s	user 0.022s	sys 0.014s
W20260709 13:07:00.070175 20841 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.102s	user 0.023s	sys 0.014s
I20260709 13:07:00.070348 16562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 1036) took 1880 ms. Trace:
I20260709 13:07:00.072348 16565 raft_consensus.cc:1275] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 13:07:00.072890 16565 pending_rounds.cc:85] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting all ops after (but not including) 10
I20260709 13:07:00.073179 16565 pending_rounds.cc:107] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:07:00.073483 16565 raft_consensus.cc:2889] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:00.019447 16561 rpcz_store.cc:276] 0709 13:06:58.049156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.049275 (+   119us) service_pool.cc:224] Handling call
0709 13:06:58.049552 (+   277us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:58.123733 (+ 74181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:58.123783 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:58.137460 (+ 13677us) log.cc:844] Serialized 26 byte log entry
0709 13:06:58.268486 (+131026us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:58.268522 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:58.268582 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:59.838803 (+1570221us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b24002f6648. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71419c4 00007f73e713facb 00007f73e72fadc9 00007f73e72faa55 00007f73e72fa09d 00007f73e9f2c10e 00007f73e9f4388c 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221
0709 13:06:59.937586 (+ 98783us) raft_consensus.cc:1685] finished
0709 13:06:59.937608 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:59.937817 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":148455,"raft.queue_time_us":79970,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":12498,"spinlock_wait_cycles":26254208,"thread_start_us":101951,"threads_started":1,"wal-append.queue_time_us":152887}
I20260709 13:07:00.057734 21013 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:07:00.069275 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:00.057595 21058 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:00.066926 15932 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 2152) took 1289 ms (client timeout 1649 ms). Trace:
I20260709 13:07:00.070808 16562 rpcz_store.cc:276] 0709 13:06:58.189789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.189946 (+   157us) service_pool.cc:224] Handling call
0709 13:06:58.190320 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:58.225442 (+ 35122us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:58.225483 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:58.225913 (+   430us) log.cc:844] Serialized 26 byte log entry
0709 13:06:58.596664 (+370751us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:06:58.666605 (+ 69941us) log.cc:844] Serialized 14 byte log entry
0709 13:06:58.666805 (+   200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:58.666873 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:59.928101 (+1261228us) spinlock_profiling.cc:243] Waited 1.44 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:07:00.070035 (+141934us) raft_consensus.cc:1685] finished
0709 13:07:00.070132 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:00.070290 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67891,"prepare.queue_time_us":37007,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":89773,"raft.run_cpu_time_us":333,"raft.run_wall_time_us":35174,"spinlock_wait_cycles":3308416,"thread_start_us":315248,"threads_started":1,"wal-append.queue_time_us":332883}
I20260709 13:07:00.081898 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:00.082355 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:00.082885 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:00.081934 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 90385.9us
I20260709 13:07:00.082964 15931 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 12.
I20260709 13:07:00.093147 15937 raft_consensus.cc:1240] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:00.096812 15856 raft_consensus.cc:1275] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:07:00.078693 15932 rpcz_store.cc:269] 0709 13:06:58.758896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.759036 (+   140us) service_pool.cc:224] Handling call
0709 13:07:00.047842 (+1288806us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b54003b8b54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:00.047958 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":549363072}
I20260709 13:07:00.098430 16561 raft_consensus.cc:1275] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:07:00.014096 20982 raft_consensus.cc:740] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:00.100759 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:00.101497 16562 raft_consensus.cc:1275] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:07:00.101552 15933 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
I20260709 13:07:00.102905 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:00.103654 15825 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:00.102048 15827 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:00.107793 16566 raft_consensus.cc:1275] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:07:00.108146 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:00.108803 15936 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:07:00.109393 15928 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:07:00.126480 15827 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:00.125679 15929 raft_consensus.cc:1275] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:07:00.124528 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:00.097224 21058 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:00.129814 15827 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:00.132085 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:00.101107 20930 raft_consensus.cc:2804] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:07:00.168452 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 496409us
I20260709 13:07:00.168867 15827 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
W20260709 13:07:00.169204 15974 leader_election.cc:336] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 1.778s (SENT)
I20260709 13:07:00.101660 20850 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:07:00.168082 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 64532.2us
I20260709 13:07:00.171218 15825 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:00.111531 20825 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.107430 21010 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:00.174876 15827 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:00.201934 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:00.202275 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:00.174355 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:07:00.206034 15974 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 2.131s (SENT) [suppressed 46 similar messages]
I20260709 13:07:00.117187 20734 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:00.263660 16008 raft_consensus.cc:1240] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:07:00.263777 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:00.265641 15935 raft_consensus.cc:1275] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:07:00.264024 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:00.263777 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:00.209863 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.264719 15900 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:00.267990 16011 raft_consensus.cc:2393] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 8: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:00.115291 21056 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.116171 20917 consensus_queue.cc:1048] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.085s
I20260709 13:07:00.169775 20979 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:00.110951 20752 consensus_queue.cc:1048] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.099s
I20260709 13:07:00.168027 20868 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:00.270803 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 96029.3us
W20260709 13:07:00.312943 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 48336.2us
I20260709 13:07:00.170636 20930 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:00.309211 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.209633 20912 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:07:00.208139 20773 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 13:07:00.316221 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:00.341810 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.419835 16005 raft_consensus.cc:1240] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 14. Current term is 15. Ops: [12.10-14.12]
I20260709 13:07:00.169803 20999 raft_consensus.cc:3055] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:00.129024 20946 consensus_queue.cc:1048] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.181s
I20260709 13:07:00.371428 16008 raft_consensus.cc:1240] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:00.305351 16002 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
I20260709 13:07:00.173200 20677 raft_consensus.cc:2749] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:00.208710 21028 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:00.201839 20975 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:00.411873 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:00.201665 20853 raft_consensus.cc:2804] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:00.412623 15859 raft_consensus.cc:1275] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:07:00.357648 20938 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:07:00.418712 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.273154 20983 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:07:00.307453 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 136301us
I20260709 13:07:00.422035 16010 raft_consensus.cc:1240] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:00.421624 15827 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:00.422497 20912 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:00.423446 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:00.424042 21042 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:00.424501 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:00.424446 21045 consensus_queue.cc:1048] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.255s
I20260709 13:07:00.416115 20930 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:07:00.425769 21065 log.cc:927] Time spent T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.317s	user 0.000s	sys 0.000s
I20260709 13:07:00.426388 20773 consensus_queue.cc:1059] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:00.427474 21000 consensus_queue.cc:1059] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:00.477948 21028 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:00.436106 20768 consensus_queue.cc:1048] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:07:00.436864 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.486361 20853 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:00.440554 16011 raft_consensus.cc:1240] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:00.440428 20982 consensus_queue.cc:260] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:00.438473 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:00.518776 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:00.446691 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:00.447052 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.448494 21013 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
I20260709 13:07:00.315760 20734 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:00.264271 20825 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:07:00.305847 20810 consensus_queue.cc:237] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:00.524792 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:00.525421 16011 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 12.
I20260709 13:07:00.526320 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:00.527276 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:00.306201 21038 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.312247 20936 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:07:00.466193 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 260178us
I20260709 13:07:00.413853 21040 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:00.479228 16003 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:00.317278 20979 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:00.480154 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:00.271795 20850 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:00.490779 16009 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:00.489306 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:00.551110 16010 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:00.493296 16005 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:00.458714 20999 raft_consensus.cc:740] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:00.305107 21010 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:00.497664 20752 consensus_queue.cc:1048] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:07:00.489673 15825 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:00.504159 20677 consensus_queue.cc:1048] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:07:00.428313 15900 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:00.478943 20975 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:00.357610 20981 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:00.437518 20369 consensus_meta.cc:287] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.540s	user 0.009s	sys 0.000s
I20260709 13:07:00.357570 21056 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:00.488373 20983 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:00.516618 16561 raft_consensus.cc:1240] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:00.525460 20825 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:00.464038 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:00.584492 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:00.584981 20981 leader_election.cc:290] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:00.580729 20369 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:00.585983 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:00.586354 16535 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:07:00.357273 21053 log.cc:927] Time spent T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:07:00.527544 20734 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:00.589979 20648 raft_consensus.cc:2804] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:00.590389 20648 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:00.590706 20648 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:00.591854 16564 raft_consensus.cc:1240] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:00.587246 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:00.459059 20868 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:00.536154 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.534231 15902 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:00.545775 16002 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:07:00.546391 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:00.512463 21028 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:07:00.495245 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 78466us
I20260709 13:07:00.605711 15827 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:00.555594 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:00.597520 16535 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:00.532259 20944 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.532506 20853 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:07:00.558614 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 69041.3us
I20260709 13:07:00.500464 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:00.538188 21040 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:00.634922 21040 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:00.513155 21071 raft_consensus.cc:2749] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:00.641326 21071 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:00.641994 21071 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:00.549589 20979 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:00.644955 21010 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:00.503888 21042 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:00.648043 20937 raft_consensus.cc:3055] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:00.648574 20937 raft_consensus.cc:740] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:00.648778 21042 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:00.563872 20768 raft_consensus.cc:2804] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:00.649607 20768 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:00.650144 20768 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:00.649730 20937 consensus_queue.cc:260] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:00.570169 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 141923us
I20260709 13:07:00.653403 21000 consensus_queue.cc:1048] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:00.653741 20773 consensus_queue.cc:1059] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:00.557618 20920 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.655997 20920 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:00.658679 20920 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:00.537457 21038 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:00.661096 21038 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:00.569262 20975 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:00.662732 20968 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:00.598191 16569 raft_consensus.cc:1240] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 13. Current term is 14. Ops: []
W20260709 13:07:00.601063 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 66932.4us
I20260709 13:07:00.667740 15902 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:00.668828 20556 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:00.669310 15902 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:00.670699 15902 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:00.551815 21044 consensus_queue.cc:1048] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:07:00.608273 16535 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:00.550952 20936 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:00.587549 20369 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:00.623512 20912 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:00.625730 20850 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:00.616580 20981 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:00.632412 21019 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:00.673221 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:00.661100 16006 raft_consensus.cc:1240] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:00.692457 16009 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:07:00.659438 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:00.620889 16011 raft_consensus.cc:1240] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:00.592672 20983 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:00.581501 20938 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:07:00.677985 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 61172.9us
W20260709 13:07:00.682211 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 74048.9us
I20260709 13:07:00.651489 20937 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:00.717233 16005 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 13.
I20260709 13:07:00.613698 21006 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:07:00.703410 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.697237 16533 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:00.718040 16008 raft_consensus.cc:1240] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:00.723721 16533 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:00.724785 16533 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:00.715276 16535 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:00.719064 16011 raft_consensus.cc:1275] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:07:00.724404 15974 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:00.627293 21058 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:00.733687 16533 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:00.739192 16005 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:00.735577 16009 raft_consensus.cc:1275] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:07:00.689749 20927 consensus_queue.cc:1048] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:07:00.723570 21019 raft_consensus.cc:697] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:00.726835 21010 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:00.758009 16562 raft_consensus.cc:1240] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:00.618647 20999 consensus_queue.cc:260] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:00.714820 20369 consensus_queue.cc:1048] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:07:00.761583 20917 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:00.756670 20883 consensus_queue.cc:1059] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
W20260709 13:07:00.764213 15962 heartbeater.cc:646] Failed to heartbeat to 127.14.103.62:42985 (1 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.14.103.62:42985 timed out after 15.000s (SENT)
I20260709 13:07:00.765287 20704 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.766001 21071 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:00.770201 21013 raft_consensus.cc:3055] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:07:00.772192 21072 log.cc:927] Time spent T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 13:07:00.678037 20944 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:00.734624 20797 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:00.817541 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:00.935879 16009 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
W20260709 13:07:00.821990 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.729058 20858 consensus_queue.cc:1048] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:07:00.725760 20850 consensus_queue.cc:1048] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:07:00.861172 15974 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:00.746203 20731 raft_consensus.cc:2749] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:07:00.920954 20934 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:00.864900 21045 consensus_queue.cc:1059] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 13:07:00.906387 16005 raft_consensus.cc:1275] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:07:00.981101 21045 raft_consensus.cc:3055] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:00.981468 21045 raft_consensus.cc:740] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:00.981931 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:00.982275 21045 consensus_queue.cc:260] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:00.740093 21025 consensus_queue.cc:1059] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.117s
I20260709 13:07:00.984362 21025 raft_consensus.cc:3055] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:00.984824 21025 raft_consensus.cc:740] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:00.985786 21025 consensus_queue.cc:260] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:00.987301 21025 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:00.988224 21058 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:00.740425 21006 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:07:00.921502 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:00.989324 15931 raft_consensus.cc:2410] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 15 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:07:00.990401 21056 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:00.772646 21062 raft_consensus.cc:2749] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:00.926533 21013 raft_consensus.cc:740] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:00.758723 21016 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:00.926872 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:01.198974 15936 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:07:00.824936 20917 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:00.916741 16010 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 14.
I20260709 13:07:00.756834 20499 raft_consensus.cc:2804] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:00.781001 20980 consensus_queue.cc:1048] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.146s
I20260709 13:07:00.937548 15860 raft_consensus.cc:1240] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:00.862226 20912 consensus_queue.cc:1048] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.103s
I20260709 13:07:00.917079 20803 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:00.917263 20999 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:00.751214 21009 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:00.886023 16003 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
W20260709 13:07:00.949172 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:01.227741 15862 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.000s	sys 0.020s
W20260709 13:07:00.956241 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 95133us
W20260709 13:07:01.209496 20768 consensus_meta.cc:287] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.559s	user 0.009s	sys 0.001s
I20260709 13:07:00.963825 21019 consensus_queue.cc:237] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:00.966229 20936 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:00.969492 20868 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:00.966174 20937 raft_consensus.cc:2749] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:00.979938 21071 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:00.910493 15929 raft_consensus.cc:1275] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:07:00.983434 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:00.931190 20704 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:00.991742 21045 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:07:01.070165 20930 consensus_meta.cc:287] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.559s	user 0.004s	sys 0.004s
I20260709 13:07:00.948797 21038 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:01.132694 16007 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.013s	sys 0.016s
W20260709 13:07:01.285743 16007 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.759s	user 0.015s	sys 0.017s
W20260709 13:07:01.183130 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:01.126036 21072 log.cc:927] Time spent T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.187s	user 0.000s	sys 0.000s
I20260709 13:07:01.021602 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:01.199864 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:01.185338 21070 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.014s	sys 0.006s
W20260709 13:07:01.208993 15902 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.005s	sys 0.019s
W20260709 13:07:01.293591 15902 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.005s	sys 0.019s
W20260709 13:07:01.294219 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 623521us
I20260709 13:07:01.216224 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:01.200654 20556 consensus_meta.cc:287] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.523s	user 0.010s	sys 0.010s
I20260709 13:07:01.221529 15936 raft_consensus.cc:1275] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 13:07:01.224265 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 490585us
I20260709 13:07:01.299705 16533 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:01.181861 20810 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:01.186204 21042 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:01.205727 20773 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.011s	sys 0.011s
I20260709 13:07:01.226325 16010 raft_consensus.cc:1240] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:01.022290 20934 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:01.227651 16003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41662 (request call id 2335) took 1019 ms. Trace:
W20260709 13:07:01.228132 15862 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.000s	sys 0.020s
I20260709 13:07:00.947906 20946 raft_consensus.cc:2749] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:07:00.957664 21069 log.cc:927] Time spent T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.201s	user 0.000s	sys 0.000s
W20260709 13:07:01.228210 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.300892 20704 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:07:01.323518 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:01.323947 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.245488 20858 consensus_queue.cc:1048] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.070s
W20260709 13:07:01.237424 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.325532 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:01.214848 20648 consensus_meta.cc:287] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.624s	user 0.005s	sys 0.005s
I20260709 13:07:01.231180 15974 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:01.211082 20917 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:01.337167 16009 raft_consensus.cc:1275] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:07:01.209908 20499 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:01.222018 21040 consensus_meta.cc:287] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.587s	user 0.009s	sys 0.001s
I20260709 13:07:01.127882 20979 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:01.270725 16011 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.015s	sys 0.007s
W20260709 13:07:01.346436 16011 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.015s	sys 0.008s
W20260709 13:07:01.240978 21028 consensus_meta.cc:287] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.569s	user 0.009s	sys 0.001s
W20260709 13:07:01.281026 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.275811 21062 consensus_queue.cc:1048] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:07:01.284895 20930 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:01.288229 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:01.245836 20768 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:01.295257 15902 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:01.304154 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:01.314123 16003 rpcz_store.cc:276] 0709 13:07:00.208746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:00.312053 (+103307us) service_pool.cc:224] Handling call
0709 13:07:01.227596 (+915543us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:07:01.293380 21070 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.014s	sys 0.006s
I20260709 13:07:01.320060 20929 consensus_queue.cc:1048] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:01.320745 20934 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:01.320909 20920 consensus_queue.cc:1048] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:01.242285 21024 consensus_queue.cc:1048] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.060s
W20260709 13:07:01.326713 16535 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.004s	sys 0.024s
I20260709 13:07:01.327939 16008 raft_consensus.cc:1240] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:01.329209 15931 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:07:01.326157 15929 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
I20260709 13:07:01.126538 20975 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:01.292945 20868 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:01.346863 15862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2362) took 1028 ms. Trace:
I20260709 13:07:01.340430 20937 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:01.382884 15862 rpcz_store.cc:276] 0709 13:07:00.363032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:00.411818 (+ 48786us) service_pool.cc:224] Handling call
0709 13:07:00.419227 (+  7409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:00.419407 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:07:00.680632 (+261225us) log.cc:844] Serialized 14 byte log entry
0709 13:07:01.346532 (+665900us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:01.346587 (+    55us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:01.346631 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:01.346693 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:01.346805 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59192,"prepare.queue_time_us":61266,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":4462,"thread_start_us":852853,"threads_started":2}
I20260709 13:07:01.351472 16009 raft_consensus.cc:1275] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:07:01.351666 16564 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
W20260709 13:07:01.313752 20773 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.012s	sys 0.012s
I20260709 13:07:01.285746 21038 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:01.311299 21009 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:01.405766 21101 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.311717 20556 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.407056 20773 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.326182 20648 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:01.407151 20556 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:01.407709 20773 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:01.409101 20648 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:01.409196 20556 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:01.409817 20773 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:01.410238 20944 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:01.410378 21082 log.cc:927] Time spent T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:07:01.360886 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:01.409787 20556 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:01.411926 21077 log.cc:927] Time spent T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 13:07:01.412305 20556 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:01.414526 20556 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:01.415913 15856 raft_consensus.cc:1240] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:01.372426 21087 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.417045 21087 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:01.419205 21087 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:01.350876 21040 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:01.420285 21083 log.cc:927] Time spent T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 13:07:01.282840 21071 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:01.217775 21016 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:01.421521 21040 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:01.422180 15857 raft_consensus.cc:1240] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:01.409374 21096 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.422736 20868 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:01.423388 21016 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:01.367581 16535 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.004s	sys 0.024s
I20260709 13:07:01.423453 21096 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:01.424119 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 666102us
I20260709 13:07:01.424810 20868 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:01.424980 16535 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:01.425690 21096 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:01.346073 20499 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:07:01.386981 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 155875us
W20260709 13:07:01.376936 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 190337us
I20260709 13:07:01.427258 15974 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:01.274554 21013 consensus_queue.cc:260] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:01.306799 20734 consensus_queue.cc:1059] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 13:07:01.428520 15974 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:01.394542 20937 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:01.315477 21042 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:01.425977 20868 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:01.459100 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:01.398399 20979 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:01.399468 20934 consensus_queue.cc:1048] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:07:01.429533 21075 log.cc:927] Time spent T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.517s	user 0.000s	sys 0.001s
I20260709 13:07:01.468890 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:01.383976 20704 raft_consensus.cc:2749] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:07:01.406308 21101 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:01.311892 20810 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:01.422878 21071 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:01.371342 15853 raft_consensus.cc:1240] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:01.388728 16567 raft_consensus.cc:1240] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:07:01.354518 21085 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:01.363010 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:01.407075 21038 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:01.454949 20879 raft_consensus.cc:3055] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:01.434789 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:01.436828 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:01.456209 15974 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:01.504400 16568 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
W20260709 13:07:01.504868 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 48750us
I20260709 13:07:01.396945 20768 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:01.460384 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:01.456104 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:07:01.400990 20983 consensus_meta.cc:287] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.675s	user 0.007s	sys 0.009s
I20260709 13:07:01.356434 21028 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:01.467363 15827 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:01.427600 21010 raft_consensus.cc:3055] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:01.429839 20858 raft_consensus.cc:2804] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:01.458630 21016 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:01.521175 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 54017.9us
W20260709 13:07:01.504235 20853 consensus_meta.cc:287] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.823s	user 0.011s	sys 0.000s
I20260709 13:07:01.478824 15855 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:07:01.432384 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:01.458819 21062 raft_consensus.cc:2749] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:01.463580 20868 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:01.518020 21025 consensus_meta.cc:287] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.526s	user 0.010s	sys 0.000s
I20260709 13:07:01.517375 15928 raft_consensus.cc:1275] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:07:01.458925 21009 consensus_queue.cc:1048] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
W20260709 13:07:01.522634 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.477002 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:01.477456 21093 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:01.465880 21040 consensus_queue.cc:1048] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 13:07:01.526028 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:01.508149 21085 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:01.576574 15937 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 12: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:01.535177 20853 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:01.577435 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:01.578111 15861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2358) took 1480 ms. Trace:
I20260709 13:07:01.578284 21085 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:01.555697 21094 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:01.578698 20853 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:01.578940 15858 raft_consensus.cc:1240] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:01.579073 21094 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:01.528692 16533 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:01.503422 20879 raft_consensus.cc:740] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:01.526888 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 2344) took 1087 ms. Trace:
I20260709 13:07:01.519253 21010 raft_consensus.cc:740] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:01.505880 20825 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:01.565199 16005 raft_consensus.cc:2410] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:07:01.566191 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:01.588640 15936 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:01.568789 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:01.570039 16561 raft_consensus.cc:2393] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 11: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:01.501360 20979 consensus_queue.cc:1048] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 13:07:01.573192 15930 raft_consensus.cc:1240] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:01.593672 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:01.576350 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:01.506824 15974 leader_election.cc:400] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Vote denied by peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) with higher term. Message: Invalid argument: T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 10. Current term is 11.
W20260709 13:07:01.581440 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.564045 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:01.582617 15931 raft_consensus.cc:1240] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:01.583952 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:01.584052 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:07:01.596689 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 301501us
I20260709 13:07:01.600975 15902 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:01.520108 20983 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:01.602512 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 244364us
I20260709 13:07:01.578490 15861 rpcz_store.cc:276] 0709 13:07:00.097640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:00.097765 (+   125us) service_pool.cc:224] Handling call
0709 13:07:00.098102 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:00.125956 (+ 27854us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:00.125986 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:00.126431 (+   445us) log.cc:844] Serialized 26 byte log entry
0709 13:07:00.545486 (+419055us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:00.545532 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:00.545605 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:01.429022 (+883417us) spinlock_profiling.cc:243] Waited 45.9 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:07:01.562879 (+133857us) raft_consensus.cc:1685] finished
0709 13:07:01.562900 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:01.578050 (+ 15150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27428,"raft.queue_time_us":30730,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":40848,"spinlock_wait_cycles":105618816,"thread_start_us":236512,"threads_started":1,"wal-append.queue_time_us":529432}
I20260709 13:07:01.533458 20768 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.580758 16002 rpcz_store.cc:276] 0709 13:07:00.439202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:00.612531 (+173329us) service_pool.cc:224] Handling call
0709 13:07:00.612846 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:00.613071 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:00.613111 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:00.674262 (+ 61151us) log.cc:844] Serialized 26 byte log entry
0709 13:07:01.193733 (+519471us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:01.513156 (+319423us) log.cc:844] Serialized 14 byte log entry
0709 13:07:01.523467 (+ 10311us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:01.523554 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:01.523594 (+    40us) raft_consensus.cc:1685] finished
0709 13:07:01.523612 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:01.526738 (+  3126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16178,"prepare.queue_time_us":129695,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":10064,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":31915,"thread_start_us":768912,"threads_started":2,"wal-append.queue_time_us":613907}
I20260709 13:07:01.770897 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:01.555884 20868 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:07:01.610098 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 321042us
W20260709 13:07:01.612361 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.605942 15974 leader_election.cc:403] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:01.581928 20975 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:01.581434 21102 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.580299 20810 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:01.594868 21085 consensus_queue.cc:1048] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:07:01.533635 21016 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:01.590863 21096 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.649627 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:01.660723 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:01.566179 20858 raft_consensus.cc:697] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:01.565812 20936 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:01.601212 21099 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:01.780109 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:01.806110 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:01.850996 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:01.881108 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:01.881640 15744 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 83a1b79cdcf046c78453695c798a14a5 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 8430db80a39d4868bcf7641c16de35ea is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:07:01.882807 15860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 2269) took 1933 ms. Trace:
I20260709 13:07:01.580446 21093 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:02.122061 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:02.126746 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:01.973071 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:02.173856 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:02.003850 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 475217us
I20260709 13:07:02.173813 15855 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:02.174235 15860 rpcz_store.cc:276] 0709 13:06:59.949011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.951219 (+  2208us) service_pool.cc:224] Handling call
0709 13:06:59.951477 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:01.210215 (+1258738us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:01.882524 (+672309us) spinlock_profiling.cc:243] Waited 986 ms on lock 0x7b54003b5e54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:01.882731 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2267221760}
I20260709 13:07:01.883764 16562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 1253) took 1446 ms. Trace:
W20260709 13:07:02.172485 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 665721us
W20260709 13:07:02.172325 15744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 290730us
W20260709 13:07:02.122257 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 370472us
W20260709 13:07:01.847977 20968 consensus_meta.cc:287] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 1.185s	user 0.007s	sys 0.007s
I20260709 13:07:01.796306 20944 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.177989 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:01.796803 20773 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.125779 15976 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:02.181073 15974 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:02.177631 15935 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:07:01.948346 20825 raft_consensus.cc:2749] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) with higher term. Message: Invalid argument: T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 10. Current term is 11.
W20260709 13:07:02.096833 21100 log.cc:927] Time spent T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.530s	user 0.000s	sys 0.001s
W20260709 13:07:02.095286 21106 log.cc:927] Time spent T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.448s	user 0.001s	sys 0.000s
W20260709 13:07:02.117278 21103 log.cc:927] Time spent T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
W20260709 13:07:02.100885 21108 log.cc:927] Time spent T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
W20260709 13:07:01.596145 20938 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.857s	user 0.015s	sys 0.026s
I20260709 13:07:02.175432 15936 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
I20260709 13:07:02.179180 16562 rpcz_store.cc:276] 0709 13:07:00.437408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:00.437581 (+   173us) service_pool.cc:224] Handling call
0709 13:07:00.437873 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:00.772989 (+335116us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:01.883482 (+1110493us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b5400550ad4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:01.883698 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":735803264}
W20260709 13:07:02.099866 21088 log.cc:927] Time spent T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:07:02.220095 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:02.220795 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:02.258934 15860 raft_consensus.cc:2410] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:07:02.258843 16002 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:02.234930 20773 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:02.213414 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:02.182026 15902 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:02.093489 20499 consensus_meta.cc:287] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.667s	user 0.009s	sys 0.000s
I20260709 13:07:02.225790 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:02.227255 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:01.808375 20999 raft_consensus.cc:2749] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:02.229380 21107 raft_consensus.cc:2749] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:02.100034 21089 raft_consensus.cc:2749] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:07:02.236897 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:02.213009 21120 log.cc:927] Time spent T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.605s	user 0.002s	sys 0.000s
I20260709 13:07:02.175941 20930 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:02.262274 15857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 972) took 1473 ms. Trace:
W20260709 13:07:02.238484 16010 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.009s	sys 0.069s
W20260709 13:07:02.292707 16010 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.009s	sys 0.069s
I20260709 13:07:02.240166 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:02.248304 15855 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
I20260709 13:07:02.293479 16011 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:07:02.250872 21025 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.264106 16005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60156 (request call id 971) took 1512 ms. Trace:
I20260709 13:07:02.296046 16007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2158) took 2306 ms. Trace:
I20260709 13:07:02.266016 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:02.298372 15935 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:07:02.296600 16007 rpcz_store.cc:276] 0709 13:06:59.989489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.989625 (+   136us) service_pool.cc:224] Handling call
0709 13:07:00.110945 (+121320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:00.526102 (+415157us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:00.526131 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:00.526547 (+   416us) log.cc:844] Serialized 26 byte log entry
0709 13:07:01.286187 (+759640us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:01.286238 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:02.267886 (+981648us) spinlock_profiling.cc:243] Waited 415 ms on lock 0x7b54000aa0d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:02.267943 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:02.267997 (+    54us) raft_consensus.cc:1685] finished
0709 13:07:02.268016 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:02.295973 (+ 27957us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5742,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":95,"spinlock_wait_cycles":953983360,"thread_start_us":759215,"threads_started":1,"wal-append.queue_time_us":822965}
W20260709 13:07:02.258076 20938 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.520s	user 0.016s	sys 0.033s
I20260709 13:07:02.294480 20999 raft_consensus.cc:2749] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:01.798205 20768 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:02.230003 20975 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:02.268976 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:02.261382 15854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2364) took 1840 ms. Trace:
I20260709 13:07:02.236452 21111 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.299743 20938 raft_consensus.cc:697] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 12 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:02.236409 20797 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.853009 21102 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:02.237083 21028 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:01.968336 21010 consensus_queue.cc:260] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:02.197710 20936 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:02.274993 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 93071.4us
I20260709 13:07:02.273437 16002 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:07:02.331053 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:02.331782 16002 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:07:02.303784 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:02.240463 20979 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:01.880455 20879 consensus_queue.cc:260] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:02.225397 20944 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:02.310882 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:02.263130 20773 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:02.312747 15862 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:07:02.314059 20768 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:02.212199 21096 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:02.316229 15936 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:02.296277 16005 rpcz_store.cc:276] 0709 13:07:00.752048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:00.983124 (+231076us) service_pool.cc:224] Handling call
0709 13:07:02.263926 (+1280802us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b54000aafd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:02.264050 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":665404288}
W20260709 13:07:02.252211 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 450246us
I20260709 13:07:02.292435 15857 rpcz_store.cc:276] 0709 13:07:00.788529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:00.788654 (+   125us) service_pool.cc:224] Handling call
0709 13:07:00.788961 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:01.422557 (+633596us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:02.262047 (+839490us) spinlock_profiling.cc:243] Waited 633 ms on lock 0x7b5400219154. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:02.262219 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1455840000}
I20260709 13:07:02.344317 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:02.316192 15854 rpcz_store.cc:276] 0709 13:07:00.421422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:00.421555 (+   133us) service_pool.cc:224] Handling call
0709 13:07:00.421798 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:00.672279 (+250481us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:00.672321 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:00.672374 (+    53us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:00.672417 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:02.261136 (+1588719us) spinlock_profiling.cc:243] Waited 250 ms on lock 0x7b54003886d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:02.261183 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:02.261326 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":575539456}
I20260709 13:07:02.236760 21101 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:02.362671 15933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2381) took 1436 ms. Trace:
I20260709 13:07:02.324419 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:02.306907 20825 raft_consensus.cc:2749] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:02.301115 21042 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:02.322387 20975 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:02.306370 21116 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:02.334585 15902 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:02.264997 20499 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:02.370339 15861 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:07:02.338253 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:02.371173 15855 raft_consensus.cc:1240] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:02.371845 15858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 974) took 1581 ms. Trace:
I20260709 13:07:02.322585 20983 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:02.346068 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:02.362202 15857 raft_consensus.cc:2393] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 12: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:02.362442 15932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2378) took 1471 ms. Trace:
I20260709 13:07:02.259290 21110 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:07:02.380983 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:02.363462 16011 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
I20260709 13:07:02.363237 15933 rpcz_store.cc:276] 0709 13:07:00.926515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:00.928922 (+  2407us) service_pool.cc:224] Handling call
0709 13:07:00.929263 (+   341us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:01.292335 (+363072us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:01.292380 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:01.302618 (+ 10238us) log.cc:844] Serialized 52 byte log entry
0709 13:07:01.519888 (+217270us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:01.519943 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:01.520055 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:02.362374 (+842319us) raft_consensus.cc:1685] finished
0709 13:07:02.362402 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:02.362597 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":372064,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":8204,"thread_start_us":555699,"threads_started":2,"wal-append.queue_time_us":624217}
I20260709 13:07:02.365736 15854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 977) took 1573 ms. Trace:
I20260709 13:07:02.382035 15854 rpcz_store.cc:276] 0709 13:07:00.792459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:02.364965 (+1572506us) service_pool.cc:224] Handling call
0709 13:07:02.365269 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:02.365439 (+   170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:02.365464 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:02.365502 (+    38us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:02.365533 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:02.365585 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:02.365687 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:02.306203 21025 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:02.367733 16567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1110) took 1798 ms. Trace:
I20260709 13:07:02.368518 15827 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:02.334443 20929 consensus_queue.cc:1059] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:07:02.371778 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:02.389786 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:02.390380 16002 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 8.
I20260709 13:07:02.391183 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:02.372293 15858 rpcz_store.cc:276] 0709 13:07:00.790562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:00.790692 (+   130us) service_pool.cc:224] Handling call
0709 13:07:00.791006 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:01.579367 (+788361us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:02.371621 (+792254us) spinlock_profiling.cc:243] Waited 788 ms on lock 0x7b540008e154. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:02.371790 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1811661568}
W20260709 13:07:02.339445 16565 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.757s	user 0.055s	sys 0.056s
W20260709 13:07:02.392009 16565 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.810s	user 0.055s	sys 0.056s
I20260709 13:07:02.392145 16002 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:07:02.392344 15936 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:07:02.392889 16565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1112) took 1616 ms. Trace:
I20260709 13:07:02.393003 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:02.365711 20773 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:02.328922 20797 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:02.377915 21113 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.328081 20930 consensus_queue.cc:1048] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:07:02.379541 21101 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:02.380195 15860 raft_consensus.cc:1240] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:02.375788 15932 rpcz_store.cc:276] 0709 13:07:00.865970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:00.917431 (+ 51461us) service_pool.cc:224] Handling call
0709 13:07:00.917826 (+   395us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:01.346501 (+428675us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:01.346534 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:01.358422 (+ 11888us) log.cc:844] Serialized 52 byte log entry
0709 13:07:01.572295 (+213873us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:01.572333 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:01.572397 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:02.336999 (+764602us) raft_consensus.cc:1685] finished
0709 13:07:02.337021 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:02.337187 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4146,"raft.queue_time_us":356366,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":9847,"thread_start_us":609473,"threads_started":2,"wal-append.queue_time_us":185239}
I20260709 13:07:02.385299 16567 rpcz_store.cc:276] 0709 13:07:00.569501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:00.569710 (+   209us) service_pool.cc:224] Handling call
0709 13:07:00.570122 (+   412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:01.476643 (+906521us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:02.367451 (+890808us) spinlock_profiling.cc:243] Waited 814 ms on lock 0x7b5400573ad4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:02.367650 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1872978176}
I20260709 13:07:02.366403 20879 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:02.304510 20858 consensus_queue.cc:237] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:02.370237 20768 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.337669 20979 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:02.426406 16002 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:07:02.373072 21116 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:02.377413 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:02.396836 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:02.430110 16011 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:02.393539 16565 rpcz_store.cc:276] 0709 13:07:00.776701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:00.782527 (+  5826us) service_pool.cc:224] Handling call
0709 13:07:00.782911 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:00.783287 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:00.783326 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:00.783888 (+   562us) log.cc:844] Serialized 26 byte log entry
0709 13:07:01.291081 (+507193us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:01.582080 (+290999us) log.cc:844] Serialized 14 byte log entry
0709 13:07:02.392484 (+810404us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:02.392586 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:02.392646 (+    60us) raft_consensus.cc:1685] finished
0709 13:07:02.392666 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:02.392819 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10964,"prepare.queue_time_us":694727,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":770544,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":14007,"thread_start_us":1390090,"threads_started":3,"wal-append.queue_time_us":610239}
I20260709 13:07:02.324894 21111 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:02.401094 20825 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:02.379526 16009 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.911s	user 0.003s	sys 0.025s
I20260709 13:07:02.378196 21107 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:02.423048 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:02.374241 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 197148us
W20260709 13:07:02.400511 20917 consensus_meta.cc:287] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 1.040s	user 0.008s	sys 0.002s
I20260709 13:07:02.428695 15827 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:02.432605 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:02.441332 15860 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 12: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:02.441533 21132 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:07:02.429492 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 248487us
W20260709 13:07:02.379292 15929 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.741s	user 0.006s	sys 0.022s
W20260709 13:07:02.443269 15929 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.808s	user 0.006s	sys 0.022s
W20260709 13:07:02.436435 16009 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.968s	user 0.003s	sys 0.025s
I20260709 13:07:02.436796 15859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 971) took 1648 ms. Trace:
I20260709 13:07:02.404618 21127 raft_consensus.cc:2749] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:02.406535 21016 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:02.444937 15859 rpcz_store.cc:276] 0709 13:07:00.787978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:00.788168 (+   190us) service_pool.cc:224] Handling call
0709 13:07:00.864193 (+ 76025us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:00.921716 (+ 57523us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:07:00.948220 (+ 26504us) log.cc:844] Serialized 14 byte log entry
0709 13:07:01.531926 (+583706us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:01.584585 (+ 52659us) log.cc:844] Serialized 52 byte log entry
0709 13:07:01.584755 (+   170us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:01.584992 (+   237us) log.cc:844] Serialized 14 byte log entry
0709 13:07:01.585176 (+   184us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:01.585249 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:02.436587 (+851338us) raft_consensus.cc:1685] finished
0709 13:07:02.436611 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:02.436740 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":34933,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":89,"raft.queue_time_us":51270,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":6554,"spinlock_wait_cycles":7982976,"thread_start_us":319558,"threads_started":1,"wal-append.queue_time_us":570235}
I20260709 13:07:02.403574 21101 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:02.405624 20938 consensus_queue.cc:237] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:02.434902 21133 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:02.444913 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 73383.9us
I20260709 13:07:02.450064 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:02.402601 20999 consensus_queue.cc:1059] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:07:02.401996 20773 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:02.450883 15974 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:02.496491 15902 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:02.458017 15932 raft_consensus.cc:1240] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:02.434428 21116 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:02.394937 21110 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:02.446388 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:02.494443 15825 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:02.522838 16002 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
W20260709 13:07:02.523178 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 69740us
I20260709 13:07:02.453619 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:02.450984 15827 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:02.467257 15930 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
W20260709 13:07:02.524147 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 73236.6us
W20260709 13:07:02.521453 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 70626.1us
I20260709 13:07:02.525046 15974 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:02.525120 15827 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:02.523348 16011 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:07:02.526729 15827 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:02.527927 15827 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:02.465173 20797 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:02.534241 15827 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:02.460168 21025 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:07:02.532382 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 406669us
I20260709 13:07:02.457358 20936 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:02.452390 21132 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:02.456621 20975 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:02.462698 20499 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:02.530952 15929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 1028) took 1007 ms. Trace:
I20260709 13:07:02.469334 21093 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:02.405932 21113 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:02.501364 21101 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:02.459228 21096 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:02.464185 21111 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:02.530227 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:02.459825 20983 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 13:07:02.434574 20768 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:02.447110 21122 consensus_queue.cc:1048] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.102s
I20260709 13:07:02.445322 20825 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:02.530128 21127 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:07:02.585426 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:02.644428 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:02.533805 21042 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:02.450917 20917 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:02.447393 21107 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:02.597182 15929 rpcz_store.cc:276] 0709 13:07:01.523042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:01.523169 (+   127us) service_pool.cc:224] Handling call
0709 13:07:01.561115 (+ 37946us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:01.561559 (+   444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:01.561597 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:01.573807 (+ 12210us) log.cc:844] Serialized 26 byte log entry
0709 13:07:02.443976 (+870169us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:02.444046 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:02.444173 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:02.530715 (+ 86542us) raft_consensus.cc:1685] finished
0709 13:07:02.530737 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:02.530876 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":262,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":19253,"thread_start_us":808507,"threads_started":1,"wal-append.queue_time_us":845817}
W20260709 13:07:02.612239 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:02.534171 20810 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:02.469286 21131 consensus_queue.cc:1059] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.097s
W20260709 13:07:02.613674 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 88715.4us
I20260709 13:07:02.612986 21016 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:02.539088 20797 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.621184 16002 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:02.545938 21025 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:02.627863 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 154124us
I20260709 13:07:02.618676 20975 raft_consensus.cc:2804] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:07:02.641986 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 101199us
I20260709 13:07:02.614159 20999 raft_consensus.cc:2749] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:02.641870 20944 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:02.648660 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:02.715575 16567 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:07:02.503706 21133 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
W20260709 13:07:02.657351 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:02.697005 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 162811us
I20260709 13:07:02.654575 21042 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:02.705070 16569 raft_consensus.cc:1240] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:02.706404 16010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 968) took 2117 ms. Trace:
I20260709 13:07:02.694386 21137 raft_consensus.cc:2749] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:07:02.643251 21128 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.700944 20810 raft_consensus.cc:697] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:02.641736 21093 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:02.645725 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 149307us
I20260709 13:07:02.644135 21127 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:02.632855 21143 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:02.597726 21132 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:02.713552 16002 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
W20260709 13:07:02.716811 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:02.717044 15827 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:02.719396 16010 rpcz_store.cc:276] 0709 13:07:00.589424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:01.311079 (+721655us) service_pool.cc:224] Handling call
0709 13:07:01.311370 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:01.311569 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:07:01.583122 (+271553us) log.cc:844] Serialized 14 byte log entry
0709 13:07:02.690708 (+1107586us) log.cc:844] Serialized 14 byte log entry
0709 13:07:02.706097 (+ 15389us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:02.706145 (+    48us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:02.706190 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:02.706250 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:02.706355 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70708,"prepare.queue_time_us":12089,"prepare.run_cpu_time_us":22,"prepare.run_wall_time_us":23,"raft.queue_time_us":341311,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":8255,"thread_start_us":1301924,"threads_started":4,"wal-append.queue_time_us":811520}
I20260709 13:07:02.644006 21117 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:07:02.656239 21135 raft_consensus.cc:2749] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:02.724126 20773 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:02.707911 21125 log.cc:927] Time spent T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.280s	user 0.001s	sys 0.000s
W20260709 13:07:02.726334 21136 log.cc:927] Time spent T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
W20260709 13:07:02.728843 20868 consensus_meta.cc:287] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.876s	user 0.011s	sys 0.000s
I20260709 13:07:02.707638 20975 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:02.702131 21016 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:02.733029 21134 log.cc:927] Time spent T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:07:02.735800 20825 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:02.738891 21107 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:02.702939 20797 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:03.051363 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 328010us
I20260709 13:07:03.324450 20975 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:03.324646 20917 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:02.722661 21042 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:03.326144 15933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2391) took 1090 ms. Trace:
W20260709 13:07:03.322842 21116 consensus_meta.cc:287] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.790s	user 0.010s	sys 0.000s
I20260709 13:07:03.326570 15933 rpcz_store.cc:276] 0709 13:07:02.236334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:02.396123 (+159789us) service_pool.cc:224] Handling call
0709 13:07:02.396426 (+   303us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:02.402871 (+  6445us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:02.402902 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:02.403639 (+   737us) log.cc:844] Serialized 52 byte log entry
0709 13:07:02.700625 (+296986us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:02.700683 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:02.702698 (+  2015us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:03.325911 (+623213us) raft_consensus.cc:1685] finished
0709 13:07:03.325930 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:03.326085 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58520,"raft.queue_time_us":105091,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":27004,"thread_start_us":218712,"threads_started":1,"wal-append.queue_time_us":892281}
I20260709 13:07:03.327749 21116 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:03.328315 16003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2161) took 2714 ms. Trace:
I20260709 13:07:03.328749 20858 consensus_queue.cc:1059] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:03.329654 21137 raft_consensus.cc:3055] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:02.721988 21146 consensus_queue.cc:1059] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.621s
I20260709 13:07:03.330054 21137 raft_consensus.cc:740] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:03.330646 21144 raft_consensus.cc:3055] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:03.331053 21144 raft_consensus.cc:740] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:03.330926 21137 consensus_queue.cc:260] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:03.261770 21101 consensus_meta.cc:287] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.637s	user 0.004s	sys 0.005s
I20260709 13:07:03.332432 21101 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:03.328699 16003 rpcz_store.cc:276] 0709 13:07:00.613693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:01.357672 (+743979us) service_pool.cc:224] Handling call
0709 13:07:01.358007 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:01.358352 (+   345us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:07:01.358393 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:01.358902 (+   509us) log.cc:844] Serialized 26 byte log entry
0709 13:07:02.233488 (+874586us) spinlock_profiling.cc:243] Waited 2.77 ms on lock 0x7b18003867a0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f480d1 00007f73ea363a0f
0709 13:07:02.433059 (+199571us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:02.433099 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:02.433154 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:03.328029 (+894875us) raft_consensus.cc:1685] finished
0709 13:07:03.328049 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:03.328255 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":75,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":8525,"spinlock_wait_cycles":6376832,"thread_start_us":198206,"threads_started":1,"wal-append.queue_time_us":1178761}
I20260709 13:07:03.332137 21144 consensus_queue.cc:260] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:03.334352 21144 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:03.334884 21101 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:02.908934 21110 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:03.336464 21128 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:02.707533 21025 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:03.339131 21128 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:02.916119 20768 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:02.915113 20934 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:03.340538 20934 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:03.340839 15902 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:02.915256 21113 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:03.342640 20934 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:03.343014 21113 raft_consensus.cc:2804] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:03.343577 21113 raft_consensus.cc:697] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:07:03.064044 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:03.324101 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:03.323743 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 600047us
W20260709 13:07:03.344961 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 627935us
I20260709 13:07:02.741084 20979 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:03.332916 21116 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:03.364111 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:03.347805 15974 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:03.352262 16010 raft_consensus.cc:1240] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:03.351291 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:07:03.355998 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:03.356383 15827 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:03.370234 15827 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:03.371647 15827 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:07:03.358223 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:03.374418 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:03.375025 16011 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:03.373692 15902 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:03.408735 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:03.425619 15855 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.010s	sys 0.010s
W20260709 13:07:03.426002 15855 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.010s	sys 0.010s
I20260709 13:07:03.462338 16003 raft_consensus.cc:1240] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 12. Current term is 13. Ops: []
W20260709 13:07:03.462635 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 87826.1us
W20260709 13:07:02.914631 21130 log.cc:927] Time spent T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.224s	user 0.000s	sys 0.000s
W20260709 13:07:03.464548 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 90930.2us
I20260709 13:07:03.469230 15902 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:03.462385 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:03.463783 15827 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:03.258920 21028 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.000s	sys 0.014s
I20260709 13:07:03.467593 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:03.472992 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 322246us
I20260709 13:07:03.119791 21149 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.271636 21148 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:03.478785 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:03.480275 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:03.363481 20768 raft_consensus.cc:2804] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:03.355065 21117 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:03.362823 20934 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.111s
I20260709 13:07:03.355353 20917 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:03.499982 20917 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:03.500810 21132 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:03.502197 20917 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:03.490556 20810 consensus_queue.cc:237] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:03.503005 21016 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:03.463045 21101 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.499383 20797 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:03.503934 21101 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:03.505896 21101 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:03.506902 21161 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.507392 21161 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:03.507936 21147 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:03.508638 21147 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.509087 21147 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:03.509542 21161 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:03.494266 21149 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:03.511255 21147 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:03.511926 21113 consensus_queue.cc:237] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:03.512576 21149 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:03.514519 16009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2162) took 2900 ms. Trace:
I20260709 13:07:03.514873 16009 rpcz_store.cc:276] 0709 13:07:00.614020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:01.385473 (+771453us) service_pool.cc:224] Handling call
0709 13:07:01.389751 (+  4278us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:01.390229 (+   478us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:07:01.390269 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:01.455585 (+ 65316us) log.cc:844] Serialized 26 byte log entry
0709 13:07:02.444706 (+989121us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:02.444745 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:02.444810 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:03.514303 (+1069493us) raft_consensus.cc:1685] finished
0709 13:07:03.514323 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:03.514450 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4045,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":81,"thread_start_us":975801,"threads_started":1,"wal-append.queue_time_us":1127475}
I20260709 13:07:03.357595 21093 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:03.516571 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:03.475898 21028 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.856s	user 0.000s	sys 0.014s
I20260709 13:07:03.518585 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:03.519187 16009 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:07:03.197592 21133 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:03.357378 21127 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:03.352240 21128 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:03.463548 21110 raft_consensus.cc:2804] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:03.475322 21143 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:03.464543 21152 raft_consensus.cc:2804] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:03.499383 20934 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:03.326910 20868 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:03.477766 20773 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:07:03.489398 21136 log.cc:927] Time spent T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:07:03.481114 21146 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:03.536561 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:03.556041 16003 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:07:03.497583 20768 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:07:03.523157 20879 consensus_meta.cc:287] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 1.099s	user 0.006s	sys 0.006s
I20260709 13:07:03.539381 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:03.559352 16009 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
I20260709 13:07:03.536748 16011 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
I20260709 13:07:03.517683 21117 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:03.561893 20768 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:03.569195 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:03.574710 21042 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:03.575292 21025 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:03.575867 21093 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:03.497284 21148 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:03.578233 21148 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:03.536063 21110 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:03.579304 21110 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:03.580307 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:03.580900 16003 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:07:03.553344 21128 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:03.581528 15930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 2386) took 1403 ms. Trace:
I20260709 13:07:03.581902 15930 rpcz_store.cc:276] 0709 13:07:02.178575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:02.178721 (+   146us) service_pool.cc:224] Handling call
0709 13:07:03.581373 (+1402652us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b540008a2d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:03.581474 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":426774912}
I20260709 13:07:03.563230 20773 raft_consensus.cc:697] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 13 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:03.564139 21146 raft_consensus.cc:697] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 14 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:03.566072 20868 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:03.566700 16562 raft_consensus.cc:2393] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 12: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:03.538746 20975 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:03.638101 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 167348us
I20260709 13:07:03.645630 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:03.651901 16570 raft_consensus.cc:1275] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:07:03.654650 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:03.655267 16561 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:07:03.603554 21128 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:03.658246 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 175735us
I20260709 13:07:03.660025 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:03.660516 21152 raft_consensus.cc:697] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:03.660655 16570 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:07:03.661795 21152 consensus_queue.cc:237] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:03.662493 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:03.663693 16570 raft_consensus.cc:2393] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 14: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:03.667591 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:03.670652 20938 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:03.668362 16566 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
W20260709 13:07:03.690165 15861 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 2282) took 2254 ms (client timeout 2015 ms). Trace:
W20260709 13:07:03.709658 15861 rpcz_store.cc:269] 0709 13:07:01.436147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:02.177702 (+741555us) service_pool.cc:224] Handling call
0709 13:07:03.689986 (+1512284us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b540008d254. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:03.690094 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":348705664}
I20260709 13:07:03.700505 15827 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:03.696622 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:03.711578 16570 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:07:03.700809 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:07:03.726240 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 378718us
I20260709 13:07:03.727200 15974 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:03.711833 15827 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:07:03.733017 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 269523us
I20260709 13:07:03.682613 21042 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:03.677909 21146 consensus_queue.cc:237] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:03.677160 20773 consensus_queue.cc:237] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:03.735556 15974 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:03.736686 15974 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:03.737815 15974 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:03.689543 21137 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:07:03.693239 21076 log.cc:927] Time spent T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 2.772s	user 0.008s	sys 0.018s
I20260709 13:07:03.740177 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:03.740067 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:03.742812 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:03.741345 15827 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:03.746371 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:03.746945 16564 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:07:03.747237 15974 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:03.743456 16566 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 14.
I20260709 13:07:03.695937 20975 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:03.701745 21113 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.753201 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:03.667160 21143 raft_consensus.cc:697] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 12 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:03.754240 15974 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:03.754262 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:03.753873 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:03.757828 21113 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:03.758522 20982 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:07:03.759225 21081 log.cc:927] Time spent T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 2.453s	user 0.013s	sys 0.029s
W20260709 13:07:03.759649 21068 log.cc:927] Time spent T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 3.210s	user 0.017s	sys 0.022s
I20260709 13:07:03.759873 20879 raft_consensus.cc:2749] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:07:03.755277 21025 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:03.761854 15855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 2276) took 2327 ms. Trace:
I20260709 13:07:03.826054 15855 rpcz_store.cc:276] 0709 13:07:01.434702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:02.371978 (+937276us) service_pool.cc:224] Handling call
0709 13:07:02.374950 (+  2972us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:02.693181 (+318231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:02.693213 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:02.693651 (+   438us) log.cc:844] Serialized 26 byte log entry
0709 13:07:03.426340 (+732689us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:03.426380 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:03.761497 (+335117us) spinlock_profiling.cc:243] Waited 271 ms on lock 0x7b5400218c54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:03.761542 (+    45us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:03.761585 (+    43us) raft_consensus.cc:1685] finished
0709 13:07:03.761599 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:03.761801 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":239055,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":153343,"spinlock_wait_cycles":622383488,"thread_start_us":685472,"threads_started":1,"wal-append.queue_time_us":914059}
I20260709 13:07:03.762377 15934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2369) took 3825 ms. Trace:
I20260709 13:07:03.763085 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:03.733064 20979 raft_consensus.cc:2804] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:03.766698 16003 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:07:03.830768 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:03.831375 16003 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
W20260709 13:07:03.771148 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:03.773016 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:03.767361 15827 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:03.832168 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:03.745270 21176 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:03.741079 21056 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:03.786228 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:03.787134 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:03.787046 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:03.744765 20670 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:03.754416 21042 raft_consensus.cc:2804] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:03.788237 21135 raft_consensus.cc:2749] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:03.818511 21133 consensus_queue.cc:1048] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:03.740478 20938 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:03.744190 21142 raft_consensus.cc:2749] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:07:03.825486 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:03.832206 16564 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 13
W20260709 13:07:03.845588 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 78320.8us
W20260709 13:07:03.780901 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:03.834601 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:03.833117 20979 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:03.759891 21167 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.836433 16566 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 LEARNER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:07:03.840348 15859 raft_consensus.cc:1240] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:03.848088 16004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2159) took 3786 ms. Trace:
I20260709 13:07:03.838184 15858 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:03.848580 16004 rpcz_store.cc:276] 0709 13:07:00.061893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:00.062039 (+   146us) service_pool.cc:224] Handling call
0709 13:07:00.062396 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:00.102872 (+ 40476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:00.102903 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:00.109811 (+  6908us) log.cc:844] Serialized 26 byte log entry
0709 13:07:00.308627 (+198816us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:00.308670 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:00.308744 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:03.758996 (+3450252us) spinlock_profiling.cc:243] Waited 1.47 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:07:03.840856 (+ 81860us) raft_consensus.cc:1685] finished
0709 13:07:03.840878 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:03.848025 (+  7147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40549,"raft.queue_time_us":41385,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":5636,"spinlock_wait_cycles":3376896,"thread_start_us":198440,"threads_started":1,"wal-append.queue_time_us":393668}
I20260709 13:07:03.827822 15934 rpcz_store.cc:276] 0709 13:06:59.937218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.959877 (+ 22659us) service_pool.cc:224] Handling call
0709 13:06:59.960150 (+   273us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:00.112370 (+152220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:00.112441 (+    71us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:00.128507 (+ 16066us) log.cc:844] Serialized 26 byte log entry
0709 13:07:00.534979 (+406472us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:00.535178 (+   199us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:00.679925 (+144747us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b54002c1854. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:00.679976 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:03.692661 (+3012685us) spinlock_profiling.cc:243] Waited 32.1 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:07:03.762189 (+ 69528us) raft_consensus.cc:1685] finished
0709 13:07:03.762210 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:03.762329 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10324,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":4948,"spinlock_wait_cycles":410326016,"thread_start_us":264712,"threads_started":1,"wal-append.queue_time_us":561578}
I20260709 13:07:03.778955 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:03.752712 21128 consensus_queue.cc:1059] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.081s
I20260709 13:07:03.830322 20934 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.846254 15827 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:03.857484 16003 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:03.836261 21176 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:03.868464 15827 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:07:03.868674 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:03.871587 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:03.843061 21042 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:07:03.872025 15974 connection.cc:256] RPC call timeout handler was delayed by -1.8003363829999444s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19740000000000002s for a response
I20260709 13:07:03.872694 16564 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:07:03.872787 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:03.847323 20979 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:03.873787 15827 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:03.890792 15974 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:03.891146 15827 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:03.893560 15827 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:03.895531 21170 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:03.841974 21127 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:03.867116 21019 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:03.904328 20752 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:03.896130 21170 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:03.868168 15934 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:46118
I20260709 13:07:03.889582 21152 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:03.867400 21176 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:03.910449 16003 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
I20260709 13:07:03.915246 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:03.955119 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:03.980374 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 82651.7us
I20260709 13:07:03.891474 21042 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:03.986116 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:03.990110 15974 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:03.908847 20841 raft_consensus.cc:2804] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:03.980003 21071 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 13:07:03.993887 21168 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:03.915010 20752 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:03.994678 21168 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:03.995010 20752 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:03.995124 21168 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:03.908934 21019 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:03.906927 21056 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:03.989982 15934 raft_consensus.cc:606] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Received request to transfer leadership to TS eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:03.996788 15934 raft_consensus.cc:614] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:03.987957 15861 raft_consensus.cc:1240] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:03.998737 15974 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:03.992820 15902 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:03.915970 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:03.995956 21019 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:03.856005 21167 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:04.000136 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 97853us
I20260709 13:07:03.864526 20934 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:03.975308 21152 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:03.979282 21181 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:04.014922 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:03.981079 21113 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:03.990480 21170 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:03.907411 20768 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:04.001648 21025 raft_consensus.cc:2749] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:03.996503 20841 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:04.009166 21056 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:03.981271 21110 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:04.047083 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:03.872035 21143 consensus_queue.cc:237] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:04.017068 21152 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:04.053988 16002 raft_consensus.cc:2410] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:07:04.054095 20841 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:04.022810 21113 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:04.025261 21127 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:04.016054 21154 consensus_queue.cc:1059] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:07:04.044137 21056 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:04.015640 21117 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:04.106273 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:04.068295 16008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 967) took 3479 ms. Trace:
I20260709 13:07:04.106460 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:04.111694 15934 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:07:04.060261 21164 log.cc:927] Time spent T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
I20260709 13:07:04.080106 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:04.077500 15857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 2281) took 2641 ms. Trace:
I20260709 13:07:04.086512 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:04.068843 21067 log.cc:927] Time spent T 00000000000000000000000000000000 P d33cebb189b14a989836214fdbafb898: Append to log took a long time: real 3.128s	user 0.019s	sys 0.027s
W20260709 13:07:04.109562 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 116780us
I20260709 13:07:04.118294 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:04.082049 21173 log.cc:927] Time spent T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.192s	user 0.001s	sys 0.001s
W20260709 13:07:04.069427 21183 log.cc:927] Time spent T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
I20260709 13:07:04.088624 21111 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:04.084045 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:04.072710 21000 raft_consensus.cc:3055] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:07:04.071033 21178 log.cc:927] Time spent T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:07:04.082440 21135 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.105082 20768 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:04.110723 21056 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:04.080473 21179 log.cc:927] Time spent T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
W20260709 13:07:04.070063 21182 log.cc:927] Time spent T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:07:04.111388 16008 rpcz_store.cc:276] 0709 13:07:00.589128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:01.316744 (+727616us) service_pool.cc:224] Handling call
0709 13:07:01.317103 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:01.367923 (+ 50820us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:04.068027 (+2700104us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b54003edd54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:04.068212 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24350080}
I20260709 13:07:04.023108 21181 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:04.120692 15933 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:07:04.074910 20938 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:04.129923 21010 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:07:04.121306 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 226814us
I20260709 13:07:04.086413 21167 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:04.131207 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:04.121181 15857 rpcz_store.cc:276] 0709 13:07:01.435911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:02.384654 (+948743us) service_pool.cc:224] Handling call
0709 13:07:02.384972 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:02.543225 (+158253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:02.543269 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:02.543320 (+    51us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:02.543353 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:04.077270 (+1533917us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b540032d554. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:04.077313 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:04.077445 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":363409152}
I20260709 13:07:04.197376 15930 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:04.197669 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:04.133520 15931 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:07:04.092733 21092 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.135934 15902 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:04.166584 20934 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:04.200743 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:04.201485 16011 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:04.196367 16008 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad"
dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
mode: GRACEFUL
new_leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:43700
I20260709 13:07:04.167968 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:04.202193 16008 raft_consensus.cc:606] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Received request to transfer leadership to TS eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:04.202476 15932 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:07:04.170046 20670 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:04.171597 16565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 1077) took 1527 ms. Trace:
I20260709 13:07:04.205447 16565 rpcz_store.cc:276] 0709 13:07:02.644462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:02.644600 (+   138us) service_pool.cc:224] Handling call
0709 13:07:02.644907 (+   307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:02.702619 (+ 57712us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:02.702644 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:02.712863 (+ 10219us) log.cc:844] Serialized 26 byte log entry
0709 13:07:03.664399 (+951536us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:03.669665 (+  5266us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:03.671116 (+  1451us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:04.171344 (+500228us) raft_consensus.cc:1685] finished
0709 13:07:04.171373 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:04.171524 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18714,"raft.queue_time_us":476551,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":21444,"spinlock_wait_cycles":391936,"thread_start_us":548239,"threads_started":2,"wal-append.queue_time_us":709976}
I20260709 13:07:04.139925 21000 raft_consensus.cc:740] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:04.210217 21000 consensus_queue.cc:260] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:04.211817 21000 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:04.212092 16006 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:04.126829 21117 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:04.114362 21113 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:04.149603 21135 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:04.159902 21180 consensus_queue.cc:1048] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 13:07:04.185590 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:04.192711 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:04.195576 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:04.196751 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:04.194602 16535 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:07:04.203663 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 67819.2us
I20260709 13:07:04.204828 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:04.233984 15931 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:07:04.234705 15902 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:04.225241 16562 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
W20260709 13:07:04.241478 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 46950.2us
I20260709 13:07:04.248265 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:04.198422 20930 consensus_queue.cc:1059] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 13:07:04.249676 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:04.250407 15936 raft_consensus.cc:2393] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 14: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:04.218156 21092 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:04.251192 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:04.252127 21122 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.253057 20803 raft_consensus.cc:2804] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:04.252211 21181 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:04.262310 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:04.254874 16533 leader_election.cc:304] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:04.257282 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:04.257473 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:07:04.259966 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:04.264101 15933 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:04.264678 16006 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
W20260709 13:07:04.279664 15825 connection.cc:256] RPC call timeout handler was delayed by -1.2744264789999988s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:07:04.277626 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:04.277310 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:07:04.289031 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:04.272558 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:04.278689 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:04.266582 21146 raft_consensus.cc:2804] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:04.219012 21167 raft_consensus.cc:2804] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:04.289240 16565 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 14.
I20260709 13:07:04.291599 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:04.292225 15936 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:07:04.288372 15825 connection.cc:256] RPC call timeout handler was delayed by -1.1523091359999853s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1668s for a response
I20260709 13:07:04.293951 21146 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:04.294966 16533 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:04.297166 16533 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:04.297633 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:04.298285 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:04.182106 21019 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:04.301262 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:04.305717 16008 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:07:04.193480 20752 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:04.307416 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:04.308089 20752 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:04.308462 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:04.309630 15857 raft_consensus.cc:1240] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:04.302205 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:04.311563 15931 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:04.312790 15930 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:07:04.313815 16011 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:07:04.313804 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:04.253022 21122 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:04.327375 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:04.328123 16011 raft_consensus.cc:2393] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 12: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:04.318615 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:04.299569 20803 raft_consensus.cc:697] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:04.331562 15933 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:07:04.295292 21167 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:04.278694 21132 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:04.267405 21171 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:04.334906 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:04.340497 15854 raft_consensus.cc:2393] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 13: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:04.335906 15936 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:07:04.225826 20841 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:04.301695 20369 raft_consensus.cc:2749] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:04.348439 20841 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:04.351126 21102 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:04.352008 21171 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:04.352451 21170 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:04.353566 15860 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:07:04.352358 15928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2371) took 3576 ms. Trace:
I20260709 13:07:04.354425 21171 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:04.354476 15928 rpcz_store.cc:276] 0709 13:07:00.776087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:00.777394 (+  1307us) service_pool.cc:224] Handling call
0709 13:07:00.777757 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:01.558639 (+780882us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:04.352140 (+2793501us) spinlock_profiling.cc:243] Waited 739 ms on lock 0x7b540034b554. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:04.352303 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1700138880}
I20260709 13:07:04.300402 21146 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:04.309046 20982 raft_consensus.cc:2749] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:04.339054 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:04.341573 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:04.343400 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:04.267760 21042 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:04.344697 21019 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:04.345768 21094 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:04.336264 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:04.279640 21176 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:04.355800 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:04.435688 15928 raft_consensus.cc:2393] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 8: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:04.436645 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:04.437350 21056 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:04.438462 21152 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:04.440240 21056 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:04.419189 21113 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:04.441892 15933 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:07:04.442138 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:04.443259 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:04.444244 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:04.355664 16569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1113) took 3334 ms. Trace:
I20260709 13:07:04.448227 16569 rpcz_store.cc:276] 0709 13:07:01.021281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:01.190654 (+169373us) service_pool.cc:224] Handling call
0709 13:07:01.190986 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:02.722662 (+1531676us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:04.355432 (+1632770us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b54000ce754. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:04.355607 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3481628416}
I20260709 13:07:04.449110 21176 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:04.445775 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:04.449559 15933 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:07:04.374727 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:04.450457 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:04.451233 15902 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:04.451478 21170 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:04.451560 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:04.401247 20979 consensus_meta.cc:287] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.507s	user 0.010s	sys 0.001s
I20260709 13:07:04.452423 20979 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:04.453600 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:04.451893 16004 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:07:04.455583 21094 raft_consensus.cc:2804] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:04.452423 15933 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:07:04.450039 15974 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:04.458189 21056 raft_consensus.cc:2749] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:04.458310 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:04.458714 15974 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:04.457538 15936 raft_consensus.cc:2376] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12. Re-sending same reply.
I20260709 13:07:04.459700 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:04.460292 21056 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:07:04.460834 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:04.461436 15933 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:07:04.462234 15931 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:07:04.450057 16011 raft_consensus.cc:2393] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 12: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:04.462647 15933 raft_consensus.cc:1240] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:07:04.463328 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:04.341434 21167 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:04.464571 15974 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:04.464835 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:04.456022 21094 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:04.466032 21094 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:04.467139 15974 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:04.374308 16563 raft_consensus.cc:1275] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:07:04.305229 20981 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:04.468400 20981 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:04.468768 15974 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:04.469655 21154 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:04.470021 21154 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:04.470335 21154 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:04.445056 15930 raft_consensus.cc:2410] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:07:04.473212 15974 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:04.474208 20879 consensus_queue.cc:1059] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:04.475091 20879 raft_consensus.cc:2749] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:04.475657 20879 raft_consensus.cc:3055] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:04.476016 20879 raft_consensus.cc:740] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:04.476099 21094 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:04.451743 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:04.476889 20879 consensus_queue.cc:260] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:04.477768 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:04.477843 21154 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:04.478412 16008 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 15.
I20260709 13:07:04.447732 21042 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:04.467909 21169 raft_consensus.cc:2749] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:04.478155 20879 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:04.363909 21132 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:04.468869 20981 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:04.454998 20979 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:04.465487 21056 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:04.487372 16011 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:04.487574 21056 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:04.488086 21056 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:04.455999 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:04.492197 16004 raft_consensus.cc:2393] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 15: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:04.493057 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:04.494205 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:04.494769 16004 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
I20260709 13:07:04.495513 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:04.496239 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:04.501019 16008 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:07:04.502058 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:04.503790 16008 raft_consensus.cc:2410] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:07:04.497154 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:04.461504 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:04.514596 16011 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
W20260709 13:07:04.505646 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 208546us
I20260709 13:07:04.524808 16533 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:04.525265 15929 raft_consensus.cc:1275] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:07:04.526096 16533 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:04.505218 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:04.520069 15936 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:46118
I20260709 13:07:04.528977 16533 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:04.530740 16533 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:04.537634 15934 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:04.492954 21132 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:04.540110 20982 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:04.540297 21180 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.546986 21180 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:04.549101 21180 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:04.550084 21056 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:04.550664 21121 consensus_queue.cc:1059] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:04.551211 16004 raft_consensus.cc:2393] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 15: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:04.551342 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:04.552495 16569 raft_consensus.cc:2393] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 14: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:04.542908 20930 consensus_queue.cc:1059] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 13:07:04.526804 21174 raft_consensus.cc:2749] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:04.560874 15930 raft_consensus.cc:1240] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:04.565371 21169 consensus_queue.cc:1059] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:04.566186 21169 raft_consensus.cc:3055] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:04.566490 21169 raft_consensus.cc:740] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:04.567260 21169 consensus_queue.cc:260] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:04.571688 21169 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:04.574311 15930 raft_consensus.cc:1240] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:07:04.583726 21137 consensus_queue.cc:1059] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:04.539198 21154 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:04.551467 21121 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:04.552630 20369 raft_consensus.cc:2749] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:04.554899 21135 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:04.605634 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 61365.3us
I20260709 13:07:04.524142 20803 consensus_queue.cc:237] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:04.611461 15936 raft_consensus.cc:606] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Received request to transfer leadership to TS eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:04.611903 15936 raft_consensus.cc:614] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:04.600405 20982 raft_consensus.cc:697] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:04.615643 21198 raft_consensus.cc:3055] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:04.615768 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:04.616513 15930 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
W20260709 13:07:04.619338 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:04.616070 20858 raft_consensus.cc:3055] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:04.628221 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:04.631055 20975 raft_consensus.cc:2955] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Committing config change with OpId 13.10: config changed from index 3 to 10, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:07:04.638779 21123 log.cc:927] Time spent T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 2.255s	user 0.017s	sys 0.000s
I20260709 13:07:04.622973 20369 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:07:04.639798 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:04.616204 21198 raft_consensus.cc:740] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:04.658718 15937 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 1033) took 3130 ms. Trace:
I20260709 13:07:04.624418 21174 consensus_queue.cc:1059] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:07:04.664463 21174 raft_consensus.cc:3055] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:04.664793 21174 raft_consensus.cc:740] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:04.665542 21174 consensus_queue.cc:260] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:04.666647 21174 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:04.668190 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:04.642242 20858 raft_consensus.cc:740] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:04.660210 15937 rpcz_store.cc:276] 0709 13:07:01.528139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:01.578249 (+ 50110us) service_pool.cc:224] Handling call
0709 13:07:01.578514 (+   265us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:01.587982 (+  9468us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:01.588034 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:01.588482 (+   448us) log.cc:844] Serialized 26 byte log entry
0709 13:07:02.127086 (+538604us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:02.127143 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:02.127238 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:04.638268 (+2511030us) spinlock_profiling.cc:243] Waited 4.48 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:07:04.657327 (+ 19059us) raft_consensus.cc:1685] finished
0709 13:07:04.657361 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:04.658635 (+  1274us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1859,"raft.queue_time_us":2052,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":7482,"spinlock_wait_cycles":10294016,"thread_start_us":463867,"threads_started":1,"wal-append.queue_time_us":748233}
I20260709 13:07:04.673534 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:04.674296 15937 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:04.677991 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:04.678496 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:04.672690 21196 log.cc:927] Time spent T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.240s	user 0.000s	sys 0.001s
I20260709 13:07:04.668099 21152 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:04.659986 20369 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:04.663311 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:04.697981 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:04.699520 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:04.676184 21115 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:07:04.701671 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 282123us
I20260709 13:07:04.705300 20369 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:04.682848 21056 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:04.677845 20982 consensus_queue.cc:237] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:04.712096 21115 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:04.717042 15937 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
I20260709 13:07:04.718031 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:04.718874 15933 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
W20260709 13:07:04.721967 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 126038us
I20260709 13:07:04.716642 21198 consensus_queue.cc:260] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:04.730723 21167 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:04.737367 15976 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:04.745134 21167 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:04.763783 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:04.766006 21169 raft_consensus.cc:2804] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:04.796149 21169 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:04.796557 21169 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:07:04.789389 21168 consensus_meta.cc:287] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.794s	user 0.009s	sys 0.003s
I20260709 13:07:04.746414 20981 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:04.798707 21198 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:07:04.799310 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 62036.3us
W20260709 13:07:04.799887 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 70163.4us
I20260709 13:07:04.800377 21056 consensus_queue.cc:1059] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:04.800719 20981 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:04.801219 21133 raft_consensus.cc:3055] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:04.801616 21133 raft_consensus.cc:740] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:04.802407 21133 consensus_queue.cc:260] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:04.805095 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:04.805873 15854 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
I20260709 13:07:04.806829 15902 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:04.807612 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:04.803521 21133 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:04.809661 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:04.810814 15902 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:04.812093 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:04.813657 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:04.814313 16562 raft_consensus.cc:1240] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:04.789036 21160 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:04.816828 15857 raft_consensus.cc:2393] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 12: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:04.815425 15974 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:04.820866 15974 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:04.821933 21056 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:04.817999 15902 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:04.822471 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:04.822881 20773 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:04.823230 20773 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:04.823597 20773 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:04.823556 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:04.824772 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:04.825363 16562 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:07:04.826419 15902 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:04.827157 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:04.827708 15902 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:04.827152 21180 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:04.828541 15974 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:04.828887 21113 raft_consensus.cc:2749] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:04.828148 15858 raft_consensus.cc:2393] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 11: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:04.830084 15902 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:04.830543 21056 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:04.830796 21113 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:04.831180 21113 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:04.831527 21113 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:04.832580 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:04.828506 15976 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:04.841151 21113 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:04.844520 15902 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:04.846000 15902 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:04.846938 21000 raft_consensus.cc:2749] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:04.847352 15902 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:04.852202 15902 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:04.853710 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:04.797467 21168 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:04.819773 21089 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:04.849057 21098 raft_consensus.cc:2749] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:04.849996 21135 raft_consensus.cc:2749] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:04.856861 21000 raft_consensus.cc:2764] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:07:04.882833 16569 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:04.856462 21160 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:04.884501 21113 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:04.876820 21214 raft_consensus.cc:2804] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:04.912559 21214 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:04.916309 21214 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:04.919569 21168 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:04.924404 21160 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:04.928874 21169 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:04.929479 21092 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:04.930795 21214 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:04.933741 21146 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:04.937911 21146 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:04.939468 21214 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:04.944180 21094 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:04.945155 21110 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:04.946676 21169 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:04.919881 21223 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:04.934913 21210 log.cc:927] Time spent T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.223s	user 0.000s	sys 0.001s
I20260709 13:07:04.951574 21223 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:04.958318 21188 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.888s	user 0.024s	sys 0.040s
W20260709 13:07:04.958765 21188 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.935s	user 0.024s	sys 0.040s
I20260709 13:07:04.961220 21223 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:05.001793 15827 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:05.009749 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 157443us
I20260709 13:07:05.012877 15902 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:05.019536 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:05.022125 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:07:05.028357 15862 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 2283) took 3591 ms (client timeout 1718 ms). Trace:
W20260709 13:07:05.035787 15862 rpcz_store.cc:269] 0709 13:07:01.436374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:01.593388 (+157014us) service_pool.cc:224] Handling call
0709 13:07:05.027752 (+3434364us) spinlock_profiling.cc:243] Waited 718 ms on lock 0x7b540008b954. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:05.028064 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1652057856}
I20260709 13:07:05.029906 15856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 970) took 4242 ms. Trace:
I20260709 13:07:05.037537 15856 rpcz_store.cc:276] 0709 13:07:00.786959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:00.787146 (+   187us) service_pool.cc:224] Handling call
0709 13:07:00.787499 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:01.416447 (+628948us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:05.029276 (+3612829us) spinlock_profiling.cc:243] Waited 628 ms on lock 0x7b54003b5954. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:05.029702 (+   426us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1444648704}
I20260709 13:07:05.032425 15860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 2383) took 1126 ms. Trace:
I20260709 13:07:05.039537 15860 rpcz_store.cc:276] 0709 13:07:03.906179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.022675 (+116496us) service_pool.cc:224] Handling call
0709 13:07:05.032028 (+1009353us) spinlock_profiling.cc:243] Waited 306 ms on lock 0x7b54002490d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:05.032233 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":703256320}
I20260709 13:07:05.043526 15861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2367) took 1259 ms. Trace:
I20260709 13:07:05.044131 15861 rpcz_store.cc:276] 0709 13:07:03.784059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:03.784220 (+   161us) service_pool.cc:224] Handling call
0709 13:07:03.784553 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:04.008095 (+223542us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:05.042891 (+1034796us) spinlock_profiling.cc:243] Waited 203 ms on lock 0x7b5400219654. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:05.043298 (+   407us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":467380992}
I20260709 13:07:05.047134 15853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 973) took 4257 ms. Trace:
I20260709 13:07:05.047765 15853 rpcz_store.cc:276] 0709 13:07:00.789412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:00.789556 (+   144us) service_pool.cc:224] Handling call
0709 13:07:00.789853 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:01.477176 (+687323us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:05.046369 (+3569193us) spinlock_profiling.cc:243] Waited 581 ms on lock 0x7b54002481d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:05.046856 (+   487us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1336854400}
I20260709 13:07:05.050505 15931 raft_consensus.cc:2393] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 12: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
W20260709 13:07:05.053603 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 223232us
W20260709 13:07:05.058111 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 186700us
W20260709 13:07:05.063547 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:05.066313 16533 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:05.058461 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:05.070210 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:05.072563 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:05.073683 15932 raft_consensus.cc:2393] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 12: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:05.073467 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:05.074797 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:07:05.049338 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:05.076992 15974 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:05.077338 15932 raft_consensus.cc:2376] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11. Re-sending same reply.
I20260709 13:07:05.079802 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:05.081581 16002 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60156 (request call id 978) took 1730 ms (client timeout 1555 ms). Trace:
W20260709 13:07:05.082036 16002 rpcz_store.cc:269] 0709 13:07:03.350665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:03.350853 (+   188us) service_pool.cc:224] Handling call
0709 13:07:05.081041 (+1730188us) spinlock_profiling.cc:243] Waited 686 ms on lock 0x7b54003ece54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:05.081309 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1578552064}
I20260709 13:07:05.082651 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:05.085414 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:05.086570 16009 raft_consensus.cc:2410] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:07:05.088430 15933 raft_consensus.cc:2393] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 14: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:05.094137 16011 raft_consensus.cc:1275] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:07:05.094897 16011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2190) took 1316 ms. Trace:
I20260709 13:07:05.097112 16003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41674 (request call id 2191) took 1308 ms. Trace:
I20260709 13:07:05.097590 16003 rpcz_store.cc:276] 0709 13:07:03.789062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:03.831962 (+ 42900us) service_pool.cc:224] Handling call
0709 13:07:05.080343 (+1248381us) spinlock_profiling.cc:243] Waited 24.5 ms on lock 0x7b54001dcc54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:05.096922 (+ 16579us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":56291072}
I20260709 13:07:05.074923 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:05.095263 16011 rpcz_store.cc:276] 0709 13:07:03.778074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.092750 (+1314676us) service_pool.cc:224] Handling call
0709 13:07:05.093348 (+   598us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:05.094571 (+  1223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:05.094790 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:05.077334 15937 raft_consensus.cc:1240] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:07:05.109789 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:05.110395 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:05.110742 15931 raft_consensus.cc:2393] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 14: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:05.111469 21153 raft_consensus.cc:2804] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:05.111922 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:05.112353 21153 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:05.112883 15937 raft_consensus.cc:2368] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 11. Current term is 12.
I20260709 13:07:05.112921 21153 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:07:04.913529 21010 consensus_meta.cc:287] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.709s	user 0.006s	sys 0.006s
I20260709 13:07:05.116972 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:05.118149 16002 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 13.
I20260709 13:07:05.119655 21180 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:05.114106 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:05.123045 15937 raft_consensus.cc:2393] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 12: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:05.111995 15929 raft_consensus.cc:2376] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12. Re-sending same reply.
I20260709 13:07:05.124878 21122 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:05.124948 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:05.125479 21139 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:05.126366 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:05.127478 21223 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:05.127743 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:05.113369 20934 raft_consensus.cc:2804] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:05.132095 21096 raft_consensus.cc:2804] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:05.132652 21096 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:05.133090 21096 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:05.135793 16567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1119) took 1766 ms. Trace:
I20260709 13:07:05.126381 21139 raft_consensus.cc:697] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:05.136235 16567 rpcz_store.cc:276] 0709 13:07:03.369233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:03.369330 (+    97us) service_pool.cc:224] Handling call
0709 13:07:03.374394 (+  5064us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:03.503281 (+128887us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:07:03.595030 (+ 91749us) log.cc:844] Serialized 14 byte log entry
0709 13:07:03.697734 (+102704us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:03.697858 (+   124us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:03.697991 (+   133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:05.135252 (+1437261us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b5400695ad4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:05.135324 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:05.135539 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3480,"prepare.queue_time_us":149381,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":144,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":5826,"spinlock_wait_cycles":295926656,"thread_start_us":193481,"threads_started":2,"wal-append.queue_time_us":256937}
I20260709 13:07:05.137729 21139 consensus_queue.cc:237] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:05.146236 21122 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:05.147295 21122 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:05.148583 16006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41674 (request call id 2189) took 1383 ms. Trace:
I20260709 13:07:05.149024 16006 rpcz_store.cc:276] 0709 13:07:03.765551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:03.985777 (+220226us) service_pool.cc:224] Handling call
0709 13:07:05.148178 (+1162401us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b54003ee754. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:05.148404 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":508609792}
I20260709 13:07:05.150924 16006 raft_consensus.cc:1240] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:07:05.151106 16568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 1070) took 3717 ms. Trace:
I20260709 13:07:05.152622 20917 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.153509 20917 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:05.155262 21122 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:05.156345 16011 raft_consensus.cc:1240] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:05.161870 20642 raft_consensus.cc:993] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee: : Instructing follower eb1e0493af1e46dbb6dec7e25525bab5 to start an election
I20260709 13:07:05.162724 21154 raft_consensus.cc:1081] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Signalling peer eb1e0493af1e46dbb6dec7e25525bab5 to start an election
I20260709 13:07:05.163106 16011 raft_consensus.cc:1240] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:05.151571 16568 rpcz_store.cc:276] 0709 13:07:01.434061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:01.505282 (+ 71221us) service_pool.cc:224] Handling call
0709 13:07:01.505649 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:02.230514 (+724865us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:07:02.252995 (+ 22481us) log.cc:844] Serialized 14 byte log entry
0709 13:07:02.442236 (+189241us) log.cc:844] Serialized 14 byte log entry
0709 13:07:02.442662 (+   426us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:02.442710 (+    48us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:02.442760 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:05.150577 (+2707817us) spinlock_profiling.cc:243] Waited 724 ms on lock 0x7b54006a1654. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:05.150673 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:05.150921 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7376,"prepare.queue_time_us":22820,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":2991,"raft.queue_time_us":7596,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":184,"spinlock_wait_cycles":1666310016,"thread_start_us":179792,"threads_started":1,"wal-append.queue_time_us":289733,"wal-append.run_cpu_time_us":15118,"wal-append.run_wall_time_us":2019467}
I20260709 13:07:05.130476 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:05.167117 20934 raft_consensus.cc:697] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 12 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
W20260709 13:07:05.185293 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 97592.5us
I20260709 13:07:05.211199 20927 raft_consensus.cc:3055] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:05.211280 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:05.211525 20670 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.215306 21154 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.218184 20670 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:05.215601 20927 raft_consensus.cc:740] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:05.221467 20937 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:05.221639 20670 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:05.221282 20927 consensus_queue.cc:260] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:05.222236 20937 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:05.222487 16567 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:05.222730 20927 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:05.224982 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:05.226779 21013 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:05.231161 21226 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.232018 21226 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:05.234776 16567 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 15.
I20260709 13:07:05.235576 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:05.235802 21226 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:05.236476 16561 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:05.218282 21154 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:05.223017 20937 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:07:05.212453 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:05.245205 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:05.256948 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 177725us
I20260709 13:07:05.260252 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:05.261196 15861 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:07:05.261790 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:05.262214 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:05.262814 15858 raft_consensus.cc:2393] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 16: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:05.264000 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:05.264962 15931 raft_consensus.cc:2376] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13. Re-sending same reply.
I20260709 13:07:05.266973 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:05.267880 21154 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:05.270721 16006 raft_consensus.cc:1240] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:05.271486 16006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 992) took 1029 ms. Trace:
I20260709 13:07:05.272058 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:05.272740 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:05.273916 16003 raft_consensus.cc:1275] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:07:05.271806 16006 rpcz_store.cc:276] 0709 13:07:04.242279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.203306 (+961027us) service_pool.cc:224] Handling call
0709 13:07:05.270394 (+ 67088us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:05.271231 (+   837us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:05.271409 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:05.274402 15854 raft_consensus.cc:1240] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:05.276437 15854 raft_consensus.cc:1240] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:05.276928 20917 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:05.278228 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:05.282452 20868 raft_consensus.cc:3055] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:07:05.282745 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:05.283126 20868 raft_consensus.cc:740] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:05.284703 20868 consensus_queue.cc:260] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:05.287829 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:05.288128 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:05.289042 15936 raft_consensus.cc:2376] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9. Re-sending same reply.
I20260709 13:07:05.285776 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:05.290304 15862 raft_consensus.cc:2393] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 12: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:05.281076 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:05.291206 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:05.291618 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:05.286170 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:05.267827 15858 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:05.292713 15860 raft_consensus.cc:2393] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 11: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:05.293147 15853 raft_consensus.cc:1240] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:05.293902 15860 raft_consensus.cc:1240] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:05.285390 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:05.296284 15855 raft_consensus.cc:2393] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 12: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:05.296775 20858 consensus_queue.cc:260] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:05.291402 15854 raft_consensus.cc:2393] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 15: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:05.298461 20858 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:05.298394 21154 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.299299 21154 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:05.300889 20934 consensus_queue.cc:237] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:05.301824 21154 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:05.307555 15858 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:05.310031 16562 raft_consensus.cc:1240] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:05.311647 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:05.312686 16562 raft_consensus.cc:2393] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 16: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:05.315759 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:05.316329 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:05.322292 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:05.323302 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:05.279163 15931 raft_consensus.cc:2376] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11. Re-sending same reply.
I20260709 13:07:05.286760 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:05.289038 15861 raft_consensus.cc:2393] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 13: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:05.274530 16003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2193) took 1294 ms. Trace:
I20260709 13:07:05.335602 16003 rpcz_store.cc:276] 0709 13:07:03.979923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.273212 (+1293289us) service_pool.cc:224] Handling call
0709 13:07:05.273538 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:05.274308 (+   770us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:05.274484 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:05.332762 16569 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:05.343572 16561 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
I20260709 13:07:05.352623 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:05.285934 20868 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:05.361578 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:05.356276 16533 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:05.335229 21115 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:05.286232 20773 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:05.379027 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:05.381671 16562 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:05.389554 15929 raft_consensus.cc:2393] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 12: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:05.390481 21115 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:05.390739 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:05.391158 21115 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:05.391700 16570 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 13.
I20260709 13:07:05.392979 20929 raft_consensus.cc:2749] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:05.393760 21115 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:05.394681 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:05.396008 15902 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:05.397112 20773 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:05.397554 15902 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:05.397826 20773 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:05.399276 21181 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:05.399746 21181 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:05.400135 21181 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:05.400617 20937 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
I20260709 13:07:05.404989 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:05.406930 16561 raft_consensus.cc:1275] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:07:05.342090 15936 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:46118
I20260709 13:07:05.409770 15936 raft_consensus.cc:606] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Received request to transfer leadership to TS 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:05.410223 15936 raft_consensus.cc:614] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:05.358124 16567 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad"
dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:45232
I20260709 13:07:05.419767 16004 raft_consensus.cc:1240] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 9. Current term is 10. Ops: []
I20260709 13:07:05.421144 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:05.422039 16004 raft_consensus.cc:2393] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 16: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:05.422520 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:07:05.422930 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:05.423301 16005 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:07:05.423853 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:05.425520 16535 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:05.425797 15902 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:05.426970 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:05.427158 15902 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:05.427583 16007 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:07:05.431888 21167 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.432255 21157 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:05.432593 16007 raft_consensus.cc:1240] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:07:05.432500 21167 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:05.433928 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:05.435055 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:05.435789 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:05.436506 16565 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:07:05.437584 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:05.440124 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:05.440723 16003 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:05.441885 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:05.443266 16007 raft_consensus.cc:2368] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 14. Current term is 15.
I20260709 13:07:05.445583 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:05.447731 15928 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:07:05.448119 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:05.451932 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:05.452745 16566 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:07:05.453804 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:05.454667 16566 raft_consensus.cc:2393] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 14: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:05.457823 16005 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 9.
I20260709 13:07:05.445822 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:05.415966 15861 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:07:05.461603 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:05.462514 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:05.466710 15934 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:05.357110 21226 consensus_queue.cc:1048] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:07:05.457582 16567 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:05.479255 16569 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
I20260709 13:07:05.475781 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:07:05.477434 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 50367.6us
I20260709 13:07:05.484582 15902 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:05.472589 20773 raft_consensus.cc:2749] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:05.486258 20773 raft_consensus.cc:2804] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:05.486078 21177 consensus_queue.cc:1048] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:05.486884 20773 raft_consensus.cc:697] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 15 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:05.488323 15827 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:05.489367 15827 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:05.489619 20768 raft_consensus.cc:2804] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:07:05.490111 20768 raft_consensus.cc:697] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 10 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:05.491829 16003 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:07:05.492010 20853 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:05.492542 20853 raft_consensus.cc:697] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:05.492861 21071 consensus_queue.cc:1048] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:07:05.493705 20853 consensus_queue.cc:237] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:05.492985 21141 consensus_queue.cc:1048] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:05.492898 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:05.495617 16003 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:07:05.496315 15936 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:46118
I20260709 13:07:05.496691 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:05.496905 15936 raft_consensus.cc:606] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Received request to transfer leadership to TS eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:05.497421 15936 raft_consensus.cc:614] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:05.491387 15827 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:05.486593 21113 raft_consensus.cc:2804] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:05.500241 16567 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:07:05.507977 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 93587.1us
I20260709 13:07:05.507439 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:05.510224 16535 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:05.528381 16007 raft_consensus.cc:1275] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:07:05.521337 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:05.531809 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:05.532085 20797 raft_consensus.cc:2804] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:05.532541 16003 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:05.532809 20797 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:05.533404 20797 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:05.521585 21113 raft_consensus.cc:697] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 12 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:05.536801 21113 consensus_queue.cc:237] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:05.539077 21223 raft_consensus.cc:2749] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:05.539921 16006 raft_consensus.cc:1240] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:05.540021 20773 consensus_queue.cc:237] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:05.541443 21153 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:05.543992 16563 raft_consensus.cc:2393] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 12: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
W20260709 13:07:05.545262 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 53979.4us
I20260709 13:07:05.547103 15827 leader_election.cc:400] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 14. Current term is 15.
I20260709 13:07:05.547394 15827 leader_election.cc:403] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:07:05.547859 21141 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:05.548288 21141 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:05.548208 21177 consensus_queue.cc:1059] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:05.491107 20768 consensus_queue.cc:237] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:05.548969 21141 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:07:05.552793 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:05.558451 16567 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:05.567641 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:05.569588 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:05.568895 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:05.573287 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:05.575110 16567 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:05.552767 21153 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:05.581377 16564 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:07:05.582397 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:05.583133 16564 raft_consensus.cc:2393] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 12: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
W20260709 13:07:05.585829 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 337389us
I20260709 13:07:05.587802 16003 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:07:05.605742 15900 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:05.572798 21180 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:05.569839 21223 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:05.610440 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:05.611032 15900 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:05.576431 21115 raft_consensus.cc:2749] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 14. Current term is 15.
I20260709 13:07:05.579006 20929 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:05.617762 15827 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:05.609174 21157 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:07:05.629459 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 72104.9us
I20260709 13:07:05.631400 16564 raft_consensus.cc:1275] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:07:05.631944 21010 consensus_queue.cc:1059] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:05.633152 21010 raft_consensus.cc:3055] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:05.633492 21010 raft_consensus.cc:740] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:07:05.634145 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 207013us
I20260709 13:07:05.634436 21010 consensus_queue.cc:260] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:05.635020 16535 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:05.637645 16569 raft_consensus.cc:1240] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:07:05.637915 20917 consensus_queue.cc:1059] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:05.639038 21122 consensus_queue.cc:1059] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:05.639566 20917 raft_consensus.cc:3055] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:05.640210 20917 raft_consensus.cc:740] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:05.572345 20642 consensus_queue.cc:1059] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
W20260709 13:07:05.631317 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:05.650241 16570 raft_consensus.cc:2393] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 14: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:05.650539 15936 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:46118
I20260709 13:07:05.651345 15934 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:05.651340 15936 raft_consensus.cc:606] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Received request to transfer leadership to TS 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:05.652477 15936 raft_consensus.cc:614] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:07:05.648911 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 72994.9us
I20260709 13:07:05.637661 20499 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:07:05.691812 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:05.681941 15825 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:05.689623 15827 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:05.655247 21010 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:05.657516 21013 raft_consensus.cc:2749] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:07:05.706069 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 44575us
I20260709 13:07:05.711035 21099 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:07:05.714172 16568 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
W20260709 13:07:05.715819 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 108112us
I20260709 13:07:05.716862 15900 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:05.719278 21226 raft_consensus.cc:2749] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:05.697239 16533 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:05.644466 21223 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:05.700317 21177 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:07:05.732999 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 77956.4us
I20260709 13:07:05.685078 21118 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.743829 16011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 991) took 1502 ms. Trace:
I20260709 13:07:05.744289 16011 rpcz_store.cc:276] 0709 13:07:04.241864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.164052 (+922188us) service_pool.cc:224] Handling call
0709 13:07:05.164565 (+   513us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:05.165577 (+  1012us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:05.165612 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:05.193620 (+ 28008us) log.cc:844] Serialized 26 byte log entry
0709 13:07:05.519910 (+326290us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:05.519964 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:05.520054 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:05.743556 (+223502us) raft_consensus.cc:1685] finished
0709 13:07:05.743577 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:05.743741 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1926,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":208,"thread_start_us":305067,"threads_started":1,"wal-append.queue_time_us":535413}
I20260709 13:07:05.747901 16569 raft_consensus.cc:1275] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:07:05.739975 21099 raft_consensus.cc:2804] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:05.748638 21099 raft_consensus.cc:697] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:05.749660 21099 consensus_queue.cc:237] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:05.751569 20752 raft_consensus.cc:3055] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:05.751981 20752 raft_consensus.cc:740] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
W20260709 13:07:05.752643 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 66043.8us
I20260709 13:07:05.753027 20752 consensus_queue.cc:260] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:05.753533 16533 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:05.754289 20868 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:05.754765 20868 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:05.755138 20868 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:05.757102 15928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 2412) took 1341 ms. Trace:
I20260709 13:07:05.757563 15928 rpcz_store.cc:276] 0709 13:07:04.415526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.436385 (+ 20859us) service_pool.cc:224] Handling call
0709 13:07:05.756869 (+1320484us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54002cc754. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:05.757033 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2323395968}
I20260709 13:07:05.760648 21167 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:05.762622 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 192331us
I20260709 13:07:05.763336 15974 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:05.764433 15974 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:05.767000 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2195) took 1681 ms. Trace:
I20260709 13:07:05.754339 20752 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:05.772498 16006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 2366) took 1172 ms. Trace:
I20260709 13:07:05.772930 16006 rpcz_store.cc:276] 0709 13:07:04.595161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.274968 (+679807us) service_pool.cc:224] Handling call
0709 13:07:05.275308 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:05.540331 (+265023us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:05.767349 (+227018us) spinlock_profiling.cc:243] Waited 264 ms on lock 0x7b54001ba154. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:05.767526 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":608100096}
I20260709 13:07:05.741482 15827 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:05.757071 16533 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:05.784127 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 53498.9us
I20260709 13:07:05.786252 21231 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:05.666067 21181 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:05.787618 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 49626us
I20260709 13:07:05.767331 16002 rpcz_store.cc:276] 0709 13:07:04.085632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.119009 (+1033377us) service_pool.cc:224] Handling call
0709 13:07:05.119571 (+   562us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:05.164586 (+ 45015us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:07:05.240381 (+ 75795us) log.cc:844] Serialized 14 byte log entry
0709 13:07:05.766683 (+526302us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:05.766744 (+    61us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:05.766790 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:05.766850 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:05.766952 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":184978,"prepare.queue_time_us":221270,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":57,"thread_start_us":348226,"threads_started":2}
I20260709 13:07:05.795442 20879 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:05.795965 15827 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:05.709690 16004 raft_consensus.cc:2955] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Committing config change with OpId 13.10: config changed from index 3 to 10, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:07:05.797387 15827 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:05.798549 21071 raft_consensus.cc:2749] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:05.799353 21071 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:05.799988 21071 raft_consensus.cc:697] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 12 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:05.800767 15974 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:05.801220 21071 consensus_queue.cc:237] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:05.807061 21056 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.796039 20879 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:05.663166 20917 consensus_queue.cc:260] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:05.653802 21157 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:05.856468 20917 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:05.858632 21157 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:05.858850 21122 raft_consensus.cc:2749] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:05.859891 16564 raft_consensus.cc:2955] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Committing config change with OpId 13.10: config changed from index 3 to 10, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:07:05.773083 21188 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:07:05.867318 20858 consensus_meta.cc:287] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.568s	user 0.006s	sys 0.003s
I20260709 13:07:05.787086 21177 raft_consensus.cc:697] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 9 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:05.812546 20499 raft_consensus.cc:2749] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:05.868467 21177 consensus_queue.cc:237] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:05.815574 21231 raft_consensus.cc:697] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:05.873255 21231 consensus_queue.cc:237] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:05.815001 21096 consensus_meta.cc:287] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.681s	user 0.013s	sys 0.001s
I20260709 13:07:05.876070 21096 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:05.878532 21096 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:05.884014 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:07:05.892939 20642 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:05.831828 20944 raft_consensus.cc:2804] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:05.899525 20944 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:05.899950 20944 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:05.796013 21118 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:05.902423 21118 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:05.819072 20879 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:05.807720 21056 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:05.837837 21223 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:05.839257 16005 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
W20260709 13:07:05.840350 21235 log.cc:927] Time spent T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 13:07:05.909421 21056 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:05.910744 16010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 986) took 1670 ms. Trace:
I20260709 13:07:05.842831 21181 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:05.854562 20797 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:05.916993 16562 consensus_meta.cc:287] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.527s	user 0.008s	sys 0.000s
W20260709 13:07:05.917639 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 64811.3us
W20260709 13:07:05.858623 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 101829us
I20260709 13:07:05.920548 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:05.931162 20797 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:05.938715 21181 consensus_queue.cc:1048] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:05.939996 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:05.938959 21153 raft_consensus.cc:3055] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:05.941956 16562 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:07:05.940599 16563 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:05.945883 21153 raft_consensus.cc:740] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:05.948127 16568 raft_consensus.cc:1240] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:07:05.911168 16010 rpcz_store.cc:276] 0709 13:07:04.240681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.140834 (+900153us) service_pool.cc:224] Handling call
0709 13:07:05.141811 (+   977us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:07:05.142835 (+  1024us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:05.142876 (+    41us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:07:05.180337 (+ 37461us) log.cc:844] Serialized 78 byte log entry
0709 13:07:05.464809 (+284472us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:05.511351 (+ 46542us) log.cc:844] Serialized 14 byte log entry
0709 13:07:05.565673 (+ 54322us) log.cc:844] Serialized 14 byte log entry
0709 13:07:05.565842 (+   169us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:05.565916 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:05.747243 (+181327us) spinlock_profiling.cc:243] Waited 2.65 ms on lock 0x7b180031b040. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb3f3 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd 000000000038f354 000000000038f199 00007f73e729e83a
0709 13:07:05.762382 (+ 15139us) spinlock_profiling.cc:243] Waited 3 ms on lock 0x7b180031b040. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e7235c0a 00007f73f0975219 00007f73ee27c58f 00007f73e72bb571 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd
0709 13:07:05.910407 (+148025us) raft_consensus.cc:1685] finished
0709 13:07:05.910428 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:05.910579 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":239658,"prepare.queue_time_us":87052,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":85,"raft.queue_time_us":41165,"raft.run_cpu_time_us":727,"raft.run_wall_time_us":18486,"spinlock_wait_cycles":13001344,"thread_start_us":257179,"threads_started":1,"wal-append.queue_time_us":427788}
I20260709 13:07:05.950994 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:05.949167 15929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 2421) took 1100 ms. Trace:
I20260709 13:07:05.952558 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:05.951946 15929 rpcz_store.cc:276] 0709 13:07:04.840781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.124484 (+283703us) service_pool.cc:224] Handling call
0709 13:07:05.941211 (+816727us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b54003b8b54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:05.941334 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":603241856}
I20260709 13:07:05.921346 21141 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:05.963641 15825 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:05.969321 16533 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:05.994122 20734 raft_consensus.cc:3055] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:05.994716 20734 raft_consensus.cc:740] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:05.995800 20734 consensus_queue.cc:260] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:05.997400 20734 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:07:06.014065 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:06.025048 20499 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:07:06.035742 20934 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.038s	sys 0.117s
W20260709 13:07:06.042264 20934 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.039s	sys 0.117s
I20260709 13:07:06.035539 20868 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:06.054400 16563 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 14.
I20260709 13:07:06.055938 15902 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:07:06.062040 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 103624us
I20260709 13:07:06.041803 20879 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:06.073992 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:06.119558 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:06.131736 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 190296us
I20260709 13:07:06.130882 16533 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:06.060519 21181 raft_consensus.cc:2804] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:06.059180 21013 consensus_queue.cc:1059] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.072s
I20260709 13:07:06.165189 21155 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:07:06.179226 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 376845us
I20260709 13:07:06.170503 21167 raft_consensus.cc:3055] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:07:06.181208 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 106392us
W20260709 13:07:06.182459 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 385159us
I20260709 13:07:06.170006 21181 raft_consensus.cc:697] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 14 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:06.171695 21188 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:06.202845 16533 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:06.178087 20670 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:06.182086 21167 raft_consensus.cc:740] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:06.193673 21141 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:06.201665 21139 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.974s	user 0.006s	sys 0.017s
W20260709 13:07:06.213759 20773 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.031s	sys 0.061s
I20260709 13:07:06.196146 21155 raft_consensus.cc:697] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
W20260709 13:07:06.219149 20773 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.031s	sys 0.062s
I20260709 13:07:06.228713 16010 raft_consensus.cc:1275] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 13:07:06.217239 21139 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.991s	user 0.006s	sys 0.017s
I20260709 13:07:06.222913 21122 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:06.225165 20868 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:06.245548 16005 raft_consensus.cc:1275] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:07:06.249289 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:07:06.253224 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 55278us
I20260709 13:07:06.216675 21188 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:06.220960 21028 consensus_queue.cc:1048] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 13:07:06.258443 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:06.233738 20670 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:06.233131 21124 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:06.444998 16567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 1100) took 1018 ms. Trace:
I20260709 13:07:06.299579 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:06.242096 20879 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:07:06.388026 15900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.001s	sys 0.033s
I20260709 13:07:06.433857 20773 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:07:06.299268 16009 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.000s	sys 0.026s
I20260709 13:07:06.546895 16005 raft_consensus.cc:2393] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 13: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
W20260709 13:07:06.396366 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 200847us
W20260709 13:07:06.296041 15901 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.006s	sys 0.009s
I20260709 13:07:06.301406 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:07:06.519393 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 306126us
I20260709 13:07:06.280292 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:07:06.555328 16009 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.787s	user 0.011s	sys 0.036s
I20260709 13:07:06.542440 16006 raft_consensus.cc:2410] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:07:06.558025 16009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2192) took 2631 ms. Trace:
W20260709 13:07:06.307904 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:06.558581 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:06.438531 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:06.398710 15935 raft_consensus.cc:1240] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 10. Current term is 11. Ops: []
W20260709 13:07:06.557148 15901 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.865s	user 0.023s	sys 0.056s
I20260709 13:07:06.523999 16008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2188) took 2770 ms. Trace:
W20260709 13:07:06.477571 21241 log.cc:927] Time spent T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.272s	user 0.001s	sys 0.002s
W20260709 13:07:06.463565 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 265823us
W20260709 13:07:06.478873 21238 log.cc:927] Time spent T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.405s	user 0.003s	sys 0.000s
W20260709 13:07:06.551760 15900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.834s	user 0.019s	sys 0.034s
W20260709 13:07:06.250686 21113 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.000s	sys 0.022s
I20260709 13:07:06.302709 15976 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:06.280757 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:06.523993 21244 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:06.542294 16567 rpcz_store.cc:276] 0709 13:07:05.413555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.642366 (+228811us) service_pool.cc:224] Handling call
0709 13:07:05.660345 (+ 17979us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:05.768032 (+107687us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:07:06.191931 (+423899us) log.cc:844] Serialized 14 byte log entry
0709 13:07:06.428119 (+236188us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:06.431719 (+  3600us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:06.431783 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:06.431874 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:06.432044 (+   170us) inbound_call.cc:177] Queueing success response
0709 13:07:06.522470 (+ 90426us) spinlock_profiling.cc:243] Waited 7.96 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
Metrics: {"prepare.queue_time_us":680356,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":953,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":8102,"spinlock_wait_cycles":18308864,"thread_start_us":596105,"threads_started":2,"wal-append.queue_time_us":278700}
I20260709 13:07:06.579813 21244 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:06.300891 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:07:06.389322 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 91410us
I20260709 13:07:06.581704 16533 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
W20260709 13:07:06.553750 21268 log.cc:927] Time spent T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.066s	user 0.001s	sys 0.002s
I20260709 13:07:06.567930 16010 raft_consensus.cc:2393] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 12: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:06.571385 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:06.584079 16002 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:06.584125 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:06.585047 15861 raft_consensus.cc:2393] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 13: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:07:06.588754 21251 log.cc:927] Time spent T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 13:07:06.592052 16002 raft_consensus.cc:2410] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
W20260709 13:07:06.574312 21262 log.cc:927] Time spent T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:07:06.574656 21137 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:06.390497 20868 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:06.575455 16004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2211) took 1114 ms. Trace:
W20260709 13:07:06.572448 21113 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.940s	user 0.000s	sys 0.053s
I20260709 13:07:06.572723 20773 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:06.577783 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 860979us
I20260709 13:07:06.260339 21181 consensus_queue.cc:237] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:06.578480 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:06.562268 16008 rpcz_store.cc:276] 0709 13:07:03.753696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.132168 (+1378472us) service_pool.cc:224] Handling call
0709 13:07:05.132610 (+   442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:05.132950 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:07:05.238115 (+105165us) log.cc:844] Serialized 14 byte log entry
0709 13:07:06.248717 (+1010602us) log.cc:844] Serialized 14 byte log entry
0709 13:07:06.523379 (+274662us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:06.523442 (+    63us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:06.523505 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:06.523596 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:06.523776 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":422646,"prepare.queue_time_us":233963,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":76,"raft.queue_time_us":477431,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":112,"thread_start_us":836512,"threads_started":3,"wal-append.queue_time_us":704569}
I20260709 13:07:06.442085 21188 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:06.303439 21157 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:06.608320 21157 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:06.609336 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:06.609954 21180 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:06.419256 21167 consensus_queue.cc:260] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:06.610630 21180 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:06.611294 21157 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:06.390739 20944 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:06.580199 21259 log.cc:927] Time spent T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.108s	user 0.000s	sys 0.002s
I20260709 13:07:06.612864 21180 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:06.606655 20858 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:06.613420 20944 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:06.613564 20858 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:06.615938 21188 consensus_queue.cc:1059] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:06.616149 15900 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:06.616802 21248 raft_consensus.cc:3055] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:06.582149 21244 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:06.617223 21248 raft_consensus.cc:740] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:06.617774 15900 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:06.583385 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:06.618283 21248 consensus_queue.cc:260] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:06.619836 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:06.574854 20670 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:06.620297 21096 raft_consensus.cc:2749] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:06.518764 21135 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:06.621071 21096 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:06.621258 20858 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:06.621090 21135 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:06.624027 21135 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:06.558391 16009 rpcz_store.cc:276] 0709 13:07:03.926815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.098910 (+1172095us) service_pool.cc:224] Handling call
0709 13:07:05.099455 (+   545us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:05.108833 (+  9378us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:07:05.240775 (+131942us) log.cc:844] Serialized 14 byte log entry
0709 13:07:05.766964 (+526189us) log.cc:844] Serialized 14 byte log entry
0709 13:07:05.767263 (+   299us) log.cc:844] Serialized 14 byte log entry
0709 13:07:05.972829 (+205566us) spinlock_profiling.cc:243] Waited 9.92 ms on lock 0x7b1800362140. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb3f3 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd 000000000038f354 000000000038f199 00007f73e729e83a
0709 13:07:05.999434 (+ 26605us) spinlock_profiling.cc:243] Waited 29.7 ms on lock 0x7b1800362140. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb3f3 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd 000000000038f354 000000000038f199 00007f73e729e83a
0709 13:07:06.557657 (+558223us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:06.557770 (+   113us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:06.557810 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:06.557885 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:06.557982 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":171168,"prepare.queue_time_us":255829,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":103,"raft.queue_time_us":230886,"raft.run_cpu_time_us":388,"raft.run_wall_time_us":15712,"spinlock_wait_cycles":91161216,"thread_start_us":1187691,"threads_started":3,"wal-append.queue_time_us":509038}
I20260709 13:07:06.575771 15825 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:06.339656 21239 log.cc:927] Time spent T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.218s	user 0.001s	sys 0.000s
I20260709 13:07:06.600014 16004 rpcz_store.cc:276] 0709 13:07:05.461473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.608053 (+146580us) service_pool.cc:224] Handling call
0709 13:07:05.621061 (+ 13008us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:05.656561 (+ 35500us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:07:05.898186 (+241625us) log.cc:844] Serialized 14 byte log entry
0709 13:07:06.216331 (+318145us) log.cc:844] Serialized 14 byte log entry
0709 13:07:06.518411 (+302080us) spinlock_profiling.cc:243] Waited 1.22 ms on lock 0x7b180028b700. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e7235c0a 00007f73f0975219 00007f73ee27c58f 00007f73e72bb571 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd
0709 13:07:06.519566 (+  1155us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:07:06.575115 (+ 55549us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:06.575169 (+    54us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:06.575218 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:06.575290 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:06.575402 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":368070,"prepare.queue_time_us":20478,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":1557,"raft.queue_time_us":362101,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":4851,"spinlock_wait_cycles":30028288,"thread_start_us":294282,"threads_started":1,"wal-append.queue_time_us":566297}
I20260709 13:07:06.602615 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:06.626147 15900 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:06.604521 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:06.637674 15936 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:06.616207 16569 raft_consensus.cc:1275] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:07:06.619541 21248 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:07:06.243283 20768 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.002s	sys 0.014s
I20260709 13:07:06.601341 15827 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:06.603890 20917 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:06.340862 21155 consensus_queue.cc:237] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:06.644716 21247 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:06.654559 20944 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:06.655932 21135 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:06.660887 21028 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:06.661221 21135 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:06.662313 20879 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:06.662444 21252 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:06.662961 20879 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:06.663036 21252 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:06.660542 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 103548us
I20260709 13:07:06.660506 21247 raft_consensus.cc:697] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:06.664577 15900 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:06.665217 20879 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:06.640254 21154 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:06.665520 21247 consensus_queue.cc:237] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:06.666128 21154 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:06.654474 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 78827.6us
W20260709 13:07:06.654378 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 351794us
I20260709 13:07:06.661516 21028 raft_consensus.cc:697] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:06.655849 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:06.654496 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:06.642220 15861 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
I20260709 13:07:06.606304 20929 raft_consensus.cc:2804] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:06.674430 15976 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:06.628124 20734 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:06.642115 20768 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.981s	user 0.003s	sys 0.014s
I20260709 13:07:06.281783 21153 consensus_queue.cc:260] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:06.665334 21252 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:06.627238 21137 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:06.663486 21135 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:06.683861 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:06.683347 15825 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:06.685590 15976 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:06.668231 21154 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:06.693528 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:06.694726 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:06.695604 15858 raft_consensus.cc:1240] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 11. Current term is 13. Ops: []
I20260709 13:07:06.697191 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:06.672912 21274 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:06.698829 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:06.761410 15862 raft_consensus.cc:2393] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 12: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:06.675453 21113 raft_consensus.cc:2749] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:06.760392 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:06.679301 21180 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:06.766045 15855 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:06.629683 20868 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:06.699460 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 87561.1us
W20260709 13:07:06.700465 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 56418.8us
I20260709 13:07:06.694387 15929 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
I20260709 13:07:06.798086 15853 raft_consensus.cc:1240] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:06.696921 21244 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:06.798791 21244 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:06.799180 21244 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:06.799304 21254 raft_consensus.cc:2804] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:06.799876 21254 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:06.800223 21254 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:06.803999 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:06.804085 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:06.686160 20917 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:06.677856 20929 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:06.805616 20929 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:06.806608 20917 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:06.760453 20734 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:06.807267 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 45881.2us
I20260709 13:07:06.808348 15827 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:06.808863 20934 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:06.809083 20734 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:06.809633 20934 raft_consensus.cc:697] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:06.809829 15827 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:06.811125 15859 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 14.
I20260709 13:07:06.810942 20934 consensus_queue.cc:237] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:06.779925 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 94402us
I20260709 13:07:06.699328 15860 raft_consensus.cc:1240] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:06.817282 15936 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
I20260709 13:07:06.817431 15976 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:06.686632 21122 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:07:06.763227 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 98745.8us
I20260709 13:07:06.816246 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:06.821382 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:06.821807 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:06.765734 21239 log.cc:927] Time spent T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:07:06.759895 21096 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:06.762817 20858 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:06.794961 21188 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:06.826392 15902 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 1.697s (SENT) [suppressed 19 similar messages]
I20260709 13:07:06.836673 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:06.699034 21177 consensus_queue.cc:1048] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.134s
I20260709 13:07:06.820061 20917 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:06.828877 21071 consensus_queue.cc:1059] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.008s
I20260709 13:07:06.847319 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:06.833922 21099 raft_consensus.cc:2749] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:06.832065 21266 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:06.766682 21274 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:06.839942 20773 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:06.838652 21181 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:06.859579 15976 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:06.795794 21226 consensus_queue.cc:1048] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.049s
W20260709 13:07:06.866636 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 183382us
I20260709 13:07:06.859969 21230 raft_consensus.cc:2804] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:06.855118 21153 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:06.878140 15974 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:06.878409 15827 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:06.882797 15825 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:06.848711 20858 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:06.846050 21180 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:06.874434 21252 raft_consensus.cc:2804] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:06.907855 21252 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:06.908334 21252 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:06.892388 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:06.893921 20773 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:06.894552 21269 raft_consensus.cc:2749] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:06.895402 21274 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:06.849707 20917 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:06.884693 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 58352.4us
W20260709 13:07:06.896813 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:06.867619 21275 consensus_queue.cc:1059] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:07:06.895756 20642 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:06.897559 21266 raft_consensus.cc:2749] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:06.900094 21137 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:06.930572 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:06.931267 15937 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:07:06.901103 20868 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:06.932160 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:06.932955 16569 raft_consensus.cc:2393] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 14: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:06.934722 16561 raft_consensus.cc:1275] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:07:06.935757 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:06.936090 20917 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:06.936640 16010 raft_consensus.cc:2393] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 13: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
W20260709 13:07:06.937104 21270 log.cc:927] Time spent T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.462s	user 0.000s	sys 0.003s
I20260709 13:07:06.938403 21124 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:06.939141 21266 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:06.939309 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:06.939720 21266 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:06.941823 16008 raft_consensus.cc:1275] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:07:06.941983 21266 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:06.942797 21253 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:06.911201 21230 raft_consensus.cc:697] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 13 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:06.943743 20642 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:06.944278 20642 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:06.945057 16567 raft_consensus.cc:1240] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:06.944705 21230 consensus_queue.cc:237] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:06.945864 20929 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:06.867054 21028 consensus_queue.cc:237] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:06.901161 21071 consensus_queue.cc:1048] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:06.952960 21180 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:06.954216 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:06.954473 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:06.955081 15861 raft_consensus.cc:2393] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 13: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:06.866856 21188 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:06.955253 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 85294.3us
I20260709 13:07:06.902491 21099 consensus_queue.cc:1048] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:06.956965 20929 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:06.957211 21188 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:06.919450 15976 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:06.957692 15825 leader_election.cc:400] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Vote denied by peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) with higher term. Message: Invalid argument: T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 11. Current term is 12.
I20260709 13:07:06.929358 15934 raft_consensus.cc:1275] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:07:06.958009 15825 leader_election.cc:403] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
W20260709 13:07:06.958267 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 65741.5us
W20260709 13:07:06.958794 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:06.954607 20858 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:06.958029 16564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 1101) took 1537 ms. Trace:
I20260709 13:07:06.868927 21181 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:06.961848 16564 rpcz_store.cc:276] 0709 13:07:05.420495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.633578 (+213083us) service_pool.cc:224] Handling call
0709 13:07:05.633880 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:05.859662 (+225782us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:07:05.941417 (+ 81755us) log.cc:844] Serialized 14 byte log entry
0709 13:07:06.671215 (+729798us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b180028a8c0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72b8455 00007f73edda03c7 00007f73ea3b29a7 00007f73ea3c20bf 00007f73ea3c2038
0709 13:07:06.819528 (+148313us) log.cc:844] Serialized 14 byte log entry
0709 13:07:06.820180 (+   652us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:06.820228 (+    48us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:06.820347 (+   119us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:06.957751 (+137404us) spinlock_profiling.cc:243] Waited 225 ms on lock 0x7b540050f0d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:06.957823 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:06.957965 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":371,"prepare.queue_time_us":162787,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":9922,"raft.run_cpu_time_us":442,"raft.run_wall_time_us":32488,"spinlock_wait_cycles":1020620800,"thread_start_us":494024,"threads_started":2,"wal-append.queue_time_us":538287}
W20260709 13:07:06.969465 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:06.970016 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:06.974211 15929 raft_consensus.cc:1275] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:07:06.975375 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:06.980396 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:06.982111 15860 raft_consensus.cc:3055] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:06.982560 15860 raft_consensus.cc:740] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:06.983011 16561 raft_consensus.cc:1240] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:06.983368 15860 consensus_queue.cc:260] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:06.984546 15860 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:06.987823 15937 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:58226
I20260709 13:07:06.988520 15937 raft_consensus.cc:606] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 10 LEADER]: Received request to transfer leadership
I20260709 13:07:06.988715 16565 raft_consensus.cc:3055] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:06.989219 16565 raft_consensus.cc:740] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:06.990164 16565 consensus_queue.cc:260] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:06.991334 16565 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:07:06.992200 21242 log.cc:927] Time spent T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.489s	user 0.001s	sys 0.003s
I20260709 13:07:06.993288 21249 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:06.993628 15900 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:06.993822 21113 consensus_queue.cc:1059] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:06.994694 21226 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:06.994827 21113 raft_consensus.cc:3055] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:06.995177 21226 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:06.995828 21226 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:06.996292 15935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 1045) took 2369 ms. Trace:
I20260709 13:07:06.996649 15935 rpcz_store.cc:276] 0709 13:07:04.627331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.245592 (+618261us) service_pool.cc:224] Handling call
0709 13:07:05.246042 (+   450us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:06.569111 (+1323069us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:06.921512 (+352401us) spinlock_profiling.cc:243] Waited 27.6 ms on lock 0x7b54000db4a0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea31fdc0 00007f73ea3aba02 00007f73ea3a7dd6 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2
0709 13:07:06.996058 (+ 74546us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400328554. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:06.996207 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2712982016}
I20260709 13:07:07.003170 21279 consensus_queue.cc:1059] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:07:07.004282 21279 raft_consensus.cc:3055] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:07.004693 21279 raft_consensus.cc:740] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:07.005860 21279 consensus_queue.cc:260] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:07.009456 20929 consensus_queue.cc:1048] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:06.974391 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:07.004096 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:06.995296 21113 raft_consensus.cc:740] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:07:07.012884 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 55304.5us
I20260709 13:07:06.987659 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:06.993950 21249 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:07.032624 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:07.035199 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:06.943316 21253 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:07.035943 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:07.036772 16561 raft_consensus.cc:2393] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 14: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:07.043177 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:07.042428 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:07.044036 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:07.044416 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:07.044826 15934 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
I20260709 13:07:07.042604 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:06.974525 21247 raft_consensus.cc:2804] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:06.978967 21188 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.047102 16569 raft_consensus.cc:1240] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:07.048243 15928 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:07:07.048524 15859 raft_consensus.cc:1240] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:07.049971 15935 raft_consensus.cc:1275] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:07:07.043929 16009 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:07:07.003603 21167 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:07.046797 15929 raft_consensus.cc:2410] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 13 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:07:07.049048 16008 raft_consensus.cc:1275] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:07:07.062774 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:07.054287 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:07.055352 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:06.943802 20773 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:07.010257 20929 consensus_queue.cc:1243] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer 8430db80a39d4868bcf7641c16de35ea log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:07:07.066921 16568 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:07:07.069900 15931 raft_consensus.cc:2393] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 12: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
W20260709 13:07:07.070847 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:07.048158 21177 consensus_queue.cc:1048] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 13:07:07.072340 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:07.072661 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:07.073040 15928 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:07.073261 15937 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:07.074771 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:07.075773 15929 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
I20260709 13:07:07.080041 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:07.080832 15858 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 15.
I20260709 13:07:07.081527 21280 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.082335 21280 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:07.088332 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:07.092176 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:07.097585 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:07.098650 16563 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:07:07.099488 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:07.042687 21275 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.104125 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:07.106878 15933 raft_consensus.cc:2393] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 16: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:07.107301 15934 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 12.
I20260709 13:07:06.979796 21244 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:07.108350 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 88800.6us
I20260709 13:07:07.108843 16567 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
I20260709 13:07:07.109081 15827 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:07.103127 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:07.111158 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:07.114246 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:07.114826 15935 raft_consensus.cc:2393] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 14: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:07.115180 16003 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
I20260709 13:07:07.115751 15934 raft_consensus.cc:1240] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:07.042371 20858 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:07.116806 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:07.064744 20670 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:07.110985 21137 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.069113 21279 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:07.120388 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:07.121955 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:07.052606 21071 consensus_queue.cc:1048] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.062s
W20260709 13:07:07.116050 21155 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 13:07:07.111317 21135 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.122004 20858 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:07.117792 21266 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:07.145577 21266 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:07.145656 20642 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:07.146142 21266 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:07:07.148742 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:07.133152 16009 raft_consensus.cc:1240] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:07.128746 15931 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:07.137189 21226 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:07.124012 20670 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:07.139554 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:07.120354 21247 raft_consensus.cc:697] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:07.141078 16561 raft_consensus.cc:1240] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:07.142320 21254 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:07.143780 21113 consensus_queue.cc:260] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:07.144240 21181 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:07.150743 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:07.151391 15937 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:07:07.140007 21244 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:07.106536 21188 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:07.182174 21188 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:07.153716 21135 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:07.184998 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:07.185397 15931 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:58226
I20260709 13:07:07.186121 15931 raft_consensus.cc:606] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 10 LEADER]: Received request to transfer leadership
I20260709 13:07:07.126454 21137 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:07.175365 21249 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:07.177435 20773 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:07.188315 21124 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.192198 15928 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 12.
I20260709 13:07:07.199007 15932 raft_consensus.cc:1240] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:07.200363 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:07.112512 21275 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:07.208526 16006 raft_consensus.cc:2368] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 14. Current term is 15.
W20260709 13:07:07.210367 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:07.218847 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:07.199499 20642 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:07.188880 21124 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:07.224339 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:07.242110 16008 raft_consensus.cc:2393] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 15: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:07.223971 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:07.242765 15861 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:07:07.245625 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 79413.9us
W20260709 13:07:07.247802 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 136255us
I20260709 13:07:07.248878 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: true dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:07.250098 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:07.235066 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:07.247928 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:07.253167 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:07.233829 15900 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:07.258306 20773 raft_consensus.cc:2749] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:07.225404 21113 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:07.255831 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:07.260046 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:07.262749 15825 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:07.264500 15855 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
I20260709 13:07:07.260453 15900 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:07.282358 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:07.285380 16007 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:07:07.290925 16008 raft_consensus.cc:1275] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:07:07.297348 15929 raft_consensus.cc:1240] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:07:07.303947 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:07.304698 16005 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
I20260709 13:07:07.304879 21137 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:07.305362 21283 log.cc:927] Time spent T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.292s	user 0.000s	sys 0.002s
W20260709 13:07:07.307144 21286 log.cc:927] Time spent T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 13:07:07.308502 21289 log.cc:927] Time spent T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
W20260709 13:07:07.309573 21285 log.cc:927] Time spent T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:07:07.310698 15861 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
W20260709 13:07:07.311736 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 49097.3us
I20260709 13:07:07.312395 16003 raft_consensus.cc:1240] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:07.277743 20868 raft_consensus.cc:3055] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:07.278676 21141 raft_consensus.cc:2749] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:07.277237 21099 raft_consensus.cc:2749] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) with higher term. Message: Invalid argument: T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 11. Current term is 12.
I20260709 13:07:07.286633 21181 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:07.259353 21188 consensus_queue.cc:1059] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:07:07.320875 15825 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:07.327227 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:07.328501 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:07.329505 21267 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:07.330615 15825 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:07.331262 21267 raft_consensus.cc:2764] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 13:07:07.353842 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:07.316941 20868 raft_consensus.cc:740] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:07.334146 21280 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:07.324232 21181 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:07.320600 21153 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:07.393890 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:07.394053 20868 consensus_queue.cc:260] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:07.394560 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 62899.3us
I20260709 13:07:07.394810 15902 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:07.394984 15744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 128419us
I20260709 13:07:07.395273 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:07.429415 15928 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:07.396924 21266 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:07.418610 16004 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 12.
I20260709 13:07:07.395308 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:07.440943 15930 raft_consensus.cc:2393] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 12: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:07.441376 16533 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:07.395892 21290 log.cc:927] Time spent T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:07:07.320358 21252 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:07.445874 15930 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:58226
I20260709 13:07:07.448666 15930 raft_consensus.cc:606] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 10 LEADER]: Received request to transfer leadership
W20260709 13:07:07.447577 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 52852.3us
I20260709 13:07:07.312141 21247 consensus_queue.cc:237] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:07.429023 21181 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:07.449096 21226 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:07.447999 20917 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.463002 20917 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:07.465027 20917 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:07.447132 20868 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:07.450176 21280 raft_consensus.cc:2749] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:07.443718 21153 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:07.477694 16011 raft_consensus.cc:1240] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:07:07.478101 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:07.481254 21275 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:07.483182 21248 consensus_meta.cc:287] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.840s	user 0.010s	sys 0.000s
W20260709 13:07:07.484171 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 125298us
I20260709 13:07:07.478940 16007 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
I20260709 13:07:07.486618 16009 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
I20260709 13:07:07.486801 16533 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:07.460637 15928 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:07:07.460813 21272 raft_consensus.cc:2749] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:07:07.499997 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:07.487944 21254 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:07.503556 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:07.499751 16533 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:07.504985 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:07.508255 16533 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:07.499305 20917 consensus_queue.cc:1059] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 13:07:07.519459 21274 raft_consensus.cc:2804] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:07.523841 16535 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:07.524242 21274 raft_consensus.cc:697] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:07.525054 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:07.525179 21274 consensus_queue.cc:237] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:07.526271 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:07.526847 16565 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:07:07.527942 16566 raft_consensus.cc:1275] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:07:07.528264 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:07.529485 16563 raft_consensus.cc:1275] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:07:07.530530 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:07.531122 16568 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:07.532948 21252 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:07.498932 20929 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:07.554143 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:07.555073 16007 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:07:07.500458 21275 consensus_queue.cc:1048] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:07:07.512810 21157 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:07.569008 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:07.575044 15932 raft_consensus.cc:1240] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 13. Current term is 14. Ops: []
W20260709 13:07:07.602741 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:07.603291 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:07.605765 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 155033us
I20260709 13:07:07.604524 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:07.608476 15932 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
I20260709 13:07:07.608969 15974 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:07.546200 20929 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:07.612679 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:07.613942 16009 raft_consensus.cc:2410] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:07:07.618712 15928 raft_consensus.cc:1240] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:07.606252 20670 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:07.601446 21124 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:07.628888 15937 raft_consensus.cc:1240] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:07.629961 20670 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:07.630033 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:07.630904 15928 raft_consensus.cc:2393] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 15: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:07.609308 20642 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.616395 20917 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:07.615531 20929 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:07.617846 20768 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:07.620756 21157 raft_consensus.cc:697] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:07.656469 15934 raft_consensus.cc:1240] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 15 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:07.615502 21248 consensus_queue.cc:1059] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:07:07.660741 20768 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:07.663296 20768 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:07.647519 20642 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:07.666070 20642 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:07.669351 15928 raft_consensus.cc:2955] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Committing config change with OpId 13.10: config changed from index 3 to 10, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:07:07.627244 21266 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:07.673698 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: true dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:07.673728 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:07.675055 16568 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:07.676433 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:07.626698 20944 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:07.644318 20670 raft_consensus.cc:697] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:07:07.697073 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 43547.7us
I20260709 13:07:07.705955 15937 raft_consensus.cc:2393] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 13: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:07.694483 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:07.696382 15855 raft_consensus.cc:1240] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:07.707518 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:07.707763 15861 raft_consensus.cc:2393] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 12: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:07.706826 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:07.716673 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:07.717410 15854 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 16.
I20260709 13:07:07.726118 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:07.728266 15853 raft_consensus.cc:1240] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 12 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:07.720638 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:07.796605 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:07.873116 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:07.876149 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:07.877393 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:07.876498 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:07.878962 15860 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:07.881577 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:07.859622 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 295250us
I20260709 13:07:07.884493 16535 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:07.885572 21124 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:07.886247 21124 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:07.874917 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:07.890071 21274 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:07.890533 21274 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:07.890666 16535 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:07.734467 15859 raft_consensus.cc:2393] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 13: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:07.867780 15934 raft_consensus.cc:1275] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:07:07.868155 21275 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.889077 21124 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:07.892228 21124 consensus_queue.cc:1048] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:07.674403 21269 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:07.872781 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:07.890980 21274 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:07.893899 21269 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:07.892136 21275 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:07.868701 21167 consensus_meta.cc:287] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.789s	user 0.012s	sys 0.008s
I20260709 13:07:07.790513 20944 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:07.899726 15853 raft_consensus.cc:2393] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 14: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:07:07.873827 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 263884us
I20260709 13:07:07.902438 15860 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
W20260709 13:07:07.865902 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:07.930598 15936 raft_consensus.cc:2410] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:07:07.930416 15974 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:07.897564 21275 leader_election.cc:290] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:07.932916 15974 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
W20260709 13:07:07.941465 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 50880us
I20260709 13:07:07.943382 16535 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:07.944936 21188 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:07.945741 21188 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:07.937211 21154 consensus_queue.cc:1059] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.011s
I20260709 13:07:07.899596 21253 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:07.948873 20937 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.899533 21181 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:07.949469 20937 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:07.950470 15931 raft_consensus.cc:1240] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 15. Current term is 16. Ops: [13.11-15.12]
I20260709 13:07:07.899808 21157 consensus_queue.cc:237] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:07.957695 15976 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:07.970243 15929 raft_consensus.cc:1275] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:07:07.980410 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 47609.3us
W20260709 13:07:07.980839 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 114466us
I20260709 13:07:07.981364 15974 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:07.983629 15974 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:07.926299 21124 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.948863 21154 raft_consensus.cc:3055] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:07.962311 21275 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.937804 21247 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:07.954591 20868 raft_consensus.cc:3055] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:07.951556 21181 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:07.989185 21300 consensus_queue.cc:1048] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:07.990377 16562 raft_consensus.cc:1275] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:07:07.990703 15976 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:07.934314 21180 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.995249 15902 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:08.013984 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:07.951467 20937 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:07.934165 21137 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.959798 21320 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:08.026181 15976 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:08.002940 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:08.027390 15860 raft_consensus.cc:2410] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:07:07.991209 21253 raft_consensus.cc:2749] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:07.998314 15974 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:07.952575 21135 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:07.989305 21312 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:07.952898 20944 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:08.008375 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 65056.3us
I20260709 13:07:07.987474 21252 raft_consensus.cc:2804] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:08.009368 16006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41674 (request call id 2221) took 1083 ms. Trace:
I20260709 13:07:08.084832 16006 rpcz_store.cc:276] 0709 13:07:06.925634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.925766 (+   132us) service_pool.cc:224] Handling call
0709 13:07:08.005197 (+1079431us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b54003ece54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:08.009293 (+  4096us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":604532608}
I20260709 13:07:08.081348 16535 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:08.009654 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:07:08.013173 15827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:08.086817 16535 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:08.015537 16565 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:07:08.018846 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:08.022221 15902 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:08.024217 21318 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:08.024260 21167 raft_consensus.cc:2749] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:08.001633 16569 raft_consensus.cc:1240] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:08.027577 15976 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:07.938241 21230 consensus_queue.cc:1048] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.072s
I20260709 13:07:08.049199 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:08.050513 16561 raft_consensus.cc:1275] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:07:07.999063 21247 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:08.010337 21275 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:08.101871 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:07.999315 21124 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:08.011153 21254 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:08.054908 21269 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:08.059995 20670 consensus_queue.cc:237] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:08.025966 21320 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:07.993090 21248 consensus_queue.cc:1048] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:07:08.064059 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:08.064654 21244 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:07.994555 21154 raft_consensus.cc:740] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:08.069653 16011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2220) took 1144 ms. Trace:
I20260709 13:07:08.146219 16011 rpcz_store.cc:276] 0709 13:07:06.925240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.925348 (+   108us) service_pool.cc:224] Handling call
0709 13:07:06.933921 (+  8573us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:07.478244 (+544323us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:08.028856 (+550612us) spinlock_profiling.cc:243] Waited 543 ms on lock 0x7b5400306454. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:08.069575 (+ 40719us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1249559808}
I20260709 13:07:08.005504 20868 raft_consensus.cc:740] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:08.083189 16533 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:08.084069 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 79844.6us
I20260709 13:07:08.090909 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:08.150381 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 67277.7us
I20260709 13:07:08.151353 16533 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:08.096495 21274 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:08.098618 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 76490.6us
W20260709 13:07:08.152551 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 154300us
I20260709 13:07:08.073871 20879 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:07:08.102118 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:08.034878 21180 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:08.136691 20934 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.095485 21135 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:08.084801 21252 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:08.094223 20917 raft_consensus.cc:2749] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:08.142767 21320 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:08.091116 21318 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:08.156044 20879 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:08.099442 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:08.174311 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:08.102103 21304 raft_consensus.cc:2749] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:07:08.102890 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 75380.2us
I20260709 13:07:08.181876 15976 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:08.140362 15858 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 12.
I20260709 13:07:08.158658 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:08.160148 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:08.182966 16564 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
W20260709 13:07:08.160697 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 58581.8us
I20260709 13:07:08.161939 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:08.184312 16569 raft_consensus.cc:2393] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 15: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:08.163089 16561 raft_consensus.cc:1240] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:08.164461 16562 raft_consensus.cc:1275] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:07:08.102396 21247 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:08.136969 21177 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.073612 21137 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:08.136842 21312 raft_consensus.cc:2749] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:08.141280 21188 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:08.143810 15853 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
I20260709 13:07:08.168169 21308 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:08.173655 21318 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:08.168469 21135 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:08.092098 20929 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:08.196182 16533 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:08.180547 20879 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:08.170380 21252 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:08.172523 21330 raft_consensus.cc:2749] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:08.201697 15931 raft_consensus.cc:1275] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:07:08.169757 20934 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:08.196385 20917 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:08.203114 21135 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:08.227722 16563 raft_consensus.cc:1240] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:08.227061 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:08.235953 21275 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:08.196954 20868 consensus_queue.cc:260] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:08.236326 21137 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:08.195904 21177 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:08.236997 16011 raft_consensus.cc:1275] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:07:08.237787 21317 log.cc:927] Time spent T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 13:07:08.238514 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:08.246610 21153 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:08.247980 20858 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:08.249715 21244 raft_consensus.cc:2749] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:08.247494 21247 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:08.256050 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:08.259249 15936 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53636 (request call id 1051) took 1438 ms. Trace:
I20260709 13:07:08.205264 21308 raft_consensus.cc:697] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:07:08.275117 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 74867.6us
I20260709 13:07:08.200423 21154 consensus_queue.cc:260] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:08.305761 16568 raft_consensus.cc:1240] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:08.306743 21326 raft_consensus.cc:2749] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:08.307113 20929 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:08.292402 21275 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:08.309773 21275 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:08.311970 21275 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:08.312574 21274 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:08.313202 21124 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:08.302829 20868 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:08.314883 21177 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:08.319380 21308 consensus_queue.cc:237] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:08.320777 21324 log.cc:927] Time spent T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 13:07:08.303754 15827 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:07:08.297288 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 143796us
I20260709 13:07:08.243523 21254 raft_consensus.cc:2749] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:08.273841 20917 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:08.301697 16006 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:07:08.301680 15936 rpcz_store.cc:276] 0709 13:07:06.821017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.821150 (+   133us) service_pool.cc:224] Handling call
0709 13:07:08.259040 (+1437890us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54002c2c54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:08.259192 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2548942080}
I20260709 13:07:08.348268 15974 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:08.373896 21312 raft_consensus.cc:2749] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:08.376895 20934 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:08.306253 21333 raft_consensus.cc:2749] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:08.383597 21320 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:08.378631 15974 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:08.393328 21247 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:08.397742 15757 catalog_manager.cc:5697] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 7 to 10, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 10 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:07:08.398912 21177 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.404798 21320 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:08.404991 21177 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:08.405656 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 252538us
I20260709 13:07:08.406330 21247 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:08.406554 15902 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:08.407130 21177 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:08.407846 21249 raft_consensus.cc:2749] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:08.399976 21096 raft_consensus.cc:3055] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:08.408432 21096 raft_consensus.cc:740] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:08.409325 21096 consensus_queue.cc:260] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:08.405725 15974 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:08.410614 21096 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:08.411267 16566 raft_consensus.cc:2393] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 14: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
W20260709 13:07:08.411868 21280 consensus_queue.cc:1175] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "85e997ba5003439781ec4f1f8db0c3ee" responder_term: 10 status { last_received { term: 10 index: 6 } last_committed_idx: 6 last_received_current_leader { term: 10 index: 6 } }
W20260709 13:07:08.401666 15808 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 13:07:08.412730 15808 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 15301761b2ec47fe82ae2f431c8b34ee already in progress
W20260709 13:07:08.413044 15808 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 15301761b2ec47fe82ae2f431c8b34ee already in progress
I20260709 13:07:08.414220 15859 raft_consensus.cc:1275] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:07:08.414966 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:08.416033 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:08.416258 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:08.416604 15862 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:08.417161 15858 raft_consensus.cc:2410] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:07:08.418000 21247 raft_consensus.cc:2749] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:08.418071 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:08.418835 21247 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:08.419497 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:08.380225 21318 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:08.421635 15860 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:07:08.422434 21318 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:08.408219 15902 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:08.423864 21249 raft_consensus.cc:2749] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:08.425287 20879 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:08.426465 15902 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:08.427246 20879 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:08.338006 21339 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:08.443547 21339 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:08.444130 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:08.445264 15858 raft_consensus.cc:1275] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:07:08.447268 21249 raft_consensus.cc:2749] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:08.447979 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:08.448215 21230 consensus_queue.cc:1048] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:08.444098 21339 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:08.303556 21334 raft_consensus.cc:2749] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:08.430015 15860 raft_consensus.cc:1275] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:07:08.432830 15855 raft_consensus.cc:1240] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 14. Current term is 15. Ops: []
W20260709 13:07:08.433323 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 51328.5us
I20260709 13:07:08.442543 21253 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:08.433040 21247 raft_consensus.cc:697] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:08.455519 15827 leader_election.cc:400] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 14. Current term is 15.
I20260709 13:07:08.458351 15827 leader_election.cc:403] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:07:08.439778 20917 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:08.469412 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:08.470810 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:08.476258 16562 raft_consensus.cc:1240] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:08.461915 16006 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:07:08.477440 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:08.485858 16569 raft_consensus.cc:1240] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:08.337011 21269 consensus_queue.cc:1048] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.136s
I20260709 13:07:08.490926 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:08.492715 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:08.493242 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:08.498145 15902 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:08.494251 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:08.500027 15931 raft_consensus.cc:1275] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:07:08.500540 15931 pending_rounds.cc:85] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 9
I20260709 13:07:08.501030 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:08.501957 15862 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:08.506142 15902 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:08.508204 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:08.509145 15934 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:08.458194 21230 consensus_queue.cc:1048] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:07:08.500798 15931 pending_rounds.cc:107] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:07:08.518298 15931 raft_consensus.cc:2889] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:08.463258 20934 consensus_queue.cc:1059] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:07:08.462145 21334 consensus_queue.cc:1059] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.052s
I20260709 13:07:08.495256 20773 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.519726 21139 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:07:08.534852 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 353056us
I20260709 13:07:08.512087 21331 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 14. Current term is 15.
I20260709 13:07:08.495445 21318 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:08.516268 21167 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:08.539142 15976 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:08.531734 21248 consensus_queue.cc:1048] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 13:07:08.536580 20773 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:08.563216 15976 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:08.560865 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:08.561944 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:08.543936 21318 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:08.508222 21249 consensus_queue.cc:1048] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:07:08.518321 21267 consensus_queue.cc:1048] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:07:08.571190 15976 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:08.579402 16535 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:08.586472 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:08.593808 15902 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:08.534734 21135 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:08.535291 21139 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:08.565764 21244 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:08.598217 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:08.600561 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:08.603268 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:08.603977 16565 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:07:08.573038 21253 raft_consensus.cc:2749] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:08.612953 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:08.566248 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 370112us
I20260709 13:07:08.614285 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:08.591322 15934 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:08.616636 16003 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:08.615597 16565 raft_consensus.cc:1275] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:07:08.618556 16533 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:08.609045 15974 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:08.595608 21181 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.623358 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:08.598718 20937 raft_consensus.cc:2749] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:08.628726 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:08.629554 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:08.625228 16007 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
I20260709 13:07:08.574187 21188 consensus_queue.cc:1059] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 13:07:08.632630 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:08.554785 21247 consensus_queue.cc:237] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:08.638764 16010 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
I20260709 13:07:08.633363 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:08.568468 20879 consensus_queue.cc:1048] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:07:08.636090 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:08.608314 21244 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:08.609962 21139 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:08.577631 21334 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 13:07:08.600848 21344 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:08.614696 20670 raft_consensus.cc:2804] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:08.645090 15974 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:08.609506 21254 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:08.647532 16565 raft_consensus.cc:1240] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:07:08.620997 21341 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:08.659103 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:08.673381 16007 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 13.
I20260709 13:07:08.674387 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:08.657757 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:08.661898 16533 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:08.674449 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:08.672847 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:08.676430 16010 raft_consensus.cc:2393] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 14: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:08.676479 16564 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:07:08.659873 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 199622us
I20260709 13:07:08.678282 16533 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:08.638293 21320 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:08.641707 21244 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:08.679427 15974 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:08.682034 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:08.638793 21337 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.677181 16003 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:07:08.679425 15827 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:08.637975 21181 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:08.687945 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:08.697188 16565 raft_consensus.cc:2393] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 14: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:08.700598 16011 raft_consensus.cc:1275] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:07:08.685560 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:08.686900 16002 raft_consensus.cc:2393] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 14: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:08.693792 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:08.656471 21253 raft_consensus.cc:2749] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:08.661065 21279 raft_consensus.cc:3055] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:08.710285 21336 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:08.710455 21177 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:08.710692 21355 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:08.701016 16011 pending_rounds.cc:85] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee: Aborting all ops after (but not including) 10
I20260709 13:07:08.714605 16011 pending_rounds.cc:107] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:07:08.714912 16011 raft_consensus.cc:2889] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:08.708014 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:08.672830 21341 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:08.719838 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:08.648247 21252 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:08.728284 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:08.733860 16002 raft_consensus.cc:1240] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 13. Current term is 14. Ops: [13.11-13.11]
W20260709 13:07:08.743059 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 63728.4us
I20260709 13:07:08.744594 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:08.745003 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:08.745206 15860 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:08.746413 15858 raft_consensus.cc:1240] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:08.747426 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:08.748481 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:08.749063 15854 raft_consensus.cc:2376] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12. Re-sending same reply.
I20260709 13:07:08.749696 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:08.750366 16563 raft_consensus.cc:2368] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 12. Current term is 13.
I20260709 13:07:08.751521 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:08.752182 15861 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:08.744379 21028 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:08.762260 21028 raft_consensus.cc:697] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:08.762058 15860 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 15.
I20260709 13:07:08.732098 15974 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:08.687352 21113 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:08.688840 20670 raft_consensus.cc:697] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:08.695475 21137 raft_consensus.cc:2749] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:07:08.785373 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 53436us
I20260709 13:07:08.768064 16007 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 10.
I20260709 13:07:08.682503 21342 raft_consensus.cc:2749] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:08.706663 21279 raft_consensus.cc:740] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:08.791154 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:08.792757 16004 raft_consensus.cc:1240] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:08.791874 15861 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 16.
I20260709 13:07:08.800009 15856 raft_consensus.cc:1240] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:08.798489 16008 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 13.
I20260709 13:07:08.802161 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:08.802835 16563 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:07:08.802953 16564 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:08.721230 21336 raft_consensus.cc:697] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:08.681895 21254 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:08.734905 21341 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:08.794266 16565 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:07:08.665016 21344 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:08.806421 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:08.769163 21320 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:08.714239 21312 consensus_queue.cc:1048] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.084s
I20260709 13:07:08.793746 21253 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:08.785868 21333 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:08.854671 16011 raft_consensus.cc:1240] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:07:08.711275 21177 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:08.801769 21180 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:08.686798 21337 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:08.870383 15902 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:08.847985 21318 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:08.844913 21300 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:08.896169 15976 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:08.790526 21113 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:08.849336 21249 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:07:08.901031 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:08.906167 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:08.715858 21355 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:08.910734 16011 raft_consensus.cc:1275] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:07:08.912981 21252 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:08.914116 21335 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:07:08.915455 21338 log.cc:927] Time spent T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.279s	user 0.000s	sys 0.001s
W20260709 13:07:08.916318 21358 log.cc:927] Time spent T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:07:08.877712 21333 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:08.917369 21347 log.cc:927] Time spent T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 13:07:08.918988 21333 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:08.920596 21351 log.cc:927] Time spent T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:07:08.898180 21253 raft_consensus.cc:697] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:08.903664 21269 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.902768 20879 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.868808 21320 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:08.942291 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:08.906137 20773 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:08.946815 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 50960.5us
W20260709 13:07:08.964113 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 51223.8us
I20260709 13:07:08.909852 21135 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:08.971168 16010 raft_consensus.cc:1240] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:08.921133 21180 raft_consensus.cc:2749] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:08.968197 21333 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.972285 15976 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
W20260709 13:07:08.973165 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:08.975509 15976 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:08.982542 16003 raft_consensus.cc:1240] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:08.983964 16010 raft_consensus.cc:1275] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:07:08.984408 20934 consensus_queue.cc:1048] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:08.906222 21300 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:08.921192 21249 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:09.022050 15900 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:08.997521 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:09.003304 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:08.981794 21252 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:08.911643 21279 consensus_queue.cc:260] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:08.996328 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:08.897652 21028 consensus_queue.cc:237] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:08.976644 21359 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:09.028926 16003 raft_consensus.cc:1275] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:07:08.964246 20879 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:08.925917 20670 consensus_queue.cc:237] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:09.039026 15900 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:08.984843 21369 consensus_queue.cc:1059] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:07:09.042163 15900 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:08.985778 21267 consensus_queue.cc:1059] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:07:08.975742 21269 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:08.994483 21135 consensus_queue.cc:1048] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:07:09.067530 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:09.069739 16011 raft_consensus.cc:1275] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:07:09.030000 21252 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:08.985037 21333 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:09.035988 21367 raft_consensus.cc:3055] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:07:09.044770 21345 log.cc:927] Time spent T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.309s	user 0.001s	sys 0.000s
I20260709 13:07:09.032963 21181 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:09.080176 21337 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:09.083943 21343 raft_consensus.cc:2749] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:09.085250 21254 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:09.114130 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 132793us
I20260709 13:07:09.086000 21314 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:09.087821 21300 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:09.115757 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:09.088686 21363 log.cc:927] Time spent T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
W20260709 13:07:09.089816 21364 log.cc:927] Time spent T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 13:07:09.090790 21368 log.cc:927] Time spent T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 13:07:09.091537 21353 log.cc:927] Time spent T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.188s	user 0.000s	sys 0.001s
I20260709 13:07:09.061589 21180 raft_consensus.cc:2804] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:09.093036 20879 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:09.061003 20773 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:09.165302 20773 raft_consensus.cc:697] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 15 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:09.166559 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:09.172093 21181 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:09.172760 21181 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:09.067065 21279 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:09.040444 21253 consensus_queue.cc:237] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:09.182137 21375 consensus_queue.cc:1059] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
W20260709 13:07:09.104700 21339 consensus_meta.cc:287] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.654s	user 0.004s	sys 0.007s
I20260709 13:07:09.183192 21339 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:09.184432 15853 raft_consensus.cc:1240] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:09.185333 21269 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:09.111758 21137 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:09.148542 21360 consensus_queue.cc:1048] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:07:09.186141 21137 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:09.085657 21357 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.981990 21336 consensus_queue.cc:237] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:09.146847 21320 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:09.078446 21252 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:09.035387 21359 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:07:09.105275 21177 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:09.106101 21367 raft_consensus.cc:740] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:09.075775 20937 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:09.119068 21267 consensus_queue.cc:1243] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer eb1e0493af1e46dbb6dec7e25525bab5 log is divergent from this leader: its last log entry 12.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
W20260709 13:07:09.162147 21325 log.cc:927] Time spent T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 1.072s	user 0.006s	sys 0.000s
I20260709 13:07:09.164646 21180 raft_consensus.cc:697] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 12 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:09.166270 20773 consensus_queue.cc:237] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:09.031773 21362 consensus_queue.cc:1059] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.078s
W20260709 13:07:09.101150 21361 log.cc:927] Time spent T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:07:09.185451 21339 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:09.105340 21355 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:09.186472 21113 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:09.117062 15976 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:09.187834 20917 consensus_meta.cc:287] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.702s	user 0.011s	sys 0.001s
I20260709 13:07:09.203850 21369 consensus_queue.cc:1059] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:09.158156 21226 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:09.218693 21181 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:09.218933 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:09.220319 21374 raft_consensus.cc:3055] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:09.195516 21344 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:09.222334 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:09.222886 21141 raft_consensus.cc:3055] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:09.223078 21366 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:09.223467 21137 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:09.283797 21141 raft_consensus.cc:740] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:09.224017 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:09.284049 21366 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:09.224651 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:09.284775 21141 consensus_queue.cc:260] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:09.285491 15862 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 14.
I20260709 13:07:09.286041 21141 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:09.286392 21366 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:09.286734 21139 consensus_meta.cc:287] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.626s	user 0.008s	sys 0.007s
I20260709 13:07:09.287159 21139 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:09.287635 15900 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:09.289003 21248 raft_consensus.cc:2804] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:09.289417 21139 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:09.289503 21248 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:09.290189 21248 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:09.208957 21357 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:09.224735 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:09.292062 16003 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
I20260709 13:07:09.292346 21357 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:09.237377 21244 consensus_meta.cc:287] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.521s	user 0.009s	sys 0.002s
I20260709 13:07:09.293174 21244 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:09.294613 16006 raft_consensus.cc:2393] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 13: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:09.295266 21244 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:09.225765 21341 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:09.297757 21341 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:09.249019 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 132023us
I20260709 13:07:09.253688 16564 raft_consensus.cc:1240] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:09.299049 15976 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:09.257628 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:09.299882 21137 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:09.300943 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:09.231190 20937 raft_consensus.cc:697] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 16 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:09.229933 21099 consensus_queue.cc:1059] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:07:09.215382 21354 consensus_queue.cc:1048] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:07:09.301838 16562 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:07:09.302282 16570 raft_consensus.cc:1240] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:09.304690 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:09.305548 16562 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:07:09.302933 20937 consensus_queue.cc:237] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:09.309711 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:09.276748 21359 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:09.265337 21374 raft_consensus.cc:740] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:09.277539 21367 consensus_queue.cc:260] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:09.280285 21226 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:09.280140 21180 consensus_queue.cc:237] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:09.261521 20917 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:09.310262 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:09.311280 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:09.247383 21362 consensus_queue.cc:1243] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer eb1e0493af1e46dbb6dec7e25525bab5 log is divergent from this leader: its last log entry 14.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:07:09.237043 21177 consensus_queue.cc:1048] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.073s
W20260709 13:07:09.324077 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 117369us
I20260709 13:07:09.261829 21113 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:09.262987 20670 raft_consensus.cc:2749] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:09.327323 16562 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
I20260709 13:07:09.318493 21028 raft_consensus.cc:3055] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:09.341007 16568 raft_consensus.cc:1240] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:07:09.330861 21367 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:09.368645 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:09.339732 21028 raft_consensus.cc:740] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:09.376075 16562 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:09.384110 21249 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:09.393044 16569 raft_consensus.cc:1275] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:07:09.400245 21028 consensus_queue.cc:260] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:09.402577 20917 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:09.407053 16566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45246 (request call id 1125) took 1245 ms. Trace:
I20260709 13:07:09.410982 21369 consensus_queue.cc:1059] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:07:09.413019 21392 log.cc:927] Time spent T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:07:09.408598 16566 rpcz_store.cc:276] 0709 13:07:08.158268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.158397 (+   129us) service_pool.cc:224] Handling call
0709 13:07:09.403227 (+1244830us) spinlock_profiling.cc:243] Waited 228 ms on lock 0x7b54006a22d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:09.403508 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":524795136}
I20260709 13:07:09.414615 15827 leader_election.cc:400] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 12. Current term is 13.
I20260709 13:07:09.414974 15827 leader_election.cc:403] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:09.415308 16009 raft_consensus.cc:1240] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:07:09.415490 20768 raft_consensus.cc:2749] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 12. Current term is 13.
I20260709 13:07:09.416290 15827 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:09.417027 20768 raft_consensus.cc:2804] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:09.417140 15827 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:09.417433 20768 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:09.417855 20768 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:09.419268 16009 raft_consensus.cc:1240] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:07:09.415370 15856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 1011) took 1540 ms. Trace:
I20260709 13:07:09.432143 16004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2232) took 1195 ms. Trace:
I20260709 13:07:09.421222 16569 raft_consensus.cc:1240] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:09.421846 15902 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:09.432747 16004 rpcz_store.cc:276] 0709 13:07:08.236835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.237011 (+   176us) service_pool.cc:224] Handling call
0709 13:07:08.237379 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:08.794272 (+556893us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:09.431968 (+637696us) spinlock_profiling.cc:243] Waited 555 ms on lock 0x7b54001b9754. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:09.432092 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1276789504}
I20260709 13:07:09.434139 15902 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:09.434561 21279 raft_consensus.cc:2804] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:09.424815 21226 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:09.424886 21267 raft_consensus.cc:3055] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:09.436400 21267 raft_consensus.cc:740] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:09.435045 21154 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:09.431953 15856 rpcz_store.cc:276] 0709 13:07:07.875280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.876193 (+   913us) service_pool.cc:224] Handling call
0709 13:07:08.789168 (+912975us) spinlock_profiling.cc:243] Waited 67.6 ms on lock 0x7b5800005798. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283f8c 00007f73e7283ef6 0000000000375939 00007f73ef2f71e0 00007f73ef3c00c4 00007f73ef3c0131 00007f73ef381fc0 00007f73ef36c929 00007f73ef36c3d2 00007f73e88e3bd4
0709 13:07:08.799827 (+ 10659us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:08.801671 (+  1844us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:09.415192 (+613521us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b5400218250. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:09.415310 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":179525632}
I20260709 13:07:09.335635 21113 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:09.435395 15902 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:09.440013 15757 catalog_manager.cc:5697] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 11 to 13, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 13 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:09.441216 21226 raft_consensus.cc:2749] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:09.442031 21113 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:09.425755 21252 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:09.442502 15757 catalog_manager.cc:5697] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 8 to 12, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 12 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:09.435868 15933 raft_consensus.cc:1240] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:09.434996 21279 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:09.445055 21279 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:09.426084 21333 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:09.360961 21362 consensus_queue.cc:1059] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 13:07:09.331720 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:07:09.449746 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 74740.2us
I20260709 13:07:09.452338 15825 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:09.449155 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:09.448884 15902 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:09.446818 21252 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:09.468822 15827 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:09.469241 15902 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:09.434950 21249 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:09.456760 21354 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:09.476114 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:09.463649 21304 raft_consensus.cc:2749] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:09.482754 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:09.477248 16010 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 15.
I20260709 13:07:09.478847 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:09.479561 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:09.465097 21113 consensus_queue.cc:1048] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:07:09.476858 21362 raft_consensus.cc:2804] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:09.484998 21362 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:09.485421 21362 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:09.486162 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:09.487519 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:09.474855 21339 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:09.484225 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:09.491585 16005 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:09.492615 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:09.492749 16005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41674 (request call id 2230) took 1266 ms. Trace:
I20260709 13:07:09.482103 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:09.465817 21226 consensus_queue.cc:1059] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 13:07:09.494285 21226 raft_consensus.cc:3055] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:09.494664 21226 raft_consensus.cc:740] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:07:09.495783 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:09.495469 21226 consensus_queue.cc:260] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:09.481886 20768 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:09.493214 16005 rpcz_store.cc:276] 0709 13:07:08.226594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.226731 (+   137us) service_pool.cc:224] Handling call
0709 13:07:09.492598 (+1265867us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b54001dbd54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:09.492694 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2786876160}
I20260709 13:07:09.501561 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:09.493237 16561 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 15.
I20260709 13:07:09.504143 21226 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:09.505513 20768 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:09.507951 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:09.475932 21354 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:09.509336 21354 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:09.512945 15933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 1072) took 1364 ms. Trace:
I20260709 13:07:09.513397 15933 rpcz_store.cc:276] 0709 13:07:08.148726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.159092 (+ 10366us) service_pool.cc:224] Handling call
0709 13:07:08.166460 (+  7368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:09.444901 (+1278441us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:09.512691 (+ 67790us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b540034bf54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:09.512881 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2918882048}
I20260709 13:07:09.514457 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:09.515268 16004 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 14: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:09.521483 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:09.522099 15937 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:09.526361 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:09.527068 15936 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 2282) took 1034 ms. Trace:
I20260709 13:07:09.528039 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:09.528684 15931 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:09.531719 15827 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:09.522924 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:09.532460 15933 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
I20260709 13:07:09.532953 21391 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:09.532941 15827 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:09.533875 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:09.534322 21391 raft_consensus.cc:2804] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:09.534763 21391 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:09.535033 15859 raft_consensus.cc:2393] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 13: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:09.535091 21391 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:09.535995 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:09.536722 16561 raft_consensus.cc:2393] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 15: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:09.536847 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:09.537473 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:09.537389 16570 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:09.540140 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:09.537649 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:09.540715 15932 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 12.
I20260709 13:07:09.542665 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:09.543490 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:09.543916 21177 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:09.544404 21177 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:09.545198 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:09.545506 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:09.545917 15861 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
I20260709 13:07:09.546346 21177 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:09.546689 15857 raft_consensus.cc:1275] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:07:09.547340 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:09.547868 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:09.539533 15934 raft_consensus.cc:1275] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:07:09.527477 15936 rpcz_store.cc:276] 0709 13:07:08.492743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.493338 (+   595us) service_pool.cc:224] Handling call
0709 13:07:08.514291 (+ 20953us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:08.514764 (+   473us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:08.514804 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:08.518210 (+  3406us) log.cc:844] Serialized 52 byte log entry
0709 13:07:08.630923 (+112713us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:08.647944 (+ 17021us) log.cc:844] Serialized 14 byte log entry
0709 13:07:09.526519 (+878575us) spinlock_profiling.cc:243] Waited 8.73 ms on lock 0x7b18002b1440. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e9f504bc 00007f73e729e48b 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72b8455 00007f73ea3a0060 00007f73ea3b2716 00007f73ea3c3287
0709 13:07:09.526653 (+   134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:09.526725 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:09.526768 (+    43us) raft_consensus.cc:1685] finished
0709 13:07:09.526861 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:09.526997 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13968,"prepare.queue_time_us":16801,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":13177,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":20477,"spinlock_wait_cycles":20073216,"thread_start_us":136547,"threads_started":2,"wal-append.queue_time_us":279142}
I20260709 13:07:09.551564 21312 consensus_queue.cc:1059] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:09.552652 21312 raft_consensus.cc:3055] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:09.553036 21312 raft_consensus.cc:740] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:09.553658 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:09.553821 21312 consensus_queue.cc:260] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:09.556079 21267 consensus_queue.cc:260] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:09.556653 21248 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:09.557104 21267 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:09.557219 21312 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:09.558498 15937 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
I20260709 13:07:09.562454 16570 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:07:09.565557 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:09.566538 16570 raft_consensus.cc:2393] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 13: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:09.475276 20937 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:09.562541 15931 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:07:09.562559 16003 raft_consensus.cc:1240] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
W20260709 13:07:09.563714 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:09.578477 16562 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
I20260709 13:07:09.604936 15976 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:09.606864 16562 raft_consensus.cc:1275] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:07:09.620486 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:09.621567 15931 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
I20260709 13:07:09.635320 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:09.642488 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:09.643747 16562 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 14: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:09.646119 15902 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:09.647495 21280 raft_consensus.cc:2749] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:07:09.664564 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 95464.4us
I20260709 13:07:09.665513 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:09.665513 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:09.666782 16561 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:09.670557 16533 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:09.672394 16533 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:09.673626 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:09.674181 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:09.674226 15931 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:09.683549 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:09.522547 15825 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:09.684449 15937 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
I20260709 13:07:09.685810 16004 raft_consensus.cc:1240] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:07:09.497807 21367 consensus_queue.cc:1048] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.171s
I20260709 13:07:09.663154 21249 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:09.696316 15827 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:09.697099 15931 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
W20260709 13:07:09.697715 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 175243us
I20260709 13:07:09.661520 20937 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:07:09.700155 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 95373.7us
I20260709 13:07:09.703562 15856 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 13.
I20260709 13:07:09.706794 15902 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:09.699272 20768 consensus_queue.cc:1059] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 13:07:09.709288 21336 raft_consensus.cc:2749] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:09.711972 21336 consensus_queue.cc:1059] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:09.712890 21336 consensus_queue.cc:1243] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer 8430db80a39d4868bcf7641c16de35ea log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:07:09.713369 21249 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:09.716286 20858 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:09.720463 21028 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:09.721009 15933 raft_consensus.cc:2410] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:07:09.721457 21249 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:09.713709 21336 raft_consensus.cc:3055] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:09.722079 15854 raft_consensus.cc:1275] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:07:09.694231 21153 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:09.738195 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:09.726351 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:09.740070 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:09.740710 16010 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:09.700438 21355 raft_consensus.cc:2749] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:09.756884 16563 raft_consensus.cc:1275] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:07:09.685277 21341 consensus_queue.cc:1059] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 13:07:09.759406 16570 raft_consensus.cc:1240] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:09.705413 20937 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:09.705425 21180 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:09.764083 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:07:09.776296 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:09.789853 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 60030.9us
I20260709 13:07:09.709152 21362 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:09.781347 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 77952.7us
I20260709 13:07:09.804118 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:09.724800 21141 raft_consensus.cc:2764] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:07:09.725860 21280 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:09.786631 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:09.725428 21099 consensus_queue.cc:1059] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.082s
I20260709 13:07:09.733536 21336 raft_consensus.cc:740] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:09.804780 16005 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:07:09.781909 15861 raft_consensus.cc:1275] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:07:09.745173 20858 raft_consensus.cc:697] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:09.804845 15974 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:09.777321 16561 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
W20260709 13:07:09.809628 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 94779.8us
I20260709 13:07:09.728647 21354 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:09.731272 21320 consensus_queue.cc:1048] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.072s
W20260709 13:07:09.817492 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 110908us
I20260709 13:07:09.842983 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:09.791736 21307 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:09.751282 21135 consensus_queue.cc:1048] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 13:07:09.850718 16533 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:07:09.904058 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 53426.4us
I20260709 13:07:09.904774 16533 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:07:09.802556 21390 log.cc:927] Time spent T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:07:09.845191 21226 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:09.856509 16561 raft_consensus.cc:1240] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 11. Current term is 12. Ops: [11.12-11.12]
I20260709 13:07:09.858595 16011 raft_consensus.cc:1275] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:07:09.863431 21410 log.cc:927] Time spent T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.052s	user 0.001s	sys 0.001s
I20260709 13:07:09.863765 16006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60156 (request call id 1018) took 1253 ms. Trace:
I20260709 13:07:09.801496 21180 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:09.866936 21305 raft_consensus.cc:3055] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:07:09.873730 21397 log.cc:927] Time spent T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:07:09.879189 21362 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:09.760205 21375 consensus_queue.cc:1048] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 13:07:09.760892 21153 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:09.888832 15902 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:09.934747 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 46029.7us
W20260709 13:07:09.897588 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 92818.1us
W20260709 13:07:09.845427 21351 log.cc:927] Time spent T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:07:09.854940 15757 catalog_manager.cc:5697] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 7 to 11, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 11 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:07:09.909629 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 100467us
I20260709 13:07:09.914053 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:09.915015 16006 rpcz_store.cc:276] 0709 13:07:08.610291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.610421 (+   130us) service_pool.cc:224] Handling call
0709 13:07:09.863513 (+1253092us) spinlock_profiling.cc:243] Waited 659 ms on lock 0x7b54000eddd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:09.863695 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1515475840}
I20260709 13:07:09.865027 16004 raft_consensus.cc:1240] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:09.937456 15902 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:09.807518 21279 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:09.908465 21253 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:09.903753 21141 consensus_queue.cc:1048] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:07:09.912708 21226 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:09.914853 21333 raft_consensus.cc:2764] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:07:09.915869 15859 raft_consensus.cc:2393] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 13: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:09.942351 16535 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:09.822245 21280 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:09.947476 15974 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:09.948195 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:09.948423 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:09.922459 21305 raft_consensus.cc:740] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:09.931241 21248 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:09.955159 16535 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:09.958727 16561 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:09.909034 21307 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:09.944665 20773 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:09.995850 16006 raft_consensus.cc:1275] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:07:09.943528 21253 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:09.995415 15857 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 14.
I20260709 13:07:10.002039 21252 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:09.954033 21249 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:10.018199 21267 consensus_queue.cc:1059] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:09.935465 20768 consensus_queue.cc:1059] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 13:07:10.021803 20768 raft_consensus.cc:3055] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:10.022284 20768 raft_consensus.cc:740] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
W20260709 13:07:10.005117 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 50026.9us
W20260709 13:07:10.028651 21400 log.cc:927] Time spent T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 13:07:10.029156 15827 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:07:10.030179 21405 log.cc:927] Time spent T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:07:10.030594 21177 raft_consensus.cc:3055] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:10.030992 21177 raft_consensus.cc:740] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:10.031658 20773 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:10.031847 21177 consensus_queue.cc:260] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 7, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:09.997722 21167 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:09.992901 21320 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:10.033483 21320 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:10.033943 21320 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:10.033999 21177 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:09.985510 21411 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:10.036206 21411 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:10.036574 21411 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:10.008177 21403 log.cc:927] Time spent T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:07:09.917438 20858 consensus_queue.cc:237] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:10.009548 16570 raft_consensus.cc:1275] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:07:10.042721 15900 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:10.042977 16005 raft_consensus.cc:1240] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:07:10.009387 21305 consensus_queue.cc:260] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:09.991775 21337 raft_consensus.cc:2749] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:10.017119 21153 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:10.032208 20773 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:09.956346 21226 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:09.955152 21312 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.002732 21252 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:10.050504 16561 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
W20260709 13:07:10.052870 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 147263us
I20260709 13:07:09.947630 20879 consensus_queue.cc:1048] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.100s
I20260709 13:07:09.955819 20670 consensus_queue.cc:1048] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.097s
I20260709 13:07:10.006711 21253 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:09.943318 21269 consensus_queue.cc:1048] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.103s
I20260709 13:07:10.086040 15931 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0b3676041725488fb6ffad37889b386b"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:46118
I20260709 13:07:10.049108 21249 raft_consensus.cc:2764] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:07:10.091480 15931 raft_consensus.cc:606] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 14 LEADER]: Received request to transfer leadership to TS 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:10.082598 16533 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:10.096216 15827 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:10.098374 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:10.106959 16010 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
I20260709 13:07:10.033308 21167 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:10.079874 21153 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:10.146602 21153 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:10.147069 21153 raft_consensus.cc:697] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:10.051479 21416 raft_consensus.cc:2804] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:10.148084 21416 raft_consensus.cc:697] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:10.147902 21153 consensus_queue.cc:237] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:10.148932 21416 consensus_queue.cc:237] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:10.152057 21249 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:10.152607 21249 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:10.154562 21249 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:10.156877 20934 raft_consensus.cc:3055] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:10.157294 20934 raft_consensus.cc:740] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:10.158104 20934 consensus_queue.cc:260] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:10.023298 20768 consensus_queue.cc:260] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:10.164079 21391 consensus_meta.cc:287] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.598s	user 0.010s	sys 0.000s
I20260709 13:07:10.164572 21391 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:10.167335 21391 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:10.116863 21374 consensus_queue.cc:260] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:10.119202 21280 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:10.125542 21180 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:10.132596 21409 raft_consensus.cc:2804] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:10.085038 21312 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:10.133124 21307 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:10.134344 21354 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:10.136605 21333 consensus_queue.cc:1048] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:10.208328 21312 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:10.209488 21409 raft_consensus.cc:697] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:10.210510 21409 consensus_queue.cc:237] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:10.139109 15853 raft_consensus.cc:2410] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:07:10.090288 21412 consensus_queue.cc:1048] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 13:07:10.089061 21331 consensus_queue.cc:1059] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:07:09.986371 21336 consensus_queue.cc:260] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:10.141570 20773 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:10.088651 21305 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:10.053344 21340 consensus_queue.cc:1059] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 13:07:10.143929 21279 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:10.145874 21167 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:10.115386 21252 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:10.169344 21366 consensus_queue.cc:1175] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" responder_term: 14 status { last_received { term: 14 index: 12 } last_committed_idx: 10 last_received_current_leader { term: 14 index: 12 } }
I20260709 13:07:10.159399 20934 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:10.173303 20937 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:10.174134 15855 raft_consensus.cc:2368] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 11. Current term is 12.
W20260709 13:07:10.174682 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 222423us
I20260709 13:07:10.188265 15900 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:10.192080 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:10.284693 16561 raft_consensus.cc:2393] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 13: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:10.189626 21388 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:10.248402 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:10.253247 15853 raft_consensus.cc:1275] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:07:10.256294 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:10.291199 16569 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:10.251660 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:10.259536 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:10.293099 16008 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 12
W20260709 13:07:10.294159 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 331851us
I20260709 13:07:10.292492 15857 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 13.
I20260709 13:07:10.259045 15931 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:46118
W20260709 13:07:10.260028 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 233972us
I20260709 13:07:10.246593 21312 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:10.279546 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:10.258447 21374 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:07:10.287245 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 99057.3us
I20260709 13:07:10.288074 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:07:10.288296 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 205757us
I20260709 13:07:10.300913 15937 raft_consensus.cc:1240] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:10.307336 16533 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:10.308286 16533 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:10.309288 16533 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:10.310097 16533 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:07:10.295590 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 358172us
I20260709 13:07:10.302491 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:10.303215 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:10.311513 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:10.296372 15931 raft_consensus.cc:606] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Received request to transfer leadership to TS eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:10.311189 15902 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:10.313033 15931 raft_consensus.cc:614] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:10.313069 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:10.301837 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:10.314931 15932 raft_consensus.cc:2393] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 15: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:10.275614 21336 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:10.315989 15929 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:07:10.316066 15902 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:10.312250 16005 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 16.
I20260709 13:07:10.314077 16010 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
I20260709 13:07:10.249939 21320 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:10.321587 15902 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:10.321866 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:10.323043 15937 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:10.323827 15931 raft_consensus.cc:1275] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:07:10.325974 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:10.326740 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:10.327786 15936 raft_consensus.cc:2410] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:07:10.326951 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:10.328931 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:10.294344 21279 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:10.355048 15930 raft_consensus.cc:1240] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:10.345433 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:10.357092 15853 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
I20260709 13:07:10.358292 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:10.346099 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:10.332288 16011 raft_consensus.cc:1275] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:07:10.339166 15937 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:07:10.333688 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:10.346985 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:10.350713 15854 raft_consensus.cc:1240] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:10.350857 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:10.351475 15928 raft_consensus.cc:1275] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:07:10.351605 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:10.297600 21423 raft_consensus.cc:3055] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:10.331125 16535 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:07:10.374590 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 43542.8us
I20260709 13:07:10.362972 15931 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:10.363248 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:10.363765 15858 raft_consensus.cc:2410] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
W20260709 13:07:10.361995 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 51958.3us
I20260709 13:07:10.364178 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:10.369138 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:10.380254 15937 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:07:10.381430 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:10.382171 15937 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 16.
I20260709 13:07:10.382679 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:10.369876 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:10.384546 15936 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:10.375459 16535 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:10.326704 21307 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:10.386476 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:10.325855 20773 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:10.387744 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:10.383479 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:10.387185 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:10.326238 21180 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:10.403178 15931 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:07:10.405876 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:10.343992 21280 raft_consensus.cc:2804] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:10.406559 15928 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:07:10.325794 21415 raft_consensus.cc:2804] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:10.406087 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:10.407936 15858 raft_consensus.cc:1275] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:07:10.394397 15900 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:10.386155 20642 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.386404 20917 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:10.410862 16569 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:07:10.315837 21312 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:10.416204 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:10.361392 21279 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:10.412837 15935 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:07:10.385726 21344 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:10.454523 15934 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
W20260709 13:07:10.455073 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 45989.4us
I20260709 13:07:10.459831 15936 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:10.419977 15858 raft_consensus.cc:1275] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
W20260709 13:07:10.387893 16533 connection.cc:256] RPC call timeout handler was delayed by -1.0941126349999877s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17400000000000002s for a response
I20260709 13:07:10.461567 16533 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:10.361620 21339 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.381731 21421 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.384778 21423 raft_consensus.cc:740] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:10.463624 15827 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:10.299853 21388 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:10.471110 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:10.418772 21248 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:10.422830 20917 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:10.504748 20917 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:10.505630 15930 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7c0fef632f9a4485838855b553a79163"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:46118
I20260709 13:07:10.506280 15930 raft_consensus.cc:606] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Received request to transfer leadership to TS 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:10.506623 15930 raft_consensus.cc:614] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:10.422657 20642 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:10.463267 21279 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:10.509367 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:07:10.511734 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.498472 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:10.516024 15934 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:10.468720 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:10.412273 21415 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:10.504273 21248 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:10.411876 21280 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:10.342298 21253 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:10.516974 15937 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
I20260709 13:07:10.361483 21425 consensus_queue.cc:1048] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.138s
I20260709 13:07:10.517522 15930 raft_consensus.cc:1275] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:07:10.501075 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:10.529772 15825 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:10.408456 21340 consensus_queue.cc:1048] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.094s
I20260709 13:07:10.420044 21418 raft_consensus.cc:2764] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:07:10.569743 15933 raft_consensus.cc:1275] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:07:10.572315 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:10.529378 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:10.587736 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:10.588832 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 213476us
W20260709 13:07:10.500648 21422 log.cc:927] Time spent T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
W20260709 13:07:10.576898 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 47214.2us
I20260709 13:07:10.586733 15934 raft_consensus.cc:1240] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:10.574704 15935 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:07:10.640400 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:10.646284 16569 raft_consensus.cc:1240] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:10.411051 21113 consensus_queue.cc:1048] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.177s
I20260709 13:07:10.463510 21424 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.658601 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:10.524930 21267 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:10.662612 15933 raft_consensus.cc:1240] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:07:10.525264 21248 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:10.525012 21415 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:10.516759 20773 consensus_queue.cc:1048] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.120s
I20260709 13:07:10.609295 21269 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:10.685812 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:10.669123 15825 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:10.586220 21167 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.673801 21426 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.676937 21279 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:10.678853 21253 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:10.664408 21430 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.592360 21333 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:10.574602 21280 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:10.702780 15859 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
I20260709 13:07:10.474668 21339 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:10.679148 20917 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:10.475705 21411 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:10.499150 21421 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:10.710506 21267 raft_consensus.cc:2749] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:10.662211 21423 consensus_queue.cc:260] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:10.720675 15934 raft_consensus.cc:1240] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 13:07:10.718222 15825 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:10.723469 15853 raft_consensus.cc:1275] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:07:10.710368 21333 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:10.731003 16003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 2401) took 1029 ms. Trace:
I20260709 13:07:10.731462 16003 rpcz_store.cc:276] 0709 13:07:09.701323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:09.701642 (+   319us) service_pool.cc:224] Handling call
0709 13:07:09.701957 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:09.813076 (+111119us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:09.813107 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:09.813562 (+   455us) log.cc:844] Serialized 26 byte log entry
0709 13:07:10.193591 (+380029us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:10.193648 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:10.193736 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:10.730765 (+537029us) raft_consensus.cc:1685] finished
0709 13:07:10.730786 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:10.730937 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":110752,"raft.queue_time_us":111468,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":27959,"thread_start_us":288478,"threads_started":1,"wal-append.queue_time_us":478993}
I20260709 13:07:10.723850 21416 consensus_queue.cc:1048] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 13:07:10.739583 21423 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:10.716161 21429 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.741055 21429 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:10.741951 21433 log.cc:927] Time spent T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:07:10.725454 21320 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:10.743060 21320 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.743741 21429 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:10.743498 21320 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:10.686556 21424 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:10.708137 21167 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:10.763602 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 45464.1us
I20260709 13:07:10.765399 15825 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:10.731359 21333 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:10.784809 16008 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
W20260709 13:07:10.790064 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 328560us
I20260709 13:07:10.791898 16533 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:10.714352 21430 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:10.799472 16533 leader_election.cc:304] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:10.710001 21426 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:10.762482 21307 raft_consensus.cc:697] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:10.769021 21344 raft_consensus.cc:697] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:10.768541 21416 raft_consensus.cc:2749] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:10.809160 16533 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:10.781049 21435 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:10.775303 21439 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:10.815716 16533 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:10.820160 16533 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:10.822011 21441 raft_consensus.cc:2804] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:10.822511 21441 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:10.822844 21441 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:10.823128 16533 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:07:10.826011 21374 consensus_meta.cc:287] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.512s	user 0.009s	sys 0.000s
W20260709 13:07:10.822063 21436 log.cc:927] Time spent T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:07:10.828363 21344 consensus_queue.cc:237] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:10.817003 21435 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:10.833026 21428 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:10.844807 21435 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:10.859041 21341 raft_consensus.cc:2749] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:10.859215 21446 raft_consensus.cc:2749] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:10.842442 21439 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:10.863161 21339 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:10.729939 21362 consensus_queue.cc:1175] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8430db80a39d4868bcf7641c16de35ea" responder_term: 12 status { last_received { term: 10 index: 11 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 11. Current term is 12. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:07:10.848588 21428 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:10.885911 21428 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:10.890069 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2244) took 1814 ms. Trace:
I20260709 13:07:10.890496 16002 rpcz_store.cc:276] 0709 13:07:09.075329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:09.206254 (+130925us) service_pool.cc:224] Handling call
0709 13:07:09.539136 (+332882us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:09.539378 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:09.539402 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:09.539842 (+   440us) log.cc:844] Serialized 26 byte log entry
0709 13:07:09.743186 (+203344us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:09.743239 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:09.743331 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:10.033584 (+290253us) raft_consensus.cc:1685] finished
0709 13:07:10.033605 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:10.889808 (+856203us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b54001dcc50. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:10.890004 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":736980352,"thread_start_us":202972,"threads_started":1,"wal-append.queue_time_us":293021}
I20260709 13:07:10.870420 21307 consensus_queue.cc:237] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:10.876415 21446 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:10.894606 15855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 2432) took 1350 ms. Trace:
I20260709 13:07:10.894539 21446 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:10.894901 21441 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:10.895089 15855 rpcz_store.cc:276] 0709 13:07:09.544574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:09.545258 (+   684us) service_pool.cc:224] Handling call
0709 13:07:10.894360 (+1349102us) spinlock_profiling.cc:243] Waited 628 ms on lock 0x7b540032d554. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:10.894525 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1444188416}
I20260709 13:07:10.896243 21424 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:10.853710 21445 raft_consensus.cc:2749] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:07:10.861946 21336 consensus_meta.cc:287] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.535s	user 0.007s	sys 0.002s
I20260709 13:07:10.898187 16561 raft_consensus.cc:1275] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 13:07:10.898618 16561 pending_rounds.cc:85] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5: Aborting all ops after (but not including) 11
I20260709 13:07:10.898880 16561 pending_rounds.cc:107] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:07:10.898953 21426 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:10.899225 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:10.922168 15856 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:10.879010 15928 raft_consensus.cc:1240] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:07:10.881068 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:10.881728 21305 consensus_meta.cc:287] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.609s	user 0.008s	sys 0.000s
I20260709 13:07:10.882196 21411 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:10.882633 20917 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:10.883899 21401 log.cc:927] Time spent T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.956s	user 0.004s	sys 0.000s
I20260709 13:07:10.896313 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:10.938689 15855 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:07:10.877508 21334 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.900202 21446 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:10.902832 15902 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:10.904090 21336 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:10.904865 21421 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:10.906103 21279 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:10.948975 21334 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:10.906713 21167 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:10.920436 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:10.953361 16569 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:10.921341 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:10.923069 15853 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
I20260709 13:07:10.899240 16561 raft_consensus.cc:2889] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:10.868085 21341 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:10.926333 16007 raft_consensus.cc:1240] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:10.926263 16568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1167) took 1956 ms. Trace:
I20260709 13:07:10.958750 16568 rpcz_store.cc:276] 0709 13:07:08.969601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:09.167643 (+198042us) service_pool.cc:224] Handling call
0709 13:07:09.212058 (+ 44415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:09.341579 (+129521us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:10.926045 (+1584466us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b54007017d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:10.926212 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":295704448}
W20260709 13:07:10.872793 20934 consensus_meta.cc:287] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.585s	user 0.009s	sys 0.000s
I20260709 13:07:10.927528 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:10.965265 21341 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:10.932453 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:10.941679 15827 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
W20260709 13:07:10.974571 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.938248 15974 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:10.946589 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:11.036890 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 98730us
W20260709 13:07:10.955511 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 52748.1us
I20260709 13:07:11.043429 15902 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:11.067807 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 126189us
I20260709 13:07:11.035593 16004 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:10.968456 16003 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:07:10.949532 21320 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:11.074221 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.076373 15857 raft_consensus.cc:1275] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:07:11.077566 15902 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:11.094296 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.088330 21446 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:11.096143 21446 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:11.098258 21446 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:11.073459 21432 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:11.099092 21432 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:11.099591 21432 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:11.117364 21441 leader_election.cc:290] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:11.119935 21341 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:11.123660 20944 raft_consensus.cc:3055] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:11.124197 21254 consensus_queue.cc:1048] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:10.952904 21336 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:11.132790 21248 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:10.933595 21415 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:11.138581 15757 catalog_manager.cc:5697] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 9 to 12, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 12 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:07:11.077350 20773 raft_consensus.cc:2749] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:11.067070 21279 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:10.949939 21367 consensus_queue.cc:1059] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.140s
I20260709 13:07:11.144774 16564 raft_consensus.cc:1275] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:07:11.155073 15856 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
W20260709 13:07:11.158694 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 81188.6us
I20260709 13:07:11.183547 16565 raft_consensus.cc:1240] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:11.077126 21412 raft_consensus.cc:2804] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:11.163596 16009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2255) took 1962 ms. Trace:
I20260709 13:07:11.186594 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:11.142665 21391 raft_consensus.cc:3055] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:11.185588 16009 rpcz_store.cc:276] 0709 13:07:09.200736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:09.433114 (+232378us) service_pool.cc:224] Handling call
0709 13:07:09.433532 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:09.434317 (+   785us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:09.434358 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:09.452829 (+ 18471us) log.cc:844] Serialized 26 byte log entry
0709 13:07:09.695861 (+243032us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:07:09.695922 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:09.696027 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:10.883293 (+1187266us) spinlock_profiling.cc:243] Waited 3.5 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:07:11.163320 (+280027us) raft_consensus.cc:1685] finished
0709 13:07:11.163343 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:11.163537 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":243,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":83,"spinlock_wait_cycles":8050432,"thread_start_us":214763,"threads_started":1,"wal-append.queue_time_us":344635}
I20260709 13:07:11.201115 15936 raft_consensus.cc:2393] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 14: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:11.180253 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:11.207397 15935 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:11.181109 21355 consensus_queue.cc:1059] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:11.181396 21415 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:11.082899 21435 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:11.187326 16568 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:07:11.185068 15902 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
W20260709 13:07:11.228932 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 43961.9us
I20260709 13:07:11.124233 20944 raft_consensus.cc:740] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:11.217900 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:11.233062 16569 raft_consensus.cc:2393] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 17: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:10.961112 21304 raft_consensus.cc:2749] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:11.164160 21113 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:11.188391 20768 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:07:11.228854 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 405761us
W20260709 13:07:11.163555 21177 consensus_meta.cc:287] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 1.129s	user 0.016s	sys 0.003s
I20260709 13:07:11.237268 15825 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:11.202913 21391 raft_consensus.cc:740] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:11.277602 21391 consensus_queue.cc:260] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:11.279034 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:11.280310 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:11.281225 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:11.164554 21320 consensus_queue.cc:1048] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.062s
I20260709 13:07:11.224924 21412 raft_consensus.cc:697] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 12 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:11.285015 21336 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:11.284574 21412 consensus_queue.cc:237] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:11.273911 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:11.288067 21248 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:11.275336 15853 raft_consensus.cc:1275] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:07:11.188745 21269 raft_consensus.cc:2749] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:11.289553 15858 raft_consensus.cc:1240] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:11.289582 15854 raft_consensus.cc:1240] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 13:07:11.288282 15856 raft_consensus.cc:2410] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 16 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:07:11.290616 21435 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:11.223194 21334 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:11.186754 21333 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:11.289114 21280 consensus_meta.cc:287] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.573s	user 0.010s	sys 0.000s
W20260709 13:07:11.232322 21226 consensus_meta.cc:287] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 1.147s	user 0.013s	sys 0.000s
I20260709 13:07:11.301128 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:11.304343 15855 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:07:11.182015 21279 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:11.306344 21432 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:11.235929 21426 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:11.260654 20773 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:11.301023 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:11.320369 15932 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:11.326241 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:11.327181 15933 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:11.332425 15862 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:07:11.301740 21429 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:11.349249 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:11.352192 16010 raft_consensus.cc:1240] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:11.372318 15929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 2461) took 1070 ms. Trace:
I20260709 13:07:11.372980 15929 rpcz_store.cc:276] 0709 13:07:10.302141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.302273 (+   132us) service_pool.cc:224] Handling call
0709 13:07:11.372097 (+1069824us) spinlock_profiling.cc:243] Waited 4.83 ms on lock 0x7b54003b7c54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:11.372245 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11101568}
I20260709 13:07:11.374553 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:11.375319 15929 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
I20260709 13:07:11.324753 21253 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:11.333927 21355 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:11.385974 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:11.346422 21429 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:11.395589 21429 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:11.401465 21333 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:11.334663 21426 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:11.405761 21426 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:11.342903 20773 raft_consensus.cc:697] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 12 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:11.393704 21279 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:11.394948 21428 consensus_meta.cc:287] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.508s	user 0.011s	sys 0.000s
I20260709 13:07:11.387869 15929 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 12.
I20260709 13:07:11.367107 21336 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:11.312742 21280 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:11.311375 21226 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:11.417074 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:11.428457 15930 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:11.429358 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 351086us
I20260709 13:07:11.429906 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:11.430938 15974 leader_election.cc:400] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 11. Current term is 12.
I20260709 13:07:11.430707 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:11.431481 15974 leader_election.cc:403] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:11.433563 15935 raft_consensus.cc:1240] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:07:11.444600 15974 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:11.450865 15974 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:11.408759 21414 raft_consensus.cc:3055] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:11.476168 15929 raft_consensus.cc:2393] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 16: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
W20260709 13:07:11.479997 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 245028us
I20260709 13:07:11.407686 21253 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:11.368157 20944 consensus_queue.cc:260] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:11.433131 21334 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 11. Current term is 12.
I20260709 13:07:11.491261 15902 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:11.475657 21414 raft_consensus.cc:740] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:11.513964 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:11.515098 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:11.515874 15856 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 14.
I20260709 13:07:11.516896 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:11.517745 15856 raft_consensus.cc:2393] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 14: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:11.433080 21336 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:11.496122 21421 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:11.499115 21167 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:07:11.430044 21428 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:11.501395 21280 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:11.504983 21388 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:11.510609 21226 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:11.564153 15900 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:07:11.545002 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.519598 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:11.565975 15856 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:11.520038 15930 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 17.
I20260709 13:07:11.494673 15902 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:11.569016 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 74580.4us
I20260709 13:07:11.569069 15930 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:46118
I20260709 13:07:11.569993 15930 raft_consensus.cc:606] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Received request to transfer leadership to TS eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:11.570465 15930 raft_consensus.cc:614] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:11.442735 21355 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:11.490974 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 217939us
I20260709 13:07:11.524011 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:11.519176 15932 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:07:11.572371 15825 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:07:11.522289 21434 log.cc:927] Time spent T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.857s	user 0.006s	sys 0.004s
I20260709 13:07:11.577302 16568 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:07:11.577733 16564 raft_consensus.cc:1275] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:07:11.522570 21421 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:11.583539 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:11.586572 21336 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:11.587862 21464 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:11.588299 21464 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:11.588621 21464 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:11.563359 20944 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:11.589143 15856 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 15.
I20260709 13:07:11.590273 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:11.590740 21428 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:11.575594 21177 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:11.579346 16533 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:11.610121 16533 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:11.611475 16533 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:11.611666 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:11.596844 21320 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:11.597662 21446 consensus_queue.cc:1059] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:11.602368 15928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2472) took 1188 ms. Trace:
I20260709 13:07:11.604725 15900 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:11.493738 21113 consensus_queue.cc:1059] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.073s
I20260709 13:07:11.578451 15931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2474) took 1070 ms. Trace:
I20260709 13:07:11.612564 15932 raft_consensus.cc:1240] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:11.626648 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:11.576170 21388 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:11.511859 20773 consensus_queue.cc:237] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:11.633912 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:11.610919 15933 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 12.
I20260709 13:07:11.573050 21279 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:11.622152 15928 rpcz_store.cc:276] 0709 13:07:10.413695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.947779 (+534084us) service_pool.cc:224] Handling call
0709 13:07:10.966090 (+ 18311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:10.966413 (+   323us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:10.966443 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:10.966867 (+   424us) log.cc:844] Serialized 26 byte log entry
0709 13:07:11.342069 (+375202us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:11.367265 (+ 25196us) log.cc:844] Serialized 14 byte log entry
0709 13:07:11.367575 (+   310us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:11.367693 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:11.602076 (+234383us) raft_consensus.cc:1685] finished
0709 13:07:11.602110 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:11.602298 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":24852,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":1662,"raft.queue_time_us":7322,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":29252,"thread_start_us":374777,"threads_started":1,"wal-append.queue_time_us":555766}
I20260709 13:07:11.606072 21421 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:11.628648 16563 raft_consensus.cc:2393] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 16: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:11.615986 16533 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:11.627893 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:11.625083 15931 rpcz_store.cc:276] 0709 13:07:10.507878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:11.260812 (+752934us) service_pool.cc:224] Handling call
0709 13:07:11.261232 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:11.261510 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:07:11.298991 (+ 37481us) log.cc:844] Serialized 14 byte log entry
0709 13:07:11.578128 (+279137us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:11.578185 (+    57us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:11.578235 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:11.578300 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:11.578403 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33694,"prepare.queue_time_us":38127,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":58,"raft.queue_time_us":842,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":7437,"thread_start_us":276269,"threads_started":1}
I20260709 13:07:11.647043 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:11.611440 21304 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:11.620509 21336 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:11.620865 21280 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:11.628883 21441 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:07:11.622812 21409 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.005s	sys 0.033s
I20260709 13:07:11.627723 21428 raft_consensus.cc:2804] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:07:11.693910 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 78027.2us
I20260709 13:07:11.633023 21177 raft_consensus.cc:697] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:11.637369 21279 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:11.748131 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:11.748812 15928 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
I20260709 13:07:11.695885 21446 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:11.644209 21435 consensus_queue.cc:1059] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.101s
I20260709 13:07:11.705972 21336 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:11.628507 21320 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:11.715018 21426 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:11.715866 21409 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.749s	user 0.005s	sys 0.033s
W20260709 13:07:11.765925 21455 log.cc:927] Time spent T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 13:07:11.715739 21466 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:11.773665 20642 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:11.784529 16561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 1325) took 1144 ms. Trace:
I20260709 13:07:11.792766 15934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2473) took 1322 ms. Trace:
I20260709 13:07:11.715116 21428 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:11.711184 21469 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:11.821842 15934 rpcz_store.cc:276] 0709 13:07:10.470697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:11.122648 (+651951us) service_pool.cc:224] Handling call
0709 13:07:11.123096 (+   448us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:11.152201 (+ 29105us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:11.152246 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:11.161094 (+  8848us) log.cc:844] Serialized 26 byte log entry
0709 13:07:11.606266 (+445172us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:07:11.606323 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:11.606415 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:11.792490 (+186075us) raft_consensus.cc:1685] finished
0709 13:07:11.792520 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:11.792687 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22950,"raft.queue_time_us":28504,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":9983,"thread_start_us":380583,"threads_started":1,"wal-append.queue_time_us":570122}
I20260709 13:07:11.746807 21279 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:11.798744 15902 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:11.767171 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:11.719816 20944 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:11.770627 15935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2476) took 1262 ms. Trace:
I20260709 13:07:11.747100 21441 raft_consensus.cc:697] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:11.812383 15827 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:11.819537 15855 raft_consensus.cc:1240] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:11.837002 15935 rpcz_store.cc:276] 0709 13:07:10.508501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:11.260908 (+752407us) service_pool.cc:224] Handling call
0709 13:07:11.300167 (+ 39259us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:11.442897 (+142730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:11.770207 (+327310us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b5400328f54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:11.770536 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":306176768}
I20260709 13:07:11.841768 15935 raft_consensus.cc:1240] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:11.632473 21414 consensus_queue.cc:260] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:11.820819 16561 rpcz_store.cc:276] 0709 13:07:10.640299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.645178 (+  4879us) service_pool.cc:224] Handling call
0709 13:07:10.668585 (+ 23407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:10.962276 (+293691us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:11.784040 (+821764us) spinlock_profiling.cc:243] Waited 229 ms on lock 0x7b5400550ad4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:11.784336 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":526448512}
W20260709 13:07:11.829380 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.762291 21336 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:11.844772 15934 raft_consensus.cc:1240] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:11.710616 21280 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:11.707847 21304 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:11.859323 15937 raft_consensus.cc:3055] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:11.859845 15937 raft_consensus.cc:740] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:11.763067 21454 consensus_queue.cc:1048] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.076s
I20260709 13:07:11.828612 21428 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:11.856765 15934 raft_consensus.cc:1275] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:07:11.826191 21466 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:11.852998 20944 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:11.912775 20642 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:11.914443 21441 consensus_queue.cc:237] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:11.838809 21248 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:11.854467 21477 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:11.812469 21113 consensus_queue.cc:1048] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.052s
I20260709 13:07:11.860743 15937 consensus_queue.cc:260] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:11.954135 15937 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:11.927872 21414 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:11.936924 15934 raft_consensus.cc:1240] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 14. Current term is 15. Ops: []
W20260709 13:07:11.945540 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.812657 21426 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:11.850381 21469 raft_consensus.cc:697] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 15 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:07:11.948730 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.855507 21435 raft_consensus.cc:2804] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:11.930791 15935 raft_consensus.cc:1275] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 13:07:11.949065 21429 consensus_meta.cc:287] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.553s	user 0.003s	sys 0.011s
I20260709 13:07:11.984210 15934 raft_consensus.cc:1240] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:11.996470 15935 raft_consensus.cc:1240] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
W20260709 13:07:12.005139 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 252799us
I20260709 13:07:12.007118 16011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2276) took 1079 ms. Trace:
I20260709 13:07:11.971911 21435 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:12.037056 15974 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 1.583s (SENT) [suppressed 14 similar messages]
W20260709 13:07:12.010859 15862 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58214 (request call id 1024) took 1665 ms (client timeout 1513 ms). Trace:
W20260709 13:07:12.051812 15862 rpcz_store.cc:269] 0709 13:07:10.345762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.346700 (+   938us) service_pool.cc:224] Handling call
0709 13:07:12.010691 (+1663991us) spinlock_profiling.cc:243] Waited 933 ms on lock 0x7b5400427cd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:12.010795 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2146272512}
I20260709 13:07:12.014446 21466 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:12.018409 21336 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:12.018814 21464 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:11.952811 20642 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:11.999243 21429 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:12.029287 21474 log.cc:927] Time spent T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
W20260709 13:07:12.037935 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 433304us
I20260709 13:07:11.964414 21477 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:12.042714 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:12.048324 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 236047us
I20260709 13:07:11.864246 21177 consensus_queue.cc:237] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:12.050206 16011 rpcz_store.cc:276] 0709 13:07:10.927793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.928395 (+   602us) service_pool.cc:224] Handling call
0709 13:07:10.940756 (+ 12361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:10.940945 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:07:11.350314 (+409369us) log.cc:844] Serialized 14 byte log entry
0709 13:07:12.005655 (+655341us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:12.006494 (+   839us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:12.006546 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:12.006940 (+   394us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:12.007061 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16657,"prepare.queue_time_us":570302,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":26,"raft.queue_time_us":201249,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":106,"thread_start_us":1036421,"threads_started":3,"wal-append.queue_time_us":476760}
I20260709 13:07:12.085098 15856 raft_consensus.cc:1275] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 13:07:12.074028 15857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 1020) took 1729 ms. Trace:
I20260709 13:07:12.042562 21312 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:12.085479 15856 pending_rounds.cc:85] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8: Aborting all ops after (but not including) 9
I20260709 13:07:12.085619 15857 rpcz_store.cc:276] 0709 13:07:10.344678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.344792 (+   114us) service_pool.cc:224] Handling call
0709 13:07:10.345050 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:11.092607 (+747557us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:12.060694 (+968087us) spinlock_profiling.cc:243] Waited 731 ms on lock 0x7b540008dc54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:12.073942 (+ 13248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1680821120}
I20260709 13:07:12.087908 15856 pending_rounds.cc:107] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:07:12.088224 15856 raft_consensus.cc:2889] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:12.088284 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:12.088969 15862 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
I20260709 13:07:12.047135 21320 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:12.054348 21435 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:12.056391 21304 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:12.094363 15859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2446) took 1708 ms. Trace:
I20260709 13:07:12.092438 16533 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:12.099063 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:12.100929 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:12.075284 21113 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.098212 15859 rpcz_store.cc:276] 0709 13:07:10.385883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.712228 (+326345us) service_pool.cc:224] Handling call
0709 13:07:10.712567 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:10.893759 (+181192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:10.893789 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:10.894279 (+   490us) log.cc:844] Serialized 26 byte log entry
0709 13:07:11.368527 (+474248us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:11.368574 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:12.080585 (+712011us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b540008e654. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:12.080656 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:12.080725 (+    69us) raft_consensus.cc:1685] finished
0709 13:07:12.080745 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:12.094300 (+ 13555us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":293,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":83,"spinlock_wait_cycles":415848704,"thread_start_us":473840,"threads_started":1,"wal-append.queue_time_us":678622}
I20260709 13:07:12.102170 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:12.096745 21280 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:12.054603 21446 consensus_queue.cc:1048] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:07:12.104117 21336 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:12.101557 16011 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:07:12.106932 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:12.084323 21360 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.125116 21360 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:12.126055 21464 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:12.127141 21360 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:12.042527 21469 consensus_queue.cc:237] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:12.100014 20944 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:12.092139 21366 consensus_queue.cc:1048] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 13:07:12.129329 20937 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:12.130558 15858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2449) took 1250 ms. Trace:
I20260709 13:07:12.131039 15858 rpcz_store.cc:276] 0709 13:07:10.879971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.902823 (+ 22852us) service_pool.cc:224] Handling call
0709 13:07:10.903070 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:11.289945 (+386875us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:12.130255 (+840310us) spinlock_profiling.cc:243] Waited 386 ms on lock 0x7b540032df54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:12.130482 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":888425344}
I20260709 13:07:12.132890 16565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 1162) took 1213 ms. Trace:
I20260709 13:07:12.133093 21428 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:12.133394 16565 rpcz_store.cc:276] 0709 13:07:10.919972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.929317 (+  9345us) service_pool.cc:224] Handling call
0709 13:07:10.929593 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:11.184112 (+254519us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:12.132627 (+948515us) spinlock_profiling.cc:243] Waited 254 ms on lock 0x7b54006a09d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:12.132822 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":583480704}
W20260709 13:07:12.102656 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 248345us
I20260709 13:07:12.135883 15936 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 2294) took 1801 ms. Trace:
I20260709 13:07:12.112692 15854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2447) took 1726 ms. Trace:
I20260709 13:07:12.112290 21435 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:12.115922 21253 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:12.117028 20642 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:12.117772 21426 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:12.118024 21430 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:12.122975 21113 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:12.123837 15937 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:12.103744 21312 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.089632 20768 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.144390 15936 rpcz_store.cc:276] 0709 13:07:10.334205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.517829 (+183624us) service_pool.cc:224] Handling call
0709 13:07:12.135621 (+1617792us) spinlock_profiling.cc:243] Waited 614 ms on lock 0x7b54002cdb54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:12.135807 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1411990784}
I20260709 13:07:12.150171 15930 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:46118
I20260709 13:07:12.154122 15930 raft_consensus.cc:606] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Received request to transfer leadership to TS eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:12.154518 15930 raft_consensus.cc:614] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:12.145509 15854 rpcz_store.cc:276] 0709 13:07:10.386114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.900734 (+514620us) service_pool.cc:224] Handling call
0709 13:07:10.901026 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:11.290532 (+389506us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:12.112450 (+821918us) spinlock_profiling.cc:243] Waited 388 ms on lock 0x7b5400219654. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:12.112635 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":893040000}
I20260709 13:07:12.159014 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:12.161602 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:07:12.144805 21320 raft_consensus.cc:2804] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:12.170809 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:12.135644 21428 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:12.181689 15937 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 16.
I20260709 13:07:12.097831 21307 consensus_queue.cc:1048] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.062s
W20260709 13:07:12.165258 21479 log.cc:927] Time spent T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 13:07:12.185120 21253 consensus_queue.cc:1059] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 13:07:12.166322 21320 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:12.203015 21320 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:12.185832 15976 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:12.185820 21336 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:12.204959 21336 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:12.207036 21336 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:07:12.138334 15744 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 83a1b79cdcf046c78453695c798a14a5 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 8430db80a39d4868bcf7641c16de35ea is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:07:12.210525 15744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 72251.3us
I20260709 13:07:12.210638 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:12.211532 15936 raft_consensus.cc:2410] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:07:12.141691 21279 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:12.183352 21446 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:12.237475 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 200444us
I20260709 13:07:12.162254 20642 consensus_queue.cc:1048] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.068s
W20260709 13:07:12.238957 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 53234.6us
I20260709 13:07:12.159277 20768 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:12.252139 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:12.253769 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:12.254436 16007 raft_consensus.cc:1240] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:07:12.254477 16006 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:12.256098 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:12.256798 16009 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:12.212455 21428 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.243547 15933 raft_consensus.cc:2393] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 15: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:12.157171 21312 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:12.260021 21428 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:12.261792 21312 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:12.262486 21428 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:12.266220 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:12.267179 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:12.267990 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:12.268112 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:12.269065 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:12.269753 16002 raft_consensus.cc:2393] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 15: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:12.270519 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:12.273983 16002 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:12.269482 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:12.275632 16007 raft_consensus.cc:2393] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 14: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:12.276497 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:12.277097 16007 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:07:12.277935 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:12.283566 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:12.284896 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:12.285567 16008 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
I20260709 13:07:12.286374 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:12.288756 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:12.224812 21486 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:12.323846 16533 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:12.301957 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:12.305652 16006 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 16.
I20260709 13:07:12.308725 15858 raft_consensus.cc:1240] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:07:12.224821 21420 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:12.328217 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:12.328414 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:12.329330 16006 raft_consensus.cc:2393] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 14: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:12.322193 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:12.247206 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 155568us
W20260709 13:07:12.336905 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 117309us
I20260709 13:07:12.331687 16533 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:12.341288 16002 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 16.
I20260709 13:07:12.245318 21485 raft_consensus.cc:3055] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:12.320567 21492 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.347461 21425 consensus_queue.cc:1048] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:12.320226 21253 raft_consensus.cc:2804] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:12.243484 21446 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:12.353601 16533 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:12.349762 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:12.418388 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:12.365733 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:12.430419 16011 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 12.
I20260709 13:07:12.431041 15854 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:12.418787 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:12.433903 15859 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:12.339275 21336 raft_consensus.cc:2804] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:07:12.457108 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:12.356468 16009 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 17.
I20260709 13:07:12.473196 16006 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:07:12.451140 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 97602.8us
I20260709 13:07:12.371923 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:12.400502 16566 raft_consensus.cc:1240] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:12.475929 15858 raft_consensus.cc:2410] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
W20260709 13:07:12.464643 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 45309.6us
I20260709 13:07:12.477443 15929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53636 (request call id 1094) took 1183 ms. Trace:
I20260709 13:07:12.478032 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:12.483596 15929 rpcz_store.cc:276] 0709 13:07:11.293757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:11.430458 (+136701us) service_pool.cc:224] Handling call
0709 13:07:12.477241 (+1046783us) spinlock_profiling.cc:243] Waited 44.4 ms on lock 0x7b54002c2754. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:12.477386 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":102181632}
I20260709 13:07:12.357740 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:12.493100 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:12.496330 16007 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
I20260709 13:07:12.497529 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:12.337853 21420 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:12.365617 21279 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:12.335057 21486 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:12.424193 21489 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:12.483651 15974 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:12.367244 21307 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:12.358843 21492 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:12.677878 21496 log.cc:927] Time spent T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.278s	user 0.000s	sys 0.000s
I20260709 13:07:12.557930 21280 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.501744 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:07:12.590605 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:12.674494 16006 raft_consensus.cc:2410] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
W20260709 13:07:12.680202 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 196618us
W20260709 13:07:12.650518 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:12.500545 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:12.502985 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:12.682466 15858 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
I20260709 13:07:12.682641 15862 raft_consensus.cc:2376] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13. Re-sending same reply.
I20260709 13:07:12.502468 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:12.683782 15856 raft_consensus.cc:2393] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 13: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:12.683884 15974 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:12.605401 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:12.688611 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:12.689620 16002 raft_consensus.cc:2393] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 14: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:12.621855 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:12.372390 21484 log.cc:927] Time spent T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.296s	user 0.000s	sys 0.001s
I20260709 13:07:12.355095 21485 raft_consensus.cc:740] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:12.367156 21435 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:12.364696 21355 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:12.424176 20773 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:12.705657 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:12.477190 21253 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:12.728154 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:12.752491 21253 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:12.707579 21307 raft_consensus.cc:697] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:12.754142 21307 consensus_queue.cc:237] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:12.728365 21312 consensus_queue.cc:1048] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:12.729267 21492 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:12.574577 21336 raft_consensus.cc:697] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:12.435549 21180 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:12.732954 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:12.768133 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:12.769232 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:12.597468 21438 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:12.701047 20642 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:12.678633 21280 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:12.737030 21341 raft_consensus.cc:3055] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:12.743789 21487 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:12.499075 21441 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.748243 21485 consensus_queue.cc:260] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:12.637205 21360 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:12.498785 21428 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:12.704931 21489 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:12.761315 16003 raft_consensus.cc:1275] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:07:12.775841 21113 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:12.578311 21491 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.812374 21491 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:12.734088 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:12.813288 16010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2278) took 1721 ms. Trace:
I20260709 13:07:12.813719 16010 rpcz_store.cc:276] 0709 13:07:11.091856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:11.092093 (+   237us) service_pool.cc:224] Handling call
0709 13:07:11.096860 (+  4767us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:11.352993 (+256133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:12.813035 (+1460042us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x7b54003ee254. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:12.813228 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":586759424}
I20260709 13:07:12.814179 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:12.814811 21491 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:12.692163 21415 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.815503 21415 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:12.817806 21415 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:12.782665 21438 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:12.794415 21501 log.cc:927] Time spent T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.220s	user 0.003s	sys 0.005s
I20260709 13:07:12.735368 20773 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:12.795398 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:12.785490 21341 raft_consensus.cc:740] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:12.778904 21435 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:12.700322 21509 log.cc:927] Time spent T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.352s	user 0.001s	sys 0.000s
W20260709 13:07:12.789274 21414 consensus_meta.cc:287] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.825s	user 0.011s	sys 0.000s
I20260709 13:07:12.789455 20642 raft_consensus.cc:697] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:12.802194 21360 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:12.786059 21369 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:12.784266 21180 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:12.811061 21391 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:12.781036 16006 raft_consensus.cc:2393] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 16: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
W20260709 13:07:12.819605 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:12.833635 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:12.827732 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:12.830003 15853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 2356) took 1316 ms. Trace:
I20260709 13:07:12.831382 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:12.832032 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:12.788797 21487 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:12.874517 15935 raft_consensus.cc:1240] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:12.825001 16005 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:12.796250 21505 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:12.794463 21441 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:12.972348 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 261607us
I20260709 13:07:12.918704 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:13.076608 15861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2445) took 2684 ms. Trace:
I20260709 13:07:13.118940 15853 rpcz_store.cc:276] 0709 13:07:11.513874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:11.520839 (+  6965us) service_pool.cc:224] Handling call
0709 13:07:11.521096 (+   257us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:11.565368 (+ 44272us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:11.565622 (+   254us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:11.611440 (+ 45818us) log.cc:844] Serialized 52 byte log entry
0709 13:07:11.955997 (+344557us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:11.956063 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:11.956160 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:12.829789 (+873629us) raft_consensus.cc:1685] finished
0709 13:07:12.829809 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:12.829943 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":38809,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":9996,"thread_start_us":247291,"threads_started":1,"wal-append.queue_time_us":407133}
I20260709 13:07:12.883431 16570 raft_consensus.cc:1275] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:07:13.121078 15861 rpcz_store.cc:276] 0709 13:07:10.385617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:10.418092 (+ 32475us) service_pool.cc:224] Handling call
0709 13:07:10.465212 (+ 47120us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:10.465611 (+   399us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:10.465651 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:12.365979 (+1900328us) spinlock_profiling.cc:243] Waited 4.57 ms on lock 0x7b2c00000f48. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e8ca1a11 00007f73e8cb1372 00007f73ea3e3a01 00007f73ea3a843a 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2
0709 13:07:12.366363 (+   384us) log.cc:844] Serialized 26 byte log entry
0709 13:07:12.783546 (+417183us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:12.783582 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:12.783648 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:13.063817 (+280169us) raft_consensus.cc:1685] finished
0709 13:07:13.063848 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:13.069564 (+  5716us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3586,"raft.queue_time_us":52492,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":48304,"spinlock_wait_cycles":10516224,"thread_start_us":416837,"threads_started":1,"wal-append.queue_time_us":544861}
I20260709 13:07:12.919322 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:07:12.946867 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:12.919802 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:12.842867 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:12.806030 21425 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.842308 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:12.832921 15854 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:07:13.141709 16565 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:07:13.170032 15862 raft_consensus.cc:1240] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:07:13.170460 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:13.159970 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:13.159569 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:13.244490 15861 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:12.824260 21446 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:13.159467 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:07:12.827131 21504 log.cc:927] Time spent T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 13:07:13.169344 16562 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
I20260709 13:07:12.824599 21248 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:12.807624 21344 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:12.829149 21486 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:13.191498 15855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 2360) took 1676 ms. Trace:
I20260709 13:07:13.185120 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:13.271404 15855 rpcz_store.cc:276] 0709 13:07:11.514873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:11.593360 (+ 78487us) service_pool.cc:224] Handling call
0709 13:07:11.593784 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:11.839534 (+245750us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:13.191189 (+1351655us) spinlock_profiling.cc:243] Waited 226 ms on lock 0x7b540008b1d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:13.191425 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":518724352}
I20260709 13:07:12.830188 21428 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.824705 21279 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.827821 21469 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:13.204589 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:13.073601 21415 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:13.290771 15854 raft_consensus.cc:2410] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
W20260709 13:07:12.821905 21320 consensus_meta.cc:287] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.618s	user 0.008s	sys 0.000s
I20260709 13:07:13.109833 21334 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:13.294194 21320 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:13.294364 21334 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:13.190457 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:13.296943 21320 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:12.836790 21113 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.822232 21485 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:13.242602 15936 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:13.220938 15932 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
I20260709 13:07:12.826905 21420 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:13.218894 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:13.101320 21254 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:13.168362 16533 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.007s	sys 0.015s
W20260709 13:07:13.307922 16533 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.731s	user 0.007s	sys 0.015s
I20260709 13:07:13.174079 15827 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:13.309024 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 135010us
I20260709 13:07:13.310163 16566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 1164) took 2131 ms. Trace:
I20260709 13:07:13.310590 16566 rpcz_store.cc:276] 0709 13:07:11.178632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:11.178809 (+   177us) service_pool.cc:224] Handling call
0709 13:07:11.214543 (+ 35734us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:12.495855 (+1281312us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:13.309901 (+814046us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b540050f0d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:13.310097 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2726907776}
I20260709 13:07:13.311197 21427 raft_consensus.cc:2764] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:07:13.312021 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:13.313778 21478 consensus_queue.cc:1048] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:13.297559 21412 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.318631 16533 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:07:13.298501 21253 consensus_meta.cc:287] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.536s	user 0.010s	sys 0.000s
I20260709 13:07:12.824476 21495 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.246397 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:13.175279 21367 raft_consensus.cc:3055] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:13.255713 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:07:13.259845 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 458285us
I20260709 13:07:13.334710 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:13.335732 16562 raft_consensus.cc:2410] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:07:13.261741 15928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2480) took 2114 ms. Trace:
I20260709 13:07:13.268142 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:13.336776 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:13.337618 16566 raft_consensus.cc:2410] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:07:13.273990 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:13.336184 15928 rpcz_store.cc:276] 0709 13:07:11.147383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:11.838011 (+690628us) service_pool.cc:224] Handling call
0709 13:07:11.925145 (+ 87134us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:12.152440 (+227295us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:12.152468 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:12.154268 (+  1800us) log.cc:844] Serialized 26 byte log entry
0709 13:07:12.248088 (+ 93820us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:12.248139 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:12.248230 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:13.261547 (+1013317us) raft_consensus.cc:1685] finished
0709 13:07:13.261564 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:13.261688 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":76055,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":1639,"thread_start_us":248027,"threads_started":2,"wal-append.queue_time_us":121670}
I20260709 13:07:13.190799 21360 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:13.208067 21425 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:13.169212 21369 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:12.826268 21438 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:13.185982 21304 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:13.131910 21487 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:13.371140 21304 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:13.373927 21304 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:12.882572 21507 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:13.376039 21507 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:13.376626 20773 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:13.378101 21464 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:12.967129 21489 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:13.301448 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:13.298836 21428 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:13.381520 16563 raft_consensus.cc:1240] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:13.381863 16569 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:07:13.383208 16563 raft_consensus.cc:1240] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:07:13.304637 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:12.929651 21435 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:13.337549 21367 raft_consensus.cc:740] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:13.333207 21420 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:12.884264 21266 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.281047 15861 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 15.
I20260709 13:07:12.828191 21477 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:12.819967 21421 consensus_meta.cc:287] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 1.113s	user 0.009s	sys 0.002s
I20260709 13:07:13.165341 21505 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:13.390952 15933 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 2309) took 1768 ms (client timeout 2052 ms). Trace:
W20260709 13:07:13.391506 15933 rpcz_store.cc:269] 0709 13:07:11.622220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:11.646715 (+ 24495us) service_pool.cc:224] Handling call
0709 13:07:13.390720 (+1744005us) spinlock_profiling.cc:243] Waited 591 ms on lock 0x7b5400328a54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:13.390883 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1358986624}
I20260709 13:07:13.298576 21414 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:13.031862 21341 consensus_queue.cc:260] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:12.937422 21336 consensus_queue.cc:237] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:13.289680 21344 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:13.320770 21320 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:13.286324 21248 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:13.338395 16565 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 14.
I20260709 13:07:13.321907 21254 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:13.363336 21527 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:13.368367 21516 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.378706 21464 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:13.367125 16535 leader_election.cc:304] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:13.369386 15859 raft_consensus.cc:1275] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 12 index: 12. (term mismatch)
I20260709 13:07:13.526518 15859 pending_rounds.cc:85] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8: Aborting all ops after (but not including) 11
I20260709 13:07:13.315619 21412 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:13.526839 15859 pending_rounds.cc:107] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8: Aborting uncommitted NO_OP operation due to leader change: 11.12
I20260709 13:07:13.527215 15859 raft_consensus.cc:2889] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:13.301198 21279 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:13.478164 20773 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:13.369297 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:13.532626 16567 raft_consensus.cc:2393] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 13: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:13.387041 15853 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 13.
I20260709 13:07:13.308511 21415 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:13.394043 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:13.535095 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:13.395434 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:13.303582 21469 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:13.386106 21420 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:13.366748 21446 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:13.494964 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:13.401254 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 283517us
I20260709 13:07:13.544589 15928 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:07:13.304759 21113 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:13.336361 21253 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:13.497133 16564 raft_consensus.cc:1275] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:07:13.317584 20642 consensus_queue.cc:237] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:13.382493 21280 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:13.385756 21334 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:13.386636 21435 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:13.469942 21438 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:13.343715 21333 consensus_queue.cc:1048] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
W20260709 13:07:13.586014 21531 log.cc:927] Time spent T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 13:07:13.589823 21360 consensus_queue.cc:1059] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:13.529810 21527 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:13.543946 21428 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:13.469956 20768 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:13.386674 21180 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:13.342258 15936 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 17.
I20260709 13:07:13.387235 21441 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:13.386457 21266 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:13.531646 16003 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 14.
I20260709 13:07:13.537053 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:13.648348 15858 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
I20260709 13:07:13.649380 21266 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:13.537542 15933 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:07:13.537889 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:13.650298 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 98756us
I20260709 13:07:13.650936 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:13.485189 21489 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:13.651628 15974 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:13.651993 21489 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:13.652901 15974 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:13.369858 21526 consensus_queue.cc:1059] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.174s
I20260709 13:07:13.653739 21492 consensus_queue.cc:1059] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:13.654076 21334 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:13.654670 21492 raft_consensus.cc:3055] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:07:13.470999 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 63193us
I20260709 13:07:13.363754 21495 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:13.387285 21421 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:13.543468 21538 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.531392 21391 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.385813 21367 consensus_queue.cc:260] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:13.479579 21414 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:13.550675 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 239851us
I20260709 13:07:13.655815 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:13.649590 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:13.661247 15858 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 12.
I20260709 13:07:13.653945 21333 raft_consensus.cc:3055] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:13.378739 21507 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:13.662616 15861 raft_consensus.cc:2410] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:07:13.663162 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:13.693532 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 327561us
I20260709 13:07:13.693466 15974 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:13.696055 16568 raft_consensus.cc:1240] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:13.697708 15974 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:13.696733 16535 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:13.699071 21536 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:13.520782 21320 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:13.705083 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:07:13.708141 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 117869us
W20260709 13:07:13.640995 21528 log.cc:927] Time spent T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:07:13.711169 16535 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:13.699630 15974 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:13.655138 21492 raft_consensus.cc:740] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:13.559988 21466 consensus_queue.cc:1059] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.159s
I20260709 13:07:13.713729 16008 raft_consensus.cc:2393] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 15: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:13.541652 20773 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:13.714243 16535 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:13.569170 21478 consensus_queue.cc:1059] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.148s
I20260709 13:07:13.661374 21530 raft_consensus.cc:2749] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:13.718564 15929 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:13.692901 21333 raft_consensus.cc:740] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:13.532259 21516 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:13.876874 21414 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:13.716190 21527 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:13.878432 21279 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:13.707361 21391 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:13.878852 21516 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:13.880769 21391 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:13.882316 21253 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:13.882967 21248 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:13.884086 21333 consensus_queue.cc:260] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:13.885126 21333 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:13.637832 21438 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:13.963260 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:13.966753 15858 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:07:13.695367 21254 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:13.693836 21412 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:13.706982 20858 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:13.709731 21428 raft_consensus.cc:2749] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:13.702389 21320 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:13.699748 21536 raft_consensus.cc:697] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 14 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:07:13.864120 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 150000us
I20260709 13:07:14.022835 16535 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:14.024250 16535 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:13.704713 21420 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:13.744760 21526 raft_consensus.cc:2804] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:13.865335 21344 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:13.866353 21421 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:13.705387 21538 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:13.872354 21492 consensus_queue.cc:260] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:13.874687 21425 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:13.874971 21495 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:13.708197 20768 consensus_queue.cc:1059] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.170s
I20260709 13:07:13.876595 21469 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:13.710176 21266 consensus_queue.cc:1048] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.241s
I20260709 13:07:13.779829 21334 raft_consensus.cc:2764] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:07:13.714020 21535 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:07:14.016631 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:13.882356 21415 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:13.993925 21113 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:14.052155 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:14.042006 16535 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:14.156771 16007 raft_consensus.cc:2393] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 16: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
W20260709 13:07:14.157104 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 115183us
I20260709 13:07:13.998646 21412 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:14.020197 21521 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:14.039018 21505 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:14.037550 21307 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:14.041311 21320 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:14.031848 21526 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:07:14.141156 21487 consensus_meta.cc:287] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.770s	user 0.007s	sys 0.004s
I20260709 13:07:14.160842 16007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41662 (request call id 2407) took 1861 ms. Trace:
I20260709 13:07:13.997568 21516 consensus_queue.cc:1059] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 13:07:14.075228 21540 raft_consensus.cc:3055] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:14.165794 21432 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:14.171453 16007 rpcz_store.cc:276] 0709 13:07:12.300886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.017658 (+1716772us) service_pool.cc:224] Handling call
0709 13:07:14.160775 (+143117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:14.148905 21535 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:07:14.177882 15827 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.003s	sys 0.002s
I20260709 13:07:14.191007 21535 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:14.178507 20773 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:14.192209 21485 consensus_meta.cc:287] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.891s	user 0.001s	sys 0.009s
I20260709 13:07:14.166643 21549 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:14.168026 21307 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:14.193197 21307 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:14.192991 21549 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:14.195142 21549 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:14.179723 21487 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:14.171095 21536 consensus_queue.cc:237] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:14.196758 21438 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:14.197165 21487 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:14.175184 21526 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:14.198721 16566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 1335) took 1326 ms. Trace:
I20260709 13:07:14.199321 16566 rpcz_store.cc:276] 0709 13:07:12.872636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:13.375998 (+503362us) service_pool.cc:224] Handling call
0709 13:07:13.376385 (+   387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:13.376764 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:13.376796 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:13.377260 (+   464us) log.cc:844] Serialized 26 byte log entry
0709 13:07:13.710263 (+333003us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:13.710307 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:13.710378 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:14.198498 (+488120us) raft_consensus.cc:1685] finished
0709 13:07:14.198517 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:14.198662 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7874,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":16186,"thread_start_us":331937,"threads_started":1,"wal-append.queue_time_us":696177}
I20260709 13:07:14.164988 21428 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:14.168484 21412 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:14.166599 21464 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:14.223641 21428 raft_consensus.cc:697] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:14.215222 21320 raft_consensus.cc:697] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:14.192481 21538 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:14.224915 21428 consensus_queue.cc:237] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:14.173666 21540 raft_consensus.cc:740] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:07:14.281832 21551 log.cc:927] Time spent T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:07:14.225240 21320 consensus_queue.cc:237] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:14.284238 21553 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:14.330608 15929 raft_consensus.cc:1275] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:07:14.331342 21412 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:14.190994 15827 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.008s	sys 0.002s
W20260709 13:07:14.358060 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 697160us
W20260709 13:07:14.359172 15932 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 2310) took 1881 ms (client timeout 1793 ms). Trace:
I20260709 13:07:14.359633 21438 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:07:14.360226 16005 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41674 (request call id 2290) took 1992 ms (client timeout 1766 ms). Trace:
W20260709 13:07:14.360862 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:14.359705 15932 rpcz_store.cc:269] 0709 13:07:12.477620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.477765 (+   145us) service_pool.cc:224] Handling call
0709 13:07:14.358975 (+1881210us) spinlock_profiling.cc:243] Waited 737 ms on lock 0x7b5400327654. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:14.359096 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1694755200}
W20260709 13:07:14.360893 16005 rpcz_store.cc:269] 0709 13:07:12.368198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.514848 (+146650us) service_pool.cc:224] Handling call
0709 13:07:14.354784 (+1839936us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b54000aafd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:14.360158 (+  5374us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":314431872}
I20260709 13:07:14.365159 15827 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:07:14.365290 21226 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
W20260709 13:07:14.363777 15974 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.007s	sys 0.084s
W20260709 13:07:14.371906 15974 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.007s	sys 0.084s
W20260709 13:07:14.372149 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 641396us
I20260709 13:07:14.370214 15827 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:14.372888 21560 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:14.374431 21526 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:14.283103 21540 consensus_queue.cc:260] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:14.285087 21553 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:14.378780 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 197355us
I20260709 13:07:14.360256 21438 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:14.417153 15827 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:07:14.420101 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:14.421546 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:14.427757 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:14.428339 16005 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:14.429224 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:14.429867 16006 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 17.
I20260709 13:07:14.371556 21412 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:14.431715 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:14.441516 16007 raft_consensus.cc:1240] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:14.443084 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:14.452210 16563 raft_consensus.cc:2393] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 17: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:14.444058 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:14.455078 15974 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:14.458230 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:14.461367 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:14.415465 21550 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:14.461995 16010 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:07:14.435812 21549 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:14.463653 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:14.463923 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:14.464035 21485 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:14.464522 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:14.415307 20934 consensus_queue.cc:1048] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:07:14.436223 21412 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:14.469538 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:14.472880 16005 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 17.
I20260709 13:07:14.501039 15857 raft_consensus.cc:2410] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:07:14.474334 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:14.499303 15827 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:14.479848 21540 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:14.422565 20642 consensus_queue.cc:1048] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
W20260709 13:07:14.526604 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:14.507220 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:14.488781 21438 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:14.493979 15974 connection.cc:256] RPC call timeout handler was delayed by -1.5786331369999971s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20520000000000002s for a response
I20260709 13:07:14.465030 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:14.505013 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:14.543958 16563 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:07:14.506497 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:14.545181 15854 raft_consensus.cc:2393] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 15: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:14.526456 15827 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:14.464767 21497 log.cc:927] Time spent T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 1.788s	user 0.033s	sys 0.022s
I20260709 13:07:14.490818 21553 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:14.560307 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:14.563100 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:14.563253 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:14.563905 16005 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:07:14.556066 21307 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:14.507125 21412 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:14.557310 21464 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:14.565976 21464 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:14.567793 21466 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.568341 21466 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:14.494845 21549 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:14.569497 21549 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:14.508189 21560 consensus_queue.cc:1059] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.062s
I20260709 13:07:14.570916 21466 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:14.571331 21413 raft_consensus.cc:3055] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:14.571735 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:14.524932 21226 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.574165 21226 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:14.447962 21552 consensus_queue.cc:1059] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.092s
I20260709 13:07:14.576124 21226 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:14.571771 21413 raft_consensus.cc:740] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:14.577382 21413 consensus_queue.cc:260] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:14.578334 21413 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:14.578114 21535 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:14.500763 21485 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:14.579438 21485 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:14.581519 21464 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:14.501922 21550 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:14.585803 21550 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:14.590600 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:07:14.590782 16009 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41662 (request call id 2412) took 2054 ms (client timeout 1954 ms). Trace:
W20260709 13:07:14.591178 16009 rpcz_store.cc:269] 0709 13:07:12.536045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.590580 (+2054535us) service_pool.cc:211] Skipping call since client already timed out
0709 13:07:14.590700 (+   120us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:07:14.593478 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:14.507418 21266 raft_consensus.cc:2749] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:14.440872 21555 consensus_queue.cc:1048] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.133s
I20260709 13:07:14.507231 21561 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.590378 21412 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:14.656309 21556 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:14.656987 21535 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:14.665235 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:07:14.666585 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 106118us
I20260709 13:07:14.670246 15827 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:07:14.671379 21478 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.008s	sys 0.033s
I20260709 13:07:14.671888 21570 raft_consensus.cc:2804] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:07:14.672160 21478 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.009s	sys 0.033s
I20260709 13:07:14.672582 21570 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:14.673427 21570 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:14.673207 21432 consensus_queue.cc:1059] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:07:14.690341 15827 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:14.691512 21413 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:14.692054 21413 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:14.692494 21413 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:14.615640 21561 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:14.700287 21565 log.cc:927] Time spent T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:07:14.701560 21561 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:14.703465 21412 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:14.704442 15854 raft_consensus.cc:2393] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 14: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:14.705677 21413 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:14.708878 15827 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:14.710024 21557 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:14.710688 21557 raft_consensus.cc:697] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 17 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:14.711822 21557 consensus_queue.cc:237] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:14.719012 21570 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:14.721352 15827 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:14.725016 21412 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:14.725193 21526 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:14.725049 21307 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:14.727120 15827 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:14.727196 21549 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:14.728579 15861 raft_consensus.cc:2393] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 16: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:14.723783 21413 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:14.733264 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:14.733821 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:14.744628 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:14.746781 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:14.752274 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:14.752879 16563 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:07:14.753101 21429 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:14.755964 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:14.756695 16561 raft_consensus.cc:2393] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 17: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:14.758556 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:14.760453 16563 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:14.761883 16005 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 17.
I20260709 13:07:14.765043 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:14.765921 16009 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:07:14.767014 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:14.771642 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:14.729547 21549 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:14.734581 21412 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:14.774694 16007 raft_consensus.cc:2393] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 14: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:14.775426 16006 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:14.776279 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:14.781200 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:14.781827 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:14.782105 15827 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:14.781823 16563 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
W20260709 13:07:14.785185 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:14.786842 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:14.788522 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:14.789563 16563 raft_consensus.cc:2393] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 13: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:14.791333 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:14.792342 21557 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:14.805975 21557 raft_consensus.cc:697] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:14.806982 21557 consensus_queue.cc:237] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:14.772859 21485 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:14.808411 21499 log.cc:927] Time spent T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 2.108s	user 0.012s	sys 0.053s
I20260709 13:07:14.810019 21485 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:14.794735 21544 log.cc:927] Time spent T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.612s	user 0.000s	sys 0.014s
I20260709 13:07:14.798756 20768 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:07:14.804656 21524 log.cc:927] Time spent T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 1.559s	user 0.010s	sys 0.013s
I20260709 13:07:14.814826 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:14.816545 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:14.816663 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:14.819823 16002 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:14.823597 15902 connection.cc:256] RPC call timeout handler was delayed by -1.8074679970000034s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.28450000000000003s for a response
W20260709 13:07:14.824288 15902 connection.cc:256] RPC call timeout handler was delayed by -1.7983249640000167s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15410000000000001s for a response
W20260709 13:07:14.824690 15902 connection.cc:256] RPC call timeout handler was delayed by -1.7524249639999425s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1592s for a response
W20260709 13:07:14.825140 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:14.826570 15902 connection.cc:256] RPC call timeout handler was delayed by -1.6669249640000317s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16870000000000002s for a response
W20260709 13:07:14.827149 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 182576us
W20260709 13:07:14.827297 15902 connection.cc:256] RPC call timeout handler was delayed by -1.4266249639999842s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19540000000000002s for a response
I20260709 13:07:14.831346 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:14.832245 16563 raft_consensus.cc:2393] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 16: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:14.829763 21226 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.836668 21226 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:14.838635 15825 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.100s	user 0.018s	sys 0.013s
W20260709 13:07:14.839062 15825 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.101s	user 0.018s	sys 0.013s
I20260709 13:07:14.839139 21226 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:14.839365 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 1.10106e+06us
W20260709 13:07:14.841315 15902 connection.cc:256] RPC call timeout handler was delayed by -1.2979249640000035s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2097s for a response
W20260709 13:07:14.841750 15902 connection.cc:256] RPC call timeout handler was delayed by -1.2898249639999904s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2106s for a response
I20260709 13:07:14.843688 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:14.844457 15932 raft_consensus.cc:2368] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 16. Current term is 17.
I20260709 13:07:14.844497 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:14.845131 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:14.845304 15933 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:14.846794 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:14.847033 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:14.847697 15930 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 9.
I20260709 13:07:14.849022 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:14.849831 15930 raft_consensus.cc:2410] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:07:14.847579 15933 raft_consensus.cc:2376] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12. Re-sending same reply.
I20260709 13:07:14.850756 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:14.851099 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:14.851644 15935 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:14.843211 15825 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:14.854483 15825 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:14.855185 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:14.856334 15932 raft_consensus.cc:2393] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 15: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:14.857784 15825 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:14.825814 21430 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:14.882997 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:14.883718 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:14.862109 15825 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:14.886442 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:14.888069 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:14.888768 15936 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:07:14.889842 16533 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:14.891018 21454 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:14.891443 21454 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:14.891870 21454 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:14.893289 16533 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:14.894351 21486 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:14.894842 21486 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:14.895251 21486 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:14.881985 21267 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:14.898852 21267 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:14.899281 21267 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:07:14.900589 21573 log.cc:927] Time spent T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
W20260709 13:07:14.901625 21574 log.cc:927] Time spent T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 13:07:14.902992 21561 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:14.903420 21561 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:14.903842 21561 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:14.906678 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:14.907522 15934 raft_consensus.cc:2393] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 11: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:14.887261 15928 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:07:14.816102 20768 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:14.856678 21167 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:14.856899 21549 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:14.860596 21429 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:14.852705 21226 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:14.915786 16002 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:14.931908 15928 raft_consensus.cc:1275] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 13:07:14.932327 15928 pending_rounds.cc:85] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 6
I20260709 13:07:14.932562 15928 pending_rounds.cc:107] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:07:14.932796 15928 raft_consensus.cc:2889] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:14.932714 21454 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:14.908720 21430 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:07:14.945933 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:07:14.947046 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:14.945094 21570 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:14.950263 16562 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:14.952067 15934 raft_consensus.cc:1275] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
W20260709 13:07:14.956130 16011 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60156 (request call id 1046) took 2683 ms (client timeout 1881 ms). Trace:
W20260709 13:07:14.957705 16011 rpcz_store.cc:269] 0709 13:07:12.265711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.265886 (+   175us) service_pool.cc:224] Handling call
0709 13:07:14.949306 (+2683420us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b540010efd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:14.949411 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":375970688}
I20260709 13:07:14.958277 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:14.959556 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:14.964496 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:14.965373 15934 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:14.966624 21585 consensus_queue.cc:1048] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:07:14.967872 21576 log.cc:927] Time spent T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:07:14.922534 21167 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:14.970956 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:14.986516 15937 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:14.970974 15825 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:14.943475 21429 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.975479 15931 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
W20260709 13:07:14.977234 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:14.950498 21557 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.979370 15935 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
I20260709 13:07:14.989964 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:14.990604 15825 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:14.990702 15934 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:07:14.991614 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:14.992367 15935 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:14.992895 15930 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:14.994980 21518 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:14.991489 21557 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:14.995296 21454 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:14.995453 21518 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:14.995916 21518 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:14.991336 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:14.981102 15902 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:14.929843 21226 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:14.985347 21585 consensus_queue.cc:1243] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer eb1e0493af1e46dbb6dec7e25525bab5 log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:07:14.998432 15933 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:07:14.999534 15934 pending_rounds.cc:85] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 10
I20260709 13:07:15.001116 15934 pending_rounds.cc:107] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:07:15.002058 15932 raft_consensus.cc:1240] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:07:15.003645 15934 raft_consensus.cc:2889] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:15.006106 21167 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:15.006546 21588 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:14.978783 21427 consensus_queue.cc:1048] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:07:15.008567 15825 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:15.009508 15825 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:15.008267 15902 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:15.011214 15930 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
I20260709 13:07:15.011224 16562 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:15.019326 15827 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:15.019603 15825 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:15.022958 21412 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:07:15.025770 16008 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41662 (request call id 2408) took 2722 ms (client timeout 1541 ms). Trace:
I20260709 13:07:15.000269 21549 raft_consensus.cc:2749] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:15.007037 21588 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:15.025275 21266 raft_consensus.cc:2804] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:07:15.027992 16008 rpcz_store.cc:269] 0709 13:07:12.304441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.321956 (+ 17515us) service_pool.cc:224] Handling call
0709 13:07:15.025572 (+2703616us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54000eecd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:15.025702 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3166736768}
I20260709 13:07:15.029182 21430 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
I20260709 13:07:15.006165 21553 raft_consensus.cc:3055] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:15.031016 15935 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:07:15.013535 21413 raft_consensus.cc:2804] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:15.032606 21588 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:15.036334 21167 raft_consensus.cc:2804] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:15.037843 21486 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:15.024058 21560 raft_consensus.cc:2804] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:15.045120 21226 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:15.027978 21412 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:07:15.055228 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 47008.8us
I20260709 13:07:15.047214 15935 raft_consensus.cc:1240] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:15.048036 21557 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:15.048434 21561 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:15.032784 21266 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:14.991834 21429 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:15.073982 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:15.076095 15902 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:15.080425 15935 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:07:15.081550 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:15.082643 21579 log.cc:927] Time spent T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:07:15.050849 21167 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:15.050799 21413 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:15.084897 15825 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:15.086270 15902 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:15.032441 20768 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:15.046321 21553 raft_consensus.cc:740] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:15.076184 21412 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:15.106704 15902 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:15.116971 15902 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:15.118969 15902 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:15.120591 15902 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:15.122965 15974 leader_election.cc:400] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Vote denied by peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) with higher term. Message: Invalid argument: T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 16. Current term is 17.
I20260709 13:07:15.123292 15974 leader_election.cc:403] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:07:15.123442 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:15.129647 15974 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:15.078498 21560 raft_consensus.cc:697] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
W20260709 13:07:15.140641 16569 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45246 (request call id 1172) took 2309 ms (client timeout 2041 ms). Trace:
I20260709 13:07:15.141353 21560 consensus_queue.cc:237] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:15.142719 21581 log.cc:927] Time spent T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.168s	user 0.000s	sys 0.000s
I20260709 13:07:15.145426 16563 raft_consensus.cc:1240] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:15.148175 16564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 1174) took 2312 ms. Trace:
I20260709 13:07:15.148589 15858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58214 (request call id 1055) took 1611 ms. Trace:
I20260709 13:07:15.149030 15858 rpcz_store.cc:276] 0709 13:07:13.537139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:13.704822 (+167683us) service_pool.cc:224] Handling call
0709 13:07:15.148428 (+1443606us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b540008e154. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:15.148536 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":395018496}
I20260709 13:07:15.150296 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:15.151124 16565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1202) took 1825 ms. Trace:
I20260709 13:07:15.151419 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:15.153381 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:15.151823 16565 rpcz_store.cc:276] 0709 13:07:13.325289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:13.513616 (+188327us) service_pool.cc:224] Handling call
0709 13:07:13.513952 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:13.515544 (+  1592us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:13.515586 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:14.796227 (+1280641us) spinlock_profiling.cc:243] Waited 1.64 ms on lock 0x7b1800387340. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72892a6 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e8ca1a11 00007f73e8cb1372 00007f73ea3e3a01 00007f73ea3a843a
0709 13:07:14.796967 (+   740us) spinlock_profiling.cc:243] Waited 5.05 ms on lock 0x7b2c00047e58. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e8ca1a11 00007f73e8cb1372 00007f73ea3e3a01 00007f73ea3a843a 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2
0709 13:07:14.797466 (+   499us) log.cc:844] Serialized 26 byte log entry
0709 13:07:14.830162 (+ 32696us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:14.830197 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:14.830276 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:15.142188 (+311912us) spinlock_profiling.cc:243] Waited 4.63 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:07:15.150828 (+  8640us) raft_consensus.cc:1685] finished
0709 13:07:15.150859 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:15.151051 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9043,"raft.queue_time_us":1138,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":7149,"spinlock_wait_cycles":26046848,"thread_start_us":32337,"threads_started":1,"wal-append.queue_time_us":132074}
I20260709 13:07:15.083983 21167 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:15.084121 21266 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:15.132308 15974 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:15.124459 21430 raft_consensus.cc:2749] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:15.114748 21454 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.135031 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:15.164743 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:15.115273 21226 raft_consensus.cc:2749] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:07:15.136679 21446 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:15.137166 21540 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:07:15.172315 15974 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:15.110699 21553 consensus_queue.cc:260] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:15.138913 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:15.140139 21587 raft_consensus.cc:2749] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:15.154418 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:15.156741 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:15.132769 21485 raft_consensus.cc:2749] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) with higher term. Message: Invalid argument: T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 16. Current term is 17.
I20260709 13:07:15.155491 15935 raft_consensus.cc:1240] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:15.156193 16568 raft_consensus.cc:1240] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:15.148609 16564 rpcz_store.cc:276] 0709 13:07:12.835452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.835613 (+   161us) service_pool.cc:224] Handling call
0709 13:07:12.836037 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:13.549715 (+713678us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:15.147913 (+1598198us) spinlock_profiling.cc:243] Waited 648 ms on lock 0x7b540048a8d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:15.148111 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1490347904}
W20260709 13:07:15.141122 16569 rpcz_store.cc:269] 0709 13:07:12.832876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.833718 (+   842us) service_pool.cc:224] Handling call
0709 13:07:15.140302 (+2306584us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b5400502154. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:15.140519 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":321035520}
I20260709 13:07:15.080037 21549 consensus_queue.cc:1059] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.077s
I20260709 13:07:15.106455 21413 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:15.115434 21557 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.112399 21570 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:15.104892 21591 raft_consensus.cc:2749] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:15.251257 16563 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:07:15.157441 16533 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:15.304102 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:15.304780 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 147455us
W20260709 13:07:15.315369 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:15.324645 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:15.326072 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:15.251652 21522 raft_consensus.cc:2804] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:15.330093 21435 consensus_queue.cc:1048] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:07:15.331107 21592 raft_consensus.cc:2749] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:07:15.333117 21589 log.cc:927] Time spent T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.254s	user 0.001s	sys 0.000s
I20260709 13:07:15.333333 21549 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:15.334070 21549 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:15.334838 21590 log.cc:927] Time spent T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
I20260709 13:07:15.300664 21553 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:15.349507 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:15.350705 15856 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:07:15.352823 15853 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 2367) took 2440 ms (client timeout 1989 ms). Trace:
I20260709 13:07:15.306118 21570 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:15.353253 15853 rpcz_store.cc:269] 0709 13:07:12.912540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:13.159741 (+247201us) service_pool.cc:224] Handling call
0709 13:07:15.352596 (+2192855us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b540032e454. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:15.352734 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":328519936}
I20260709 13:07:15.359071 15853 raft_consensus.cc:1240] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:15.361672 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:15.318869 21428 raft_consensus.cc:2764] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader election decision vote started in defunct term 14: won
I20260709 13:07:15.250268 21454 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:15.365849 15853 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:15.327481 21485 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:15.322319 21336 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.315026 21267 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:15.359176 21591 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.358356 21570 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:15.312263 21557 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:15.361224 21522 raft_consensus.cc:697] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:15.400216 21489 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:15.403970 21341 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:07:15.403573 15900 connection.cc:256] RPC call timeout handler was delayed by -1.4529697230000238s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2081s for a response
W20260709 13:07:15.421074 15900 connection.cc:256] RPC call timeout handler was delayed by -1.2133873059999587s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20240000000000002s for a response
W20260709 13:07:15.421942 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:15.406885 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:15.410362 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:15.423072 15934 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 10.
I20260709 13:07:15.422425 15900 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:15.427572 15827 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:15.428437 15900 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:15.429188 21592 raft_consensus.cc:2749] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:15.429853 21592 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:15.430320 21226 consensus_queue.cc:1048] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:15.432044 21593 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:15.432583 21593 raft_consensus.cc:697] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 17 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:15.433462 21593 consensus_queue.cc:237] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:15.376523 21485 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:15.229614 21446 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:15.436547 16533 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:15.438015 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:15.428725 15827 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:15.438624 15934 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
I20260709 13:07:15.444108 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:07:15.445686 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:15.445761 15859 raft_consensus.cc:1240] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:15.229929 21540 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:15.446585 15856 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:07:15.251919 21585 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:15.441572 21428 raft_consensus.cc:2804] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:15.503861 21428 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:15.504357 21428 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:15.506913 21519 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.507606 21519 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:15.508905 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:15.509239 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:15.510687 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:15.512163 16570 raft_consensus.cc:2410] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 16 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:07:15.519438 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:15.520464 15928 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
I20260709 13:07:15.522699 15853 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
I20260709 13:07:15.388417 21336 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:15.439812 21485 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:15.440593 21592 raft_consensus.cc:697] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 15 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:15.448552 21486 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:07:15.530916 21518 consensus_meta.cc:287] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.535s	user 0.006s	sys 0.005s
I20260709 13:07:15.470110 21556 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:15.550853 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:15.554584 15934 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:07:15.422791 21591 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:15.537271 21561 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:15.567144 16010 raft_consensus.cc:3055] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:15.590174 16010 raft_consensus.cc:740] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:15.569682 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:15.570753 16003 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41662 (request call id 2415) took 2840 ms (client timeout 2163 ms). Trace:
W20260709 13:07:15.591933 16003 rpcz_store.cc:269] 0709 13:07:12.729962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.803024 (+ 73062us) service_pool.cc:224] Handling call
0709 13:07:15.570501 (+2767477us) spinlock_profiling.cc:243] Waited 412 ms on lock 0x7b5400304654. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:15.570623 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":947571584}
I20260709 13:07:15.573706 15929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2490) took 2742 ms. Trace:
I20260709 13:07:15.593281 15929 rpcz_store.cc:276] 0709 13:07:12.831139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.831929 (+   790us) service_pool.cc:224] Handling call
0709 13:07:12.832199 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:14.331832 (+1499633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:15.573402 (+1241570us) spinlock_profiling.cc:243] Waited 886 ms on lock 0x7b54003b7c54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:15.573619 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2038084352}
I20260709 13:07:15.579684 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:15.595037 15937 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:07:15.596755 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:15.455252 21540 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:15.607923 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:15.601634 16010 consensus_queue.cc:260] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:15.556514 21549 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:15.559054 21518 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:15.604679 21522 consensus_queue.cc:237] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:15.610146 15931 raft_consensus.cc:2393] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 13: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:15.630323 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:15.700659 15856 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 13.
W20260709 13:07:15.706208 15930 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.037s	sys 0.057s
I20260709 13:07:15.700659 15858 raft_consensus.cc:1240] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 15. Current term is 16. Ops: []
W20260709 13:07:15.724500 15930 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.037s	sys 0.057s
W20260709 13:07:15.706835 15932 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.117s	sys 0.187s
W20260709 13:07:15.725319 15932 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.117s	sys 0.187s
W20260709 13:07:15.708827 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 536432us
I20260709 13:07:15.733494 21336 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:15.614120 16010 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:15.735507 15974 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:15.740952 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:15.743032 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:15.706923 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:15.750685 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:15.714537 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:15.769236 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:15.770721 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:15.772228 15934 raft_consensus.cc:2393] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 13: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:15.773307 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:15.774654 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:15.776408 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:15.777913 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:15.801399 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:15.808121 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:15.779345 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:15.815037 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:15.820681 21446 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:15.825177 21446 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:15.636444 21454 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:15.827853 21477 consensus_queue.cc:1048] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:07:15.831436 21477 raft_consensus.cc:2804] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:15.832407 21477 raft_consensus.cc:697] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:15.833943 21477 consensus_queue.cc:237] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:15.834025 21428 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:15.739557 15974 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:15.880246 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:15.664036 21588 consensus_meta.cc:287] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.586s	user 0.009s	sys 0.003s
I20260709 13:07:15.633118 21592 consensus_queue.cc:237] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:15.890358 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 151060us
I20260709 13:07:15.894731 21518 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:15.892143 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:07:15.629001 21594 log.cc:927] Time spent T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.194s	user 0.000s	sys 0.001s
I20260709 13:07:15.903291 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:15.629789 21557 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:15.655416 21591 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:15.914824 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:15.916460 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:15.922565 21454 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:15.923477 21454 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:15.924049 21428 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:15.656934 21570 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:15.926175 21454 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:15.926797 21570 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:15.929778 21588 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:15.944372 15936 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 2327) took 1036 ms. Trace:
I20260709 13:07:15.945616 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:15.949606 21588 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:15.950626 15859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 1051) took 2414 ms. Trace:
I20260709 13:07:15.951437 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:15.952490 15859 rpcz_store.cc:276] 0709 13:07:13.535951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:13.536126 (+   175us) service_pool.cc:224] Handling call
0709 13:07:15.357030 (+1820904us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:15.446508 (+ 89478us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:15.948995 (+502487us) spinlock_profiling.cc:243] Waited 88.5 ms on lock 0x7b540032e454. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:15.950047 (+  1052us) spinlock_profiling.cc:243] Waited 1.82 s on lock 0x7b540032e450. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:15.950375 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4389954176}
I20260709 13:07:15.943320 21477 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:15.962795 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:15.963923 15860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 1043) took 3453 ms. Trace:
I20260709 13:07:15.964502 15862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 1047) took 3459 ms. Trace:
I20260709 13:07:15.964982 15862 rpcz_store.cc:276] 0709 13:07:12.505859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.752991 (+247132us) service_pool.cc:224] Handling call
0709 13:07:13.169754 (+416763us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:13.170484 (+   730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:15.964075 (+2793591us) spinlock_profiling.cc:243] Waited 416 ms on lock 0x7b54003b6350. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:15.964341 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":957549824}
I20260709 13:07:15.966475 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:15.968711 15855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 2365) took 3086 ms. Trace:
I20260709 13:07:15.969223 15855 rpcz_store.cc:276] 0709 13:07:12.882283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:13.288716 (+406433us) service_pool.cc:224] Handling call
0709 13:07:13.311985 (+ 23269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:13.715322 (+403337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:13.715367 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:13.715427 (+    60us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:13.715485 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:15.968169 (+2252684us) spinlock_profiling.cc:243] Waited 403 ms on lock 0x7b54000e9cd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:15.968252 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:15.968465 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":927009664}
I20260709 13:07:15.969973 21519 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:07:15.970242 16535 leader_election.cc:336] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 1.809s (SENT)
I20260709 13:07:15.970862 16535 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:15.967598 15862 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:07:15.944916 15936 rpcz_store.cc:276] 0709 13:07:14.905759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.906667 (+   908us) service_pool.cc:224] Handling call
0709 13:07:14.907086 (+   419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:14.980927 (+ 73841us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:14.980961 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:14.984445 (+  3484us) log.cc:844] Serialized 26 byte log entry
0709 13:07:15.162668 (+178223us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:15.162695 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:15.162744 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:15.939792 (+777048us) raft_consensus.cc:1685] finished
0709 13:07:15.939830 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:15.942390 (+  2560us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":58825,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":99,"thread_start_us":243748,"threads_started":2,"wal-append.queue_time_us":375124}
I20260709 13:07:15.964545 15860 rpcz_store.cc:276] 0709 13:07:12.500758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.502098 (+  1340us) service_pool.cc:224] Handling call
0709 13:07:12.589615 (+ 87517us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:13.481139 (+891524us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:13.481177 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:13.481227 (+    50us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:13.481276 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:15.953721 (+2472445us) spinlock_profiling.cc:243] Waited 891 ms on lock 0x7b54003b5e54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:15.953854 (+   133us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:15.954124 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":123,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":27048,"spinlock_wait_cycles":2049361792}
I20260709 13:07:15.976950 16568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 1341) took 1203 ms. Trace:
I20260709 13:07:15.977483 16568 rpcz_store.cc:276] 0709 13:07:14.773556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.774486 (+   930us) service_pool.cc:224] Handling call
0709 13:07:15.155956 (+381470us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:15.287056 (+131100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:15.976457 (+689401us) spinlock_profiling.cc:243] Waited 350 ms on lock 0x7b5400502150. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:15.976720 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":805017472}
I20260709 13:07:15.974246 15861 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 15.
I20260709 13:07:15.981848 15860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2475) took 1491 ms. Trace:
I20260709 13:07:15.980916 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:15.982478 15860 rpcz_store.cc:276] 0709 13:07:14.489357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.977654 (+1488297us) service_pool.cc:224] Handling call
0709 13:07:15.979859 (+  2205us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:15.980142 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:15.980178 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:15.980255 (+    77us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:15.980290 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:15.980346 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:15.980450 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:07:15.984835 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:15.992223 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:15.983613 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:15.992697 16010 raft_consensus.cc:1275] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 6. (term mismatch)
I20260709 13:07:15.997440 16010 pending_rounds.cc:85] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee: Aborting all ops after (but not including) 5
I20260709 13:07:15.995800 21477 raft_consensus.cc:697] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:07:15.999775 21167 consensus_meta.cc:287] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.770s	user 0.013s	sys 0.012s
W20260709 13:07:15.981308 15902 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.734s	user 0.008s	sys 0.011s
I20260709 13:07:16.000638 16010 pending_rounds.cc:107] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 12.6
W20260709 13:07:16.002403 15902 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.844s	user 0.009s	sys 0.011s
I20260709 13:07:16.006721 16010 raft_consensus.cc:2889] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:16.011350 21360 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:16.012216 21360 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:16.015259 16009 raft_consensus.cc:1240] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:16.020715 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:16.021978 16009 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:16.016729 21391 raft_consensus.cc:3055] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:16.001739 21519 raft_consensus.cc:2749] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:16.031953 15934 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:16.028615 21391 raft_consensus.cc:740] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:16.035017 21367 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:16.036110 21391 consensus_queue.cc:260] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:16.037762 21391 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:16.042189 21601 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:16.042347 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:16.043210 21478 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:16.043171 21601 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:16.044346 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.044276 21478 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:16.044701 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:16.045605 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.046795 21478 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:16.046880 21601 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:16.047992 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.048041 21454 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:07:16.048766 21607 log.cc:927] Time spent T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.095s	user 0.000s	sys 0.002s
I20260709 13:07:16.048925 21454 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:16.050097 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:16.050845 16568 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
W20260709 13:07:16.040288 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 882150us
W20260709 13:07:16.031167 21413 consensus_meta.cc:287] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.643s	user 0.017s	sys 0.005s
I20260709 13:07:16.047214 16010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2302) took 1547 ms. Trace:
I20260709 13:07:16.053623 16010 rpcz_store.cc:276] 0709 13:07:14.499928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.500526 (+   598us) service_pool.cc:224] Handling call
0709 13:07:14.506110 (+  5584us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:15.567031 (+1060921us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b54003d3220. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea31fdc0 00007f73ea3aa1c0 00007f73ea3ab231 00007f73ea3a7d8f 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c
0709 13:07:16.040834 (+473803us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.047137 (+  6303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":386040576}
I20260709 13:07:16.054499 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:16.006925 21167 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:16.055735 15936 raft_consensus.cc:2393] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 13: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:16.055645 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:16.054653 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:16.056102 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:16.056217 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:16.053159 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:16.064407 21253 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
I20260709 13:07:16.064221 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:16.065232 16569 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:07:16.062714 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:16.059661 21417 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.066506 21369 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:16.066797 21417 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:16.062923 21601 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:16.072009 21253 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.072304 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:16.072798 21253 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:16.074119 21167 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:16.075644 21253 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:15.875389 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:16.075785 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:16.082274 21601 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:16.084671 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:16.097045 21253 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:07:16.097321 21412 consensus_meta.cc:287] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.876s	user 0.016s	sys 0.008s
I20260709 13:07:16.097795 21253 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:16.098079 21412 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:16.099035 21417 leader_election.cc:290] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:16.100112 21253 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:16.101287 15853 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 15: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:07:16.101400 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.101518 21601 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:16.114006 15933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2513) took 1145 ms. Trace:
W20260709 13:07:16.114742 21610 log.cc:927] Time spent T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
W20260709 13:07:16.114179 15854 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 2476) took 1622 ms (client timeout 1802 ms). Trace:
I20260709 13:07:16.096341 15859 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
I20260709 13:07:16.123140 16566 raft_consensus.cc:2393] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 13: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:16.125000 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:16.126149 15928 raft_consensus.cc:2393] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 15: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
W20260709 13:07:16.126521 21611 log.cc:927] Time spent T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:07:16.109738 16563 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
W20260709 13:07:16.120517 15854 rpcz_store.cc:269] 0709 13:07:14.491967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.560638 (+ 68671us) service_pool.cc:224] Handling call
0709 13:07:16.113799 (+1553161us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b54003b6d54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:16.114004 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":253269376}
I20260709 13:07:16.114609 15933 rpcz_store.cc:276] 0709 13:07:14.968107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.021134 (+ 53027us) service_pool.cc:224] Handling call
0709 13:07:15.569114 (+547980us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:15.569400 (+   286us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:15.569437 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:15.569495 (+    58us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:15.569549 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:15.569624 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:16.113593 (+543969us) spinlock_profiling.cc:243] Waited 548 ms on lock 0x7b54002c1850. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:16.113784 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1259305216}
I20260709 13:07:16.127297 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:16.134227 15860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 1065) took 1173 ms. Trace:
I20260709 13:07:16.063856 21413 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:16.137092 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:16.137275 21266 consensus_meta.cc:287] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.860s	user 0.012s	sys 0.012s
I20260709 13:07:16.134601 15860 rpcz_store.cc:276] 0709 13:07:14.961217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.987282 (+1026065us) service_pool.cc:224] Handling call
0709 13:07:15.987876 (+   594us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:15.988327 (+   451us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:07:16.029405 (+ 41078us) log.cc:844] Serialized 14 byte log entry
0709 13:07:16.133855 (+104450us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:16.133910 (+    55us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:16.133962 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.134031 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:16.134148 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":81625,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"raft.queue_time_us":180,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":98,"thread_start_us":114473,"threads_started":2}
I20260709 13:07:16.146347 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:16.143275 15931 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 12.
I20260709 13:07:16.095046 21454 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:16.137748 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:16.149614 15855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2472) took 1666 ms. Trace:
I20260709 13:07:16.155792 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:16.161351 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:16.165373 21412 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:16.166256 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:16.167052 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:16.169272 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:16.152521 21266 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:16.152536 15855 rpcz_store.cc:276] 0709 13:07:14.482858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.970579 (+1487721us) service_pool.cc:224] Handling call
0709 13:07:15.971232 (+   653us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:15.975998 (+  4766us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:07:16.030578 (+ 54580us) log.cc:844] Serialized 14 byte log entry
0709 13:07:16.149279 (+118701us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:16.149331 (+    52us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:16.149383 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.149449 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:16.149566 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":87322,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":8752,"raft.queue_time_us":4601,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":1149,"thread_start_us":136861,"threads_started":2}
I20260709 13:07:16.149914 21477 consensus_queue.cc:237] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:16.179126 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 59980.4us
I20260709 13:07:16.181954 15935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2515) took 1210 ms. Trace:
I20260709 13:07:16.182672 21429 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:16.182940 21266 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:16.183602 15861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 1060) took 1236 ms. Trace:
I20260709 13:07:16.183884 15857 raft_consensus.cc:2393] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 13: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:16.184530 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:16.184886 21485 consensus_meta.cc:287] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.619s	user 0.015s	sys 0.002s
I20260709 13:07:16.185578 15931 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
I20260709 13:07:16.185504 21485 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:16.186061 15857 raft_consensus.cc:1240] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:16.186684 15857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 1071) took 1024 ms. Trace:
I20260709 13:07:16.186604 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:16.187086 15857 rpcz_store.cc:276] 0709 13:07:15.161871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.185347 (+1023476us) service_pool.cc:224] Handling call
0709 13:07:16.185861 (+   514us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:16.186448 (+   587us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.186612 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:16.182503 15935 rpcz_store.cc:276] 0709 13:07:14.971280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.082466 (+111186us) service_pool.cc:224] Handling call
0709 13:07:15.155225 (+ 72759us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:15.286703 (+131478us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.181541 (+894838us) spinlock_profiling.cc:243] Waited 72.4 ms on lock 0x7b54003b8b50. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:16.181745 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":166520704}
I20260709 13:07:16.191790 15928 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:16.196548 15935 raft_consensus.cc:1275] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 13:07:16.197311 21554 log.cc:927] Time spent T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 2.001s	user 0.000s	sys 0.015s
I20260709 13:07:16.194406 15974 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:16.198087 15857 raft_consensus.cc:1275] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:07:16.200891 21522 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:16.201663 21522 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:16.184003 15861 rpcz_store.cc:276] 0709 13:07:14.947498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.990342 (+1042844us) service_pool.cc:224] Handling call
0709 13:07:15.990901 (+   559us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:16.010256 (+ 19355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:16.010278 (+    22us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:16.031063 (+ 20785us) log.cc:844] Serialized 26 byte log entry
0709 13:07:16.132601 (+101538us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:16.147140 (+ 14539us) log.cc:844] Serialized 14 byte log entry
0709 13:07:16.150673 (+  3533us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.150742 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:16.183390 (+ 32648us) raft_consensus.cc:1685] finished
0709 13:07:16.183407 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:16.183553 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15869,"prepare.queue_time_us":15501,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":23379,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":1289,"thread_start_us":91491,"threads_started":1,"wal-append.queue_time_us":134769}
I20260709 13:07:16.209846 15929 raft_consensus.cc:1240] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:16.210672 15861 raft_consensus.cc:1240] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:07:16.212724 21490 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.213516 21490 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:16.214260 15929 raft_consensus.cc:1275] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:07:16.217489 15929 raft_consensus.cc:1275] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:07:16.217612 21267 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:16.217902 21540 consensus_meta.cc:287] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.597s	user 0.009s	sys 0.009s
I20260709 13:07:16.219229 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:16.220196 21522 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:16.220263 15929 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:16.221737 21601 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.221913 21478 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:16.222366 21601 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:16.222633 21478 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:16.224658 21601 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:16.229453 16563 raft_consensus.cc:1240] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:16.230072 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:07:16.231541 15976 connection.cc:256] RPC call timeout handler was delayed by -1.5021708860000444s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1955s for a response
W20260709 13:07:16.232045 15976 connection.cc:256] RPC call timeout handler was delayed by -1.2600708859999941s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22240000000000004s for a response
I20260709 13:07:16.233647 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:16.234100 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:16.235175 16005 raft_consensus.cc:2410] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:07:16.235536 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:16.236457 20934 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.236317 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:16.235656 15825 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:16.218585 21540 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:16.237182 20934 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:16.245857 20934 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:16.246245 21609 log.cc:927] Time spent T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.301s	user 0.002s	sys 0.000s
I20260709 13:07:16.247551 21427 raft_consensus.cc:2804] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:16.248096 21427 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:16.248493 21427 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:16.250268 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:16.251034 15937 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:16.252110 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:16.253746 21478 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:16.253799 16006 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 15.
I20260709 13:07:16.256654 21485 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:07:16.257781 16570 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45246 (request call id 1187) took 1729 ms (client timeout 1845 ms). Trace:
I20260709 13:07:16.258168 16561 raft_consensus.cc:1275] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 13. (term mismatch)
I20260709 13:07:16.258741 16561 pending_rounds.cc:85] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting all ops after (but not including) 12
I20260709 13:07:16.258960 16561 pending_rounds.cc:107] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting uncommitted NO_OP operation due to leader change: 14.13
I20260709 13:07:16.259209 16561 raft_consensus.cc:2889] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:07:16.258257 16570 rpcz_store.cc:269] 0709 13:07:14.527984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.528235 (+   251us) service_pool.cc:224] Handling call
0709 13:07:16.257311 (+1729076us) spinlock_profiling.cc:243] Waited 920 ms on lock 0x7b54006a1654. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:16.257577 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2116926848}
I20260709 13:07:16.268442 15930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2518) took 1218 ms. Trace:
I20260709 13:07:16.269222 15930 rpcz_store.cc:276] 0709 13:07:15.050526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.149868 (+ 99342us) service_pool.cc:224] Handling call
0709 13:07:15.150283 (+   415us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:15.725197 (+574914us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:15.725275 (+    78us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:15.726297 (+  1022us) log.cc:844] Serialized 52 byte log entry
0709 13:07:15.877260 (+150963us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:07:15.942999 (+ 65739us) log.cc:844] Serialized 14 byte log entry
0709 13:07:15.954685 (+ 11686us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:15.956925 (+  2240us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:16.118807 (+161882us) spinlock_profiling.cc:243] Waited 5.76 ms on lock 0x7b18004e2b00. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb522 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd 000000000038f354 000000000038f199 00007f73e729e83a
0709 13:07:16.245510 (+126703us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:07:16.268126 (+ 22616us) raft_consensus.cc:1685] finished
0709 13:07:16.268167 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:16.268338 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":52409,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":62,"raft.queue_time_us":105479,"raft.run_cpu_time_us":451,"raft.run_wall_time_us":23009,"spinlock_wait_cycles":36907520,"thread_start_us":724590,"threads_started":2,"wal-append.queue_time_us":218271}
I20260709 13:07:16.273787 15859 raft_consensus.cc:1240] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:16.274794 15856 raft_consensus.cc:1240] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:07:16.275629 15859 raft_consensus.cc:1275] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:07:16.279520 15937 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:16.279633 15929 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:16.282191 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:16.283535 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:16.283819 16561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 1340) took 1515 ms. Trace:
I20260709 13:07:16.284330 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:16.278520 21367 consensus_queue.cc:1048] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 13:07:16.286478 21360 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:16.286913 21540 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:16.287760 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:16.288810 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:16.289796 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:16.290704 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:16.292156 15935 raft_consensus.cc:1275] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:07:16.292481 21615 log.cc:927] Time spent T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:07:16.292717 15931 raft_consensus.cc:1240] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 15 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 13:07:16.292832 21413 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:16.293061 16533 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:16.293938 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:16.294204 15825 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:16.305595 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:16.298956 21427 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:16.314580 15862 raft_consensus.cc:1240] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:07:16.322736 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:16.325994 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:16.308120 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:16.321480 15900 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:16.319830 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:16.327039 15936 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:16.317909 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:16.318482 15858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2469) took 1841 ms. Trace:
W20260709 13:07:16.312367 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 120750us
I20260709 13:07:16.326303 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:16.299228 15928 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 11.
I20260709 13:07:16.334637 16009 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:16.335459 16563 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:16.305287 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:16.327487 16561 rpcz_store.cc:276] 0709 13:07:14.768415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.768578 (+   163us) service_pool.cc:224] Handling call
0709 13:07:16.257548 (+1488970us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:16.262050 (+  4502us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.283524 (+ 21474us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b54006a1650. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:16.283739 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3366867328}
I20260709 13:07:16.342314 15858 rpcz_store.cc:276] 0709 13:07:14.476785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.355416 (+878631us) service_pool.cc:224] Handling call
0709 13:07:15.355997 (+   581us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:15.724855 (+368858us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.318170 (+593315us) spinlock_profiling.cc:243] Waited 344 ms on lock 0x7b54002490d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:16.318393 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":790681984}
I20260709 13:07:16.306543 16570 raft_consensus.cc:1275] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:07:16.307613 15855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 1069) took 1337 ms. Trace:
I20260709 13:07:16.337280 16004 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:07:16.303120 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:16.346716 16003 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:16.321964 21490 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:16.357291 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:16.357414 21553 consensus_queue.cc:1048] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:07:16.358731 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:16.326403 21522 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:16.350814 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:16.360697 15931 raft_consensus.cc:2410] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 17 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:07:16.360586 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:16.361403 15935 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:16.361718 15976 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:16.362046 15931 raft_consensus.cc:1240] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:16.364034 15974 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:16.368391 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:16.369221 16007 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:07:16.348127 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:16.323907 15757 catalog_manager.cc:5697] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 9 to 13, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 13 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:16.370733 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:16.371604 16003 raft_consensus.cc:2393] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 12: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:16.372552 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:16.371970 15757 catalog_manager.cc:5697] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 8 to 11, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 11 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:16.371974 15929 raft_consensus.cc:1240] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:07:16.374311 15757 catalog_manager.cc:5697] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 6 to 12, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 12 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:07:16.375115 16535 leader_election.cc:336] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 1.881s (SENT)
I20260709 13:07:16.376573 15757 catalog_manager.cc:5697] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 11 to 12, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 12 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:07:16.374272 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:16.378434 15757 catalog_manager.cc:5697] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 12 to 13, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 13 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:16.350234 16569 raft_consensus.cc:2393] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 14: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:16.350116 16010 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:16.379941 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:16.380760 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:16.380963 21522 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:16.381117 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:16.381933 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:16.382357 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 51313.5us
I20260709 13:07:16.382654 16010 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 14.
W20260709 13:07:16.350754 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 56644.5us
I20260709 13:07:16.350988 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:16.383819 15827 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:16.384280 16011 raft_consensus.cc:2393] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 13: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:16.385061 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:16.386226 21367 raft_consensus.cc:2749] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:16.386919 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:16.388113 16011 raft_consensus.cc:1240] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:16.321199 21492 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:16.388351 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:16.389513 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:16.390347 16011 raft_consensus.cc:1275] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:07:16.388109 15827 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:16.390779 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:16.391417 15860 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
I20260709 13:07:16.392747 21367 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:16.393179 21367 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:16.393534 21367 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:16.393978 15757 catalog_manager.cc:5697] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 8 to 11, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 11 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:16.352412 16008 raft_consensus.cc:1275] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:07:16.351856 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:16.395418 16005 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:16.395767 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:16.353179 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:07:16.356099 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 119110us
I20260709 13:07:16.396898 16533 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:16.356490 21454 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:16.397760 21417 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:16.397758 21522 consensus_queue.cc:1059] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:16.396379 15934 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:16.399333 21312 raft_consensus.cc:3055] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:16.399720 21312 raft_consensus.cc:740] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:16.400686 21312 consensus_queue.cc:260] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:16.401712 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:16.402134 21312 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:16.402410 15931 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:07:16.387061 15937 raft_consensus.cc:1240] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:16.350134 15862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 1059) took 1401 ms. Trace:
I20260709 13:07:16.364243 15932 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:16.403913 15862 rpcz_store.cc:276] 0709 13:07:14.947138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.000626 (+1053488us) service_pool.cc:224] Handling call
0709 13:07:16.314319 (+313693us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:16.344158 (+ 29839us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.347897 (+  3739us) spinlock_profiling.cc:243] Waited 297 ms on lock 0x7b5400427cd0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:16.348013 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":683991936}
I20260709 13:07:16.394644 15825 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:16.398053 21454 raft_consensus.cc:697] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:16.406394 21454 consensus_queue.cc:237] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:16.352352 21427 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:16.353775 21360 raft_consensus.cc:2804] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:16.408030 21428 consensus_queue.cc:1059] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:16.408836 21360 raft_consensus.cc:697] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:16.409888 21360 consensus_queue.cc:237] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:16.410719 21428 raft_consensus.cc:3055] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:16.411118 21413 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:16.411186 21428 raft_consensus.cc:740] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:16.411859 21413 raft_consensus.cc:697] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:16.353255 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:16.412534 21428 consensus_queue.cc:260] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:16.412938 21413 consensus_queue.cc:237] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:16.413667 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:16.393196 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:16.414331 15858 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:07:16.415006 21519 consensus_queue.cc:1059] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:16.415885 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:16.416617 16007 raft_consensus.cc:2393] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 15: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:16.359570 16002 raft_consensus.cc:2393] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 10: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
W20260709 13:07:16.418488 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.383569 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:16.346418 15855 rpcz_store.cc:276] 0709 13:07:14.969906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.177171 (+1207265us) service_pool.cc:224] Handling call
0709 13:07:16.191810 (+ 14639us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:16.305684 (+113874us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:16.305719 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:16.305761 (+    42us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:16.305802 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.307386 (+  1584us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b54002d0ad4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:16.307435 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:16.307560 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7110,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"spinlock_wait_cycles":260937984}
I20260709 13:07:16.413966 21428 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:16.428362 15932 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:07:16.433063 15937 raft_consensus.cc:1240] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:16.438469 16005 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:16.447439 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:16.448660 15859 raft_consensus.cc:2393] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 16: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:16.449438 20937 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:07:16.450778 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 86898.1us
I20260709 13:07:16.451989 21540 raft_consensus.cc:2749] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:16.454767 15936 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
W20260709 13:07:16.456451 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 68411.8us
I20260709 13:07:16.403378 21367 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:16.418915 21522 consensus_queue.cc:1059] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:07:16.455921 16564 pending_rounds.cc:85] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5: Aborting all ops after (but not including) 10
I20260709 13:07:16.470046 16564 pending_rounds.cc:107] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:07:16.470371 16564 raft_consensus.cc:2889] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:16.476679 15827 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:07:16.482061 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 160822us
W20260709 13:07:16.483562 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.484447 15900 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:16.487623 21490 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:16.487794 15825 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:16.488174 21490 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:16.488611 21490 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:16.489146 15825 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:16.489547 21432 raft_consensus.cc:3055] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:16.489956 21432 raft_consensus.cc:740] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:16.492334 15928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2520) took 1087 ms. Trace:
I20260709 13:07:16.492738 15928 rpcz_store.cc:276] 0709 13:07:15.404543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.421724 (+1017181us) service_pool.cc:224] Handling call
0709 13:07:16.468959 (+ 47235us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:16.491970 (+ 23011us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:16.491998 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:16.492054 (+    56us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:16.492103 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.492157 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:16.492278 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:16.494972 21553 consensus_queue.cc:1059] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:07:16.496119 20934 consensus_queue.cc:1048] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:16.496253 21553 consensus_queue.cc:1048] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:16.496644 21412 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:16.497123 21412 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:16.497475 21412 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:16.497828 21553 raft_consensus.cc:2804] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:16.497757 20934 consensus_queue.cc:1059] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:07:16.503216 15759 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58434 (request call id 344) took 31333 ms (client timeout 15000 ms). Trace:
I20260709 13:07:16.490967 21432 consensus_queue.cc:260] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:16.504606 21432 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:16.509218 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:16.510246 16569 raft_consensus.cc:1240] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:16.511438 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:16.520490 16570 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:16.521293 21266 raft_consensus.cc:3055] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:16.521821 21266 raft_consensus.cc:740] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:16.523877 21367 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:16.523967 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:16.524514 20934 raft_consensus.cc:3055] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:07:16.525048 20934 raft_consensus.cc:740] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:16.526108 20934 consensus_queue.cc:260] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:16.527812 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 49534.6us
I20260709 13:07:16.529016 15929 raft_consensus.cc:1275] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:07:16.478864 21540 consensus_queue.cc:1059] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 13:07:16.522914 21266 consensus_queue.cc:260] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:16.539649 21266 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:16.544742 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:16.545125 15974 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:16.544658 21522 consensus_queue.cc:1059] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 13:07:16.542270 15933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 1110) took 1182 ms. Trace:
W20260709 13:07:16.503707 15759 rpcz_store.cc:269] 0709 13:06:45.169352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.169592 (+   240us) service_pool.cc:224] Handling call
0709 13:07:16.502986 (+31333394us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:59.570732 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:59.571389 (+   657us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:59.571439 (+    50us) write_op.cc:435] Acquired schema lock
0709 13:06:59.571457 (+    18us) tablet.cc:662] Decoding operations
0709 13:06:59.574069 (+  2612us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:59.574179 (+   110us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:59.574208 (+    29us) tablet.cc:685] Acquiring locks for 36 operations
0709 13:06:59.698024 (+123816us) tablet.cc:701] Row locks acquired
0709 13:06:59.698069 (+    45us) write_op.cc:260] PREPARE: finished
0709 13:06:59.698250 (+   181us) write_op.cc:270] Start()
0709 13:06:59.698391 (+   141us) write_op.cc:276] Timestamp: P: 1783602419698189 usec, L: 0
0709 13:06:59.698432 (+    41us) op_driver.cc:348] REPLICATION: starting
0709 13:06:59.707269 (+  8837us) log.cc:844] Serialized 14134 byte log entry
0709 13:07:00.077834 (+370565us) spinlock_profiling.cc:243] Waited 5.04 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e729e0ac 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b
0709 13:07:00.858489 (+780655us) spinlock_profiling.cc:243] Waited 42.9 ms on lock 0x7b1800060040. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb3f3 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd 000000000038f354 000000000038f199 00007f73e729e83a
0709 13:07:03.907256 (+3048767us) spinlock_profiling.cc:243] Waited 9.68 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:07:04.614249 (+706993us) op_driver.cc:464] REPLICATION: finished
0709 13:07:04.681032 (+ 66783us) write_op.cc:301] APPLY: starting
0709 13:07:04.681137 (+   105us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:04.968471 (+287334us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:04.968543 (+    72us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:06.346153 (+1377610us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:06.347018 (+   865us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=72,key_file_lookups=72,delta_file_lookups=72,mrs_lookups=0
0709 13:07:06.347095 (+    77us) write_op.cc:312] APPLY: finished
0709 13:07:06.414684 (+ 67589us) log.cc:844] Serialized 308 byte log entry
0709 13:07:06.654869 (+240185us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:06.655394 (+   525us) write_op.cc:454] Released schema lock
0709 13:07:06.655493 (+    99us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:06.656045 (+   552us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:06.656079 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":807743,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":66505,"apply.run_cpu_time_us":89896,"apply.run_wall_time_us":1974947,"cfile_cache_hit":146,"cfile_cache_hit_bytes":154967,"num_ops":36,"prepare.queue_time_us":252134,"prepare.run_cpu_time_us":33991,"prepare.run_wall_time_us":738651,"raft.queue_time_us":343707,"raft.run_cpu_time_us":16031,"raft.run_wall_time_us":116393,"replication_time_us":4905743,"spinlock_wait_cycles":132539008,"thread_start_us":1055328,"threads_started":5,"wal-append.queue_time_us":899013,"wal-append.run_cpu_time_us":90307,"wal-append.run_wall_time_us":5566167}]]}
I20260709 13:07:16.547128 15825 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:16.548483 15935 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:16.549444 15930 raft_consensus.cc:1240] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 13:07:16.542526 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:16.543336 15976 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:16.547163 15933 rpcz_store.cc:276] 0709 13:07:15.360101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.193638 (+833537us) service_pool.cc:224] Handling call
0709 13:07:16.297877 (+104239us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:16.298608 (+   731us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:16.298637 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:16.299273 (+   636us) log.cc:844] Serialized 26 byte log entry
0709 13:07:16.444769 (+145496us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:16.444832 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.444928 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:16.542008 (+ 97080us) raft_consensus.cc:1685] finished
0709 13:07:16.542029 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:16.542191 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":107,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":84,"thread_start_us":145012,"threads_started":1,"wal-append.queue_time_us":240453}
I20260709 13:07:16.562392 15934 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:16.567286 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:16.512971 21553 raft_consensus.cc:697] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:16.577699 15857 raft_consensus.cc:2393] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 14: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:16.575496 15974 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:16.582345 15900 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:16.582538 21412 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:16.582538 21540 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:16.584446 21540 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:16.584889 21540 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:16.588548 21588 raft_consensus.cc:2764] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:07:16.589990 21427 consensus_queue.cc:1048] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:16.590345 15825 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:16.592041 21593 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:16.592583 21593 raft_consensus.cc:697] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:16.592895 21553 consensus_queue.cc:237] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:16.593595 21593 consensus_queue.cc:237] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:16.579602 21601 raft_consensus.cc:2804] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:16.568420 21478 raft_consensus.cc:3055] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:16.570650 21367 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:16.596704 21367 raft_consensus.cc:697] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:16.593624 21412 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:16.600037 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 53234.1us
I20260709 13:07:16.600152 21522 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:16.562390 21492 raft_consensus.cc:3055] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:07:16.600122 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 56916.4us
I20260709 13:07:16.601733 16569 raft_consensus.cc:2410] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:07:16.596071 21601 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:16.558459 21417 raft_consensus.cc:697] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:16.605981 21167 consensus_queue.cc:1048] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:16.613760 16570 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 15.
I20260709 13:07:16.578699 20934 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:16.614871 15976 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:16.613703 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 203886us
I20260709 13:07:16.596167 21478 raft_consensus.cc:740] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:16.600656 21623 raft_consensus.cc:3055] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:16.615216 21601 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:16.612208 21492 raft_consensus.cc:740] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:16.638147 15902 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:16.606859 21588 consensus_queue.cc:1059] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
W20260709 13:07:16.643432 21597 log.cc:927] Time spent T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.073s	user 0.000s	sys 0.008s
I20260709 13:07:16.624663 20937 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:16.624617 21522 consensus_queue.cc:1059] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:07:16.632547 21623 raft_consensus.cc:740] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:16.652559 21490 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:16.653915 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:16.654589 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 47270.7us
I20260709 13:07:16.657262 15976 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:16.630340 21477 consensus_queue.cc:1059] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 13:07:16.643105 21417 consensus_queue.cc:237] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:16.646391 21485 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:16.661105 15825 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:16.597908 21367 consensus_queue.cc:237] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:16.676903 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:16.677564 16003 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:16.678236 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:16.678814 16010 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:16.679263 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:16.679872 16005 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
W20260709 13:07:16.691988 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.700052 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:16.705240 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:16.705899 15929 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:07:16.705487 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 66125.6us
W20260709 13:07:16.708441 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 103183us
I20260709 13:07:16.709501 16533 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:16.710523 16006 raft_consensus.cc:1275] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:07:16.681857 21360 raft_consensus.cc:2804] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:16.713889 21360 raft_consensus.cc:697] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:16.714780 21360 consensus_queue.cc:237] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:16.720383 21558 consensus_queue.cc:1048] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.079s
I20260709 13:07:16.721220 21490 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:16.722883 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:16.723698 15857 raft_consensus.cc:2368] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 14. Current term is 15.
I20260709 13:07:16.724567 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:16.725945 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:16.727105 21492 consensus_queue.cc:260] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:16.728452 21492 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:16.711629 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:16.778239 16003 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:16.778927 16008 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:16.780529 16010 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:16.781208 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:16.781965 15857 raft_consensus.cc:2368] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 15. Current term is 16.
I20260709 13:07:16.812611 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:16.701545 21477 consensus_queue.cc:1059] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.082s
W20260709 13:07:16.840583 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:16.843835 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.845180 21625 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:16.845674 21625 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:16.846055 21625 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:16.847872 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 64982.8us
W20260709 13:07:16.841809 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 203762us
I20260709 13:07:16.851200 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:16.783401 21167 raft_consensus.cc:2749] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:16.784323 21588 raft_consensus.cc:3055] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:16.708941 21540 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:16.870553 16008 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:16.874329 15929 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:16.876515 16570 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:16.837260 21485 raft_consensus.cc:2749] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:16.707824 21478 consensus_queue.cc:260] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:16.917107 15900 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:16.871093 21588 raft_consensus.cc:740] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:07:16.933736 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 224315us
W20260709 13:07:16.940459 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 283274us
I20260709 13:07:16.935611 21426 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:16.948711 16533 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:16.948681 21426 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:16.957895 21634 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:16.963110 21417 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:16.948915 21454 consensus_queue.cc:1059] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.027s
W20260709 13:07:16.981974 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 141482us
I20260709 13:07:16.977733 21478 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:07:16.979863 21626 log.cc:927] Time spent T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 13:07:16.984440 15902 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:16.983659 21417 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.003053 21636 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.003746 21636 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:17.005738 21624 log.cc:927] Time spent T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.305s	user 0.001s	sys 0.000s
W20260709 13:07:17.007087 21627 log.cc:927] Time spent T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.202s	user 0.000s	sys 0.001s
I20260709 13:07:17.007393 21558 raft_consensus.cc:3055] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:17.007856 21558 raft_consensus.cc:740] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:07:17.008419 21628 log.cc:927] Time spent T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:07:17.008895 21558 consensus_queue.cc:260] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:17.010252 21558 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:07:17.011109 21629 log.cc:927] Time spent T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
W20260709 13:07:17.011667 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 99372us
I20260709 13:07:16.995437 21417 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:16.994320 21588 consensus_queue.cc:260] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:17.018391 21634 consensus_queue.cc:1059] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:07:17.045248 21634 raft_consensus.cc:3055] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:17.045725 21634 raft_consensus.cc:740] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:07:17.064374 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 122987us
W20260709 13:07:17.064266 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 77525.3us
W20260709 13:07:17.087417 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:17.088676 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:17.089527 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 61416.1us
I20260709 13:07:17.036846 20937 raft_consensus.cc:2804] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:07:17.094703 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 104003us
I20260709 13:07:17.096675 15827 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:17.046826 21634 consensus_queue.cc:260] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:17.091082 15902 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:17.094154 16535 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 2.898s (SENT) [suppressed 39 similar messages]
I20260709 13:07:17.100534 15825 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:17.086722 21588 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:17.100807 21652 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:17.102452 21647 log.cc:927] Time spent T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:07:17.102854 16533 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:17.106534 21652 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:17.109051 21652 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:17.111896 21652 consensus_queue.cc:1048] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:17.102846 21642 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:17.098945 20937 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:17.111871 21593 raft_consensus.cc:2749] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:17.105010 21540 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:17.125476 15974 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:17.129401 15827 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:17.117539 21553 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:17.132313 15902 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:17.124930 20937 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:17.137159 21367 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:17.139091 21553 raft_consensus.cc:697] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:17.145192 21636 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:17.155431 21428 consensus_meta.cc:287] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.701s	user 0.011s	sys 0.000s
I20260709 13:07:17.156086 21428 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:17.156196 21362 consensus_queue.cc:1048] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.020s
I20260709 13:07:17.157235 21426 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:17.159641 21644 log.cc:927] Time spent T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
W20260709 13:07:17.160209 21639 log.cc:927] Time spent T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
W20260709 13:07:17.163239 21601 consensus_meta.cc:287] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.507s	user 0.008s	sys 0.009s
I20260709 13:07:17.148345 21642 consensus_queue.cc:1059] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 13:07:17.164160 21553 consensus_queue.cc:237] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:17.165968 21642 raft_consensus.cc:3055] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:17.166410 21642 raft_consensus.cc:740] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:17.147542 21367 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:17.167953 21367 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:17.172252 21642 consensus_queue.cc:260] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:17.173470 21490 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.174191 21490 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:17.175190 20934 consensus_meta.cc:287] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.545s	user 0.011s	sys 0.004s
I20260709 13:07:17.177232 16533 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:17.178687 16533 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:17.179880 21428 raft_consensus.cc:2749] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:17.183705 15855 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:17.186345 15827 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:17.149611 21652 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:17.189087 15827 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:17.190119 21412 raft_consensus.cc:2749] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:17.191879 21652 raft_consensus.cc:697] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 15 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:17.192914 21652 consensus_queue.cc:237] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:17.195019 15902 leader_election.cc:336] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 1.667s (SENT)
I20260709 13:07:17.191915 21362 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:17.196094 16533 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:17.197058 21428 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:17.198565 16533 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:17.199231 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:17.199432 21428 raft_consensus.cc:2749] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:17.153481 21625 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:17.204764 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.206748 21651 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:17.207283 21655 raft_consensus.cc:2749] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:07:17.207571 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 93385.8us
I20260709 13:07:17.208490 15976 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:17.210047 16570 raft_consensus.cc:1240] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:07:17.195477 15902 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:17.219964 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 94628.8us
I20260709 13:07:17.173570 21642 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:17.228819 15827 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:17.230721 21426 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:17.231127 21426 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:17.231518 21426 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:17.231704 15900 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:17.234300 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:17.239713 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:17.245266 16566 raft_consensus.cc:1240] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 16. Current term is 17. Ops: []
W20260709 13:07:17.252736 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.207327 21651 raft_consensus.cc:697] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:07:17.271905 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 73414.7us
I20260709 13:07:17.272912 20937 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:17.273285 16533 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:07:17.275126 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.277308 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:17.278025 16002 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:07:17.279763 16007 raft_consensus.cc:1275] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:07:17.283078 21522 raft_consensus.cc:3055] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:07:17.283607 21657 log.cc:927] Time spent T 00000000000000000000000000000000 P d33cebb189b14a989836214fdbafb898: Append to log took a long time: real 0.115s	user 0.003s	sys 0.000s
I20260709 13:07:17.283543 21522 raft_consensus.cc:740] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:17.283963 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:17.284605 21522 consensus_queue.cc:260] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:17.286127 21522 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:17.287325 21417 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:17.284601 16009 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:17.254309 21490 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:17.163663 21601 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:17.291533 21601 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:17.291790 21655 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:07:17.292757 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 97783.8us
I20260709 13:07:17.293220 21643 raft_consensus.cc:3055] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:17.293640 21643 raft_consensus.cc:740] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:17.295104 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:17.295374 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:17.295993 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:17.296072 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:17.296917 15936 raft_consensus.cc:2368] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 14. Current term is 15.
I20260709 13:07:17.300053 16009 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:17.303083 16009 raft_consensus.cc:1240] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 9. Current term is 10. Ops: []
W20260709 13:07:17.304917 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 96491.1us
I20260709 13:07:17.260507 21625 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:17.262178 21656 raft_consensus.cc:2749] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:17.302675 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:17.297636 21659 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:17.266058 21558 raft_consensus.cc:2804] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:17.315164 15930 raft_consensus.cc:1240] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:17.318337 16007 raft_consensus.cc:1275] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:07:17.324787 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:17.325860 15934 raft_consensus.cc:2393] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 15: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:17.294517 21643 consensus_queue.cc:260] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:17.306705 21312 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.335341 16005 raft_consensus.cc:1275] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:07:17.323169 21417 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:17.341305 15855 raft_consensus.cc:1240] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:17.344122 16533 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:17.321918 21558 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:17.344410 15827 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:17.330606 21634 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:17.348333 21558 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:17.330292 21477 raft_consensus.cc:3055] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:17.351385 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:17.351404 15974 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:17.348660 21362 raft_consensus.cc:2749] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:17.356647 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:17.358851 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:17.351994 21658 raft_consensus.cc:2749] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:17.361265 21666 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:17.363360 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:17.363780 21601 raft_consensus.cc:2804] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:17.364215 21601 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:17.364583 21601 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:17.364778 16003 raft_consensus.cc:1275] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:07:17.365751 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:17.366758 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:17.367763 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:17.368036 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:17.368775 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:17.369058 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:17.369293 16003 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:17.370107 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:17.371099 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:17.362771 15827 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:17.335378 21312 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:17.372054 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:17.375666 16006 raft_consensus.cc:2410] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:07:17.376031 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:17.376497 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:17.377974 16006 raft_consensus.cc:1240] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 14 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:07:17.379105 16006 raft_consensus.cc:1240] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:17.379856 15933 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:07:17.353739 21477 raft_consensus.cc:740] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:17.385740 16003 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
I20260709 13:07:17.390520 16009 raft_consensus.cc:1240] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:17.347741 21367 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:17.359525 21417 raft_consensus.cc:2749] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:17.382458 21266 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:17.395429 16009 raft_consensus.cc:1240] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:17.388166 21643 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:17.405901 16011 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
W20260709 13:07:17.408063 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:17.412043 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.356227 21651 consensus_queue.cc:237] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:17.409775 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 73306us
I20260709 13:07:17.408670 21477 consensus_queue.cc:260] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:17.453603 21477 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:07:17.416123 21641 log.cc:927] Time spent T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.382s	user 0.004s	sys 0.000s
W20260709 13:07:17.420040 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 87251.5us
I20260709 13:07:17.427040 16005 raft_consensus.cc:1275] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:07:17.426999 16010 raft_consensus.cc:1240] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:17.410202 16004 raft_consensus.cc:1240] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:17.430045 21266 raft_consensus.cc:697] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:17.433393 21367 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:17.435323 21312 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:17.441133 21664 log.cc:927] Time spent T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.086s	user 0.002s	sys 0.000s
I20260709 13:07:17.442787 21601 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:07:17.448343 21426 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:17.452019 21369 consensus_queue.cc:1048] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:17.452895 16533 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:17.453398 21666 consensus_queue.cc:1059] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:07:17.396237 16003 raft_consensus.cc:1275] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:07:17.411005 20937 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:17.458050 16002 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:17.458336 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:17.466578 16011 raft_consensus.cc:2393] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 15: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:17.474324 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:17.474428 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:17.475572 16011 raft_consensus.cc:1275] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:07:17.476125 16009 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:17.473336 21367 consensus_queue.cc:1048] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 13:07:17.480890 15974 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:17.483083 15974 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:17.471130 21658 consensus_queue.cc:1048] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 13:07:17.477279 16565 raft_consensus.cc:1275] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:07:17.470224 21519 raft_consensus.cc:2749] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:17.477983 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:17.485581 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:17.479864 21642 consensus_queue.cc:1059] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:17.480418 15859 raft_consensus.cc:1240] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:17.465602 21266 consensus_queue.cc:237] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:17.484030 15902 leader_election.cc:336] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 1.856s (SENT)
I20260709 13:07:17.504115 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:17.504590 15902 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:17.431596 21558 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:17.506860 16003 raft_consensus.cc:3055] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:17.508946 16003 raft_consensus.cc:740] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:17.511010 15930 raft_consensus.cc:1275] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:07:17.497831 21659 raft_consensus.cc:2749] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:17.514847 15934 raft_consensus.cc:1275] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:07:17.516741 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 63962.8us
I20260709 13:07:17.516812 21642 raft_consensus.cc:3055] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:17.517434 21642 raft_consensus.cc:740] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:17.518064 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:17.518311 21642 consensus_queue.cc:260] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:17.521003 15937 raft_consensus.cc:1275] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:07:17.519309 21553 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:17.527456 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:17.527657 16008 raft_consensus.cc:1240] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:17.508494 21522 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:17.535135 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:17.507203 21485 raft_consensus.cc:2804] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:17.510526 20934 raft_consensus.cc:3055] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:07:17.539629 16566 raft_consensus.cc:1275] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 13:07:17.540145 16566 pending_rounds.cc:85] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting all ops after (but not including) 10
I20260709 13:07:17.540515 16566 pending_rounds.cc:107] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:07:17.540715 15930 raft_consensus.cc:2410] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 14 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 13:07:17.540828 16566 raft_consensus.cc:2889] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:17.541352 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:17.542261 16005 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 16.
I20260709 13:07:17.543115 15930 raft_consensus.cc:1275] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:07:17.516052 16003 consensus_queue.cc:260] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:17.545185 15974 leader_election.cc:400] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Vote denied by peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) with higher term. Message: Invalid argument: T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 14. Current term is 15.
I20260709 13:07:17.545524 15974 leader_election.cc:403] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:17.545989 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:17.553537 16535 leader_election.cc:336] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 1.617s (SENT)
I20260709 13:07:17.558748 15827 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:17.558504 15937 raft_consensus.cc:1275] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:07:17.560571 15974 leader_election.cc:400] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 14. Current term is 15.
I20260709 13:07:17.563710 15974 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:17.561383 16535 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:17.537147 21642 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:17.545487 20934 raft_consensus.cc:740] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:17.566156 21669 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:17.569130 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:17.569759 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:17.569912 16004 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:07:17.538379 20937 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:17.567617 16009 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 17.
I20260709 13:07:17.575024 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:17.569404 15974 leader_election.cc:400] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 15. Current term is 16.
I20260709 13:07:17.535756 21593 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:17.574254 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.540205 21485 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:17.577298 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:17.575937 15974 leader_election.cc:403] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:07:17.552464 21478 raft_consensus.cc:2749] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) with higher term. Message: Invalid argument: T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 14. Current term is 15.
I20260709 13:07:17.578195 15860 raft_consensus.cc:1240] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:07:17.578819 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:17.578541 15858 raft_consensus.cc:1240] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:17.576943 15825 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:17.553231 21553 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:17.606957 21553 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:17.581393 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:17.608232 21601 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:17.603551 15861 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
I20260709 13:07:17.611392 15934 raft_consensus.cc:1275] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:07:17.582093 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:17.583640 21593 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:17.584064 21426 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:17.584385 21669 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:17.585126 21648 consensus_queue.cc:1059] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 13:07:17.586046 20934 consensus_queue.cc:260] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:17.565017 21367 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:17.639351 21659 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.658449 21367 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:17.658730 21659 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:17.660861 21367 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:17.660945 21659 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:17.661584 21669 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:17.662506 16566 raft_consensus.cc:1240] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:17.662592 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:07:17.590485 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.663463 16567 raft_consensus.cc:2393] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 15: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
W20260709 13:07:17.592432 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 108436us
I20260709 13:07:17.592046 21623 consensus_queue.cc:260] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:17.664418 21623 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:17.564824 21362 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:17.665408 21362 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:17.572824 21522 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:07:17.598881 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 45392.3us
I20260709 13:07:17.552700 21312 consensus_queue.cc:1059] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:07:17.581185 21417 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:17.600883 21558 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:17.620486 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:17.634439 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:17.636058 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:17.640434 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:07:17.643918 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 65927.1us
I20260709 13:07:17.671553 16566 raft_consensus.cc:3055] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:17.672019 16566 raft_consensus.cc:740] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:17.672703 21454 consensus_queue.cc:1059] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:17.673884 21454 consensus_queue.cc:1048] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:17.674620 16533 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
W20260709 13:07:17.643816 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 66957.4us
I20260709 13:07:17.649732 21588 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.676432 21417 consensus_queue.cc:1048] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:17.676890 15825 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:17.677810 21417 consensus_queue.cc:1048] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:17.679939 20768 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:17.679883 21666 consensus_queue.cc:1048] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:07:17.680258 21427 consensus_queue.cc:1048] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:17.680857 20768 consensus_queue.cc:1048] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:17.681430 21666 consensus_queue.cc:1048] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:17.671578 16564 raft_consensus.cc:2393] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 12: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:17.679483 21417 raft_consensus.cc:2804] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:17.645430 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:17.684810 16009 raft_consensus.cc:2376] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15. Re-sending same reply.
I20260709 13:07:17.684996 21593 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:17.684711 21417 raft_consensus.cc:697] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:17.646142 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:07:17.647912 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 89219.2us
I20260709 13:07:17.579942 21485 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:17.648379 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:07:17.649497 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.649686 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:17.654024 20944 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:17.649631 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:17.654636 16563 raft_consensus.cc:1275] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:07:17.645975 21601 raft_consensus.cc:2749] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 15. Current term is 16.
W20260709 13:07:17.656571 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.568938 21369 raft_consensus.cc:2749] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:17.668298 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:07:17.670936 16535 leader_election.cc:336] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 1.552s (SENT)
I20260709 13:07:17.684000 21362 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:17.675640 16566 consensus_queue.cc:260] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:17.699491 15827 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:17.700990 21553 consensus_queue.cc:1048] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:17.699491 16568 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
W20260709 13:07:17.703389 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 137457us
I20260709 13:07:17.704221 15974 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:17.705734 16569 raft_consensus.cc:2393] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 15: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:17.706377 16535 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:17.709071 15902 leader_election.cc:336] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 1.975s (SENT)
I20260709 13:07:17.710110 21417 consensus_queue.cc:237] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:17.711015 15902 leader_election.cc:336] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 2.075s (SENT)
I20260709 13:07:17.715445 21656 consensus_queue.cc:1059] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:17.716825 21656 raft_consensus.cc:3055] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:17.717197 21656 raft_consensus.cc:740] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:17.718108 16003 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:17.718021 21656 consensus_queue.cc:260] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:17.711426 15902 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:17.721971 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:17.722836 16005 raft_consensus.cc:2393] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 16: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
W20260709 13:07:17.724815 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 53935.1us
I20260709 13:07:17.725335 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:17.725575 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:17.725972 16567 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:17.729637 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:17.730048 16004 raft_consensus.cc:1275] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:07:17.729703 16006 raft_consensus.cc:1240] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 16. Current term is 17. Ops: [14.12-16.13]
I20260709 13:07:17.730075 15974 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:17.676838 21588 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:17.706144 21490 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:17.732475 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:17.735569 15827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.733775 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:17.736660 16566 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:17.737169 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:17.731770 21454 consensus_queue.cc:1048] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 13:07:17.739118 15974 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:17.730796 21519 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:17.740818 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:17.743187 16569 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:07:17.748162 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:17.742901 21312 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:17.737612 21669 raft_consensus.cc:2749] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:17.705855 21426 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:07:17.753018 16566 raft_consensus.cc:1275] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:07:17.698473 21367 consensus_queue.cc:1059] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:07:17.764225 16003 raft_consensus.cc:1275] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 13:07:17.751950 21588 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:17.773217 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.777505 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:17.780220 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:17.788074 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:17.796514 15933 raft_consensus.cc:2410] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 13 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 13:07:17.797633 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:17.798398 15858 raft_consensus.cc:2393] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 14: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:07:17.788257 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 109014us
I20260709 13:07:17.790599 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:17.733629 21651 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:07:17.800482 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 89498.6us
I20260709 13:07:17.783521 21656 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:17.799516 15861 raft_consensus.cc:2410] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:07:17.791466 21478 raft_consensus.cc:2804] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:17.803634 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:17.787985 21312 raft_consensus.cc:697] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 15 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:17.804472 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:17.821038 16011 raft_consensus.cc:2368] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 15. Current term is 16.
I20260709 13:07:17.791487 16003 pending_rounds.cc:85] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting all ops after (but not including) 9
I20260709 13:07:17.821775 16003 pending_rounds.cc:107] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:07:17.822070 16003 raft_consensus.cc:2889] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:17.802498 15854 raft_consensus.cc:1240] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:17.812534 16569 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:17.803818 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:17.811975 21412 consensus_queue.cc:1059] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 13:07:17.813483 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:17.814275 15825 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:17.814549 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:17.826933 16008 raft_consensus.cc:2376] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 17. Re-sending same reply.
I20260709 13:07:17.814788 21362 consensus_queue.cc:1048] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:17.797526 21654 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.818804 21478 raft_consensus.cc:697] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:17.818518 16567 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:17.823156 16005 raft_consensus.cc:2393] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 15: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:17.794840 21427 raft_consensus.cc:3055] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:17.829178 15976 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:17.827180 16004 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 16.
I20260709 13:07:17.836318 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:17.837839 15976 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:17.843822 16533 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:17.844985 21519 raft_consensus.cc:2749] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:17.850152 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:17.850889 16006 raft_consensus.cc:2393] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 15: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:17.853313 15974 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:17.863476 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:17.863366 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:17.866794 16003 raft_consensus.cc:2410] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 18 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:07:17.867023 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:17.868543 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:17.868963 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:17.871446 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:17.806171 21413 consensus_queue.cc:1048] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:07:17.871332 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:17.835013 21427 raft_consensus.cc:740] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:17.877028 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:17.882944 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:17.883119 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:17.883384 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:17.851564 21593 raft_consensus.cc:2749] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:17.850661 21651 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:17.884057 16003 raft_consensus.cc:2393] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 15: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:17.831967 21417 consensus_queue.cc:1059] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 13:07:17.890213 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:17.891340 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:17.848964 21648 raft_consensus.cc:2749] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:17.836962 21654 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:17.855065 21553 consensus_queue.cc:1048] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.034s
W20260709 13:07:17.882982 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 65299us
I20260709 13:07:17.902827 21427 consensus_queue.cc:260] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:17.897970 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:17.904346 21427 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:17.905628 16003 raft_consensus.cc:1240] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:17.908550 16002 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:07:17.908133 21478 consensus_queue.cc:237] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:17.897450 21681 log.cc:927] Time spent T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:07:17.911962 21642 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.912482 21642 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:17.914487 21642 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:17.914680 16569 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
W20260709 13:07:17.916342 21362 consensus_queue.cc:1175] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "85e997ba5003439781ec4f1f8db0c3ee" responder_term: 17 status { last_received { term: 13 index: 11 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 16. Current term is 17. Ops: [14.12-16.13]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:07:17.918735 15902 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:17.921341 16535 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:17.921725 21454 consensus_queue.cc:1059] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:17.922609 16535 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:17.923278 21643 consensus_queue.cc:1048] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:07:17.922561 21454 consensus_queue.cc:1048] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:17.925375 16535 leader_election.cc:304] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:17.915084 21485 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:17.923835 21642 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:17.946990 16535 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:17.897364 21687 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:17.923843 21623 raft_consensus.cc:2749] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:07:17.951728 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 49818.6us
I20260709 13:07:17.963894 15902 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:17.893247 21637 consensus_queue.cc:1048] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:07:17.946419 21655 raft_consensus.cc:2804] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:17.973853 16002 raft_consensus.cc:1275] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 7. (term mismatch)
I20260709 13:07:17.948457 21417 raft_consensus.cc:2804] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:17.946062 21428 raft_consensus.cc:3055] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:17.974318 16002 pending_rounds.cc:85] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting all ops after (but not including) 6
I20260709 13:07:17.978760 16002 pending_rounds.cc:107] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 14.7
I20260709 13:07:17.979025 16002 raft_consensus.cc:2889] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:17.981209 16535 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:17.984896 16002 raft_consensus.cc:1275] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:07:17.986862 16002 raft_consensus.cc:1240] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:17.951833 21687 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:17.966228 21662 raft_consensus.cc:2749] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:07:17.990067 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 151390us
I20260709 13:07:17.989776 15827 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:17.992519 15976 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:17.994256 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:17.974117 21655 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:17.978328 21417 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.995955 15827 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:17.924119 21312 consensus_queue.cc:237] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:17.965569 21413 consensus_queue.cc:1048] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:07:17.982503 21669 raft_consensus.cc:2749] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:17.997792 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:17.977188 21553 consensus_queue.cc:1048] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:07:17.978579 21428 raft_consensus.cc:740] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:17.996078 21643 raft_consensus.cc:2749] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:18.010521 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:18.005128 15976 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:18.006011 15930 raft_consensus.cc:2393] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 11: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:17.992058 21687 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:18.009230 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:18.002197 21417 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:18.028712 21485 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:18.028906 16535 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:18.008703 21648 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:18.037103 21654 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:17.997869 21593 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:18.028394 15933 raft_consensus.cc:1275] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:07:17.999801 21655 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:18.072031 21648 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:18.072521 21648 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:18.078289 21428 consensus_queue.cc:260] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:18.079970 21428 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:17.999423 20937 consensus_queue.cc:1059] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.058s
I20260709 13:07:18.084043 15974 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:18.085144 21659 consensus_queue.cc:1059] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:07:18.084146 20937 raft_consensus.cc:3055] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:18.086431 20937 raft_consensus.cc:740] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:18.087335 20937 consensus_queue.cc:260] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:18.087817 21655 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:18.088613 20937 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:18.096199 20768 raft_consensus.cc:3055] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:18.096062 21417 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:18.096619 20768 raft_consensus.cc:740] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
W20260709 13:07:18.098378 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 93351.7us
I20260709 13:07:18.099349 15976 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:18.099861 15902 leader_election.cc:336] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 1.541s (SENT)
W20260709 13:07:18.103794 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 74981.9us
I20260709 13:07:18.104265 21625 raft_consensus.cc:2749] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:07:18.105326 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 109440us
I20260709 13:07:18.097498 20768 consensus_queue.cc:260] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:18.130199 15974 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:18.128991 16535 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:18.102913 21478 raft_consensus.cc:2749] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:18.112092 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:18.135211 15974 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:18.140564 15974 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:18.137092 16567 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 14.
I20260709 13:07:18.145691 16567 raft_consensus.cc:1240] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:18.108472 21485 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:18.108585 21427 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:18.133348 21522 raft_consensus.cc:3055] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:18.154562 16563 raft_consensus.cc:1275] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 13:07:18.160053 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 60211.9us
I20260709 13:07:18.162698 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:18.163187 16566 raft_consensus.cc:1240] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:07:18.163425 15902 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:18.166178 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.133519 21691 raft_consensus.cc:2749] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:18.178743 15902 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:18.142354 21643 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:18.179256 15929 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:18.148307 21637 raft_consensus.cc:2749] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:18.157215 21426 consensus_queue.cc:1059] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:07:18.194535 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:18.200834 15933 raft_consensus.cc:1240] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:18.151109 21655 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:18.158936 21636 raft_consensus.cc:2749] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:18.198653 15902 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:18.142474 21477 consensus_queue.cc:1048] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:07:18.157225 21522 raft_consensus.cc:740] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:07:18.210847 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 60165.2us
I20260709 13:07:18.212417 16535 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:18.178848 21588 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:18.187911 21427 raft_consensus.cc:697] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:18.202795 21417 leader_election.cc:290] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:18.321683 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.227689 21588 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:18.210665 21656 raft_consensus.cc:2749] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:18.201869 21669 raft_consensus.cc:2749] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:18.196035 21654 raft_consensus.cc:2749] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:18.152217 21601 consensus_queue.cc:1048] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 13:07:18.156615 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:18.154937 21412 consensus_queue.cc:1048] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
W20260709 13:07:18.409650 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.443781 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:18.155063 21266 consensus_queue.cc:1048] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:07:18.196372 21478 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:18.415987 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 213631us
W20260709 13:07:18.427337 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 214977us
W20260709 13:07:18.429055 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.444622 16566 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:18.452091 16535 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:18.407933 21659 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:18.428185 21454 raft_consensus.cc:2804] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:18.434428 21637 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.462062 21656 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.462294 16535 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:18.462568 21427 consensus_queue.cc:237] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:18.535001 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:18.534806 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:18.547533 16008 raft_consensus.cc:2393] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 13: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
W20260709 13:07:18.432225 21674 log.cc:927] Time spent T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.221s	user 0.001s	sys 0.006s
I20260709 13:07:18.468312 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:18.468626 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:18.469053 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:18.550879 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:18.428802 21588 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:18.550894 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:18.552100 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:18.476032 15976 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:07:18.476045 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 307478us
I20260709 13:07:18.473650 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:18.552745 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:18.432962 21519 raft_consensus.cc:3055] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:18.551321 16002 raft_consensus.cc:2393] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 10: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:18.464488 21654 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.557632 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:18.556046 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 415510us
I20260709 13:07:18.562569 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:18.464604 21669 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:07:18.568054 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 111352us
I20260709 13:07:18.568547 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:18.576068 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:18.582767 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:18.583967 16003 raft_consensus.cc:2410] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
W20260709 13:07:18.587558 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.534379 21454 raft_consensus.cc:697] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:18.556217 21656 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:18.598451 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:18.599253 15928 raft_consensus.cc:1275] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 12. (term mismatch)
I20260709 13:07:18.608915 15928 pending_rounds.cc:85] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 11
I20260709 13:07:18.609243 15928 pending_rounds.cc:107] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 12.12
I20260709 13:07:18.609539 15928 raft_consensus.cc:2889] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:18.605271 16008 raft_consensus.cc:1275] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:07:18.465412 21478 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:18.556926 21519 raft_consensus.cc:740] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:18.624136 15855 raft_consensus.cc:1275] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:07:18.625837 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 421351us
I20260709 13:07:18.476735 21659 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:18.626700 15902 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:18.462064 21522 consensus_queue.cc:260] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:18.576064 21654 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:18.638691 16009 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 14.
I20260709 13:07:18.555995 21637 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:18.626698 21601 consensus_queue.cc:1059] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 13:07:18.649097 21654 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:18.649892 16007 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 16.
I20260709 13:07:18.652045 21708 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:18.585573 21669 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:18.655314 21669 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:18.656360 21701 log.cc:927] Time spent T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.228s	user 0.000s	sys 0.001s
I20260709 13:07:18.656265 21519 consensus_queue.cc:260] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:18.660431 21702 log.cc:927] Time spent T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
W20260709 13:07:18.660825 21593 consensus_meta.cc:287] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.598s	user 0.003s	sys 0.007s
W20260709 13:07:18.661955 21428 consensus_meta.cc:287] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.582s	user 0.011s	sys 0.000s
I20260709 13:07:18.643579 21713 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.664986 21478 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:18.665686 21601 consensus_queue.cc:1243] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer eb1e0493af1e46dbb6dec7e25525bab5 log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:07:18.666016 21711 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:18.666235 21651 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:18.666640 21706 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:07:18.678912 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 216696us
W20260709 13:07:18.688457 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 46145.7us
I20260709 13:07:18.695470 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:18.710553 15902 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:18.711392 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:18.696401 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:18.715408 15933 raft_consensus.cc:1275] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:07:18.716704 15929 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 12: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:18.712105 15930 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:07:18.715283 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:18.678387 21593 raft_consensus.cc:2749] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:18.712036 21519 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:18.706624 21651 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:18.724577 20937 consensus_meta.cc:287] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.636s	user 0.004s	sys 0.009s
I20260709 13:07:18.711568 16535 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:18.729719 21648 consensus_meta.cc:287] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.657s	user 0.007s	sys 0.011s
I20260709 13:07:18.721089 21706 raft_consensus.cc:697] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:07:18.734696 21687 consensus_meta.cc:287] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.670s	user 0.013s	sys 0.000s
W20260709 13:07:18.753003 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 277031us
I20260709 13:07:18.729748 21708 raft_consensus.cc:3055] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:07:18.760758 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.741454 21651 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:18.781206 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 68508.9us
I20260709 13:07:18.787400 15902 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:07:18.792584 15827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.800508 15928 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:18.758822 15930 raft_consensus.cc:1275] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 13:07:18.810870 15930 pending_rounds.cc:85] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 8
I20260709 13:07:18.811292 15930 pending_rounds.cc:107] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:07:18.811667 15930 raft_consensus.cc:2889] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:18.652086 21454 consensus_queue.cc:237] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:18.824185 16566 raft_consensus.cc:1240] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:18.703701 21713 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:18.867867 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.757768 21712 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:18.751183 21427 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.935447 16008 raft_consensus.cc:1275] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:07:18.789420 21708 raft_consensus.cc:740] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:18.939389 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:18.940810 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:18.948849 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:18.963867 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.998669 15855 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:07:18.924858 21478 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:19.008272 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 220887us
I20260709 13:07:18.831395 21312 consensus_queue.cc:1048] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.133s
I20260709 13:07:18.763654 21648 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:18.945173 21643 consensus_meta.cc:287] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.751s	user 0.012s	sys 0.000s
I20260709 13:07:18.934828 21654 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.814879 21687 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:19.081821 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 370278us
W20260709 13:07:19.085176 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 324707us
I20260709 13:07:19.085405 16535 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:19.101284 16008 raft_consensus.cc:1240] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:19.007660 21718 consensus_queue.cc:1048] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.082s
I20260709 13:07:19.102429 16005 raft_consensus.cc:1240] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:19.051098 21656 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:19.010810 21427 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:19.084856 21643 raft_consensus.cc:2749] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:07:19.123587 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.124527 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:19.130991 16565 raft_consensus.cc:2410] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:07:19.032536 21478 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:19.136843 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 104691us
I20260709 13:07:19.143254 15974 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:19.105772 21369 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.122722 21360 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.110817 21659 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:19.116459 21637 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:19.155113 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:19.128933 21655 raft_consensus.cc:2749] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:19.097544 21728 consensus_queue.cc:1059] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:07:19.097853 21654 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:19.139559 21656 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:19.165481 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:19.181010 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:19.167435 16565 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 13.
W20260709 13:07:19.180583 15903 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.722s	user 0.004s	sys 0.027s
W20260709 13:07:19.190745 15903 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.004s	sys 0.028s
I20260709 13:07:19.211321 15928 raft_consensus.cc:1275] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:07:19.211828 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 53795us
I20260709 13:07:19.212536 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:19.214064 15928 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
W20260709 13:07:19.214592 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.167060 21656 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:19.190605 21588 consensus_meta.cc:287] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.620s	user 0.010s	sys 0.000s
I20260709 13:07:19.158699 21360 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:19.219347 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:19.228360 15928 raft_consensus.cc:1275] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:07:19.191278 21637 raft_consensus.cc:2749] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:07:19.222505 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 137208us
I20260709 13:07:19.187140 20937 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.244475 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:19.244601 20937 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:19.248190 20937 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:19.156364 21369 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:19.240566 21427 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:19.251407 21369 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:19.223646 21588 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:19.252333 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:19.253328 16565 raft_consensus.cc:2393] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 12: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:19.253522 21588 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:19.233322 21737 consensus_queue.cc:1048] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.084s
I20260709 13:07:19.254421 21747 consensus_queue.cc:1059] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:07:19.233723 21648 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:19.222543 21656 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:19.261613 21360 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:19.263159 21733 log.cc:927] Time spent T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:07:19.129025 21708 consensus_queue.cc:260] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:19.266249 21708 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:19.271433 21755 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:19.270922 21740 consensus_queue.cc:1175] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" responder_term: 16 status { last_received { term: 13 index: 12 } last_committed_idx: 12 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 15. Current term is 16. Ops: [15.13-15.13]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:07:19.272261 21755 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:19.274058 21478 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:07:19.234210 21714 log.cc:927] Time spent T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.423s	user 0.000s	sys 0.001s
I20260709 13:07:19.234501 21687 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:19.236102 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:19.129352 21706 consensus_queue.cc:237] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:19.238943 21654 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:19.240897 21720 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:07:19.243501 21727 log.cc:927] Time spent T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.250s	user 0.001s	sys 0.001s
I20260709 13:07:19.249074 21637 raft_consensus.cc:3055] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:19.224364 15902 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:19.275179 21755 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:19.279250 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:07:19.317247 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.270993 21728 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:07:19.404232 15932 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.011s	sys 0.019s
W20260709 13:07:19.405004 15932 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.012s	sys 0.019s
I20260709 13:07:19.279031 21723 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:19.405576 16565 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
W20260709 13:07:19.441900 15933 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.700s	user 0.027s	sys 0.079s
W20260709 13:07:19.442420 15933 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.027s	sys 0.079s
W20260709 13:07:19.443594 16003 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.009s	sys 0.007s
W20260709 13:07:19.445628 16003 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.009s	sys 0.008s
I20260709 13:07:19.398878 21637 raft_consensus.cc:740] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:19.404024 21718 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:19.444278 16010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2377) took 1303 ms. Trace:
I20260709 13:07:19.419103 21723 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:19.411186 21417 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.003s	sys 0.008s
I20260709 13:07:19.411953 21712 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:19.452915 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 136156us
I20260709 13:07:19.458799 21637 consensus_queue.cc:260] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:19.450611 16010 rpcz_store.cc:276] 0709 13:07:18.140610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.140825 (+   215us) service_pool.cc:224] Handling call
0709 13:07:18.141108 (+   283us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:18.410107 (+268999us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:18.410153 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:18.410760 (+   607us) log.cc:844] Serialized 26 byte log entry
0709 13:07:18.576387 (+165627us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:18.576418 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:18.576493 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:19.417359 (+840866us) raft_consensus.cc:1685] finished
0709 13:07:19.417386 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:19.444203 (+ 26817us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11914,"raft.queue_time_us":54294,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":8374,"thread_start_us":305272,"threads_started":2,"wal-append.queue_time_us":293482}
I20260709 13:07:19.462070 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:19.462575 21656 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:19.479444 21656 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:19.403576 21720 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:19.486858 21720 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:19.487874 21711 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:19.488679 21711 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:19.493738 20934 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:19.496840 21490 raft_consensus.cc:3055] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:19.497280 21490 raft_consensus.cc:740] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:19.498112 21490 consensus_queue.cc:260] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:07:19.499233 21490 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:19.463207 16010 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:19.463428 21741 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.457748 21712 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:19.500401 21741 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:19.454071 21718 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:19.474677 21651 consensus_meta.cc:287] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.660s	user 0.006s	sys 0.003s
I20260709 13:07:19.504359 21651 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:19.507000 21651 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:19.476578 21723 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:19.410692 21728 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:19.511083 21728 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:19.511947 21741 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:19.503520 21718 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:19.516891 21711 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:19.546227 16010 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
I20260709 13:07:19.549814 16535 leader_election.cc:304] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:19.551169 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:19.552104 16010 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
I20260709 13:07:19.555183 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:19.555898 16003 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:07:19.557252 16535 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:07:19.565413 21752 log.cc:927] Time spent T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.065s	user 0.000s	sys 0.002s
I20260709 13:07:19.566825 21737 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.567091 21749 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.547502 21720 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.568118 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:19.555783 21718 raft_consensus.cc:2804] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:19.553032 21734 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.556906 21755 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:19.567431 21737 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:19.571565 21748 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:19.572467 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 351351us
I20260709 13:07:19.574601 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:19.576970 15902 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:19.568728 21519 consensus_meta.cc:287] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.827s	user 0.012s	sys 0.000s
I20260709 13:07:19.578781 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:19.581276 21750 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:19.581445 21760 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:19.567575 21749 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:19.583663 15902 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:19.584169 21749 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:19.584736 21749 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:19.585433 21749 raft_consensus.cc:2749] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:19.585608 21767 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.586143 21767 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:19.588186 21737 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:19.594403 16570 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:19.584995 15902 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:19.569353 21711 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:19.581900 21750 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:19.607486 21266 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.608161 21750 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:19.608060 21266 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:19.576494 21720 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:19.615578 15902 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:19.581568 21748 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:19.618712 21748 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:19.578696 21734 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:19.627861 21757 raft_consensus.cc:2749] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:19.578845 21755 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:19.630764 21757 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:07:19.631791 21634 raft_consensus.cc:2749] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:19.637938 21519 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:19.635025 15902 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:19.638581 21519 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:19.639361 21741 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:19.639967 21741 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:19.640923 21519 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:19.642115 21741 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:19.580739 21718 raft_consensus.cc:697] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:19.644520 21718 consensus_queue.cc:237] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:19.604343 21711 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:19.600445 21767 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:19.648986 21711 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:19.645711 21760 raft_consensus.cc:2749] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:19.650980 21748 raft_consensus.cc:2749] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:19.653925 15902 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:19.645875 21704 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:19.661298 21669 raft_consensus.cc:2749] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:19.664932 15902 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:19.665279 16004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2376) took 1526 ms. Trace:
I20260709 13:07:19.670454 16010 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 16.
I20260709 13:07:19.671361 16007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41662 (request call id 2457) took 1202 ms. Trace:
I20260709 13:07:19.671865 16007 rpcz_store.cc:276] 0709 13:07:18.467362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.572594 (+105232us) service_pool.cc:224] Handling call
0709 13:07:19.669640 (+1097046us) spinlock_profiling.cc:243] Waited 41.1 ms on lock 0x7b54001f77d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:19.669751 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":94499200}
I20260709 13:07:19.674275 15902 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:19.669508 16004 rpcz_store.cc:276] 0709 13:07:18.138990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.139481 (+   491us) service_pool.cc:224] Handling call
0709 13:07:18.154909 (+ 15428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:19.467933 (+1313024us) spinlock_profiling.cc:243] Waited 4.27 ms on lock 0x7b18004ad2e0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e9f504bc 00007f73e729e48b 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72b8455 00007f73ea3a0060 00007f73ea3a0bfb 00007f73ea3ab64b
0709 13:07:19.468083 (+   150us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:19.468108 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:19.468564 (+   456us) log.cc:844] Serialized 26 byte log entry
0709 13:07:19.556465 (+ 87901us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:19.556517 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:19.556617 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:19.664982 (+108365us) raft_consensus.cc:1685] finished
0709 13:07:19.665003 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:19.665206 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":687,"raft.queue_time_us":44848,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":8832,"spinlock_wait_cycles":9811200,"thread_start_us":285159,"threads_started":2,"wal-append.queue_time_us":127843}
I20260709 13:07:19.675647 15902 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:19.673029 21760 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:19.672358 21734 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:19.683104 21775 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:19.679967 21669 raft_consensus.cc:2749] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:19.668716 21704 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:19.695351 16006 raft_consensus.cc:1240] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:19.680212 21723 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.700429 16006 raft_consensus.cc:1240] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:19.702028 15902 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:19.702468 16006 raft_consensus.cc:1275] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:07:19.697415 21558 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:19.677629 21706 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:19.708361 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:19.693151 21728 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:19.706401 15974 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:19.710633 15902 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:19.701575 21723 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:19.717299 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.713204 21728 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:19.713519 21654 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:19.713662 15974 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:19.712045 21734 consensus_queue.cc:1048] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 13:07:19.711968 21760 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:19.711795 21669 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.750276 21669 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:19.752429 21669 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:19.754263 21728 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:19.754959 21728 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.755482 21728 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:19.752705 21775 raft_consensus.cc:2749] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:19.757889 21728 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:19.764223 21760 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:19.764891 21760 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:19.746536 21654 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:19.766885 21654 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:19.767206 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:19.768639 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:19.769322 16003 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 16.
I20260709 13:07:19.770181 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:19.772521 21723 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:19.743016 21748 raft_consensus.cc:2804] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:19.774053 21748 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:19.774472 21748 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:19.746654 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:19.747308 21773 log.cc:927] Time spent T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.075s	user 0.000s	sys 0.006s
I20260709 13:07:19.749477 21757 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:07:19.713979 21312 consensus_queue.cc:1048] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:07:19.777348 15933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2575) took 1070 ms. Trace:
I20260709 13:07:19.704525 21558 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:19.777776 15933 rpcz_store.cc:276] 0709 13:07:18.707552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.728536 (+ 20984us) service_pool.cc:224] Handling call
0709 13:07:18.741448 (+ 12912us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:19.442943 (+701495us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:19.442985 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:19.448482 (+  5497us) log.cc:844] Serialized 26 byte log entry
0709 13:07:19.579544 (+131062us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:19.579574 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:19.579650 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:19.777106 (+197456us) raft_consensus.cc:1685] finished
0709 13:07:19.777130 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:19.777280 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":369070,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":22011,"thread_start_us":747140,"threads_started":2,"wal-append.queue_time_us":125256}
I20260709 13:07:19.784526 21558 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:19.785360 16006 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 15.
W20260709 13:07:19.745334 21774 log.cc:927] Time spent T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:07:19.712882 21706 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:19.774935 21713 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:19.803292 21728 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:07:19.815972 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 102359us
I20260709 13:07:19.817608 21558 consensus_queue.cc:1048] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:07:19.824331 21757 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:19.828866 21757 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:19.829442 21706 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:19.830295 21711 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:19.833640 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.836125 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:19.813885 21760 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:19.837520 16004 raft_consensus.cc:2393] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 16: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:19.831192 21637 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:19.844089 21711 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:19.867805 21744 log.cc:927] Time spent T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.418s	user 0.002s	sys 0.001s
I20260709 13:07:19.836854 21757 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:19.869391 21760 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:19.875783 15826 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.937s	user 0.019s	sys 0.017s
W20260709 13:07:19.876307 15826 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.938s	user 0.019s	sys 0.018s
W20260709 13:07:19.877302 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:19.878163 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.878751 20934 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.879292 20934 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:19.880416 21654 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:19.881374 20934 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:19.881510 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.881954 20934 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.882491 21654 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:19.882419 20934 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:19.883138 21706 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:19.883723 21706 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:19.885839 21706 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:19.886821 21367 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:19.896205 21367 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:19.898895 21367 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:19.893208 21765 log.cc:927] Time spent T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.422s	user 0.003s	sys 0.000s
I20260709 13:07:19.900887 16569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 1233) took 1932 ms. Trace:
I20260709 13:07:19.901108 20934 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:19.902086 16567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1230) took 1080 ms. Trace:
I20260709 13:07:19.902663 16567 rpcz_store.cc:276] 0709 13:07:18.822246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.822416 (+   170us) service_pool.cc:224] Handling call
0709 13:07:18.822854 (+   438us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:18.839062 (+ 16208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:18.926989 (+ 87927us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:18.928694 (+  1705us) log.cc:844] Serialized 26 byte log entry
0709 13:07:19.412673 (+483979us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:19.412750 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:19.412854 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:19.892493 (+479639us) spinlock_profiling.cc:243] Waited 12.6 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:07:19.901667 (+  9174us) raft_consensus.cc:1685] finished
0709 13:07:19.901712 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:19.902009 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40183,"raft.queue_time_us":204978,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":20597,"spinlock_wait_cycles":30645504,"thread_start_us":360189,"threads_started":1,"wal-append.queue_time_us":465288}
I20260709 13:07:19.901328 16569 rpcz_store.cc:276] 0709 13:07:17.968035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:17.969059 (+  1024us) service_pool.cc:224] Handling call
0709 13:07:17.969519 (+   460us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:18.105465 (+135946us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:18.105497 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:18.153435 (+ 47938us) log.cc:844] Serialized 52 byte log entry
0709 13:07:18.372850 (+219415us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:18.674745 (+301895us) log.cc:844] Serialized 14 byte log entry
0709 13:07:18.675026 (+   281us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:19.900467 (+1225441us) spinlock_profiling.cc:243] Waited 30.7 ms on lock 0x7b54007017d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:19.900516 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:19.900569 (+    53us) raft_consensus.cc:1685] finished
0709 13:07:19.900666 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:19.900812 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":304230,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":1875,"raft.queue_time_us":43684,"raft.run_cpu_time_us":295,"raft.run_wall_time_us":17766,"spinlock_wait_cycles":70556928,"thread_start_us":493956,"threads_started":3,"wal-append.queue_time_us":370850,"wal-append.run_cpu_time_us":3031,"wal-append.run_wall_time_us":1173091}
I20260709 13:07:19.906021 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2387) took 1067 ms. Trace:
I20260709 13:07:19.906481 16002 rpcz_store.cc:276] 0709 13:07:18.838428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.923615 (+ 85187us) service_pool.cc:224] Handling call
0709 13:07:18.924026 (+   411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:19.448620 (+524594us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:19.449399 (+   779us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:19.452407 (+  3008us) log.cc:844] Serialized 26 byte log entry
0709 13:07:19.579212 (+126805us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:19.579247 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:19.905585 (+326338us) spinlock_profiling.cc:243] Waited 236 ms on lock 0x7b54001ba154. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:19.905644 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:19.905700 (+    56us) raft_consensus.cc:1685] finished
0709 13:07:19.905717 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:19.905957 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2284,"raft.queue_time_us":62087,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":18956,"spinlock_wait_cycles":541718400,"thread_start_us":327081,"threads_started":2,"wal-append.queue_time_us":129160}
I20260709 13:07:19.886338 21706 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.910117 21706 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:19.913098 21729 log.cc:927] Time spent T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.797s	user 0.004s	sys 0.000s
I20260709 13:07:19.917737 21748 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:19.919359 21706 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:19.930625 21748 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:19.944139 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:19.947739 21759 consensus_queue.cc:1048] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:07:19.950989 15827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.954442 15976 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:19.956827 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:19.957641 16002 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:19.965343 16002 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
I20260709 13:07:19.971258 15855 raft_consensus.cc:1275] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:07:19.973037 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:19.973975 15854 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:07:19.975839 15859 raft_consensus.cc:1275] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:07:19.980506 21755 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:19.980882 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:19.987571 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:19.987910 15974 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:19.988708 15854 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:19.989481 21781 consensus_queue.cc:1048] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:07:19.989722 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:19.990861 21781 raft_consensus.cc:2764] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:07:19.991994 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:19.992820 15860 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:19.993959 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:19.994076 21781 consensus_queue.cc:1048] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:07:19.994478 15860 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:19.996098 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:19.996264 15974 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:19.997148 21781 raft_consensus.cc:2749] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:19.997531 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:19.997954 15974 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:19.998996 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:19.999191 21781 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:19.999581 21781 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:19.999603 15974 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:20.000025 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:20.000771 21782 raft_consensus.cc:2804] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:20.001262 21782 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:20.001643 21782 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:20.002836 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:19.981127 21755 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:19.985419 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:20.006485 21755 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:20.006975 21755 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:07:19.886565 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.984465 16568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1229) took 1539 ms. Trace:
I20260709 13:07:20.000229 21781 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:20.011474 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:20.013746 21782 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:20.015885 21748 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.016129 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:20.016412 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:20.016779 16565 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:20.016489 21748 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:20.008237 16568 rpcz_store.cc:276] 0709 13:07:18.445654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.445857 (+   203us) service_pool.cc:224] Handling call
0709 13:07:18.462931 (+ 17074us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:18.463296 (+   365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:18.463326 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:18.463846 (+   520us) log.cc:844] Serialized 26 byte log entry
0709 13:07:18.708984 (+245138us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:18.709054 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:18.709175 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:19.911209 (+1202034us) spinlock_profiling.cc:243] Waited 6.74 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:07:19.984053 (+ 72844us) raft_consensus.cc:1685] finished
0709 13:07:19.984084 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:19.984383 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":202369,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":111,"spinlock_wait_cycles":15499008,"thread_start_us":244699,"threads_started":1,"wal-append.queue_time_us":541500}
I20260709 13:07:20.018759 21782 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:20.018970 21748 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:20.020241 15900 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:20.021158 21731 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:20.023427 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:20.024049 16563 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:20.024808 15976 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:20.025583 20768 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:20.025995 21755 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:20.023383 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:20.028400 21755 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:20.028467 21781 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:20.030236 21755 raft_consensus.cc:2804] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:20.030665 21755 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:20.030900 21781 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:20.032112 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:20.032814 16568 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:20.028469 15854 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:20.041877 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:20.042629 15857 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:20.053814 15860 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:20.057524 16563 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
W20260709 13:07:20.058092 15825 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.207s	user 0.002s	sys 0.016s
I20260709 13:07:20.031060 21755 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:20.063506 15825 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.212s	user 0.002s	sys 0.017s
I20260709 13:07:20.071173 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:20.071610 21755 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:20.072157 15858 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:07:20.073463 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:20.073988 21755 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:20.074205 15858 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:20.076857 15853 raft_consensus.cc:1240] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:07:20.080627 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:20.081538 15853 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:07:20.082774 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:20.084388 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:20.085232 15853 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:20.090004 21412 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.090922 21412 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:20.093575 21412 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:20.096141 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:20.096992 15928 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:07:20.098213 15928 raft_consensus.cc:1240] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:20.098716 15976 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:20.104291 21266 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:20.135147 21755 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.136433 21755 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:07:20.106181 21636 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:20.108536 21797 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:20.142691 21636 raft_consensus.cc:697] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:20.142771 21797 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:20.145452 21797 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:20.109032 21759 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:20.145963 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:20.146428 21759 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:20.147408 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:20.148985 21759 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:20.143950 21636 consensus_queue.cc:237] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:20.117585 21704 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:20.122275 21362 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.154042 15974 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:20.154106 21362 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:20.154954 21687 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:20.155465 21687 raft_consensus.cc:697] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:20.155951 16008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2389) took 1220 ms. Trace:
I20260709 13:07:20.156416 21362 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:20.156273 21687 consensus_queue.cc:237] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:20.124938 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:20.105598 15825 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:20.113884 20934 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.157706 20934 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:20.162156 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:20.162454 21723 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.162962 21723 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:20.164352 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:20.165227 16007 raft_consensus.cc:2393] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 15: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:20.156301 16008 rpcz_store.cc:276] 0709 13:07:18.935249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.116727 (+181478us) service_pool.cc:224] Handling call
0709 13:07:19.143983 (+ 27256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:19.485165 (+341182us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:19.485195 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:19.485233 (+    38us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:19.485271 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:20.155728 (+670457us) spinlock_profiling.cc:243] Waited 341 ms on lock 0x7b5400306454. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:20.155778 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:20.155899 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":784193280}
I20260709 13:07:20.166144 21723 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:20.167227 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:20.167482 15933 raft_consensus.cc:1240] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:20.167791 16002 raft_consensus.cc:2410] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:07:20.170814 20934 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:20.171767 20934 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:20.172647 16007 raft_consensus.cc:1275] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:07:20.172787 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 67263.8us
I20260709 13:07:20.173738 15825 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:20.173846 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:20.174609 20934 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:20.174652 16008 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 16.
I20260709 13:07:20.178674 21712 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:20.178678 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:20.179627 15861 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:07:20.181509 15974 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:20.182480 21723 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:20.182916 21723 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:20.183288 21723 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:20.154202 15860 raft_consensus.cc:1240] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:07:20.188530 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:20.189383 16010 raft_consensus.cc:2393] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 14: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
W20260709 13:07:20.190224 15902 leader_election.cc:336] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Timed out: RequestConsensusVote RPC to 127.14.103.3:46251 timed out after 1.966s (SENT)
I20260709 13:07:20.190733 15825 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:20.193123 21362 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:20.193647 21362 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:20.194082 21362 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:20.195076 21754 consensus_queue.cc:1048] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:20.201937 15862 raft_consensus.cc:1275] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:07:20.191063 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:20.209250 16007 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
I20260709 13:07:20.209685 21755 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:20.210832 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:20.212941 15854 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:07:20.214241 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:20.222375 15854 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:20.139462 15858 raft_consensus.cc:2410] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:07:20.216207 20768 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:20.224004 20768 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:20.212019 15861 raft_consensus.cc:1275] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:07:20.214495 15974 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:20.219723 15853 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
I20260709 13:07:20.220504 15857 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:07:20.239723 15859 raft_consensus.cc:1240] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 15. Current term is 17. Ops: []
I20260709 13:07:20.240458 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:20.241951 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:20.243582 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:20.244230 15928 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:07:20.245908 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:20.246546 15931 raft_consensus.cc:2410] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 16 because replica has last-logged OpId of term: 15 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:07:20.248154 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:20.248682 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:20.249197 15931 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:20.228132 21642 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:20.249958 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:20.257100 15935 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
I20260709 13:07:20.238142 21756 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:20.250427 21712 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.244640 21636 raft_consensus.cc:2804] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:20.259292 15825 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:20.268491 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:20.269353 15935 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
I20260709 13:07:20.265870 21642 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:20.276749 21642 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:20.276796 21413 consensus_queue.cc:1059] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:20.278128 21791 raft_consensus.cc:3055] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:20.278497 21791 raft_consensus.cc:740] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:20.279263 21791 consensus_queue.cc:260] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:20.280778 21791 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:20.293890 16008 raft_consensus.cc:1275] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:07:20.296741 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:20.297376 15935 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:20.256265 21688 consensus_queue.cc:1048] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:07:20.295850 21636 raft_consensus.cc:697] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:20.270767 21756 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:20.304355 21636 consensus_queue.cc:237] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:20.306104 21756 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:20.306011 15928 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
W20260709 13:07:20.313688 21798 log.cc:927] Time spent T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:07:20.307241 21723 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:20.309029 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:20.285210 21712 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:20.317425 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:20.325394 16010 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 17.
I20260709 13:07:20.322948 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:20.325891 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:20.326537 15936 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:20.318734 15856 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 10.
I20260709 13:07:20.326756 15935 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
I20260709 13:07:20.296394 20934 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:20.336941 15825 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:20.338641 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:20.339854 15936 raft_consensus.cc:2410] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
W20260709 13:07:20.340466 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:20.467733 21742 log.cc:927] Time spent T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 1.058s	user 0.005s	sys 0.000s
I20260709 13:07:20.328748 21756 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.417014 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:20.414214 15827 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:20.477252 21756 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:20.442942 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:20.371958 15974 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:20.380720 15931 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:07:20.483384 15857 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 14.
I20260709 13:07:20.380311 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:20.381050 21804 log.cc:927] Time spent T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 13:07:20.379848 21454 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:20.478883 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:20.520345 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:20.521041 16002 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 10.
I20260709 13:07:20.521173 15862 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 13.
W20260709 13:07:20.442216 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:20.482810 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 68730.3us
I20260709 13:07:20.440492 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:20.399874 15933 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:07:20.361526 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:20.357849 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:20.405522 16567 raft_consensus.cc:1240] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:20.342042 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:20.345813 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:20.408699 15828 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.465s	user 0.020s	sys 0.013s
I20260709 13:07:20.478240 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:20.490670 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 118803us
I20260709 13:07:20.357703 15825 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:20.407058 16569 raft_consensus.cc:1240] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:07:20.426469 21797 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:20.377947 21731 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:20.481519 21756 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:20.505151 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:20.519645 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:20.467339 21687 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:20.521529 21654 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:20.598943 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:20.371973 21623 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:20.599962 15900 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:20.373940 21712 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:20.607738 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:20.610160 15936 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:07:20.611994 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:20.612686 15936 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:20.552299 21454 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:20.600690 15860 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:07:20.465391 21642 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:20.601280 15828 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.676s	user 0.029s	sys 0.021s
I20260709 13:07:20.616106 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:20.451612 21362 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:20.601682 15931 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:07:20.601598 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:20.599022 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:20.357028 21662 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:20.602880 15856 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 14.
I20260709 13:07:20.598920 15935 raft_consensus.cc:2410] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 13 because replica has last-logged OpId of term: 12 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 12.
I20260709 13:07:20.489995 21682 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:20.606022 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:20.627087 15855 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 17.
I20260709 13:07:20.627291 15857 raft_consensus.cc:2393] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 16: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:20.608384 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:20.600258 15974 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:20.374794 21688 raft_consensus.cc:2764] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:07:20.618546 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:20.600872 16010 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:20.618832 15900 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:20.630005 15974 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:20.631083 15900 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:20.619982 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:20.631858 15933 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:20.604151 21654 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:20.636082 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:20.636323 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:20.622139 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 264526us
I20260709 13:07:20.604310 21756 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:20.624552 15936 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:07:20.637995 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:20.639626 15858 raft_consensus.cc:1240] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:20.606979 21731 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:20.640110 15900 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:20.639559 21654 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:20.629051 21682 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:20.626438 20944 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:20.638720 15974 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:20.598182 21797 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:20.645421 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:20.646970 15933 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 10.
I20260709 13:07:20.624972 21642 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:20.665557 15974 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:20.633129 21662 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:20.651124 15853 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 14.
I20260709 13:07:20.643167 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:20.672489 15935 raft_consensus.cc:1275] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:07:20.672684 15936 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:07:20.622347 21623 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:20.607347 21687 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:20.648550 21712 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.659441 21682 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:20.658985 20944 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:20.641382 21636 consensus_queue.cc:1059] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 13:07:20.640920 21756 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:20.675158 21454 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:20.685451 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:20.670246 15974 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:20.649777 15931 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:20.686159 16004 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
W20260709 13:07:20.694873 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:20.695624 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:20.698074 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:20.684442 20944 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:20.700757 21819 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:20.687808 21636 consensus_queue.cc:1243] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer eb1e0493af1e46dbb6dec7e25525bab5 log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:07:20.686825 21712 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:20.709810 21819 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:20.719166 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 49051.3us
I20260709 13:07:20.728884 15974 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
W20260709 13:07:20.732486 21807 log.cc:927] Time spent T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:07:20.732750 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:20.733330 21797 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:20.734485 21642 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:20.735114 21642 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:20.718508 21759 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:20.735810 21642 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:20.736290 21642 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:20.736872 21623 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:20.738636 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:20.739419 16008 raft_consensus.cc:2393] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 13: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:20.740051 21687 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:20.742003 21712 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:20.743777 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:20.746541 21642 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:20.751973 15974 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:20.754290 15974 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:20.753247 21636 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:20.728247 21819 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:20.754208 21687 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:20.832594 21821 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:07:20.834105 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:20.834151 21636 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:20.865310 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:20.870013 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:20.867116 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 112898us
I20260709 13:07:20.840520 21687 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:20.862396 21636 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:20.878168 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:20.881691 21682 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:20.884045 21682 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:20.876415 21687 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:20.878487 21813 log.cc:927] Time spent T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 13:07:20.870837 21662 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:20.875240 21634 raft_consensus.cc:2804] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:20.899194 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:20.900007 15861 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:20.911172 15861 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 14.
I20260709 13:07:20.912546 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:20.885116 21266 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.931824 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:20.891044 21731 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:20.941459 15883 debug-util.cc:398] Leaking SignalData structure 0x7b08002ccec0 after lost signal to thread 14751
I20260709 13:07:20.920830 21634 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:20.934556 21756 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:20.931962 21825 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:20.945561 21731 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:20.945191 21634 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:20.946337 21821 raft_consensus.cc:697] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:20.937054 21266 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:20.966572 21830 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:20.959703 21810 raft_consensus.cc:2749] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:20.958474 21642 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:20.955624 21825 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:20.958774 21731 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:20.982203 21835 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:20.956770 21654 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:20.974947 21827 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.973944 21830 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:20.990099 21266 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:20.993172 21830 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:20.994906 21362 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:21.000399 21700 log.cc:927] Time spent T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 2.563s	user 0.006s	sys 0.003s
I20260709 13:07:20.977874 21825 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:21.008514 15825 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:21.010010 21817 raft_consensus.cc:2764] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:07:20.983855 21731 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:21.017403 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.020105 21821 consensus_queue.cc:237] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:21.018567 21637 consensus_meta.cc:287] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 1.155s	user 0.012s	sys 0.000s
I20260709 13:07:21.027132 21830 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:21.033304 16002 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 12.
I20260709 13:07:21.034389 15929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2577) took 2313 ms. Trace:
I20260709 13:07:21.034834 15929 rpcz_store.cc:276] 0709 13:07:18.721136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.725093 (+  3957us) service_pool.cc:224] Handling call
0709 13:07:18.725417 (+   324us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:18.787031 (+ 61614us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:18.787060 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:18.787699 (+   639us) log.cc:844] Serialized 52 byte log entry
0709 13:07:18.825212 (+ 37513us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:18.826595 (+  1383us) log.cc:844] Serialized 14 byte log entry
0709 13:07:18.829293 (+  2698us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:18.829381 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:21.022607 (+2193226us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b180036fe20. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb3f3 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd 000000000038f354 000000000038f199 00007f73e729e83a
0709 13:07:21.033987 (+ 11380us) raft_consensus.cc:1685] finished
0709 13:07:21.034114 (+   127us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:21.034296 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5398,"prepare.queue_time_us":328798,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":1229,"raft.queue_time_us":690208,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":7908,"spinlock_wait_cycles":34597760,"thread_start_us":98368,"threads_started":4,"wal-append.queue_time_us":318206}
I20260709 13:07:21.038060 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:07:21.038587 16009 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41662 (request call id 2458) took 2571 ms (client timeout 1992 ms). Trace:
W20260709 13:07:21.039004 16009 rpcz_store.cc:269] 0709 13:07:18.467620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.552530 (+ 84910us) service_pool.cc:224] Handling call
0709 13:07:21.038369 (+2485839us) spinlock_profiling.cc:243] Waited 85.2 ms on lock 0x7b54001dd154. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:21.038489 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":195968768}
I20260709 13:07:21.040557 15900 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:21.041428 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:21.042165 15936 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:07:21.021198 21687 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:21.014279 21827 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:21.024927 21654 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:20.984954 21819 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:21.029838 21636 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:21.052444 16566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 1238) took 2218 ms. Trace:
I20260709 13:07:21.061048 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:21.068059 16566 rpcz_store.cc:276] 0709 13:07:18.834125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.834388 (+   263us) service_pool.cc:224] Handling call
0709 13:07:18.834676 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:19.014997 (+180321us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:19.015028 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:19.015068 (+    40us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:19.015103 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:21.052138 (+2037035us) spinlock_profiling.cc:243] Waited 180 ms on lock 0x7b540048a8d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:21.052194 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:21.052351 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":414186240}
I20260709 13:07:21.068668 16008 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:21.046929 15825 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:21.061972 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:21.013980 20944 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:21.045663 16007 raft_consensus.cc:1240] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:21.044219 15900 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:21.032886 21829 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:07:21.078590 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:21.082422 15827 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:21.084921 16004 raft_consensus.cc:1240] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:21.051932 21833 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.088409 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:21.057054 21642 raft_consensus.cc:2804] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:21.091331 16007 raft_consensus.cc:1240] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:21.095398 16008 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 18.
I20260709 13:07:21.066036 21637 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:21.046690 21835 raft_consensus.cc:697] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 14 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:21.053754 21830 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:21.077054 21829 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:21.111205 15827 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:21.080924 21362 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:07:21.121402 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 77282.5us
I20260709 13:07:21.086823 21817 raft_consensus.cc:2749] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:21.124061 21786 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:21.081859 21634 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:21.133273 15974 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:21.138163 15974 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:21.142009 21786 raft_consensus.cc:697] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:21.144910 20944 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:21.144814 21786 consensus_queue.cc:237] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:21.146974 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:21.121857 21362 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:21.147881 15933 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:21.149152 21825 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:21.122597 21829 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:07:21.109815 21266 consensus_queue.cc:1059] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 13:07:21.155617 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:21.156400 21844 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:21.120342 21637 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:21.147924 21362 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:21.150238 21704 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:07:21.176723 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.152119 21655 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.132187 21688 consensus_queue.cc:1059] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.036s
W20260709 13:07:21.186288 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.166416 21830 raft_consensus.cc:2804] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:07:21.195905 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.196030 21830 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:21.196781 21830 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:21.183657 21844 raft_consensus.cc:697] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 14 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:21.196025 21655 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:21.184803 16008 raft_consensus.cc:1240] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:21.109396 21833 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:21.201949 16009 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:07:21.181406 15933 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:07:21.202980 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:21.163502 21821 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:21.203791 15860 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
I20260709 13:07:21.200452 21844 consensus_queue.cc:237] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:21.205487 15825 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:21.208530 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:21.207823 21704 raft_consensus.cc:697] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:21.209872 15933 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:21.214407 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:21.275332 15929 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:21.216559 21839 log.cc:927] Time spent T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:07:21.220207 21827 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:21.224604 15902 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:21.224700 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:21.283273 16566 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:21.225435 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:21.225576 21662 consensus_queue.cc:1059] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:21.226168 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:21.284237 16567 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:07:21.284951 16006 raft_consensus.cc:2393] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 15: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:21.226518 21712 consensus_queue.cc:1048] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:21.206406 21842 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.234965 21710 raft_consensus.cc:3055] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:07:21.236564 16533 connection.cc:256] RPC call timeout handler was delayed by -1.1655673950001528s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21520000000000003s for a response
I20260709 13:07:21.237428 15934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 1155) took 1215 ms. Trace:
I20260709 13:07:21.237735 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:21.239156 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:07:21.240095 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.294966 15932 raft_consensus.cc:2393] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 12: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:21.235836 21838 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.240170 21723 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:21.245919 15937 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 1150) took 3435 ms. Trace:
I20260709 13:07:21.185534 21835 consensus_queue.cc:237] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:21.252013 21833 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:21.253386 21687 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:21.255285 16003 raft_consensus.cc:1240] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:21.318408 16003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2401) took 1018 ms. Trace:
I20260709 13:07:21.258189 21819 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:21.208437 21821 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:21.264916 15859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 1082) took 1199 ms. Trace:
I20260709 13:07:21.323911 15859 rpcz_store.cc:276] 0709 13:07:20.065748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:20.149430 (+ 83682us) service_pool.cc:224] Handling call
0709 13:07:20.231286 (+ 81856us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:20.240206 (+  8920us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:21.264646 (+1024440us) spinlock_profiling.cc:243] Waited 81.5 ms on lock 0x7b54002486d0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:21.264817 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":187482496}
I20260709 13:07:21.203511 21654 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:21.325582 21821 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:21.325371 21654 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:21.312466 21838 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:21.276468 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.219322 21720 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:21.278053 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 72656.4us
I20260709 13:07:21.285506 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:21.287961 16006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41674 (request call id 2394) took 1126 ms. Trace:
I20260709 13:07:21.289290 15933 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 18.
I20260709 13:07:21.331295 16569 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:21.290831 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 66289.2us
I20260709 13:07:21.294868 15857 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:21.294406 15934 rpcz_store.cc:276] 0709 13:07:20.021855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:20.618872 (+597017us) service_pool.cc:224] Handling call
0709 13:07:20.629360 (+ 10488us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:20.669102 (+ 39742us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:20.669158 (+    56us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:20.669217 (+    59us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:20.669265 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:21.237139 (+567874us) spinlock_profiling.cc:243] Waited 39.5 ms on lock 0x7b54003b9054. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:21.237194 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:21.237362 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":90861952}
I20260709 13:07:21.299271 15932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 2583) took 1205 ms. Trace:
I20260709 13:07:21.302604 15929 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:21.293957 21710 raft_consensus.cc:740] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:21.331183 16006 rpcz_store.cc:276] 0709 13:07:20.161105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:21.225920 (+1064815us) service_pool.cc:224] Handling call
0709 13:07:21.287886 (+ 61966us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:21.306046 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:21.269134 21642 raft_consensus.cc:697] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 14 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:21.303476 15937 rpcz_store.cc:276] 0709 13:07:17.810149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:17.811130 (+   981us) service_pool.cc:224] Handling call
0709 13:07:17.811584 (+   454us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:17.826284 (+ 14700us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:07:17.951423 (+125139us) log.cc:844] Serialized 14 byte log entry
0709 13:07:18.144632 (+193209us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:18.145170 (+   538us) log.cc:844] Serialized 52 byte log entry
0709 13:07:18.147008 (+  1838us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:18.147045 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:18.147106 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:20.999857 (+2852751us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:07:21.245473 (+245616us) raft_consensus.cc:1685] finished
0709 13:07:21.245624 (+   151us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:21.245832 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":65640,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":1028,"raft.queue_time_us":1752,"raft.run_cpu_time_us":301,"raft.run_wall_time_us":26590,"spinlock_wait_cycles":23527168,"thread_start_us":255643,"threads_started":2,"wal-append.queue_time_us":207003}
I20260709 13:07:21.313925 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:21.337206 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:21.337764 16567 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:21.338045 16568 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:21.290365 21490 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:07:21.318948 16003 rpcz_store.cc:276] 0709 13:07:20.299590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:21.254691 (+955101us) service_pool.cc:224] Handling call
0709 13:07:21.255059 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:21.318008 (+ 62949us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:21.318323 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:21.323058 21655 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:21.343755 15929 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:21.328955 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:21.329428 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:21.345527 15859 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:21.245949 21731 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:21.347173 15936 raft_consensus.cc:1240] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:07:21.332218 15902 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:21.332317 15933 raft_consensus.cc:1240] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:21.332916 15932 rpcz_store.cc:276] 0709 13:07:20.093420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:21.237421 (+1144001us) service_pool.cc:224] Handling call
0709 13:07:21.299185 (+ 61764us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:21.334607 16563 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:21.316565 21636 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:21.357271 16003 raft_consensus.cc:1240] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:21.323073 21846 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:21.364228 16561 raft_consensus.cc:1275] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:07:21.358642 15902 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:21.366483 15902 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:21.366925 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:21.364295 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:21.368886 16533 connection.cc:256] RPC call timeout handler was delayed by -1.0102278120000392s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23740000000000003s for a response
I20260709 13:07:21.371364 16563 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
W20260709 13:07:21.377869 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 149286us
W20260709 13:07:21.379410 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.380287 16003 raft_consensus.cc:1240] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:07:21.383565 16569 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:07:21.385203 15929 raft_consensus.cc:1275] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:07:21.390021 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:21.371994 15902 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:21.395797 15902 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:21.298415 21842 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:21.400606 21847 raft_consensus.cc:2749] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:21.392287 21819 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:21.381328 21827 raft_consensus.cc:2804] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:21.405810 15976 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:21.371423 21846 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:21.364182 21490 raft_consensus.cc:697] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:21.423916 21490 consensus_queue.cc:237] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:21.430059 15929 raft_consensus.cc:1240] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 14 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:07:21.409889 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:21.431538 16569 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:07:21.404927 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:21.369979 21704 consensus_queue.cc:237] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:21.363878 21754 consensus_queue.cc:1059] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:07:21.444038 15976 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:21.445079 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:21.380184 16535 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:21.420600 16567 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:07:21.421408 21854 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:21.448493 21859 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:21.448313 21854 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:21.451200 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 55504.7us
I20260709 13:07:21.453424 15902 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:21.422021 21312 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:21.455915 21312 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:21.456475 21312 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:21.457283 15902 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:21.422598 21654 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:21.432286 15929 raft_consensus.cc:1240] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:21.461340 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:21.459951 15902 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:21.462594 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:21.464056 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 75786.5us
I20260709 13:07:21.412861 15857 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 15.
I20260709 13:07:21.449064 21859 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:21.395725 21634 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:21.412838 21827 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:21.476580 15900 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:21.392930 21266 consensus_queue.cc:1059] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.037s
I20260709 13:07:21.404913 21833 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:21.481212 21859 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:21.484710 21736 raft_consensus.cc:2749] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:21.490130 15859 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
I20260709 13:07:21.444610 21637 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:21.428071 21846 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:21.502648 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:21.482796 21687 raft_consensus.cc:2804] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:21.505616 15929 raft_consensus.cc:2393] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 15: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:21.508909 16533 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:21.421422 21477 consensus_queue.cc:1048] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.073s
I20260709 13:07:21.522946 21844 raft_consensus.cc:2749] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:21.491477 21827 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:21.523844 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 143788us
I20260709 13:07:21.524914 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:21.540596 16535 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:21.542258 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:21.544503 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 84612.6us
I20260709 13:07:21.498090 21847 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:21.548139 21854 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:21.555629 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:21.509925 21634 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:21.511142 21687 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:21.560215 15974 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:21.562107 21855 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:21.576313 21855 raft_consensus.cc:697] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:21.577661 21855 consensus_queue.cc:237] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:21.563865 15974 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:21.564363 21654 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:21.580862 21654 raft_consensus.cc:697] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:21.581429 21782 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:21.581980 21782 raft_consensus.cc:697] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:21.581672 21654 consensus_queue.cc:237] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:21.582940 21782 consensus_queue.cc:237] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:21.510501 21490 consensus_queue.cc:1059] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 13:07:21.584586 21477 consensus_queue.cc:1048] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:07:21.550340 21736 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:07:21.566541 15883 debug-util.cc:398] Leaking SignalData structure 0x7b08002f99a0 after lost signal to thread 15794
W20260709 13:07:21.567723 21851 log.cc:927] Time spent T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:07:21.569371 20768 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:21.486470 21642 consensus_queue.cc:237] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:21.571235 21522 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:21.588812 15930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 2360) took 2406 ms. Trace:
I20260709 13:07:21.501927 21833 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:21.597066 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:21.591265 16570 raft_consensus.cc:1275] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:07:21.412304 21710 consensus_queue.cc:260] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:21.589254 15930 rpcz_store.cc:276] 0709 13:07:19.183295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.183504 (+   209us) service_pool.cc:224] Handling call
0709 13:07:19.183860 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:19.444316 (+260456us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:07:19.453990 (+  9674us) log.cc:844] Serialized 14 byte log entry
0709 13:07:19.594464 (+140474us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:21.231871 (+1637407us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b2c00010cf8. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e8ca1a11 00007f73e8cb1372 00007f73ea3e3a01 00007f73ea3a843a 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2
0709 13:07:21.232414 (+   543us) log.cc:844] Serialized 26 byte log entry
0709 13:07:21.321581 (+ 89167us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:21.321652 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:21.381831 (+ 60179us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:21.588500 (+206669us) raft_consensus.cc:1685] finished
0709 13:07:21.588598 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:21.588749 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29064,"prepare.queue_time_us":8351,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":401670,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":7275,"spinlock_wait_cycles":24085888,"thread_start_us":433698,"threads_started":3,"wal-append.queue_time_us":338012,"wal-append.run_cpu_time_us":1539,"wal-append.run_wall_time_us":1038871}
I20260709 13:07:21.607883 21736 raft_consensus.cc:697] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 15 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:21.608018 15930 raft_consensus.cc:1240] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 16. Current term is 17. Ops: []
W20260709 13:07:21.608351 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:21.609000 21848 log.cc:927] Time spent T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.415s	user 0.004s	sys 0.000s
I20260709 13:07:21.608904 21736 consensus_queue.cc:237] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:21.611263 15900 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:21.611557 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:21.612396 15930 raft_consensus.cc:2393] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 14: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:21.613184 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:21.613881 15930 raft_consensus.cc:2393] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 15: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:21.614612 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:21.615023 16568 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:21.616006 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:21.617359 15930 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
W20260709 13:07:21.600741 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 135907us
W20260709 13:07:21.565910 21852 log.cc:927] Time spent T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:07:21.591353 21687 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:21.625996 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:21.626992 15930 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 12.
W20260709 13:07:21.638072 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 129267us
W20260709 13:07:21.638907 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 98405.1us
I20260709 13:07:21.639294 21871 raft_consensus.cc:2764] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 13:07:21.640277 21871 consensus_queue.cc:1048] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:21.640661 16535 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:21.642455 21871 consensus_queue.cc:1048] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:21.644229 21871 consensus_queue.cc:1048] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:21.645941 21871 consensus_queue.cc:1048] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:21.621487 21847 raft_consensus.cc:2804] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:07:21.629066 21838 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:21.657711 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:21.682699 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:21.683005 15930 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 12.
I20260709 13:07:21.660513 21759 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:07:21.662375 21840 log.cc:927] Time spent T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.442s	user 0.002s	sys 0.000s
I20260709 13:07:21.670258 15860 raft_consensus.cc:1275] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:07:21.672561 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:21.619776 21312 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:21.689093 15859 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:07:21.675962 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:21.648461 21854 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:21.652519 21731 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:21.695817 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:21.696584 15858 raft_consensus.cc:2376] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15. Re-sending same reply.
I20260709 13:07:21.697063 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:21.680092 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:21.680967 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:21.682365 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:21.683995 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:21.686508 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:21.693847 21873 consensus_queue.cc:1059] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:21.704198 21867 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:21.704751 21867 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:07:21.706868 21867 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
I20260709 13:07:21.710482 21859 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:21.650133 21830 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:07:21.701469 15861 raft_consensus.cc:2410] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:07:21.702342 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:21.650919 21754 consensus_queue.cc:1059] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.037s
I20260709 13:07:21.672299 21847 raft_consensus.cc:697] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 12 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:21.703334 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:21.688138 21759 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:21.716039 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:21.698721 21854 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:21.687311 21868 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:21.683139 21362 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:21.719189 15855 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 12.
I20260709 13:07:21.714141 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:21.728971 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:21.721256 15857 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:21.727273 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:21.729094 15854 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 16.
I20260709 13:07:21.729988 15861 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
W20260709 13:07:21.720778 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 102681us
I20260709 13:07:21.728025 21854 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:21.734445 21858 raft_consensus.cc:3055] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:21.734822 21858 raft_consensus.cc:740] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:21.711078 21871 consensus_queue.cc:1059] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 13:07:21.741355 15855 raft_consensus.cc:1240] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:07:21.733989 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:21.733963 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:21.753206 15862 raft_consensus.cc:1240] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:07:21.746088 15857 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
I20260709 13:07:21.732779 21873 raft_consensus.cc:3055] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:21.749094 16533 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:21.728180 21868 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:21.765092 16533 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:21.770010 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:07:21.773118 21829 consensus_meta.cc:287] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.603s	user 0.006s	sys 0.005s
I20260709 13:07:21.761512 21873 raft_consensus.cc:740] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:21.799732 21827 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:21.810474 21829 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:21.202673 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:21.838021 21873 consensus_queue.cc:260] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:21.838804 21827 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:21.838950 21829 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:21.812202 21312 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:21.815168 21651 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.816277 21830 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:21.840936 16007 raft_consensus.cc:2393] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 11: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:21.818835 21854 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:21.820572 21847 consensus_queue.cc:237] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:21.821759 21819 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:21.763114 21868 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:21.827907 21833 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:21.828617 21859 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:21.806874 21846 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:21.830556 21362 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:21.833324 15756 catalog_manager.cc:5697] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 7 to 10, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 10 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:07:21.769972 21871 consensus_queue.cc:1048] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 13:07:21.835770 20944 raft_consensus.cc:3055] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:21.735575 21858 consensus_queue.cc:260] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:21.839249 21873 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:21.773149 21867 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:21.859331 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:21.868134 15929 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 16.
I20260709 13:07:21.874485 21426 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:21.877429 21426 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:21.881330 21426 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:21.883749 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 118870us
I20260709 13:07:21.884604 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:21.884851 16533 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:21.856477 21651 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:21.870033 20944 raft_consensus.cc:740] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:21.899127 16533 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:21.899607 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:21.900117 16533 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:21.901465 15937 raft_consensus.cc:2393] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 17: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:21.891690 21855 raft_consensus.cc:2749] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:21.907562 21855 raft_consensus.cc:2749] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:21.907914 21854 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:21.908440 21855 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:21.908649 21854 raft_consensus.cc:2804] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:21.908983 21855 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:21.909246 21854 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:21.909531 21855 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:21.909714 21854 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:21.911826 15937 raft_consensus.cc:1240] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:21.914687 20944 consensus_queue.cc:260] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:21.915884 20944 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:21.882405 21867 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:21.890924 21662 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:21.921181 16011 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 13.
I20260709 13:07:21.898626 21858 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:21.922317 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 80990.9us
I20260709 13:07:21.926403 21867 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:21.928933 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:21.936478 15859 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:21.930752 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:21.930915 21855 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:21.942278 15930 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:07:21.942317 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:21.944919 15856 raft_consensus.cc:1275] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:07:21.947860 16005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2379) took 3794 ms. Trace:
I20260709 13:07:21.948662 15974 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:21.949473 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:07:21.950520 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 105451us
I20260709 13:07:21.948377 16005 rpcz_store.cc:276] 0709 13:07:18.149668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.149866 (+   198us) service_pool.cc:224] Handling call
0709 13:07:18.150199 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:19.117135 (+966936us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:21.933266 (+2816131us) spinlock_profiling.cc:243] Waited 948 ms on lock 0x7b54001dc754. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:21.944217 (+ 10951us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2180441984}
I20260709 13:07:21.953337 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:21.953414 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:21.955220 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:21.957392 15856 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 15.
I20260709 13:07:21.958212 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:21.958791 15856 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:21.946684 21796 raft_consensus.cc:3055] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:07:21.947315 21786 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.716s	user 0.007s	sys 0.022s
I20260709 13:07:21.949119 21601 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.955060 21782 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:21.964893 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:21.960570 15859 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:21.961375 15974 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:21.961684 21796 raft_consensus.cc:740] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:21.961741 21312 consensus_queue.cc:1048] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 13:07:21.951893 21833 consensus_queue.cc:1059] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 13:07:21.961210 21830 consensus_queue.cc:1059] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
W20260709 13:07:21.964318 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 64284.4us
I20260709 13:07:21.967604 21601 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:22.028393 21601 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:21.972517 15974 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:22.029769 21601 raft_consensus.cc:2749] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:07:22.029911 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 57475.8us
I20260709 13:07:22.030715 15974 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:22.031404 21601 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:22.032071 15974 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:22.032853 21601 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:22.033332 15935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2588) took 1741 ms. Trace:
I20260709 13:07:22.033403 21601 raft_consensus.cc:697] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:21.973456 21881 consensus_queue.cc:1059] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.060s
I20260709 13:07:22.009398 15902 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:22.010749 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:22.010416 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.036827 15858 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
I20260709 13:07:21.995134 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:07:22.003403 16011 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41662 (request call id 2462) took 3530 ms (client timeout 1966 ms). Trace:
I20260709 13:07:22.033784 15935 rpcz_store.cc:276] 0709 13:07:20.292173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:20.641247 (+349074us) service_pool.cc:224] Handling call
0709 13:07:20.652449 (+ 11202us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:20.672872 (+ 20423us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:22.033102 (+1360230us) spinlock_profiling.cc:243] Waited 19.7 ms on lock 0x7b540034ba54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:22.033276 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":45340928}
I20260709 13:07:22.039906 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.045014 15860 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 18.
W20260709 13:07:22.037794 16011 rpcz_store.cc:269] 0709 13:07:18.473772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.551863 (+ 78091us) service_pool.cc:224] Handling call
0709 13:07:22.003204 (+3451341us) spinlock_profiling.cc:243] Waited 3.37 s on lock 0x7b54000aa5d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:22.003330 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7746737536}
I20260709 13:07:22.024964 16533 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:22.052012 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.052430 15856 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 18.
I20260709 13:07:21.997128 21759 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:22.054345 21588 raft_consensus.cc:2749] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:22.001889 21651 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:22.010459 21884 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.055749 21601 consensus_queue.cc:237] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:21.995242 21782 raft_consensus.cc:2749] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:22.056228 21884 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:21.999604 21846 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:22.060070 21884 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:22.056067 21873 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.995972 21712 raft_consensus.cc:3055] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:22.070511 16533 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:22.071374 15827 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:21.985255 21855 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:21.979190 21796 consensus_queue.cc:260] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:21.980885 21833 raft_consensus.cc:3055] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:22.104079 16569 raft_consensus.cc:1275] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:07:22.095115 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:07:21.963992 21786 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.008s	sys 0.022s
I20260709 13:07:22.069917 21553 raft_consensus.cc:3055] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:21.972389 21814 raft_consensus.cc:2749] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:21.998910 21691 raft_consensus.cc:3055] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:21.983062 21687 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:22.100037 21884 consensus_queue.cc:1059] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:07:22.127521 21884 consensus_queue.cc:1243] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer 8430db80a39d4868bcf7641c16de35ea log is divergent from this leader: its last log entry 14.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:07:22.128067 21884 raft_consensus.cc:2749] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:22.128654 21884 raft_consensus.cc:3055] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:22.128736 21687 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:22.129036 21884 raft_consensus.cc:740] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:22.130851 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.055786 21759 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:22.010048 21888 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:22.106801 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.013240 21655 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:22.129907 21884 consensus_queue.cc:260] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:22.060966 21712 raft_consensus.cc:740] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:22.121968 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.098817 21881 consensus_queue.cc:1059] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:07:22.123374 21691 raft_consensus.cc:740] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:22.132843 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.012210 21868 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:22.143652 21868 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:22.144899 21879 log.cc:927] Time spent T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.251s	user 0.001s	sys 0.000s
I20260709 13:07:22.153299 15825 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:22.156697 21712 consensus_queue.cc:260] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:22.158210 21712 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:22.160476 21881 raft_consensus.cc:3055] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:22.160727 21710 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:22.009594 21854 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:22.161433 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.136387 21655 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:22.172812 21655 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:22.111531 21553 raft_consensus.cc:740] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:22.174115 21553 consensus_queue.cc:260] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:22.133857 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.175515 16564 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:07:22.169533 21816 raft_consensus.cc:3055] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:22.176241 21816 raft_consensus.cc:740] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:22.176819 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:22.177040 21816 consensus_queue.cc:260] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:22.178130 21816 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:22.137260 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.179214 16561 raft_consensus.cc:2393] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 17: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:22.179103 21846 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:22.179746 21846 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:22.180055 21691 consensus_queue.cc:260] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:22.180722 21688 raft_consensus.cc:2749] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:22.181322 21691 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:22.181906 21846 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:22.182751 16004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2408) took 1139 ms. Trace:
W20260709 13:07:22.183310 15883 debug-util.cc:398] Leaking SignalData structure 0x7b08002ae260 after lost signal to thread 15886
W20260709 13:07:22.183637 15883 debug-util.cc:398] Leaking SignalData structure 0x7b0800310000 after lost signal to thread 15887
W20260709 13:07:22.185101 15883 debug-util.cc:398] Leaking SignalData structure 0x7b08001c9860 after lost signal to thread 15961
I20260709 13:07:22.139822 16569 raft_consensus.cc:2393] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 16: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
W20260709 13:07:22.185413 15883 debug-util.cc:398] Leaking SignalData structure 0x7b08001c9880 after lost signal to thread 15962
W20260709 13:07:22.122846 15928 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 2371) took 1864 ms (client timeout 1876 ms). Trace:
I20260709 13:07:22.186124 16010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2404) took 1876 ms. Trace:
W20260709 13:07:22.186081 15928 rpcz_store.cc:269] 0709 13:07:20.243795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:20.247921 (+  4126us) service_pool.cc:224] Handling call
0709 13:07:22.108064 (+1860143us) spinlock_profiling.cc:243] Waited 57.2 ms on lock 0x7b54001a81d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:22.108245 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":131592064}
I20260709 13:07:22.187901 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60156 (request call id 1120) took 1811 ms. Trace:
I20260709 13:07:22.183094 16004 rpcz_store.cc:276] 0709 13:07:21.043006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:21.043132 (+   126us) service_pool.cc:224] Handling call
0709 13:07:21.056076 (+ 12944us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:21.090973 (+ 34897us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:22.182533 (+1091560us) spinlock_profiling.cc:243] Waited 25.1 ms on lock 0x7b54000f0354. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:22.182693 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":57804672}
I20260709 13:07:22.188284 16002 rpcz_store.cc:276] 0709 13:07:20.376524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:20.636110 (+259586us) service_pool.cc:224] Handling call
0709 13:07:22.187767 (+1551657us) spinlock_profiling.cc:243] Waited 396 ms on lock 0x7b540025a8d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:22.187850 (+    83us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":911562624}
W20260709 13:07:22.189105 15883 debug-util.cc:398] Leaking SignalData structure 0x7b08002f8fc0 after lost signal to thread 16035
W20260709 13:07:22.189421 15883 debug-util.cc:398] Leaking SignalData structure 0x7b08004170a0 after lost signal to thread 16036
W20260709 13:07:22.190605 15883 debug-util.cc:398] Leaking SignalData structure 0x7b080015ed00 after lost signal to thread 16594
W20260709 13:07:22.190912 15883 debug-util.cc:398] Leaking SignalData structure 0x7b08001e0920 after lost signal to thread 16595
I20260709 13:07:22.196316 15974 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:22.196729 21869 raft_consensus.cc:3055] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:22.197163 21869 raft_consensus.cc:740] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:22.198112 21720 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.197937 21869 consensus_queue.cc:260] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:22.198598 21720 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:22.198963 21705 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:22.198992 21869 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:22.186661 16010 rpcz_store.cc:276] 0709 13:07:20.309864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:20.630629 (+320765us) service_pool.cc:224] Handling call
0709 13:07:20.630882 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:20.695955 (+ 65073us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:07:20.755038 (+ 59083us) log.cc:844] Serialized 14 byte log entry
0709 13:07:20.960314 (+205276us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:20.960365 (+    51us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:20.960415 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:22.185917 (+1225502us) spinlock_profiling.cc:243] Waited 64.8 ms on lock 0x7b54003ed854. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:22.185957 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:22.186061 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":157173,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":40,"raft.queue_time_us":181385,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":7714,"spinlock_wait_cycles":149093632,"thread_start_us":254126,"threads_started":3,"wal-append.queue_time_us":190464,"wal-append.run_cpu_time_us":1401,"wal-append.run_wall_time_us":1020155}
I20260709 13:07:22.124856 21817 raft_consensus.cc:2804] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:22.204897 21817 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:22.205272 21817 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:07:22.206377 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.160893 21881 raft_consensus.cc:740] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:22.208745 21881 consensus_queue.cc:260] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:22.210268 21881 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:22.163252 21854 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:22.102732 21833 raft_consensus.cc:740] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:22.230163 21833 consensus_queue.cc:260] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:22.231419 21833 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:22.231822 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.232447 16569 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:07:22.169365 16568 raft_consensus.cc:1240] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:22.233213 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.234216 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.235127 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:22.235440 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:22.235908 15928 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:22.236532 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:22.137719 21888 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:22.092247 21871 consensus_queue.cc:1059] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:07:22.184703 21688 consensus_queue.cc:1048] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:22.216950 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:22.245014 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.199518 21705 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:22.253839 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.254462 16563 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 16.
W20260709 13:07:22.120079 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 48792.1us
W20260709 13:07:22.221309 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.260567 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.262661 15933 raft_consensus.cc:1240] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 16. Current term is 17. Ops: []
W20260709 13:07:22.264707 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.060973 21873 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:22.175575 21553 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:22.227065 21847 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:07:22.237048 15902 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.103.4:43643 timed out after 1.840s (SENT) [suppressed 35 similar messages]
W20260709 13:07:22.237746 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.279389 16566 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 17.
I20260709 13:07:22.280750 16011 raft_consensus.cc:1240] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 17. Current term is 18. Ops: [14.12-17.13]
I20260709 13:07:22.280932 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:22.281548 16009 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:22.281612 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:22.282850 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:22.283547 16005 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:22.286250 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:22.287273 16003 raft_consensus.cc:2368] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 15. Current term is 16.
I20260709 13:07:22.290390 15928 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:22.105639 16533 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:22.235167 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.292344 16569 raft_consensus.cc:2393] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 15: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:22.266551 16561 raft_consensus.cc:1240] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:22.260082 21705 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:22.270412 16570 raft_consensus.cc:3055] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:07:22.292598 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 187036us
I20260709 13:07:22.294620 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:22.295161 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:22.226941 21825 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:22.294808 16570 raft_consensus.cc:740] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:22.240701 21814 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:22.296073 16003 raft_consensus.cc:2393] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 17: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:22.296506 16533 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:07:22.298921 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.252471 21884 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:22.301364 15902 leader_election.cc:400] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 15. Current term is 16.
W20260709 13:07:22.301915 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.309577 15902 leader_election.cc:403] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:07:22.302536 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 43519us
I20260709 13:07:22.312547 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:22.303471 16570 consensus_queue.cc:260] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:22.320770 16533 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:22.321153 16570 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:22.321763 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:22.324744 16005 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 16.
I20260709 13:07:22.327566 16010 raft_consensus.cc:1240] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:22.326859 15902 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:22.326756 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:22.333205 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.333948 16568 raft_consensus.cc:2368] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 16. Current term is 17.
I20260709 13:07:22.294489 21871 raft_consensus.cc:2749] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:07:22.343393 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.339933 16533 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:22.337200 21867 raft_consensus.cc:2804] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:22.347633 21842 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:22.348297 21818 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.348637 21854 raft_consensus.cc:3055] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:22.349139 21854 raft_consensus.cc:740] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:22.348786 21818 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:22.350068 21854 consensus_queue.cc:260] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:22.351261 21854 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:22.324448 21846 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:07:22.373852 15862 raft_consensus.cc:1275] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 13:07:22.375118 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.364573 15932 raft_consensus.cc:1240] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:07:22.366719 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:22.367439 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.325203 21367 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.377975 16566 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
I20260709 13:07:22.384653 15862 raft_consensus.cc:1275] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:07:22.367835 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.372670 16533 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:22.368000 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:22.384402 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.362534 15976 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:22.365470 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:22.386261 16569 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 16.
I20260709 13:07:22.386423 15937 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:22.382578 16009 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 17.
I20260709 13:07:22.322273 21759 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:07:22.358848 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.367102 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.331755 21869 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:22.415982 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:22.418987 15859 raft_consensus.cc:1275] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:07:22.304919 21847 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:22.423224 21849 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.775s	user 0.005s	sys 0.022s
W20260709 13:07:22.423693 21849 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.878s	user 0.005s	sys 0.023s
I20260709 13:07:22.428412 21888 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:22.351631 21818 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:22.417485 21759 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:22.433681 21759 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:22.435204 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:22.435875 15937 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:07:22.414301 21367 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:22.385249 21880 log.cc:927] Time spent T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.408s	user 0.003s	sys 0.000s
I20260709 13:07:22.437624 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.442139 16566 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 13.
I20260709 13:07:22.437925 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:22.328019 21814 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:22.449968 21873 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:22.451052 21814 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:22.453169 21847 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:22.321013 21786 consensus_queue.cc:1059] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 13:07:22.388402 16561 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
I20260709 13:07:22.456166 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:22.414273 21601 raft_consensus.cc:2804] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:22.388571 21720 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:22.458760 16009 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 14.
I20260709 13:07:22.463178 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.465052 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.473435 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:22.466526 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.466836 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:22.476958 16566 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:07:22.477138 16569 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:22.387295 21854 raft_consensus.cc:2749] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:22.417222 21867 raft_consensus.cc:697] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 16 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:22.384359 21828 consensus_queue.cc:1048] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.085s
I20260709 13:07:22.467365 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:22.501767 15859 raft_consensus.cc:2393] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 14: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:22.468422 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.462527 15902 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:22.503046 16568 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
I20260709 13:07:22.471817 16562 raft_consensus.cc:2410] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 13:07:22.356035 21850 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:22.457788 16570 raft_consensus.cc:1275] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 16 index: 13. (term mismatch)
I20260709 13:07:22.513808 16570 pending_rounds.cc:85] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting all ops after (but not including) 12
I20260709 13:07:22.514259 16570 pending_rounds.cc:107] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 13:07:22.514544 16570 raft_consensus.cc:2889] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:22.461181 15862 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:07:22.515815 15862 raft_consensus.cc:606] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Received request to transfer leadership to TS 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:22.487955 16561 raft_consensus.cc:2410] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:07:22.371995 21872 consensus_queue.cc:1059] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.097s
I20260709 13:07:22.460245 21846 raft_consensus.cc:2749] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 15. Current term is 16.
I20260709 13:07:22.514859 15934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 1158) took 1130 ms. Trace:
I20260709 13:07:22.495463 21786 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.544710 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.505553 16569 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 13.
W20260709 13:07:22.510713 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.421814 21756 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.557479 21756 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:22.561368 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:22.562026 16561 raft_consensus.cc:2376] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 17. Re-sending same reply.
I20260709 13:07:22.562881 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:22.563942 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:22.566736 21867 consensus_queue.cc:237] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:22.567595 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:22.567762 21786 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:22.568253 15862 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 16.
I20260709 13:07:22.568856 21796 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:22.544740 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.569875 16563 raft_consensus.cc:2393] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 15: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:22.570791 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.571388 16570 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 16.
I20260709 13:07:22.572247 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.572427 21367 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:22.572769 16570 raft_consensus.cc:2376] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 13. Re-sending same reply.
I20260709 13:07:22.573531 21817 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:22.470115 21601 raft_consensus.cc:697] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:22.576347 21601 consensus_queue.cc:237] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:22.577778 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.578416 16567 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 13.
W20260709 13:07:22.583266 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 210697us
I20260709 13:07:22.584087 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:22.584389 16533 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:22.584802 15930 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:22.585507 16533 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:22.588011 21706 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:22.588971 21849 raft_consensus.cc:2804] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:22.589421 21849 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:22.589793 21849 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:22.591161 21869 raft_consensus.cc:2804] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:22.591706 21869 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:22.592043 21869 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:07:22.593531 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.593195 21705 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:22.594287 16008 raft_consensus.cc:1240] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:22.595600 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:22.596064 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:22.596419 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:22.596901 21872 raft_consensus.cc:3055] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:22.597290 21872 raft_consensus.cc:740] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:22.597514 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:22.598017 21705 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:22.598282 16005 raft_consensus.cc:2393] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 18: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:22.596360 16010 raft_consensus.cc:2368] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 13. Current term is 14.
I20260709 13:07:22.600847 21786 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:22.601748 21817 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:22.556829 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:22.602824 16569 raft_consensus.cc:2393] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 15: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:22.603645 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:22.604717 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:22.604876 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.605348 16003 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:22.606091 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.607076 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:22.607313 16003 raft_consensus.cc:1240] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:22.609386 16005 raft_consensus.cc:1240] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:22.611416 16010 raft_consensus.cc:1240] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:22.598834 21662 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:22.615859 21846 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:22.598078 21872 consensus_queue.cc:260] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:22.541147 21847 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:07:22.630522 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.630489 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:22.630890 16535 leader_election.cc:400] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 13. Current term is 14.
I20260709 13:07:22.631256 15854 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:22.631366 16535 leader_election.cc:403] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:07:22.621687 15900 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:22.622016 15902 leader_election.cc:400] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 16. Current term is 17.
I20260709 13:07:22.634754 15900 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:22.625667 16005 raft_consensus.cc:1275] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:07:22.568907 21756 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:22.626212 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.628029 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:22.637496 15861 raft_consensus.cc:2393] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 15: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:22.629878 15859 raft_consensus.cc:1240] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 14. Current term is 15. Ops: [13.10-14.11]
I20260709 13:07:22.638979 16567 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
I20260709 13:07:22.541739 15934 rpcz_store.cc:276] 0709 13:07:21.384227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:21.445860 (+ 61633us) service_pool.cc:224] Handling call
0709 13:07:21.477559 (+ 31699us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:21.477766 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:07:21.496511 (+ 18745us) log.cc:844] Serialized 14 byte log entry
0709 13:07:22.180231 (+683720us) spinlock_profiling.cc:243] Waited 4.69 ms on lock 0x7b180040dec0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f4823a 00007f73ea3b2a2a
0709 13:07:22.493015 (+312784us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:22.493074 (+    59us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:22.514549 (+ 21475us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:22.514670 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:22.514802 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":189,"prepare.queue_time_us":18568,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":14587,"raft.queue_time_us":285,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":40173,"spinlock_wait_cycles":10782592,"thread_start_us":312512,"threads_started":1}
I20260709 13:07:22.650063 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:22.653581 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:22.654814 16004 raft_consensus.cc:1240] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:07:22.636072 15902 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:22.656399 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:22.657168 15857 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:22.654011 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:22.657856 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:22.658795 15859 raft_consensus.cc:1240] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:22.616498 21846 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:22.663868 15902 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:22.668936 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:22.674602 15934 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:22.670537 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:22.671522 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:22.676651 15932 raft_consensus.cc:2393] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 13: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:22.677677 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:22.678596 15854 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 17.
I20260709 13:07:22.672960 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:22.679581 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:22.673496 15858 raft_consensus.cc:1275] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:07:22.680923 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:22.683552 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:22.685072 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:22.686574 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:22.686439 16003 raft_consensus.cc:2393] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 14: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:22.687330 15853 raft_consensus.cc:2368] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 13. Current term is 14.
I20260709 13:07:22.688515 15855 raft_consensus.cc:2368] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 17. Current term is 18.
I20260709 13:07:22.691303 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:22.660893 21850 consensus_queue.cc:1059] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:07:22.693604 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.694655 16570 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 18.
I20260709 13:07:22.555574 16566 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 18.
I20260709 13:07:22.653460 21847 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:22.699366 16533 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:22.661016 21655 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:22.660563 21731 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:22.685240 21872 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:22.682049 21756 raft_consensus.cc:2749] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:22.710983 15861 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
I20260709 13:07:22.660763 21854 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:22.668424 21710 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.715610 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:22.718396 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:22.738157 15853 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:22.707201 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 59598.8us
I20260709 13:07:22.712934 21847 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:22.710584 21858 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.728045 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:22.744058 15933 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:46118
I20260709 13:07:22.746326 15933 raft_consensus.cc:606] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Received request to transfer leadership to TS 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:22.746793 15933 raft_consensus.cc:614] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:22.744551 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:22.747831 15857 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:22.749835 15861 raft_consensus.cc:2393] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 14: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:07:22.752020 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 52752.6us
I20260709 13:07:22.754194 16533 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:22.758882 15934 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 17.
I20260709 13:07:22.765616 15856 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 18.
I20260709 13:07:22.731755 21588 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:22.734627 21731 raft_consensus.cc:697] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 18 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:22.740302 21819 raft_consensus.cc:2749] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:22.779479 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:22.768074 21850 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:22.738620 21854 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:22.770773 21866 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:22.745169 21756 consensus_queue.cc:1048] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:07:22.741457 21710 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:22.814330 15860 raft_consensus.cc:1240] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:22.801463 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.818269 15853 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:22.819324 16568 raft_consensus.cc:2410] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 13:07:22.801821 21869 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:22.802980 21817 consensus_queue.cc:1048] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:07:22.803786 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.805076 21846 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:22.805619 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:07:22.809875 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.765653 21858 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:22.868635 21858 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:22.870121 16002 raft_consensus.cc:1240] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:22.874022 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.875034 16567 raft_consensus.cc:2393] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 17: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:22.875072 16569 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:07:22.882504 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.883867 16567 raft_consensus.cc:2393] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 15: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:22.885339 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.886924 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:22.888545 15931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2594) took 2576 ms. Trace:
I20260709 13:07:22.889072 15931 rpcz_store.cc:276] 0709 13:07:20.312143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:20.726568 (+414425us) service_pool.cc:224] Handling call
0709 13:07:22.108237 (+1381669us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:22.108514 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:07:22.132738 (+ 24224us) log.cc:844] Serialized 14 byte log entry
0709 13:07:22.323438 (+190700us) log.cc:844] Serialized 14 byte log entry
0709 13:07:22.334139 (+ 10701us) log.cc:844] Serialized 14 byte log entry
0709 13:07:22.380280 (+ 46141us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:22.380328 (+    48us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:22.380373 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:22.380434 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:22.560860 (+180426us) spinlock_profiling.cc:243] Waited 4.19 ms on lock 0x7b180014f140. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb3f3 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd 000000000038f354 000000000038f199 00007f73e729e83a
0709 13:07:22.589864 (+ 29004us) spinlock_profiling.cc:243] Waited 7.26 ms on lock 0x7b180014f140. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb3f3 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd 000000000038f354 000000000038f199 00007f73e729e83a
0709 13:07:22.888337 (+298473us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54001a81d0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:22.888473 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":90235,"prepare.queue_time_us":37500,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":107,"raft.queue_time_us":57528,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":30727,"spinlock_wait_cycles":3202902912,"thread_start_us":189984,"threads_started":1,"wal-append.queue_time_us":411496}
I20260709 13:07:22.892115 15974 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:22.893852 21723 raft_consensus.cc:2804] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:22.894307 21723 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:22.894767 21723 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:22.897006 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.897406 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.897673 16564 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 16.
I20260709 13:07:22.898528 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.899473 21648 consensus_queue.cc:1048] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:22.901192 21759 raft_consensus.cc:3055] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:22.901088 21828 consensus_queue.cc:1059] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:07:22.902047 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:22.816246 15825 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:22.821190 21731 consensus_queue.cc:237] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:22.902604 21558 raft_consensus.cc:3055] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:22.903045 21558 raft_consensus.cc:740] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:22.903640 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.904291 16565 raft_consensus.cc:2376] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 18. Re-sending same reply.
I20260709 13:07:22.903980 21558 consensus_queue.cc:260] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:22.905164 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:22.905989 15974 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:22.906257 21854 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:22.906977 21720 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:22.898536 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.907366 15974 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:22.907666 21720 raft_consensus.cc:697] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:07:22.909070 21897 log.cc:927] Time spent T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.181s	user 0.002s	sys 0.000s
I20260709 13:07:22.908926 21720 consensus_queue.cc:237] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:22.909508 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.909989 15974 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:22.910494 21758 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:22.910600 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.910761 21859 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:22.911012 21758 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:22.911213 16570 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:22.911474 21758 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:22.911387 21859 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:22.912576 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.913462 16570 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:07:22.913833 21859 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:22.914574 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:22.914817 21891 raft_consensus.cc:3055] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:22.915277 21891 raft_consensus.cc:740] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:22.916047 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.916191 21891 consensus_queue.cc:260] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:22.917001 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.917312 21891 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:22.917591 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:22.918516 16570 raft_consensus.cc:2410] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:07:22.919201 15931 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:46118
I20260709 13:07:22.920077 15931 raft_consensus.cc:606] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Received request to transfer leadership to TS 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:22.920418 15931 raft_consensus.cc:614] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:07:22.920857 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.921824 15976 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:22.906257 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.923504 16565 raft_consensus.cc:2410] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:07:22.924453 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.924252 21758 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:22.926757 21758 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:22.830885 21706 raft_consensus.cc:2749] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:22.857522 15756 catalog_manager.cc:5697] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:22.928117 21854 consensus_queue.cc:1059] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:22.928743 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:22.901748 21759 raft_consensus.cc:740] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:22.930696 21846 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:22.931097 16564 raft_consensus.cc:2410] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:07:22.868671 21710 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:22.931885 21854 raft_consensus.cc:3055] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:22.929584 16561 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:07:22.931547 21846 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:22.929338 21859 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:22.933750 16567 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:22.934118 21833 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:22.934875 21691 raft_consensus.cc:2804] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:22.935472 16569 raft_consensus.cc:1275] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 13:07:22.963429 16569 pending_rounds.cc:85] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting all ops after (but not including) 11
I20260709 13:07:22.963719 16569 pending_rounds.cc:107] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 13:07:22.964043 16569 raft_consensus.cc:2889] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:22.933881 21814 consensus_queue.cc:1059] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 13:07:22.934921 21558 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:22.936143 16568 raft_consensus.cc:1275] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 13:07:22.966554 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 150423us
I20260709 13:07:22.967242 21858 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:22.968109 21691 raft_consensus.cc:697] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:22.968698 15825 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:22.969062 21691 consensus_queue.cc:237] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:07:22.969651 15825 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:22.970468 15825 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:22.971763 21266 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:22.971845 21816 raft_consensus.cc:2804] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:22.972245 21266 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:22.930801 21759 consensus_queue.cc:260] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:22.938652 16566 raft_consensus.cc:1240] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:07:22.972774 21266 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:22.941769 21648 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:22.971884 21367 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:22.973518 21648 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:22.973840 21367 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:22.974128 21648 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:22.974392 21367 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:22.951973 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.952841 16561 raft_consensus.cc:1275] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:07:22.977458 21833 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:22.979157 16566 raft_consensus.cc:1275] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:07:22.941107 15976 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:22.966733 21814 raft_consensus.cc:3055] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:22.972296 21816 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:22.942459 16564 raft_consensus.cc:1240] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:22.973037 21759 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:22.942951 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:07:23.001036 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.958348 21859 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:22.796453 15827 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:22.988906 16565 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 13: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:22.951745 21854 raft_consensus.cc:740] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:07:22.991499 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 50492us
W20260709 13:07:22.999500 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 45211.9us
W20260709 13:07:23.011883 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 215511us
I20260709 13:07:23.004068 16567 raft_consensus.cc:1275] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:07:23.004899 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:23.000305 21816 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:22.996526 21814 raft_consensus.cc:740] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:23.006263 21858 raft_consensus.cc:697] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:23.016866 21553 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:23.016194 21655 raft_consensus.cc:2749] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 13. Current term is 14.
W20260709 13:07:23.005756 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.991339 16568 raft_consensus.cc:1275] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 13:07:22.999856 21819 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.011958 15976 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:23.027735 21833 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:22.996121 21705 raft_consensus.cc:2749] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:07:23.031936 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 368116us
I20260709 13:07:23.068503 15860 raft_consensus.cc:1275] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 13:07:23.032549 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 340992us
I20260709 13:07:23.043011 16568 pending_rounds.cc:85] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting all ops after (but not including) 10
I20260709 13:07:23.069454 16568 pending_rounds.cc:107] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:07:23.069782 16568 raft_consensus.cc:2889] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:23.036580 15827 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:23.075124 15827 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:23.047987 15853 raft_consensus.cc:1275] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:07:23.016896 16533 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:23.063119 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 51234.9us
I20260709 13:07:23.077329 15976 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:22.934931 21849 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:23.064504 15930 raft_consensus.cc:1240] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 16 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:23.082311 15976 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:23.083827 21413 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:23.085201 21859 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:23.086694 21847 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:23.065534 21854 consensus_queue.cc:260] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:23.088886 21854 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:23.089280 21847 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:23.090135 21847 consensus_queue.cc:1048] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:23.092751 21828 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:23.095470 21869 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:23.097383 21846 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:23.040551 21817 consensus_queue.cc:1048] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 13:07:23.100003 15928 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:07:23.102588 15900 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:23.107995 16562 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
W20260709 13:07:23.110227 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 93409.6us
I20260709 13:07:23.111196 16533 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:23.111320 15900 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:23.068943 15902 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:23.119957 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 51104.4us
I20260709 13:07:23.121702 15902 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:23.123256 16563 raft_consensus.cc:2393] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 14: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:23.124330 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:23.129405 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 54365.1us
I20260709 13:07:23.130964 15928 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 12.
I20260709 13:07:23.132824 21828 consensus_queue.cc:1059] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:23.134049 21828 raft_consensus.cc:3055] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:23.134457 21828 raft_consensus.cc:740] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:23.109162 21814 consensus_queue.cc:260] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:23.137553 21814 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:23.142905 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.067836 21819 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:23.143848 21858 consensus_queue.cc:237] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:23.145993 21819 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:23.146746 21819 raft_consensus.cc:2804] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:23.147207 21819 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:23.118991 21868 raft_consensus.cc:2749] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:23.148605 21710 consensus_queue.cc:1048] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:07:23.149255 21849 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:23.080543 21758 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:23.150207 21758 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:23.150702 21758 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:23.151145 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.119452 21901 raft_consensus.cc:2749] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:23.156297 15931 raft_consensus.cc:1275] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:07:23.156724 21901 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:23.157446 21901 raft_consensus.cc:697] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 13 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:07:23.157573 15757 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58966 (request call id 245) took 32685 ms (client timeout 15000 ms). Trace:
I20260709 13:07:23.158571 21901 consensus_queue.cc:237] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:23.118917 21705 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 13:07:23.115753 21558 raft_consensus.cc:2804] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:23.160228 21847 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:23.116535 21846 consensus_queue.cc:1048] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:07:23.171418 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:23.173885 16003 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:07:23.165114 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.169667 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:23.172266 16011 raft_consensus.cc:1240] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 17. Current term is 18. Ops: []
W20260709 13:07:23.177192 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 66059.5us
I20260709 13:07:23.178418 16533 leader_election.cc:400] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 13. Current term is 14.
I20260709 13:07:23.179908 16533 leader_election.cc:400] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 17. Current term is 18.
I20260709 13:07:23.180295 16533 leader_election.cc:403] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:07:23.181094 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.181305 16533 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:23.135517 21828 consensus_queue.cc:260] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:23.165568 15825 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:23.033838 21796 consensus_queue.cc:1059] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.087s
I20260709 13:07:23.184701 21833 consensus_queue.cc:1048] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 13:07:23.206071 16003 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
I20260709 13:07:23.195926 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:23.277433 16011 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:23.207921 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:23.213502 16004 raft_consensus.cc:1240] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 15. Current term is 16. Ops: []
W20260709 13:07:23.196722 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 75099.3us
W20260709 13:07:23.197562 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 67201.3us
I20260709 13:07:23.281296 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:23.282027 15854 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:07:23.147614 21819 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:07:23.158118 15757 rpcz_store.cc:269] 0709 13:06:50.472478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:50.472674 (+   196us) service_pool.cc:224] Handling call
0709 13:06:51.395762 (+923088us) spinlock_profiling.cc:243] Waited 73.8 ms on lock 0x7b54003f3c54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea312090 00007f73ea3b0f3a 00007f73f01671cc 00007f73f026537e 00007f73ee7262a4 00007f73ee726232 00007f73ee726161 00007f73ee7260dc 00007f73ee725392 00007f73e86e348c 00007f73e86e28c6
0709 13:07:23.157470 (+31761708us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:16.930374 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:17.036248 (+105874us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:17.037339 (+  1091us) write_op.cc:435] Acquired schema lock
0709 13:07:17.037359 (+    20us) tablet.cc:662] Decoding operations
0709 13:07:17.038552 (+  1193us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:17.038671 (+   119us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:17.038711 (+    40us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:07:17.106870 (+ 68159us) tablet.cc:701] Row locks acquired
0709 13:07:17.106918 (+    48us) write_op.cc:260] PREPARE: finished
0709 13:07:17.107108 (+   190us) write_op.cc:270] Start()
0709 13:07:17.107280 (+   172us) write_op.cc:276] Timestamp: P: 1783602437107029 usec, L: 0
0709 13:07:17.107320 (+    40us) op_driver.cc:348] REPLICATION: starting
0709 13:07:17.108114 (+   794us) log.cc:844] Serialized 4334 byte log entry
0709 13:07:17.179595 (+ 71481us) spinlock_profiling.cc:243] Waited 8.94 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:07:17.395529 (+215934us) op_driver.cc:464] REPLICATION: finished
0709 13:07:17.485664 (+ 90135us) write_op.cc:301] APPLY: starting
0709 13:07:17.485777 (+   113us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:17.671792 (+186015us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:17.671829 (+    37us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:17.816394 (+144565us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:17.816764 (+   370us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=22,mrs_lookups=0
0709 13:07:17.816815 (+    51us) write_op.cc:312] APPLY: finished
0709 13:07:17.817218 (+   403us) log.cc:844] Serialized 105 byte log entry
0709 13:07:17.817404 (+   186us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:17.817540 (+   136us) write_op.cc:454] Released schema lock
0709 13:07:17.817607 (+    67us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:17.818016 (+   409us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:17.818045 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"dns_us":4572,"mutex_wait_us":7996566,"reported_tablets":42,"spinlock_wait_cycles":169749248,"child_traces":[["op",{"apply.queue_time_us":89836,"apply.run_cpu_time_us":18212,"apply.run_wall_time_us":332357,"cfile_cache_hit":46,"cfile_cache_hit_bytes":50217,"num_ops":11,"prepare.queue_time_us":326593,"prepare.run_cpu_time_us":22500,"prepare.run_wall_time_us":256166,"raft.queue_time_us":110284,"raft.run_cpu_time_us":4376,"raft.run_wall_time_us":76783,"replication_time_us":288020,"spinlock_wait_cycles":20562688,"thread_start_us":308357,"threads_started":4,"wal-append.queue_time_us":59534,"wal-append.run_cpu_time_us":29717,"wal-append.run_wall_time_us":1664753}]]}
I20260709 13:07:23.172987 21847 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:23.172844 21558 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:23.183437 21849 raft_consensus.cc:2749] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 17. Current term is 18.
I20260709 13:07:23.193429 21891 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:07:23.240906 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:23.246029 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 80572.4us
I20260709 13:07:23.188153 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:23.250648 15935 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.009s	sys 0.008s
W20260709 13:07:23.292784 15935 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.009s	sys 0.009s
W20260709 13:07:23.197161 21601 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.017s	sys 0.017s
W20260709 13:07:23.271154 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 90085.9us
I20260709 13:07:23.281363 15902 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:23.283104 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:23.284073 15827 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:23.295318 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:07:23.297338 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.298069 15902 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:23.298096 16533 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:23.276458 21786 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:23.341217 15853 raft_consensus.cc:1275] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:07:23.347167 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:23.347980 15933 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:07:23.294610 21601 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.017s	sys 0.017s
I20260709 13:07:23.283038 21655 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:23.349248 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:23.365882 15858 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:23.286928 21847 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:23.287123 21558 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:23.374598 16008 raft_consensus.cc:1240] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:23.358491 21796 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:23.379334 21731 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:23.357986 21846 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.343468 21786 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:23.365475 21655 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:23.379915 21731 raft_consensus.cc:697] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 17 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:23.417682 21731 consensus_queue.cc:237] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:23.426335 15856 raft_consensus.cc:1275] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:07:23.383985 21601 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:23.350575 21873 raft_consensus.cc:2749] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:07:23.356179 21904 log.cc:927] Time spent T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:07:23.348587 21723 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:23.341681 21891 raft_consensus.cc:697] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:23.452531 15859 raft_consensus.cc:1275] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:07:23.402186 21786 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:23.374944 21266 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:23.459779 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.403924 21909 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:23.438285 21655 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:23.506824 16002 raft_consensus.cc:1240] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:23.457531 21881 raft_consensus.cc:2749] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:23.402081 21367 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:23.401822 21846 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:23.497367 21413 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:23.526299 21905 log.cc:927] Time spent T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
W20260709 13:07:23.526577 21906 log.cc:927] Time spent T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.179s	user 0.000s	sys 0.000s
I20260709 13:07:23.459127 21601 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:23.561048 15861 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.021s	sys 0.052s
W20260709 13:07:23.561991 15861 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.022s	sys 0.052s
I20260709 13:07:23.363255 16011 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:07:23.605996 15933 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
W20260709 13:07:23.621737 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:23.342288 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 44279.5us
I20260709 13:07:23.529376 21413 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:23.642097 16533 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:23.639122 15858 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
W20260709 13:07:23.647574 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.619190 21849 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:23.613791 21859 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:07:23.642844 21908 log.cc:927] Time spent T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.270s	user 0.000s	sys 0.001s
I20260709 13:07:23.560259 21891 consensus_queue.cc:237] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:23.680410 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 49174.6us
W20260709 13:07:23.688868 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.701413 16533 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:23.703778 15855 raft_consensus.cc:1240] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:07:23.684140 21873 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:23.713904 16533 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:23.672435 21849 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:23.706070 21367 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:23.699856 21588 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:23.674000 21859 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:23.731760 21846 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:23.733134 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.740419 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:23.741937 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:23.742620 15862 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:23.734464 21881 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:23.736493 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:23.744392 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:23.745182 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.707427 21723 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:23.718339 21868 raft_consensus.cc:2749] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:23.729291 21266 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:23.744297 21367 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.763763 16006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 2484) took 1482 ms. Trace:
I20260709 13:07:23.752943 21710 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:23.765722 16006 rpcz_store.cc:276] 0709 13:07:22.281537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:22.285380 (+  3843us) service_pool.cc:224] Handling call
0709 13:07:22.285742 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:22.454855 (+169113us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:07:22.466399 (+ 11544us) log.cc:844] Serialized 14 byte log entry
0709 13:07:23.243035 (+776636us) log.cc:844] Serialized 14 byte log entry
0709 13:07:23.700380 (+457345us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:23.700439 (+    59us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:23.700500 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:23.763361 (+ 62861us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b5400304654. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:23.763429 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:23.763617 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":89425,"prepare.queue_time_us":79167,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":5134,"raft.queue_time_us":188488,"raft.run_cpu_time_us":240,"raft.run_wall_time_us":15391,"spinlock_wait_cycles":388439936,"thread_start_us":1111476,"threads_started":3,"wal-append.queue_time_us":567807}
I20260709 13:07:23.758107 21873 raft_consensus.cc:2804] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:07:23.775291 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.739379 21588 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:23.771456 21914 log.cc:927] Time spent T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 13:07:23.754310 21881 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:23.790444 21859 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:23.758450 21869 consensus_queue.cc:1059] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.020s
W20260709 13:07:23.792213 21816 consensus_meta.cc:287] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.761s	user 0.011s	sys 0.000s
I20260709 13:07:23.792464 21558 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:23.792711 21816 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:23.795126 21558 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:23.774389 21601 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:23.796157 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:23.796741 16568 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:23.797372 21588 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:23.778586 21710 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:23.795053 21816 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:23.799697 21710 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:23.790443 21881 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:23.800684 21849 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:23.801360 21849 raft_consensus.cc:3055] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:23.801738 21849 raft_consensus.cc:740] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:23.803054 21553 raft_consensus.cc:3055] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:23.803442 21553 raft_consensus.cc:740] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:23.804251 21553 consensus_queue.cc:260] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:23.805135 16568 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
I20260709 13:07:23.805274 21553 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:07:23.783624 21911 log.cc:927] Time spent T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.116s	user 0.001s	sys 0.001s
I20260709 13:07:23.806973 15935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2621) took 1443 ms. Trace:
I20260709 13:07:23.773413 21367 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:23.780853 21873 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:07:23.805533 21648 consensus_meta.cc:287] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.831s	user 0.012s	sys 0.000s
I20260709 13:07:23.802610 21849 consensus_queue.cc:260] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:23.807335 15935 rpcz_store.cc:276] 0709 13:07:22.363112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:22.363235 (+   123us) service_pool.cc:224] Handling call
0709 13:07:22.363563 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:22.680692 (+317129us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:22.680723 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:22.689678 (+  8955us) log.cc:844] Serialized 26 byte log entry
0709 13:07:23.293344 (+603666us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:23.293409 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:23.293503 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:23.806735 (+513232us) raft_consensus.cc:1685] finished
0709 13:07:23.806756 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:23.806917 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53868,"raft.queue_time_us":73242,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":15225,"thread_start_us":829777,"threads_started":2,"wal-append.queue_time_us":932911}
I20260709 13:07:23.811038 21873 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:23.812403 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:23.819185 15933 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:23.823894 15862 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:23.828963 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:23.829197 15933 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:23.813175 21868 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.832255 15976 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:07:23.834000 15827 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.001s	sys 0.021s
W20260709 13:07:23.824302 21758 consensus_meta.cc:287] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.673s	user 0.007s	sys 0.006s
W20260709 13:07:23.835629 15827 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.001s	sys 0.021s
W20260709 13:07:23.836493 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 552444us
I20260709 13:07:23.824687 21413 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:23.837962 15827 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:23.860306 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:23.861130 16568 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:07:23.864054 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:23.866632 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:23.869238 16011 raft_consensus.cc:1275] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:07:23.870363 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:23.883404 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:23.884248 16564 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:23.850028 21842 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.007s	sys 0.013s
I20260709 13:07:23.869829 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:23.901036 16002 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
W20260709 13:07:23.856819 21759 consensus_meta.cc:287] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.850s	user 0.010s	sys 0.002s
I20260709 13:07:23.898907 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:23.837471 21710 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.836833 21927 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.855046 21558 raft_consensus.cc:2804] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:23.819906 21648 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:23.814581 21847 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:23.905207 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:23.911840 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:23.888088 21413 raft_consensus.cc:2804] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:23.920653 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:23.921512 16570 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:07:23.900518 21796 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.851423 21868 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:23.854249 21758 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:23.924472 15902 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.018s	sys 0.003s
W20260709 13:07:23.929704 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 213619us
W20260709 13:07:23.930303 15902 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.018s	sys 0.003s
W20260709 13:07:23.933945 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 220170us
I20260709 13:07:23.929391 16566 raft_consensus.cc:1240] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 13. Current term is 14. Ops: []
W20260709 13:07:23.934629 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 636564us
I20260709 13:07:23.936342 16535 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:23.937484 15902 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:23.939023 15902 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:23.944648 15929 raft_consensus.cc:2393] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 17: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:23.948218 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:23.951028 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:23.952075 15862 raft_consensus.cc:2393] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 13: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:23.955188 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:23.956240 15860 raft_consensus.cc:2393] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 13: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:23.958042 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:23.961139 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:23.962071 16002 raft_consensus.cc:2393] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 15: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:23.963476 15902 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:23.965529 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:23.970731 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:23.972218 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:23.974720 15902 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:23.927940 21413 raft_consensus.cc:697] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:23.911775 21710 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:23.933017 21558 raft_consensus.cc:697] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 13 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:23.983063 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:23.992060 15935 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:23.992475 15853 raft_consensus.cc:1275] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:07:23.993517 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:23.995868 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:23.911983 21927 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:23.978348 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:23.990584 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.977954 15856 raft_consensus.cc:1275] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:07:23.997459 16005 raft_consensus.cc:1275] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:07:23.997614 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:24.012040 16003 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 13.
I20260709 13:07:23.998327 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:24.012627 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:24.003479 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:23.961279 21731 raft_consensus.cc:2764] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:07:23.962468 21881 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:23.952800 21929 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:24.033635 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:23.945082 21796 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:23.975104 21854 consensus_meta.cc:287] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.886s	user 0.013s	sys 0.000s
I20260709 13:07:24.012949 16007 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:07:24.035768 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:24.040373 15853 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:07:24.046569 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:24.050364 16006 raft_consensus.cc:1240] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:24.050567 15853 raft_consensus.cc:606] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Received request to transfer leadership to TS 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:23.972430 21367 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:24.043390 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:23.972260 21932 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.042443 15827 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:24.044034 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:24.056665 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 69736.2us
I20260709 13:07:23.977577 21924 raft_consensus.cc:2749] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:07:24.062865 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.036274 21846 raft_consensus.cc:2749] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:07:24.062238 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 123039us
I20260709 13:07:24.068723 16011 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 14.
I20260709 13:07:23.992126 21858 consensus_queue.cc:1048] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 13:07:24.044013 21929 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:24.043213 21901 consensus_queue.cc:1048] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
W20260709 13:07:24.071309 21923 log.cc:927] Time spent T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.244s	user 0.000s	sys 0.001s
I20260709 13:07:24.097708 16570 raft_consensus.cc:1275] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:07:24.102202 16570 pending_rounds.cc:85] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting all ops after (but not including) 9
I20260709 13:07:24.044374 21756 consensus_queue.cc:1048] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
W20260709 13:07:24.099337 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.066792 21928 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.071267 21941 raft_consensus.cc:2804] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:24.127735 21941 raft_consensus.cc:697] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:07:24.128556 21814 consensus_meta.cc:287] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.990s	user 0.013s	sys 0.000s
I20260709 13:07:24.128729 21941 consensus_queue.cc:237] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:24.129418 21849 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:24.102579 16570 pending_rounds.cc:107] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:07:24.132469 16570 raft_consensus.cc:2889] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:24.132531 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:24.133688 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:24.134348 16570 raft_consensus.cc:2368] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 12. Current term is 13.
I20260709 13:07:24.135136 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:24.106031 21942 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.136693 21942 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:24.107517 21934 log.cc:927] Time spent T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:07:24.046186 21881 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:24.074430 21940 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:24.076417 21828 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:24.045827 21937 consensus_queue.cc:1048] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 13:07:24.120225 21648 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:24.122020 21796 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:24.123901 21710 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:24.096674 21413 consensus_queue.cc:237] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:24.123930 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:24.115895 21558 consensus_queue.cc:237] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:24.175720 21847 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:24.176129 21819 consensus_meta.cc:287] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.877s	user 0.013s	sys 0.000s
I20260709 13:07:24.149869 16564 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 17.
I20260709 13:07:24.080215 21929 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:24.150517 21930 log.cc:927] Time spent T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.368s	user 0.000s	sys 0.003s
W20260709 13:07:24.149477 21931 log.cc:927] Time spent T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.237s	user 0.005s	sys 0.000s
I20260709 13:07:24.098448 21367 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:24.070919 21854 consensus_queue.cc:1059] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.075s
I20260709 13:07:24.162963 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:24.205120 16010 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 18.
I20260709 13:07:24.066419 21943 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:24.168488 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:24.150856 21939 log.cc:927] Time spent T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 13:07:24.177444 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:24.208070 16564 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:24.180155 16533 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:07:24.149098 21933 log.cc:927] Time spent T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.205s	user 0.000s	sys 0.004s
W20260709 13:07:24.193435 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:24.148885 21918 log.cc:927] Time spent T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.417s	user 0.006s	sys 0.000s
I20260709 13:07:24.195328 16570 raft_consensus.cc:1275] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:07:24.154515 15862 raft_consensus.cc:1275] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:07:24.199434 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:07:24.199865 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 128451us
I20260709 13:07:24.144861 21942 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:24.067734 21932 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:24.192984 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:24.202912 15974 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:24.209678 15976 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:24.161695 21869 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:24.218674 16564 raft_consensus.cc:1275] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 13:07:24.219228 16564 pending_rounds.cc:85] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5: Aborting all ops after (but not including) 11
I20260709 13:07:24.219528 16564 pending_rounds.cc:107] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:07:24.219918 16564 raft_consensus.cc:2889] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:24.213126 15902 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:24.214960 15853 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:24.223295 15902 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:07:24.225099 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.224926 15902 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:24.226864 15825 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:07:24.230644 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 188275us
I20260709 13:07:24.203244 21756 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.232255 15827 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:24.236215 16535 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:24.203892 21367 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:24.238967 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:24.211943 21910 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.243481 15974 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:24.224550 21655 consensus_meta.cc:287] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.704s	user 0.011s	sys 0.000s
I20260709 13:07:24.216092 21901 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:24.222828 21731 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.153066 21928 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:24.280402 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:24.287722 15929 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 17.
I20260709 13:07:24.229229 21942 raft_consensus.cc:2804] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:24.234982 21924 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:24.291785 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:24.200264 21819 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:24.257258 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.283659 16567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1281) took 1050 ms. Trace:
I20260709 13:07:24.300092 16567 rpcz_store.cc:276] 0709 13:07:23.233339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:23.233541 (+   202us) service_pool.cc:224] Handling call
0709 13:07:23.244359 (+ 10818us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:23.244781 (+   422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:23.244809 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:23.245266 (+   457us) log.cc:844] Serialized 26 byte log entry
0709 13:07:23.622833 (+377567us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:23.622894 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:23.622996 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:24.126211 (+503215us) spinlock_profiling.cc:243] Waited 27.9 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:07:24.247289 (+121078us) raft_consensus.cc:1685] finished
0709 13:07:24.247310 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:24.283588 (+ 36278us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10601,"raft.queue_time_us":113,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":28920,"spinlock_wait_cycles":64194304,"thread_start_us":280498,"threads_started":1,"wal-append.queue_time_us":454375}
I20260709 13:07:24.303514 21881 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:24.305219 15854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2563) took 1301 ms. Trace:
I20260709 13:07:24.247318 21842 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:24.164841 21940 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:24.284904 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 52781us
I20260709 13:07:24.206928 21710 consensus_queue.cc:1048] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 13:07:24.229351 21588 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:24.310007 21954 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.261473 15853 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 19.
I20260709 13:07:24.254905 21926 consensus_queue.cc:1048] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.176s
I20260709 13:07:24.305596 15854 rpcz_store.cc:276] 0709 13:07:23.003460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:23.290899 (+287439us) service_pool.cc:224] Handling call
0709 13:07:23.296258 (+  5359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:23.296523 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:23.296560 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:23.298025 (+  1465us) log.cc:844] Serialized 26 byte log entry
0709 13:07:23.698824 (+400799us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:23.698878 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:23.698971 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:24.305004 (+606033us) raft_consensus.cc:1685] finished
0709 13:07:24.305025 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:24.305164 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":354413,"threads_started":1,"wal-append.queue_time_us":557506}
I20260709 13:07:24.225831 21936 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:24.252599 21367 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:24.277169 21833 raft_consensus.cc:2804] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:24.254822 21846 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:07:24.328523 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:24.356849 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.234611 21648 consensus_queue.cc:1048] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
W20260709 13:07:24.399065 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 174181us
I20260709 13:07:24.298913 21825 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:24.225462 21948 consensus_queue.cc:1048] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.083s
I20260709 13:07:24.229189 21869 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:24.242758 21952 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:24.455097 21952 raft_consensus.cc:697] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:24.404881 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:24.456184 21917 log.cc:927] Time spent T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.717s	user 0.003s	sys 0.000s
W20260709 13:07:24.461251 21938 log.cc:927] Time spent T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.381s	user 0.000s	sys 0.001s
I20260709 13:07:24.412878 15928 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 13: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:24.226450 21909 consensus_queue.cc:1048] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 13:07:24.255179 21891 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:24.283206 21868 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:24.298559 21924 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:24.295710 21490 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:24.245325 21756 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:24.292783 15862 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:07:24.309463 21842 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:24.277501 21925 consensus_queue.cc:1059] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.168s
I20260709 13:07:24.298182 15825 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:24.217005 21943 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:24.276908 21655 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:24.498891 21891 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:24.310559 21942 raft_consensus.cc:697] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:24.297312 21704 consensus_queue.cc:1048] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.125s
I20260709 13:07:24.291659 21731 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:07:24.285307 21910 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:24.435770 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:24.522861 15931 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 11.
I20260709 13:07:24.436802 15933 raft_consensus.cc:1240] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 15. Current term is 16. Ops: [15.8-15.8]
W20260709 13:07:24.503948 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 205833us
I20260709 13:07:24.521013 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:24.436941 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:24.527889 15929 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:24.291993 21901 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:24.460613 21825 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:24.465143 21758 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:24.412777 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 99110.9us
W20260709 13:07:24.476567 21957 log.cc:927] Time spent T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.068s	user 0.000s	sys 0.002s
I20260709 13:07:24.548806 21819 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:24.478839 16010 raft_consensus.cc:2410] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:07:24.421583 21588 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:24.549633 21588 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:24.431741 21845 consensus_queue.cc:1059] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 13:07:24.522459 21868 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:24.553598 21845 raft_consensus.cc:3055] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:24.554095 21845 raft_consensus.cc:740] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
W20260709 13:07:24.432920 21916 log.cc:927] Time spent T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.740s	user 0.004s	sys 0.000s
I20260709 13:07:24.418211 21927 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:24.426997 21814 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:24.497625 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:24.561190 15937 raft_consensus.cc:2393] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 17: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
W20260709 13:07:24.499778 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 66066.1us
I20260709 13:07:24.498692 21857 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.501850 16566 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:24.503398 21842 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:24.503285 15928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53636 (request call id 1183) took 1208 ms. Trace:
W20260709 13:07:24.529230 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 129973us
I20260709 13:07:24.487957 21846 raft_consensus.cc:697] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 14 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:24.577168 15928 rpcz_store.cc:276] 0709 13:07:23.294924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:23.295073 (+   149us) service_pool.cc:224] Handling call
0709 13:07:24.503013 (+1207940us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54003b8154. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:24.503207 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2567613184}
I20260709 13:07:24.533391 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:24.580241 15933 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 17.
I20260709 13:07:24.534353 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:24.538714 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:24.516760 21956 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:24.541649 21825 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:24.430428 21954 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:24.543849 21655 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:24.558554 21938 log.cc:927] Time spent T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:07:24.545503 21950 raft_consensus.cc:3055] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:07:24.545836 21786 consensus_meta.cc:287] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 1.042s	user 0.010s	sys 0.003s
I20260709 13:07:24.515144 21816 consensus_queue.cc:1048] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:07:24.620656 15928 raft_consensus.cc:1275] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:07:24.501771 21891 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:24.499076 21924 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:24.574661 15937 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:07:24.622023 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:24.585719 15900 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:24.590106 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:24.613821 16561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1275) took 1801 ms. Trace:
I20260709 13:07:24.590281 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:24.624655 16568 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:24.554899 21845 consensus_queue.cc:260] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:24.522966 21847 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:07:24.558018 21873 consensus_meta.cc:287] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.731s	user 0.002s	sys 0.008s
I20260709 13:07:24.544324 21901 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:24.609691 15936 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:24.611128 16566 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 19.
I20260709 13:07:24.557076 21720 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.575302 21854 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:24.628219 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:24.594000 21919 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.801s	user 0.006s	sys 0.019s
I20260709 13:07:24.621346 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:24.623840 16004 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 17.
I20260709 13:07:24.630046 15827 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:24.593082 21857 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:24.630880 21928 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:24.631057 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:24.553833 21868 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:24.624037 16561 rpcz_store.cc:276] 0709 13:07:22.812187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:22.976278 (+164091us) service_pool.cc:224] Handling call
0709 13:07:22.976585 (+   307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:22.976897 (+   312us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:22.976926 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:22.977482 (+   556us) log.cc:844] Serialized 26 byte log entry
0709 13:07:23.460488 (+483006us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:23.460535 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:23.460611 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:24.426029 (+965418us) spinlock_profiling.cc:243] Waited 69.1 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:07:24.602334 (+176305us) raft_consensus.cc:1685] finished
0709 13:07:24.602363 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:24.613731 (+ 11368us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":85,"raft.run_cpu_time_us":49,"raft.run_wall_time_us":48,"spinlock_wait_cycles":158974848,"thread_start_us":482573,"threads_started":1,"wal-append.queue_time_us":718476}
W20260709 13:07:24.636574 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 50963.5us
I20260709 13:07:24.641458 21720 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:24.645732 16569 raft_consensus.cc:2393] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 14: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:24.645696 21869 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:24.646241 15900 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:24.601576 21965 raft_consensus.cc:3055] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:24.601271 21956 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:24.541925 21942 consensus_queue.cc:237] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:24.642846 15827 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:24.666800 15827 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:24.650810 15900 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:24.638985 21919 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.846s	user 0.006s	sys 0.019s
I20260709 13:07:24.631901 21648 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.670519 15900 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:24.656622 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:24.614365 21950 raft_consensus.cc:740] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:24.602850 21490 consensus_queue.cc:1048] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:07:24.673491 21964 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:24.674503 15827 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:24.674810 15928 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:46118
I20260709 13:07:24.683688 15928 raft_consensus.cc:606] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 15 LEADER]: Received request to transfer leadership to TS 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:24.676014 21937 consensus_queue.cc:1048] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:24.675710 21367 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:24.678690 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:24.679764 15933 raft_consensus.cc:1240] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 14. Current term is 15. Ops: [14.11-14.11]
W20260709 13:07:24.695541 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 66308.4us
I20260709 13:07:24.695724 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:24.695770 15934 raft_consensus.cc:2393] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 14: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:24.696544 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:24.697142 15933 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:24.681212 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:24.643945 21819 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:24.689808 15827 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:24.700969 15827 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:24.601831 21814 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:24.657007 21965 raft_consensus.cc:740] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:24.633932 21786 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:24.704034 15928 raft_consensus.cc:1275] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:07:24.655776 21955 raft_consensus.cc:2804] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:24.702750 15934 raft_consensus.cc:1240] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 13 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:24.669658 21816 raft_consensus.cc:2749] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:24.666937 21932 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:24.714164 15827 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:24.682268 21845 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:24.698824 21817 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:07:24.754472 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 258013us
I20260709 13:07:24.718935 15933 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 18.
I20260709 13:07:24.662118 21859 consensus_queue.cc:1048] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:07:24.695020 21909 raft_consensus.cc:2749] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:24.666532 21956 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:24.729331 16565 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:07:24.722858 21881 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:24.683617 21593 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:24.647617 21854 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:24.642107 21873 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:24.715317 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:07:24.755739 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:24.638198 21847 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:24.757319 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 60087.1us
W20260709 13:07:24.757642 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:24.769954 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.774837 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:24.775741 15856 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
W20260709 13:07:24.760076 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 45984.8us
I20260709 13:07:24.779870 15858 raft_consensus.cc:2368] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 14. Current term is 15.
I20260709 13:07:24.780526 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:24.766510 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:24.781819 15934 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 13: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
W20260709 13:07:24.783403 15974 leader_election.cc:336] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 1.876s (SENT)
I20260709 13:07:24.784098 15974 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:24.761453 21601 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.764858 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:24.788309 15934 raft_consensus.cc:1240] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:24.709904 21951 raft_consensus.cc:2749] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:24.690614 21648 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:24.727353 21955 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:24.744536 21731 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
W20260709 13:07:24.780814 21938 log.cc:927] Time spent T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:07:24.805490 21953 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.821405 21953 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:24.842598 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 48127.9us
W20260709 13:07:24.811470 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.817443 16570 raft_consensus.cc:1240] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:24.780128 21266 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:24.761466 21881 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:24.825824 15934 raft_consensus.cc:1275] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:07:24.846314 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 62508us
I20260709 13:07:24.805761 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:24.735500 21588 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:24.860008 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:24.863073 21854 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:24.866132 21954 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:24.698633 21846 consensus_queue.cc:237] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:24.870338 15932 raft_consensus.cc:1275] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:07:24.855433 15974 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:24.642167 21952 consensus_queue.cc:237] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:24.792694 21940 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:24.880553 15934 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:07:24.791415 21970 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:07:24.853035 21961 log.cc:927] Time spent T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.253s	user 0.000s	sys 0.001s
I20260709 13:07:24.819736 21955 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:24.789965 21601 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:24.892323 15928 raft_consensus.cc:1240] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 12. Current term is 13. Ops: []
W20260709 13:07:24.876832 21963 log.cc:927] Time spent T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.294s	user 0.000s	sys 0.001s
I20260709 13:07:24.896708 15974 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:24.855994 21881 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:24.897814 15861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2550) took 2225 ms. Trace:
W20260709 13:07:24.882349 21960 log.cc:927] Time spent T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:07:24.929724 15974 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:24.932696 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:24.852254 21558 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:24.929498 15861 rpcz_store.cc:276] 0709 13:07:22.672199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:22.766706 (+ 94507us) service_pool.cc:224] Handling call
0709 13:07:22.768169 (+  1463us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:22.948703 (+180534us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:07:23.570252 (+621549us) log.cc:844] Serialized 14 byte log entry
0709 13:07:23.907776 (+337524us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:23.917318 (+  9542us) log.cc:844] Serialized 52 byte log entry
0709 13:07:23.917479 (+   161us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:23.917758 (+   279us) log.cc:844] Serialized 14 byte log entry
0709 13:07:23.917995 (+   237us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:23.918075 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:24.117251 (+199176us) spinlock_profiling.cc:243] Waited 3.9 ms on lock 0x7b180027acc0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb3f3 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd 000000000038f354 000000000038f199 00007f73e729e83a
0709 13:07:24.897518 (+780267us) raft_consensus.cc:1685] finished
0709 13:07:24.897546 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:24.897741 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71569,"prepare.queue_time_us":400053,"prepare.run_cpu_time_us":114,"prepare.run_wall_time_us":4243,"raft.queue_time_us":180887,"raft.run_cpu_time_us":354,"raft.run_wall_time_us":5276,"spinlock_wait_cycles":8979328,"thread_start_us":854654,"threads_started":2,"wal-append.queue_time_us":357107}
I20260709 13:07:24.937595 15934 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
I20260709 13:07:24.938520 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:24.931399 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:24.883572 21266 raft_consensus.cc:697] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 17 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:24.739233 21950 consensus_queue.cc:260] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:24.933346 15933 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:24.888581 21901 raft_consensus.cc:2749] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:24.948961 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:24.950062 15933 raft_consensus.cc:2393] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 17: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:24.893450 21910 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:24.939579 21957 log.cc:927] Time spent T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:07:24.893232 21925 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:24.901328 21943 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:24.941294 21953 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:24.984369 21786 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:24.958794 21367 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:24.992295 21553 consensus_meta.cc:287] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 1.186s	user 0.013s	sys 0.000s
I20260709 13:07:24.993278 15854 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:07:24.995067 21881 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:24.947048 21928 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:24.968178 21868 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:24.969393 21648 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:24.972525 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:24.975031 21847 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:24.975975 21873 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:24.976188 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.976105 21266 consensus_queue.cc:237] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:24.893262 21970 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:25.037642 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:24.941526 21705 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:25.027840 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:25.040602 15933 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
I20260709 13:07:25.032259 15857 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 13: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:25.031060 15859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 2439) took 1079 ms. Trace:
I20260709 13:07:25.030347 21987 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:25.044193 21987 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:25.031947 21982 log.cc:927] Time spent T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.059s	user 0.002s	sys 0.017s
W20260709 13:07:25.030571 21976 log.cc:927] Time spent T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 13:07:25.042199 15859 rpcz_store.cc:276] 0709 13:07:23.951243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:23.969653 (+ 18410us) service_pool.cc:224] Handling call
0709 13:07:23.988464 (+ 18811us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:24.141538 (+153074us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:24.141572 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:24.142177 (+   605us) log.cc:844] Serialized 26 byte log entry
0709 13:07:24.230334 (+ 88157us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:24.230388 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:24.230483 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:25.030758 (+800275us) raft_consensus.cc:1685] finished
0709 13:07:25.030788 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:25.030986 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5002,"raft.queue_time_us":71303,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":6942,"thread_start_us":187181,"threads_started":2,"wal-append.queue_time_us":279537}
W20260709 13:07:25.034328 21989 log.cc:927] Time spent T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:07:24.888554 21965 consensus_queue.cc:260] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:25.030087 21928 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:25.052846 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:25.033806 21983 log.cc:927] Time spent T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.069s	user 0.004s	sys 0.000s
I20260709 13:07:25.070472 21588 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:25.035348 21929 consensus_meta.cc:287] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.827s	user 0.003s	sys 0.008s
I20260709 13:07:25.077736 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:25.029331 21925 raft_consensus.cc:697] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 19 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:25.083576 21928 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:25.084074 21833 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.017s	sys 0.000s
I20260709 13:07:25.040324 21705 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:25.127880 21705 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:25.128393 21970 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:25.127954 21814 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:25.126838 15933 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:07:25.132004 15935 raft_consensus.cc:1240] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 14 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:25.132006 15858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 2438) took 1181 ms. Trace:
I20260709 13:07:25.134569 15858 rpcz_store.cc:276] 0709 13:07:23.950903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:23.968060 (+ 17157us) service_pool.cc:224] Handling call
0709 13:07:25.131821 (+1163761us) spinlock_profiling.cc:243] Waited 807 ms on lock 0x7b540008d254. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:25.131950 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1855303424}
I20260709 13:07:25.085330 21965 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:25.142711 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:25.130750 21828 consensus_meta.cc:287] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.966s	user 0.013s	sys 0.000s
W20260709 13:07:25.137887 21833 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.018s	sys 0.000s
I20260709 13:07:25.123207 21929 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:25.150395 21925 consensus_queue.cc:237] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:25.161001 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:25.163239 21987 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:25.163866 21819 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:25.159914 21927 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.165395 21927 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:25.163877 15929 raft_consensus.cc:1240] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 16. Current term is 17. Ops: []
W20260709 13:07:25.147502 21842 consensus_meta.cc:287] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.534s	user 0.004s	sys 0.008s
I20260709 13:07:25.174662 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:25.175360 15933 raft_consensus.cc:2376] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 18. Re-sending same reply.
I20260709 13:07:25.176239 21588 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.176764 21588 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:25.177745 21857 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:25.178860 21588 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:25.184742 21924 consensus_meta.cc:287] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.542s	user 0.011s	sys 0.000s
I20260709 13:07:25.185303 21924 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:25.189750 21924 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:25.187849 15937 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
I20260709 13:07:25.196234 15931 raft_consensus.cc:1240] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:07:25.157650 21833 raft_consensus.cc:697] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:07:25.205516 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:25.212356 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:25.209043 21891 consensus_meta.cc:287] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.566s	user 0.011s	sys 0.000s
I20260709 13:07:25.215855 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:25.217918 15933 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:25.206550 21929 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:25.229177 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:25.232149 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:25.232800 15930 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 13: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:25.231463 21601 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:25.234083 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:25.174849 21842 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:25.238173 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.248309 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:25.243330 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:25.244838 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:25.249536 16005 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:25.250698 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:25.253082 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:25.249886 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:25.254055 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:25.256145 16010 raft_consensus.cc:1275] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:07:25.258930 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:25.291400 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:25.291472 16005 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:25.294492 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:25.296360 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:25.297549 16005 raft_consensus.cc:2410] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 13:07:25.221167 21891 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:25.310273 21891 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:25.260843 21956 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:25.313371 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:25.316058 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:25.317137 16002 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 13: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:25.324635 15860 raft_consensus.cc:1240] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:07:25.326067 21956 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:25.333542 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.333865 21720 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:25.337196 21842 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:25.337311 16008 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 13.
I20260709 13:07:25.345227 21705 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:25.346210 21962 log.cc:927] Time spent T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.771s	user 0.002s	sys 0.000s
I20260709 13:07:25.348655 21756 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:25.348665 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:25.350111 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:25.351056 15854 raft_consensus.cc:2393] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 13: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:25.351442 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:25.351086 21975 log.cc:927] Time spent T 00000000000000000000000000000000 P d33cebb189b14a989836214fdbafb898: Append to log took a long time: real 0.505s	user 0.000s	sys 0.001s
I20260709 13:07:25.352104 15862 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 14.
I20260709 13:07:25.353219 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:25.354274 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:25.354578 16003 raft_consensus.cc:2393] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 15: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:25.359648 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:25.360244 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:25.368058 15827 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:25.371325 21956 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:25.371894 21956 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:25.372325 21956 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:25.377378 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:25.378257 15827 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:25.352794 21955 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:25.362295 21929 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:25.347604 21705 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:25.397202 15900 leader_election.cc:400] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 election: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 14. Current term is 15.
I20260709 13:07:25.401262 15900 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:25.407541 16007 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:25.413945 15827 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:25.280654 21833 consensus_queue.cc:237] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:25.420545 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:25.421422 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:25.424880 16004 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 17.
I20260709 13:07:25.415544 15934 raft_consensus.cc:1275] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:07:25.361289 21731 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:25.386271 21593 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:25.432360 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:25.385862 21819 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:25.434069 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:25.435532 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:25.436412 16011 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:25.437428 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:25.437891 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:25.438089 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:25.439815 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:25.440168 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:25.439325 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:25.440773 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.442983 15976 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:25.447064 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:25.447732 15929 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:25.430744 21588 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:25.430922 21817 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:25.451386 16011 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 19.
I20260709 13:07:25.398351 21929 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:25.463778 21817 raft_consensus.cc:697] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:25.464706 21817 consensus_queue.cc:237] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:25.448570 21854 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:25.466637 21854 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:25.442202 21593 raft_consensus.cc:697] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
W20260709 13:07:25.472918 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 51978.5us
I20260709 13:07:25.450517 21965 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:25.435168 21731 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:25.452670 21588 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:25.485772 21854 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:25.486334 21956 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:25.487403 21967 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:25.440495 21819 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:25.489897 15929 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 19.
I20260709 13:07:25.489986 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:25.490943 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:25.506407 16566 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 12.
I20260709 13:07:25.506494 16568 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:07:25.488816 21868 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:07:25.511065 15976 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:25.513870 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:25.514911 15929 raft_consensus.cc:2393] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 17: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:25.500147 21588 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:25.500289 21965 raft_consensus.cc:697] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:25.512748 16533 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:25.519766 15859 raft_consensus.cc:1240] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:25.524122 16004 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:07:25.518082 21842 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:25.530776 21869 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:25.548884 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:25.507954 21967 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:25.563536 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.538038 21842 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:25.550213 21956 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:25.552158 21868 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.557585 21869 raft_consensus.cc:697] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:25.549852 21756 consensus_queue.cc:1059] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 13:07:25.579497 21842 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:25.575062 21955 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:25.590101 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:25.540369 21593 consensus_queue.cc:237] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:25.604183 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:25.601377 21966 log.cc:927] Time spent T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 1.009s	user 0.004s	sys 0.000s
I20260709 13:07:25.584733 21868 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:25.614331 21819 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:25.614013 21869 consensus_queue.cc:237] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:25.586158 21965 consensus_queue.cc:237] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:25.616047 22019 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.617002 21969 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.616670 22019 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:25.617517 21969 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:25.619594 21969 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:25.621294 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:25.622802 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:25.622926 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:25.623741 16563 raft_consensus.cc:1275] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:07:25.623804 16007 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 16.
W20260709 13:07:25.624558 22021 log.cc:927] Time spent T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:07:25.625464 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:25.619638 21868 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:25.617009 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:07:25.627271 22012 log.cc:927] Time spent T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:07:25.627056 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:25.628036 16565 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:25.627064 16568 raft_consensus.cc:2393] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 15: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
W20260709 13:07:25.612526 22016 log.cc:927] Time spent T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:07:25.638125 15827 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:25.611706 21731 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:25.645815 16565 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:25.641594 15827 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:25.651060 16568 raft_consensus.cc:1240] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:07:25.652336 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:25.653043 16569 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 18.
I20260709 13:07:25.621395 22019 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:25.622911 21969 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.659516 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:25.660379 15934 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 17.
I20260709 13:07:25.662124 15935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 2393) took 1227 ms. Trace:
I20260709 13:07:25.662657 15935 rpcz_store.cc:276] 0709 13:07:24.434805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:24.435187 (+   382us) service_pool.cc:224] Handling call
0709 13:07:24.479176 (+ 43989us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:25.132456 (+653280us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:25.661849 (+529393us) spinlock_profiling.cc:243] Waited 653 ms on lock 0x7b54003b8b54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:25.662052 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1500812032}
I20260709 13:07:25.665105 15827 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:25.652212 21901 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.656652 21858 raft_consensus.cc:2804] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:25.666393 15827 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:25.641913 21955 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:25.643038 21868 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.657490 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:25.683561 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:25.684855 15855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 2445) took 1706 ms. Trace:
I20260709 13:07:25.688309 16569 raft_consensus.cc:2393] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 17: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:25.688093 15855 rpcz_store.cc:276] 0709 13:07:23.978395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:23.978621 (+   226us) service_pool.cc:224] Handling call
0709 13:07:23.979080 (+   459us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:24.066505 (+ 87425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:24.066542 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:24.075460 (+  8918us) log.cc:844] Serialized 52 byte log entry
0709 13:07:24.403954 (+328494us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:24.404015 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:24.404131 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:25.539793 (+1135662us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:07:25.676687 (+136894us) raft_consensus.cc:1685] finished
0709 13:07:25.676718 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:25.684771 (+  8053us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15464,"raft.queue_time_us":65445,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":10462,"spinlock_wait_cycles":30174592,"thread_start_us":333469,"threads_started":2,"wal-append.queue_time_us":401588}
I20260709 13:07:25.674028 21490 raft_consensus.cc:2804] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:25.692667 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:25.661892 21413 consensus_queue.cc:1048] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:07:25.687093 21759 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:25.700121 21759 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:25.700489 21759 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:25.695837 21967 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:25.696234 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:25.696473 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:25.665962 21969 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:25.701982 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:25.703630 21969 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:25.704269 16569 raft_consensus.cc:1275] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 13:07:25.704711 22017 log.cc:927] Time spent T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.169s	user 0.000s	sys 0.000s
W20260709 13:07:25.696269 22024 log.cc:927] Time spent T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:07:25.706910 21819 consensus_queue.cc:1048] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:25.683552 21901 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:25.692185 21868 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:25.680297 21858 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:25.718475 16010 raft_consensus.cc:1240] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:25.722958 16535 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:25.726706 16535 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:25.699182 21929 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:25.714468 16533 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:25.729987 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:25.730860 16565 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
W20260709 13:07:25.732689 15746 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 679229ffe7114facacb8e3a2c4c6d226 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:07:25.733814 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:25.734151 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:25.734529 15854 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
I20260709 13:07:25.735417 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:25.736193 15857 raft_consensus.cc:2368] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 18. Current term is 19.
I20260709 13:07:25.736922 21901 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:25.739048 21940 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:25.739480 21940 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:25.739859 21940 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:25.733826 15855 raft_consensus.cc:1240] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:25.744285 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:25.706941 15902 leader_election.cc:400] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 12. Current term is 13.
I20260709 13:07:25.745217 16565 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:25.745270 15902 leader_election.cc:403] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:25.716724 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:25.746222 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:25.744485 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:25.751219 15862 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:07:25.752761 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:25.720077 21858 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:25.755136 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 48336.5us
I20260709 13:07:25.755604 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:25.758288 15902 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:25.756701 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:25.717969 21490 raft_consensus.cc:697] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:25.759308 15857 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
I20260709 13:07:25.760639 16535 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:25.757468 21901 raft_consensus.cc:2749] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 12. Current term is 13.
I20260709 13:07:25.761677 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:25.762800 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:25.772367 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:25.769387 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:25.770123 16006 raft_consensus.cc:2410] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 14 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:07:25.777083 16535 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:25.764534 21731 raft_consensus.cc:2749] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:07:25.764194 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.778791 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:25.783167 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:25.783895 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:25.784924 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.786038 16565 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:25.740964 21854 raft_consensus.cc:2749] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:25.743384 22019 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:25.810316 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:25.800508 16533 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:25.810936 15928 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:25.802695 15929 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 18.
W20260709 13:07:25.803314 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.816708 16535 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:25.804745 16565 raft_consensus.cc:1275] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:07:25.821089 16533 leader_election.cc:400] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 election: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 18. Current term is 19.
I20260709 13:07:25.821561 16533 leader_election.cc:403] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:07:25.829154 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:25.831413 15935 raft_consensus.cc:1275] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 13:07:25.832603 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 71259.5us
I20260709 13:07:25.834681 15902 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:25.837394 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.830266 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:25.838871 15928 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 17.
I20260709 13:07:25.839448 15902 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:25.840804 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:25.841506 15928 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:25.819593 21891 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:25.809942 21969 raft_consensus.cc:2749] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:25.825408 21950 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:25.819871 22019 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:25.811991 21942 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:07:25.838328 22026 log.cc:927] Time spent T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:07:25.842367 21756 raft_consensus.cc:2749] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:25.825798 21941 raft_consensus.cc:2749] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:25.829022 21873 raft_consensus.cc:2749] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:25.770373 21929 consensus_queue.cc:1048] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.078s
I20260709 13:07:25.790712 21819 consensus_queue.cc:1048] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 13:07:25.768666 22022 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.087s
I20260709 13:07:25.846336 21854 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.872749 21854 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:25.875037 21854 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:25.853643 21901 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:25.861123 22019 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:25.858723 21891 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:25.860512 21928 consensus_meta.cc:287] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.738s	user 0.004s	sys 0.010s
W20260709 13:07:25.881306 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.879974 15902 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:25.867719 21969 raft_consensus.cc:2749] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 18. Current term is 19.
W20260709 13:07:25.887949 22003 log.cc:927] Time spent T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.659s	user 0.004s	sys 0.000s
I20260709 13:07:25.883529 22041 consensus_queue.cc:1048] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.013s
I20260709 13:07:25.896232 15931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2649) took 1365 ms. Trace:
I20260709 13:07:25.862287 21942 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:25.896632 15931 rpcz_store.cc:276] 0709 13:07:24.530944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:24.535067 (+  4123us) service_pool.cc:224] Handling call
0709 13:07:24.535427 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:25.196887 (+661460us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:25.895966 (+699079us) spinlock_profiling.cc:243] Waited 660 ms on lock 0x7b54000997d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:25.896165 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1518862080}
I20260709 13:07:25.900875 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:25.875846 21588 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:25.906991 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:25.879076 21891 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:25.908304 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:25.908923 15862 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:07:25.904755 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.905047 15854 raft_consensus.cc:1275] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:07:25.876829 21948 consensus_queue.cc:1059] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
W20260709 13:07:25.899181 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 48323.8us
I20260709 13:07:25.906570 16567 raft_consensus.cc:1275] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:07:25.911577 16533 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:25.884531 21731 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:25.871577 21759 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:25.910013 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 96690.4us
I20260709 13:07:25.919559 15933 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:07:25.920153 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:25.921377 15902 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:25.922821 15928 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 18.
I20260709 13:07:25.882754 21928 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:25.911085 21691 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.934099 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:25.866104 21490 consensus_queue.cc:237] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:25.944329 15862 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:25.946004 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:25.946854 16563 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 17.
I20260709 13:07:25.955217 16004 raft_consensus.cc:2393] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 18: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
W20260709 13:07:25.956212 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.956877 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:25.927922 21868 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:25.938014 21969 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:07:25.972401 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 51097.8us
I20260709 13:07:25.973974 15902 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:25.953418 21731 raft_consensus.cc:2749] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:25.964316 21758 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.983657 15855 raft_consensus.cc:1240] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:25.990190 15902 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:25.950337 21691 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:25.971837 21969 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:07:25.987113 22032 log.cc:927] Time spent T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
W20260709 13:07:26.007781 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 48687us
I20260709 13:07:26.008443 15974 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:25.991859 22039 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.994333 22041 raft_consensus.cc:2749] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:25.981038 21901 consensus_queue.cc:1048] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 13:07:26.001209 21969 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:25.994984 21758 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:26.028735 21758 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:26.038378 22053 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:26.038594 21588 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:26.024582 22039 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:26.038947 22053 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:26.039445 22053 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:26.035879 21940 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:26.042111 21940 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:26.024503 22037 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:07:26.043188 22035 log.cc:927] Time spent T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 13:07:26.034608 22048 consensus_queue.cc:1059] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:07:26.057435 21842 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:26.057791 22019 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:26.059046 22053 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:26.065114 21928 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:07:26.068956 21997 log.cc:927] Time spent T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.841s	user 0.008s	sys 0.000s
I20260709 13:07:26.070261 21858 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:26.061030 22034 log.cc:927] Time spent T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 13:07:26.062848 22053 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:26.075819 21969 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:26.079741 21969 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:26.082296 22040 raft_consensus.cc:3055] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:26.082877 22040 raft_consensus.cc:740] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:26.083931 22040 consensus_queue.cc:260] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:26.088577 22040 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:26.121327 15858 raft_consensus.cc:1240] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 15. Current term is 16. Ops: [15.10-15.10]
W20260709 13:07:26.122635 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 114068us
I20260709 13:07:26.128748 16564 raft_consensus.cc:1240] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:26.145197 21691 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:26.146533 21756 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:26.148389 21593 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:26.147809 21756 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:26.149392 21593 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:26.151664 21756 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:26.059640 21842 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:26.153787 21759 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:26.159427 21858 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:26.162458 21964 raft_consensus.cc:3055] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:26.162741 21927 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:26.163103 21964 raft_consensus.cc:740] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:26.164155 21842 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:26.165454 21964 consensus_queue.cc:260] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:26.167572 21964 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:26.169504 21833 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:26.171690 21927 consensus_queue.cc:1048] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:26.198020 21593 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:26.198274 22048 consensus_queue.cc:1243] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer 31882606e87447fb9a36bbf625129fe8 log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:07:26.211411 21891 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:26.218806 21891 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:26.220984 16002 raft_consensus.cc:1275] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:07:26.228760 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:26.229609 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:26.231000 16005 raft_consensus.cc:2393] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 15: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:26.229630 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:26.232540 15862 raft_consensus.cc:1240] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:26.223917 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:26.233830 16006 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:07:26.234879 15932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 2418) took 1478 ms. Trace:
I20260709 13:07:26.235399 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:26.236366 16008 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:26.237030 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:26.237305 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:26.238236 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:26.239117 16007 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 16.
I20260709 13:07:26.229799 16563 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:26.240473 21758 consensus_queue.cc:1048] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 13:07:26.240564 16009 raft_consensus.cc:1275] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 12. (term mismatch)
W20260709 13:07:26.248721 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 258483us
I20260709 13:07:26.241786 15974 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:26.250365 15902 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:26.246078 16009 pending_rounds.cc:85] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee: Aborting all ops after (but not including) 11
I20260709 13:07:26.258948 22056 raft_consensus.cc:2749] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:26.261420 21881 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:26.261703 21814 consensus_queue.cc:1048] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:26.248808 21891 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:26.264478 21891 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:26.257344 16009 pending_rounds.cc:107] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 12.12
I20260709 13:07:26.267778 16009 raft_consensus.cc:2889] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:26.269523 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:26.271651 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:26.272985 15861 raft_consensus.cc:2368] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 14. Current term is 15.
I20260709 13:07:26.282297 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:26.284970 15900 leader_election.cc:400] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 election: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 14. Current term is 15.
I20260709 13:07:26.285434 15900 leader_election.cc:403] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:07:26.286351 21819 raft_consensus.cc:2749] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:26.292028 21756 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:26.293483 21967 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:26.294339 21967 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:26.299794 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:26.235399 15932 rpcz_store.cc:276] 0709 13:07:24.756148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:24.885981 (+129833us) service_pool.cc:224] Handling call
0709 13:07:25.411651 (+525670us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:25.425891 (+ 14240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:25.425938 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:25.426001 (+    63us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:25.426048 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:25.426132 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:26.234433 (+808301us) spinlock_profiling.cc:243] Waited 525 ms on lock 0x7b54002c1850. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:26.234629 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21974,"raft.queue_time_us":13986,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":2458,"spinlock_wait_cycles":1207689088}
W20260709 13:07:26.307775 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:26.314486 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:26.315404 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:26.318840 16006 raft_consensus.cc:1240] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:26.322189 21956 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:26.325973 16570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1292) took 2153 ms. Trace:
I20260709 13:07:26.323494 21956 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:26.326624 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:26.326483 16570 rpcz_store.cc:276] 0709 13:07:24.172389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:24.214634 (+ 42245us) service_pool.cc:224] Handling call
0709 13:07:24.233216 (+ 18582us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:24.855565 (+622349us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:26.325393 (+1469828us) spinlock_profiling.cc:243] Waited 584 ms on lock 0x7b540041d554. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:26.325746 (+   353us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1343142784}
I20260709 13:07:26.328747 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:26.332904 22063 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:26.338853 16570 raft_consensus.cc:1275] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:07:26.347906 16008 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 17.
I20260709 13:07:26.349021 16563 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 19.
W20260709 13:07:26.351564 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 101491us
I20260709 13:07:26.352069 21943 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:26.353245 15902 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:26.355398 21901 raft_consensus.cc:2749] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:26.356233 22049 raft_consensus.cc:993] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea: : Instructing follower 85e997ba5003439781ec4f1f8db0c3ee to start an election
I20260709 13:07:26.339955 21942 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:26.357300 22049 raft_consensus.cc:1081] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 15 LEADER]: Signalling peer 85e997ba5003439781ec4f1f8db0c3ee to start an election
I20260709 13:07:25.810740 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:26.353052 21943 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:26.363216 21943 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:26.365439 22039 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:26.370174 22037 raft_consensus.cc:3055] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:26.370642 22037 raft_consensus.cc:740] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:26.373098 22063 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:26.368148 15860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2576) took 1593 ms. Trace:
I20260709 13:07:26.374687 15860 rpcz_store.cc:276] 0709 13:07:24.774079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:24.776013 (+  1934us) service_pool.cc:224] Handling call
0709 13:07:24.776306 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:25.325271 (+548965us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:26.367506 (+1042235us) spinlock_profiling.cc:243] Waited 548 ms on lock 0x7b54000b5c54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:26.367929 (+   423us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1260145536}
I20260709 13:07:26.379106 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:07:26.348560 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 331275us
I20260709 13:07:26.380180 22037 consensus_queue.cc:260] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:26.375752 22019 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:26.385375 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:26.387208 16008 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:07:26.385547 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:26.388168 16002 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:07:26.396437 22047 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:26.397324 22047 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:26.358867 21967 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:26.404912 15862 raft_consensus.cc:1240] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:26.405352 15860 raft_consensus.cc:1240] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:26.403523 22047 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:26.406343 15858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2580) took 1631 ms. Trace:
I20260709 13:07:26.406829 15858 rpcz_store.cc:276] 0709 13:07:24.775051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:25.167714 (+392663us) service_pool.cc:224] Handling call
0709 13:07:25.168139 (+   425us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:26.122672 (+954533us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:26.406019 (+283347us) spinlock_profiling.cc:243] Waited 952 ms on lock 0x7b540008e654. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:26.406285 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2189680000}
I20260709 13:07:26.407394 22037 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:07:26.402989 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 49930.3us
I20260709 13:07:26.411831 15902 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
W20260709 13:07:26.412282 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 79937.3us
I20260709 13:07:26.413257 15974 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:26.414695 21965 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:26.418823 16002 raft_consensus.cc:1275] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 13:07:26.419319 16002 pending_rounds.cc:85] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting all ops after (but not including) 9
I20260709 13:07:26.419611 16002 pending_rounds.cc:107] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 13:07:26.419965 16002 raft_consensus.cc:2889] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:26.421594 16008 raft_consensus.cc:1275] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:07:26.424135 22064 log.cc:927] Time spent T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 13:07:26.425899 21756 raft_consensus.cc:2749] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 14. Current term is 15.
I20260709 13:07:26.427100 21932 raft_consensus.cc:993] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8: : Instructing follower 85e997ba5003439781ec4f1f8db0c3ee to start an election
I20260709 13:07:26.453845 21932 raft_consensus.cc:1081] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Signalling peer 85e997ba5003439781ec4f1f8db0c3ee to start an election
I20260709 13:07:26.454223 15937 raft_consensus.cc:1275] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:07:26.454276 21956 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:26.408979 16003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41674 (request call id 2461) took 1169 ms. Trace:
I20260709 13:07:26.455435 16003 rpcz_store.cc:276] 0709 13:07:25.238920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:25.250461 (+ 11541us) service_pool.cc:224] Handling call
0709 13:07:26.408499 (+1158038us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b54000f0354. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:26.408746 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":236819712}
I20260709 13:07:26.435940 21857 consensus_queue.cc:1059] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:26.458524 21956 raft_consensus.cc:3055] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:26.459299 21956 raft_consensus.cc:740] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:26.437268 16009 raft_consensus.cc:1275] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:07:26.437863 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:26.445138 15856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2578) took 1670 ms. Trace:
I20260709 13:07:26.446468 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:26.448818 22063 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:26.449730 21842 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:26.450472 21593 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:26.451494 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:26.452669 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:26.481503 21819 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:26.482246 21819 raft_consensus.cc:697] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 13 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:26.478720 21842 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:26.483305 21819 consensus_queue.cc:237] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:26.483886 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 52918.8us
I20260709 13:07:26.485316 21842 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:26.485729 21413 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:26.486454 21413 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:26.487989 21593 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:26.489064 21413 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:26.490351 21593 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:26.491153 15935 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:26.496869 15937 raft_consensus.cc:1275] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:07:26.496618 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:26.498013 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:26.498740 15934 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:07:26.500468 16567 raft_consensus.cc:1275] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:07:26.503535 21869 consensus_queue.cc:1048] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:07:26.503399 15827 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:26.515741 22047 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:26.516398 22047 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:26.505460 15825 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:26.517053 22047 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:26.531807 15935 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 18.
I20260709 13:07:26.532284 16003 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6"
dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:41674
I20260709 13:07:26.540637 16003 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:26.541038 16003 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:26.525501 21869 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:26.525980 22063 consensus_queue.cc:1059] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.011s
I20260709 13:07:26.537330 21932 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:26.538223 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:26.521749 21891 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:26.549952 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:26.550711 21924 consensus_queue.cc:1059] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:26.551093 16566 raft_consensus.cc:2393] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 18: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:26.550144 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:26.552027 16564 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:07:26.512984 21593 consensus_queue.cc:1048] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:07:26.546483 21932 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:26.548298 21869 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:26.474874 15856 rpcz_store.cc:276] 0709 13:07:24.774602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:24.776541 (+  1939us) service_pool.cc:224] Handling call
0709 13:07:24.776958 (+   417us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:24.847077 (+ 70119us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:24.847109 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:24.847661 (+   552us) log.cc:844] Serialized 52 byte log entry
0709 13:07:25.170990 (+323329us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:25.504443 (+333453us) log.cc:844] Serialized 14 byte log entry
0709 13:07:25.504813 (+   370us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:26.068424 (+563611us) spinlock_profiling.cc:243] Waited 1.82 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:07:26.205013 (+136589us) spinlock_profiling.cc:243] Waited 5.38 ms on lock 0x7b180000f280. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb3f3 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd 000000000038f354 000000000038f199 00007f73e729e83a
0709 13:07:26.444525 (+239512us) spinlock_profiling.cc:243] Waited 39.3 ms on lock 0x7b54003c1254. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:26.444575 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:26.444673 (+    98us) raft_consensus.cc:1685] finished
0709 13:07:26.444686 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:26.444948 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30383,"prepare.queue_time_us":50293,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":82,"raft.queue_time_us":30638,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":31585,"spinlock_wait_cycles":106851840,"thread_start_us":580762,"threads_started":2,"wal-append.queue_time_us":329159}
I20260709 13:07:26.572922 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:26.573839 16566 raft_consensus.cc:2393] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 14: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:26.575106 21796 consensus_queue.cc:1048] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:26.576056 22051 consensus_queue.cc:1048] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:26.552985 21964 raft_consensus.cc:3055] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:26.582830 16004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41674 (request call id 2468) took 1030 ms. Trace:
I20260709 13:07:26.583994 16004 rpcz_store.cc:276] 0709 13:07:25.548435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:25.548619 (+   184us) service_pool.cc:224] Handling call
0709 13:07:26.562939 (+1014320us) spinlock_profiling.cc:243] Waited 404 ms on lock 0x7b54000eecd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:26.579055 (+ 16116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":929346816}
I20260709 13:07:26.567131 21932 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:07:26.587432 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 175792us
W20260709 13:07:26.590070 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 87799us
W20260709 13:07:26.596088 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 90868.5us
I20260709 13:07:26.583853 21964 raft_consensus.cc:740] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:26.595920 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:26.549057 21956 consensus_queue.cc:260] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:26.600517 15825 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:26.602011 16568 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:26.603024 16533 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:26.600431 16535 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:26.605062 22022 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:26.605571 22022 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:26.606017 22022 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:26.608240 22063 raft_consensus.cc:3055] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:26.608666 22063 raft_consensus.cc:740] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:26.609638 22063 consensus_queue.cc:260] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:26.615597 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:26.617116 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:26.598635 21967 consensus_queue.cc:1048] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.019s
I20260709 13:07:26.619746 16535 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:26.620837 22040 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:26.621238 22040 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:26.621320 16535 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:26.621616 22040 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:26.622200 21943 raft_consensus.cc:2749] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:26.611889 21413 consensus_queue.cc:1048] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
W20260709 13:07:26.629060 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:26.630889 16533 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:26.604149 21891 consensus_queue.cc:1048] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:07:26.610414 22047 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:26.644132 16003 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:26.643487 15976 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:26.632669 21942 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:07:26.651194 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:26.652108 16003 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:26.652397 15902 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:26.654131 22047 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:26.663420 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:26.650115 21869 leader_election.cc:290] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:26.675454 21731 raft_consensus.cc:2749] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:26.676131 21950 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:26.672852 21413 raft_consensus.cc:2749] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:26.685372 21965 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:26.691880 21965 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:26.691191 21950 raft_consensus.cc:697] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:26.726328 21950 consensus_queue.cc:237] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:26.728025 21964 consensus_queue.cc:260] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:26.729569 21964 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:26.692337 21965 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:26.733525 15756 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58448 (request call id 310) took 40718 ms (client timeout 15000 ms). Trace:
W20260709 13:07:26.698414 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 60953.6us
I20260709 13:07:26.741173 15976 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:26.742782 21842 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:26.744869 21720 consensus_queue.cc:1048] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:26.743502 21842 raft_consensus.cc:697] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:26.747891 21842 consensus_queue.cc:237] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:26.735901 15756 rpcz_store.cc:269] 0709 13:06:46.014601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.014789 (+   188us) service_pool.cc:224] Handling call
0709 13:07:26.733292 (+40718503us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:23.743768 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:23.782902 (+ 39134us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:23.784977 (+  2075us) write_op.cc:435] Acquired schema lock
0709 13:07:23.786657 (+  1680us) tablet.cc:662] Decoding operations
0709 13:07:23.790892 (+  4235us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:23.790978 (+    86us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:23.791008 (+    30us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:07:23.791327 (+   319us) tablet.cc:701] Row locks acquired
0709 13:07:23.791348 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:07:23.791487 (+   139us) write_op.cc:270] Start()
0709 13:07:23.791613 (+   126us) write_op.cc:276] Timestamp: P: 1783602443791443 usec, L: 0
0709 13:07:23.791634 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:07:23.792333 (+   699us) log.cc:844] Serialized 918 byte log entry
0709 13:07:24.421078 (+628745us) spinlock_profiling.cc:243] Waited 17.4 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e729e0ac 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b
0709 13:07:25.350269 (+929191us) spinlock_profiling.cc:243] Waited 12.6 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:07:25.375077 (+ 24808us) op_driver.cc:464] REPLICATION: finished
0709 13:07:25.431192 (+ 56115us) write_op.cc:301] APPLY: starting
0709 13:07:25.443314 (+ 12122us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:25.633008 (+189694us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:25.633036 (+    28us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:25.766342 (+133306us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:25.766561 (+   219us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=4,mrs_lookups=0
0709 13:07:25.766613 (+    52us) write_op.cc:312] APPLY: finished
0709 13:07:25.766838 (+   225us) log.cc:844] Serialized 33 byte log entry
0709 13:07:25.767010 (+   172us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:25.767099 (+    89us) write_op.cc:454] Released schema lock
0709 13:07:25.767157 (+    58us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:25.767569 (+   412us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:25.767601 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1569612,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":4443,"apply.run_cpu_time_us":7638,"apply.run_wall_time_us":384494,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12507,"num_ops":2,"prepare.queue_time_us":622494,"prepare.run_cpu_time_us":28967,"prepare.run_wall_time_us":936128,"raft.queue_time_us":21463,"raft.run_cpu_time_us":1278,"raft.run_wall_time_us":10768,"replication_time_us":1583228,"spinlock_wait_cycles":68965504,"thread_start_us":1379763,"threads_started":4,"wal-append.queue_time_us":963424}]]}
I20260709 13:07:26.702116 21956 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:26.751811 15974 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:26.752895 21936 raft_consensus.cc:2764] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
W20260709 13:07:26.754140 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 91993.3us
I20260709 13:07:26.755095 15974 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:26.757336 22037 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:26.758126 22037 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:26.759234 16568 raft_consensus.cc:1275] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:07:26.713909 21932 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:26.671253 21942 raft_consensus.cc:697] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:26.715392 16565 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
I20260709 13:07:26.721366 22038 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:26.720839 15758 catalog_manager.cc:5697] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 11 to 15, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 15 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
W20260709 13:07:26.722920 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 70646.2us
W20260709 13:07:26.691195 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 90899.6us
I20260709 13:07:26.764118 21936 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:07:26.757383 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 82684.1us
I20260709 13:07:26.670254 22063 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:26.775321 21936 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:26.807999 15902 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:26.808199 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 193968us
I20260709 13:07:26.809441 15827 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:26.800894 21936 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:26.815727 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 60774.1us
I20260709 13:07:26.817288 15974 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:26.820060 21705 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:26.820999 21720 consensus_queue.cc:1059] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:07:26.822397 21720 raft_consensus.cc:3055] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:26.822799 21720 raft_consensus.cc:740] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:26.828318 15825 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:26.819272 15974 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:26.810851 15758 catalog_manager.cc:5697] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:07:26.778139 22040 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:26.835546 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 50686.9us
I20260709 13:07:26.839146 22047 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:26.839856 22047 raft_consensus.cc:697] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 17 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:26.841096 22047 consensus_queue.cc:237] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:26.842896 21796 consensus_queue.cc:1059] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:07:26.810007 22037 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:26.847801 15827 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:26.802151 21593 consensus_queue.cc:1048] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:07:26.855955 15758 catalog_manager.cc:5697] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 9 to 12, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 12 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:07:26.855052 21720 consensus_queue.cc:260] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:26.863178 15902 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:26.864260 21819 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:26.864688 21819 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:26.865142 21819 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:26.865271 15902 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:26.866264 21756 consensus_queue.cc:1048] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:26.867944 22048 raft_consensus.cc:2749] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:26.869750 21588 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:26.870839 21720 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:26.876430 21932 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:26.876544 21817 raft_consensus.cc:3055] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:26.877228 22033 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:26.877324 21817 raft_consensus.cc:740] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:26.847095 22040 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:26.877825 21593 consensus_queue.cc:1243] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer eb1e0493af1e46dbb6dec7e25525bab5 log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:07:26.852208 21705 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:26.883556 22033 raft_consensus.cc:697] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 19 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:26.880425 15827 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:26.846047 22038 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:07:26.894632 15758 catalog_manager.cc:5697] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:26.924263 21987 raft_consensus.cc:2804] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:26.924474 15825 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:26.912356 21413 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:26.924832 21987 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:26.934015 15746 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 679229ffe7114facacb8e3a2c4c6d226 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:07:26.939419 15928 raft_consensus.cc:1275] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:07:26.939816 21956 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:26.940431 21956 raft_consensus.cc:697] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:26.946086 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:26.951305 15825 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:26.888187 21942 consensus_queue.cc:237] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:26.956334 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:26.957151 16568 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:26.960399 16567 raft_consensus.cc:1240] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:07:26.952487 15758 catalog_manager.cc:5697] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 10 to 11, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 11 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:26.899425 22033 consensus_queue.cc:237] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:26.963212 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 144074us
I20260709 13:07:26.942442 21987 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:26.966099 16564 raft_consensus.cc:1275] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:07:26.970484 16566 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 19.
I20260709 13:07:26.975857 21266 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:26.975880 15758 catalog_manager.cc:5697] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 11 to 12, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 12 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:07:26.992581 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 158906us
W20260709 13:07:27.017963 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:27.014344 21842 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.005s
W20260709 13:07:27.024433 22074 log.cc:927] Time spent T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:07:27.005170 21956 consensus_queue.cc:237] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:27.015128 21891 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:27.045217 21891 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:27.050084 22038 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:27.051023 21842 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.051730 21842 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:27.052448 22072 log.cc:927] Time spent T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:07:27.051051 15974 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:26.976594 21266 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:27.012619 15758 catalog_manager.cc:5697] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 11 to 12, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 12 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:07:27.060464 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 66897.6us
I20260709 13:07:27.065807 21266 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:27.068374 22022 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:27.070634 22022 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:27.057853 21593 raft_consensus.cc:2764] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:07:27.096544 15976 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:07:27.106429 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 161561us
I20260709 13:07:27.116251 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:27.117565 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:27.120306 15929 raft_consensus.cc:1275] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 13:07:27.120836 15929 pending_rounds.cc:85] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 8
I20260709 13:07:27.121141 15929 pending_rounds.cc:107] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:07:27.121430 15929 raft_consensus.cc:2889] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:27.122920 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:27.124202 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:27.129835 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:27.136621 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:27.139089 15758 catalog_manager.cc:5697] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 8 to 15. New cstate: current_term: 15 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:27.146519 21927 consensus_queue.cc:1048] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:27.143829 21842 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:27.152387 21842 raft_consensus.cc:2749] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:26.990684 21817 consensus_queue.cc:260] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:27.153098 21842 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:27.152884 21927 consensus_queue.cc:1048] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:07:27.153555 21842 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:27.153957 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:27.153961 21842 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:27.156142 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:27.156179 21936 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:27.157907 21891 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:27.166738 15758 catalog_manager.cc:5697] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 10 to 11, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 11 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:07:27.168610 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:27.169029 15758 catalog_manager.cc:5697] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:27.170786 22040 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:27.171056 15758 catalog_manager.cc:5697] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 13 to 15, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 15 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:27.159420 21936 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:27.176760 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:27.178237 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 114424us
I20260709 13:07:27.097090 21965 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:27.171464 22040 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:27.190726 16008 raft_consensus.cc:1240] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:07:27.202561 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:27.203807 16008 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:27.138255 21987 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:27.181334 22063 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:27.208308 22063 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:27.210675 21941 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.211064 22063 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:27.211330 21941 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:27.213465 22040 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:27.214056 22040 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:27.214365 21941 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:27.214636 22040 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:27.220335 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:27.221228 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:07:27.236881 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 196868us
I20260709 13:07:27.217887 21731 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:27.230252 15758 catalog_manager.cc:5697] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: config changed from index 3 to 10, term changed from 13 to 14, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 14 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:07:27.254644 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:27.255896 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:27.256585 15932 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:07:27.264922 16533 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.625s (SENT) [suppressed 20 similar messages]
I20260709 13:07:27.269838 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:27.249742 21588 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:07:27.271337 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 174872us
I20260709 13:07:27.252547 21593 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.245221 22051 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.276885 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:27.284003 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:27.288936 15976 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:27.278743 15758 catalog_manager.cc:5697] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:27.290386 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:27.303314 15935 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
I20260709 13:07:27.305182 15862 raft_consensus.cc:2410] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:07:27.309659 22078 consensus_queue.cc:1048] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.022s
I20260709 13:07:27.310310 16011 raft_consensus.cc:1275] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:07:27.312835 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:27.315762 15856 raft_consensus.cc:1275] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:07:27.323896 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:27.304490 21593 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:27.300169 21731 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:27.329970 21593 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:27.300917 21819 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:27.330698 21731 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:27.331961 15933 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:07:27.332558 21819 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:27.308092 21588 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:27.337543 21588 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:27.340485 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:27.353803 15932 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
I20260709 13:07:27.354048 15930 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:27.355976 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:27.357124 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:27.358188 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:27.361656 15862 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
W20260709 13:07:27.359915 22089 log.cc:927] Time spent T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:07:27.360260 21942 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:27.362323 15858 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:07:27.303146 21936 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:27.363173 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:27.374490 16535 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:27.436787 15856 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:07:27.375730 22022 consensus_queue.cc:1048] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:27.378659 15932 raft_consensus.cc:1275] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:07:27.380175 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:27.437712 15934 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:27.383033 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:27.393949 15855 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
I20260709 13:07:27.394480 15758 catalog_manager.cc:5697] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 8 to 12, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 12 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:27.403640 21987 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:27.362293 22051 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:27.399360 16008 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:07:27.423247 21817 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:27.387122 15902 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:27.434901 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:27.445581 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:27.440366 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:27.446544 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 72145.9us
I20260709 13:07:27.452088 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:27.438498 15931 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:07:27.448969 21950 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:27.455435 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:27.436245 21936 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:27.458704 15929 raft_consensus.cc:2410] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 18 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:07:27.456604 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:27.445245 22048 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.462141 15860 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
W20260709 13:07:27.453625 21964 consensus_meta.cc:287] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.724s	user 0.013s	sys 0.002s
I20260709 13:07:27.462889 15930 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:27.462387 15827 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:27.453368 22040 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:27.466730 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:27.472954 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:27.473651 15859 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 16.
I20260709 13:07:27.474483 15854 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 17.
I20260709 13:07:27.474831 21842 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:27.467353 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:27.468467 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:27.475066 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:27.464782 22037 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:27.461690 21936 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:27.492069 16565 pending_rounds.cc:85] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5: Aborting all ops after (but not including) 10
I20260709 13:07:27.494860 16565 pending_rounds.cc:107] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:07:27.495191 16565 raft_consensus.cc:2889] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:27.460507 21731 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:07:27.485349 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 98361.6us
I20260709 13:07:27.491585 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:27.491895 16568 raft_consensus.cc:1275] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:07:27.463227 21965 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:27.490123 15758 catalog_manager.cc:5697] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 9 to 11, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 11 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:07:27.501462 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:27.502629 15859 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:27.469981 21593 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:27.452145 21942 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:27.510208 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:27.508024 16533 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:27.514887 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:27.516626 15827 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:27.520833 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:27.521615 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:27.522437 16568 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:27.523932 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:27.527022 21266 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:27.465696 21950 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:07:27.527855 21266 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:27.530382 21266 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:27.531476 21942 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:27.535609 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:27.536602 22041 consensus_queue.cc:1059] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:27.536623 15902 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:27.538408 22063 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:27.539114 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:27.540352 16005 raft_consensus.cc:2393] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 18: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:27.541631 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:27.541913 16005 raft_consensus.cc:1240] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:27.522464 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:27.543828 16010 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:27.544360 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:27.550339 16008 raft_consensus.cc:3055] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:27.550849 16008 raft_consensus.cc:740] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:27.554291 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:27.557245 21842 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:27.502849 21731 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:27.559087 21731 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:27.559839 16008 consensus_queue.cc:260] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:27.561424 16008 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:27.565538 22079 log.cc:927] Time spent T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.413s	user 0.003s	sys 0.000s
I20260709 13:07:27.569407 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:27.570425 16002 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
I20260709 13:07:27.461308 22090 consensus_queue.cc:1059] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 13:07:27.571614 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:27.576488 15758 catalog_manager.cc:5697] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 9 to 14. New cstate: current_term: 14 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
W20260709 13:07:27.581063 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 64556.8us
I20260709 13:07:27.589848 16005 raft_consensus.cc:1240] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:27.621956 15861 raft_consensus.cc:1240] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:27.621902 15825 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:27.603248 15758 catalog_manager.cc:5697] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
I20260709 13:07:27.491204 22048 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:27.629560 15860 raft_consensus.cc:1275] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:07:27.629885 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:27.630548 16561 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:27.631065 22048 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:27.628921 15758 catalog_manager.cc:5697] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 6 to 10, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 10 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:07:27.633821 15758 catalog_manager.cc:5697] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 11 to 14, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 14 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } }
I20260709 13:07:27.630950 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:27.635329 15758 catalog_manager.cc:5697] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:07:27.636819 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:27.637733 21932 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:27.638509 21266 consensus_queue.cc:1048] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:07:27.640232 22087 log.cc:927] Time spent T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 13:07:27.640645 15825 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:27.641448 16002 raft_consensus.cc:1240] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:07:27.641570 21413 raft_consensus.cc:2749] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:27.642395 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:27.643087 16002 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def"
dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:41662
I20260709 13:07:27.643754 16002 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:27.644351 16002 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:27.579526 22037 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:27.600063 21950 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:27.647578 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:27.648327 15858 raft_consensus.cc:3055] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:27.648759 15858 raft_consensus.cc:740] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:27.580605 21593 raft_consensus.cc:697] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:27.649505 15858 consensus_queue.cc:260] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:27.650475 15858 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:27.650051 21593 consensus_queue.cc:237] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:27.649946 15758 catalog_manager.cc:5697] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:27.573422 15934 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 19.
I20260709 13:07:27.653730 16010 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
I20260709 13:07:27.661875 16561 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:27.663424 15928 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 19.
I20260709 13:07:27.663844 15976 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:27.620244 21987 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:27.667248 15856 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 17.
I20260709 13:07:27.667739 16002 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:27.677750 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 88074.7us
I20260709 13:07:27.677539 16010 raft_consensus.cc:1240] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:27.680043 15827 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:27.666792 15758 catalog_manager.cc:5697] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 11 to 12, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 12 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
I20260709 13:07:27.682096 15858 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:07:27.647989 22037 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:27.676054 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:27.672255 22088 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:07:27.692008 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:27.695310 16002 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:27.677275 21987 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:27.698062 16005 raft_consensus.cc:2368] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 14. Current term is 15.
I20260709 13:07:27.698098 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:27.699909 16561 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:27.704881 16006 raft_consensus.cc:3055] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:27.705307 16006 raft_consensus.cc:740] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:27.707129 15974 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:27.701380 21987 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:27.686770 21857 consensus_queue.cc:1048] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 13:07:27.700688 22088 raft_consensus.cc:697] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:27.718858 16006 consensus_queue.cc:260] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:27.734977 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:27.719434 22051 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:27.722867 21932 consensus_queue.cc:1048] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 13:07:27.745828 21705 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:27.759083 16561 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:27.777984 15976 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:07:27.794258 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 255189us
I20260709 13:07:27.811838 16004 raft_consensus.cc:1275] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:07:27.812023 16006 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:27.822256 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:27.822693 16568 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:07:27.803731 22037 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:27.833509 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:27.837059 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:27.807258 21927 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:27.808451 22038 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:27.806483 21705 raft_consensus.cc:697] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 15 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:27.857234 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:07:27.865226 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 168732us
I20260709 13:07:27.896440 15974 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:27.900746 15929 raft_consensus.cc:1275] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:07:27.856364 21819 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.928449 15827 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:27.930300 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:27.932600 15934 raft_consensus.cc:1240] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:27.934643 16568 raft_consensus.cc:1275] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:07:27.935624 16562 raft_consensus.cc:1240] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:27.936474 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:27.896081 22041 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:27.938527 15860 raft_consensus.cc:2368] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 13. Current term is 14.
W20260709 13:07:27.939419 22093 log.cc:927] Time spent T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 13:07:27.940140 15931 raft_consensus.cc:1275] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:07:27.953439 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:27.954761 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:27.934299 21819 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:27.941645 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:27.956895 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:27.958165 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:27.942464 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:27.960989 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:27.943562 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:27.962858 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:27.961836 15929 raft_consensus.cc:2368] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 11. Current term is 12.
I20260709 13:07:27.965061 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:27.959419 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:27.969656 15931 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:27.927345 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 227493us
I20260709 13:07:27.971855 15825 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:27.952521 21927 raft_consensus.cc:697] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:27.949079 15928 raft_consensus.cc:2376] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13. Re-sending same reply.
I20260709 13:07:27.896760 22037 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:27.977434 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:27.981508 15934 raft_consensus.cc:2376] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16. Re-sending same reply.
I20260709 13:07:27.981711 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:27.985739 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:27.985867 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:27.986723 15858 raft_consensus.cc:2393] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 17: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:28.005981 15860 raft_consensus.cc:1240] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:27.937388 21266 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:28.010965 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:28.013748 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:28.015741 15900 leader_election.cc:400] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 13. Current term is 14.
I20260709 13:07:28.021100 15900 leader_election.cc:403] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:28.017326 15856 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 11.
I20260709 13:07:28.017479 15934 raft_consensus.cc:1240] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 16. Current term is 17. Ops: [15.10-16.11]
W20260709 13:07:28.025097 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 45959.2us
I20260709 13:07:28.025957 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:28.026578 15928 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:07:28.031011 15932 raft_consensus.cc:2393] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 18: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:28.033480 16564 raft_consensus.cc:2393] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 15: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:28.027144 15825 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:28.035864 15933 raft_consensus.cc:1240] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 13:07:28.051966 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:28.055472 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:28.066931 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 95172.3us
I20260709 13:07:28.017591 22091 raft_consensus.cc:2749] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:27.932142 22088 consensus_queue.cc:237] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:28.029320 21857 raft_consensus.cc:2804] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:28.080578 15825 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:28.086768 15933 raft_consensus.cc:1240] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:07:28.091893 15825 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:28.110765 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 246523us
I20260709 13:07:27.986794 22041 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:28.076954 22110 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:28.139389 15933 raft_consensus.cc:1240] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:07:28.082603 22047 consensus_queue.cc:1048] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 13:07:28.107821 21857 raft_consensus.cc:697] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:28.203449 15935 raft_consensus.cc:2376] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13. Re-sending same reply.
I20260709 13:07:28.107431 22090 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.218812 15928 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:07:28.093988 21266 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:28.228936 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 137104us
I20260709 13:07:28.245359 21593 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:28.246836 21593 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:28.248139 21593 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:28.232931 16535 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:28.198091 22048 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:28.258435 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:28.257769 22094 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:28.132529 22124 consensus_queue.cc:1048] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.145s
W20260709 13:07:28.240057 22098 log.cc:927] Time spent T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.376s	user 0.000s	sys 0.001s
I20260709 13:07:28.264739 15933 raft_consensus.cc:2393] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 17: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:28.094317 21705 consensus_queue.cc:237] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:28.258591 22095 log.cc:927] Time spent T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.090s	user 0.001s	sys 0.001s
I20260709 13:07:28.253540 22051 raft_consensus.cc:2804] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:28.243551 22090 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:28.281962 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:28.282886 15928 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:28.271706 22048 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:28.293922 21857 consensus_queue.cc:237] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:28.231063 22108 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 13. Current term is 14.
I20260709 13:07:28.284618 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:07:28.304150 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 74575.6us
I20260709 13:07:28.307646 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:28.308707 15933 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:28.297350 21266 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:28.266671 22103 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.305826 15974 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:28.278529 22051 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:28.210666 21927 consensus_queue.cc:237] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:28.313225 21819 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:28.268636 22094 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:28.334128 22099 log.cc:927] Time spent T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.386s	user 0.001s	sys 0.000s
W20260709 13:07:28.377480 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 108162us
I20260709 13:07:28.332719 22051 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:28.389108 16006 consensus_meta.cc:287] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.576s	user 0.009s	sys 0.002s
I20260709 13:07:28.388967 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:28.389393 16570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1326) took 1016 ms. Trace:
I20260709 13:07:28.334512 22110 raft_consensus.cc:697] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:28.394325 16006 raft_consensus.cc:1275] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:07:28.391707 16570 rpcz_store.cc:276] 0709 13:07:27.373512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.373993 (+   481us) service_pool.cc:224] Handling call
0709 13:07:27.374291 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:27.374699 (+   408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:27.374730 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:27.375234 (+   504us) log.cc:844] Serialized 26 byte log entry
0709 13:07:27.701974 (+326740us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:27.702018 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:27.702083 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:28.389164 (+687081us) raft_consensus.cc:1685] finished
0709 13:07:28.389184 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:28.389331 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1332,"raft.run_cpu_time_us":64,"raft.run_wall_time_us":66,"thread_start_us":326401,"threads_started":1,"wal-append.queue_time_us":433168}
W20260709 13:07:28.387878 22100 log.cc:927] Time spent T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
W20260709 13:07:28.401402 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 95675.1us
W20260709 13:07:28.402593 15827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:28.354127 21857 consensus_queue.cc:1048] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 13:07:28.390018 22037 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:28.408949 15929 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:28.393837 21950 raft_consensus.cc:2749] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:28.392768 22048 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:28.360694 22103 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:28.400902 22112 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.419320 22126 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:28.426121 22126 raft_consensus.cc:697] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 19 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:28.426764 22123 consensus_queue.cc:1048] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:28.426919 22127 consensus_queue.cc:1059] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:28.428529 22123 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:28.429152 22123 raft_consensus.cc:697] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:28.430096 22123 consensus_queue.cc:237] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:28.430735 22110 consensus_queue.cc:237] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:28.431941 22109 log.cc:927] Time spent T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.324s	user 0.000s	sys 0.001s
I20260709 13:07:28.432433 21965 raft_consensus.cc:3055] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:28.432950 21965 raft_consensus.cc:740] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:28.434010 21965 consensus_queue.cc:260] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:28.435767 21965 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:07:28.439818 22097 log.cc:927] Time spent T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.347s	user 0.001s	sys 0.000s
W20260709 13:07:28.440065 21731 consensus_meta.cc:287] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.876s	user 0.005s	sys 0.008s
W20260709 13:07:28.441105 22096 log.cc:927] Time spent T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.355s	user 0.001s	sys 0.000s
I20260709 13:07:28.440682 21731 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:28.442241 15930 raft_consensus.cc:2368] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 17. Current term is 18.
I20260709 13:07:28.443133 16566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1321) took 1079 ms. Trace:
W20260709 13:07:28.443770 22115 log.cc:927] Time spent T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:07:28.419932 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:28.445050 16569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1325) took 1072 ms. Trace:
I20260709 13:07:28.445389 16569 rpcz_store.cc:276] 0709 13:07:27.372877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.373174 (+   297us) service_pool.cc:224] Handling call
0709 13:07:27.373510 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:27.705422 (+331912us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:27.705455 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:27.730387 (+ 24932us) log.cc:844] Serialized 26 byte log entry
0709 13:07:27.953378 (+222991us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:27.953919 (+   541us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:27.954320 (+   401us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:28.444838 (+490518us) raft_consensus.cc:1685] finished
0709 13:07:28.444859 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:28.444983 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1877,"raft.queue_time_us":2308,"raft.run_cpu_time_us":47,"raft.run_wall_time_us":47,"thread_start_us":473852,"threads_started":2,"wal-append.queue_time_us":577208}
I20260709 13:07:28.446349 21731 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:28.448181 16009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2492) took 1273 ms. Trace:
I20260709 13:07:28.448702 16009 rpcz_store.cc:276] 0709 13:07:27.175504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.175647 (+   143us) service_pool.cc:224] Handling call
0709 13:07:27.181799 (+  6152us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:27.189756 (+  7957us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:27.189790 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:27.214894 (+ 25104us) log.cc:844] Serialized 26 byte log entry
0709 13:07:27.504694 (+289800us) spinlock_profiling.cc:243] Waited 4.2 ms on lock 0x7b180046e400. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f480d1 00007f73ea363a0f
0709 13:07:27.813361 (+308667us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:27.813402 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:27.813464 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:28.447767 (+634303us) raft_consensus.cc:1685] finished
0709 13:07:28.447791 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:28.448080 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":100,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":7117,"spinlock_wait_cycles":9665792,"thread_start_us":306826,"threads_started":1,"wal-append.queue_time_us":867591}
I20260709 13:07:28.419584 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:28.453627 15862 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:28.421401 22112 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:28.454761 15931 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:28.443689 16566 rpcz_store.cc:276] 0709 13:07:27.363960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.364116 (+   156us) service_pool.cc:224] Handling call
0709 13:07:27.370277 (+  6161us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:27.376372 (+  6095us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:27.376403 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:27.376855 (+   452us) log.cc:844] Serialized 26 byte log entry
0709 13:07:27.746214 (+369359us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:27.746259 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:27.746335 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:28.442786 (+696451us) raft_consensus.cc:1685] finished
0709 13:07:28.442817 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:28.443019 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5606,"raft.queue_time_us":6605,"raft.run_cpu_time_us":73,"raft.run_wall_time_us":68,"thread_start_us":368937,"threads_started":1,"wal-append.queue_time_us":488997}
I20260709 13:07:28.456182 22112 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:28.457113 21731 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.457613 21731 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:28.458889 22120 log.cc:927] Time spent T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
W20260709 13:07:28.459703 22104 log.cc:927] Time spent T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 13:07:28.460880 16007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 1175) took 1103 ms. Trace:
I20260709 13:07:28.460951 22090 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:28.391804 22125 consensus_queue.cc:1048] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 13:07:28.461304 16007 rpcz_store.cc:276] 0709 13:07:27.357949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.536877 (+178928us) service_pool.cc:224] Handling call
0709 13:07:27.537262 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:27.537659 (+   397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:27.537691 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:27.538195 (+   504us) log.cc:844] Serialized 26 byte log entry
0709 13:07:27.823324 (+285129us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:27.823350 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:27.823396 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:28.460668 (+637272us) raft_consensus.cc:1685] finished
0709 13:07:28.460689 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:28.460828 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":20497,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":101,"thread_start_us":283941,"threads_started":1,"wal-append.queue_time_us":690904}
I20260709 13:07:28.464210 22094 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:28.465143 22130 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:28.415270 22124 consensus_queue.cc:1059] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 13:07:28.459838 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:28.467016 15936 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2681) took 1184 ms. Trace:
I20260709 13:07:28.465762 22130 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:28.469405 15931 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 17.
W20260709 13:07:28.444577 21987 consensus_meta.cc:287] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.699s	user 0.010s	sys 0.001s
I20260709 13:07:28.492792 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:28.420138 22041 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:28.496484 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 80153us
W20260709 13:07:28.498585 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 430557us
W20260709 13:07:28.502022 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:28.502965 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:28.412529 15974 connection.cc:256] RPC call timeout handler was delayed by -1.1405259179999803s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23470000000000002s for a response
W20260709 13:07:28.422420 22128 log.cc:927] Time spent T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:07:28.467428 15936 rpcz_store.cc:276] 0709 13:07:27.286456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.291310 (+  4854us) service_pool.cc:224] Handling call
0709 13:07:27.308905 (+ 17595us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:27.352277 (+ 43372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:27.352307 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:27.432522 (+ 80215us) log.cc:844] Serialized 52 byte log entry
0709 13:07:27.708181 (+275659us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:27.950972 (+242791us) log.cc:844] Serialized 14 byte log entry
0709 13:07:27.954554 (+  3582us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:27.956250 (+  1696us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:28.466787 (+510537us) raft_consensus.cc:1685] finished
0709 13:07:28.466807 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:28.466944 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1602,"raft.queue_time_us":11718,"raft.run_cpu_time_us":316,"raft.run_wall_time_us":80512,"thread_start_us":468533,"threads_started":2,"wal-append.queue_time_us":399019}
I20260709 13:07:28.490423 21731 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:28.512771 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:28.494516 22048 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:28.513805 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:28.520488 16007 raft_consensus.cc:2393] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 15: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
W20260709 13:07:28.497606 21936 consensus_meta.cc:287] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.922s	user 0.008s	sys 0.005s
I20260709 13:07:28.427115 22126 consensus_queue.cc:237] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:28.517978 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 285121us
W20260709 13:07:28.512208 15974 connection.cc:256] RPC call timeout handler was delayed by -1.0712259179999819s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2424s for a response
I20260709 13:07:28.517009 15827 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:28.523083 16535 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:28.519621 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:28.526492 16006 raft_consensus.cc:1240] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:28.526849 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:28.527817 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:28.528131 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:28.529940 16009 raft_consensus.cc:1275] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:07:28.529304 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:28.533345 16535 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:28.534375 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:28.536918 16006 raft_consensus.cc:1275] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:07:28.492394 21987 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:28.539130 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:28.539951 16003 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 13.
I20260709 13:07:28.545228 16002 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 15.
I20260709 13:07:28.546980 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:28.531282 21266 raft_consensus.cc:2749] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:28.549847 22110 consensus_queue.cc:1059] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
W20260709 13:07:28.564087 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:28.525162 21936 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:28.541456 22048 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:28.577044 22148 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.577785 22148 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:28.538125 21950 raft_consensus.cc:2764] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:07:28.581573 16010 raft_consensus.cc:1240] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 13. Current term is 14. Ops: [13.12-13.12]
W20260709 13:07:28.592428 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 63294.7us
I20260709 13:07:28.551174 21942 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:28.549422 22088 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:28.608541 22145 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:28.609401 22145 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:28.612125 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 1.14002e+06us
I20260709 13:07:28.613358 16533 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:28.618285 22148 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:28.622725 15862 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:28.554534 22094 consensus_queue.cc:1048] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.071s
I20260709 13:07:28.631573 15902 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:28.638324 15902 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:28.660849 15902 leader_election.cc:400] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 election: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 14. Current term is 15.
W20260709 13:07:28.661221 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 47993.9us
I20260709 13:07:28.621639 21593 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:28.662948 22152 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:28.665700 22152 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:28.666870 21593 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:28.667295 22146 log.cc:927] Time spent T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 13:07:28.638346 22145 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:28.674170 22094 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:07:28.662204 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 68685.2us
W20260709 13:07:28.662712 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 117004us
I20260709 13:07:28.630970 22088 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:28.642122 22041 raft_consensus.cc:2749] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:28.689472 16002 raft_consensus.cc:1240] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:07:28.643391 21731 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:28.693753 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:28.696049 16003 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 11.
I20260709 13:07:28.671254 22051 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:07:28.723986 16007 pending_rounds.cc:85] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting all ops after (but not including) 10
I20260709 13:07:28.724463 16007 pending_rounds.cc:107] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 13:07:28.724785 16007 raft_consensus.cc:2889] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:28.709393 15900 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:28.637645 21950 consensus_queue.cc:1048] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
W20260709 13:07:28.726014 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 130381us
W20260709 13:07:28.726418 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 58006.7us
I20260709 13:07:28.731845 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:28.657402 22108 consensus_queue.cc:1059] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 13:07:28.744928 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:28.746479 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:28.693626 22048 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:28.750115 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:28.790827 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:28.791528 15862 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 17.
I20260709 13:07:28.699126 22125 consensus_queue.cc:1048] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.043s
W20260709 13:07:28.790617 15974 leader_election.cc:336] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 1.739s (SENT)
I20260709 13:07:28.796244 15855 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 13.
I20260709 13:07:28.701438 22154 consensus_queue.cc:1059] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.114s
I20260709 13:07:28.722200 21936 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:28.802584 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 251091us
I20260709 13:07:28.800391 15937 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 1219) took 1831 ms. Trace:
I20260709 13:07:28.698010 21942 raft_consensus.cc:697] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:28.791921 15827 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:28.793022 16563 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
I20260709 13:07:28.799499 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:28.739446 22130 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:28.749185 21731 raft_consensus.cc:2804] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:07:28.814360 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:28.841543 22155 consensus_queue.cc:1059] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.105s
I20260709 13:07:28.843098 22159 consensus_queue.cc:1059] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:07:28.844250 21588 raft_consensus.cc:3055] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:28.844727 21588 raft_consensus.cc:740] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:28.845697 21588 consensus_queue.cc:260] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:28.846722 16533 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:28.847009 21588 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:28.808804 21819 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:28.848265 16533 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:07:28.825165 22121 log.cc:927] Time spent T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:07:28.796007 22108 consensus_queue.cc:1059] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:07:28.847968 22162 raft_consensus.cc:2749] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:28.828037 22126 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:28.824745 21731 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:07:28.880304 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 140494us
W20260709 13:07:28.869867 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 127326us
W20260709 13:07:28.869983 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 78562.5us
I20260709 13:07:28.872884 22038 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:28.876317 21819 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:28.877079 21942 consensus_queue.cc:237] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:28.859236 22154 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:28.868759 22094 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:28.882550 15900 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:28.879974 21731 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:07:28.885854 15974 leader_election.cc:336] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 1.744s (SENT)
I20260709 13:07:28.886397 22038 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:28.891748 22179 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:07:28.892601 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 44437.3us
I20260709 13:07:28.883064 15902 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:28.880515 22126 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:28.916165 22126 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:28.916638 16533 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:28.917599 22162 consensus_queue.cc:1048] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:07:28.918287 16533 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:28.917263 21588 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:07:28.920186 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:28.922111 15974 leader_election.cc:336] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 1.822s (SENT)
I20260709 13:07:28.922567 15974 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:28.901804 16011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 1174) took 1544 ms. Trace:
I20260709 13:07:28.923828 16011 rpcz_store.cc:276] 0709 13:07:27.357674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.525293 (+167619us) service_pool.cc:224] Handling call
0709 13:07:27.525673 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:27.525955 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:07:27.863828 (+337873us) log.cc:844] Serialized 14 byte log entry
0709 13:07:28.894159 (+1030331us) spinlock_profiling.cc:243] Waited 5.84 ms on lock 0x7b180020c980. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e9f504bc 00007f73e729e48b 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72b8455 00007f73ea3a0060 00007f73ea3b2716 00007f73ea3c3287
0709 13:07:28.894513 (+   354us) log.cc:844] Serialized 14 byte log entry
0709 13:07:28.901342 (+  6829us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:28.901412 (+    70us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:28.901487 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:28.901575 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:28.901736 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25933,"prepare.queue_time_us":480839,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":71,"raft.queue_time_us":312150,"raft.run_cpu_time_us":295,"raft.run_wall_time_us":18933,"spinlock_wait_cycles":36799616,"thread_start_us":714021,"threads_started":3,"wal-append.queue_time_us":627608}
I20260709 13:07:28.925495 22147 raft_consensus.cc:2749] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:28.919993 21588 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:28.926684 21588 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:28.848609 22163 consensus_queue.cc:1048] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 13:07:28.905748 22108 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.940121 22156 raft_consensus.cc:2749] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:28.940274 22108 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:28.942651 22108 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:28.908053 22179 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:28.908798 22154 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:28.910370 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:28.907385 22051 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:28.953574 16533 leader_election.cc:304] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:28.931771 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 48779.6us
W20260709 13:07:28.923825 15974 leader_election.cc:336] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 1.906s (SENT)
I20260709 13:07:28.957815 15902 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:28.947966 21964 raft_consensus.cc:3055] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:28.962752 21964 raft_consensus.cc:740] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:28.963593 21964 consensus_queue.cc:260] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:28.964094 15902 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:28.964962 21964 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:28.948544 22179 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:28.950629 22154 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:28.956406 16533 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:28.939743 22162 raft_consensus.cc:2764] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader election decision vote started in defunct term 15: won
I20260709 13:07:28.959879 22157 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:28.969882 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:28.978176 15933 raft_consensus.cc:3055] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:28.978603 15933 raft_consensus.cc:740] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:07:28.982436 15974 leader_election.cc:336] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 1.976s (SENT)
I20260709 13:07:28.983260 15974 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:28.995595 22083 raft_consensus.cc:3055] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:28.996165 22083 raft_consensus.cc:740] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:28.997309 22083 consensus_queue.cc:260] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:28.998667 22083 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:29.002162 21731 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:29.003186 22118 log.cc:927] Time spent T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.747s	user 0.007s	sys 0.005s
I20260709 13:07:29.005597 16010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 2527) took 1452 ms. Trace:
I20260709 13:07:29.009843 16010 rpcz_store.cc:276] 0709 13:07:27.552982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.693504 (+140522us) service_pool.cc:224] Handling call
0709 13:07:27.702613 (+  9109us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:28.589694 (+887081us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.005273 (+415579us) spinlock_profiling.cc:243] Waited 867 ms on lock 0x7b54001f7cd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:29.005520 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1993993984}
I20260709 13:07:28.976380 22108 raft_consensus.cc:2804] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:29.010931 22108 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:28.988512 22038 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:29.011518 22108 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:29.016084 15932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 2447) took 1074 ms. Trace:
I20260709 13:07:28.946080 22126 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:29.017484 22174 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:29.018143 22174 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:29.020754 22174 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:29.021353 21731 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:29.023833 22119 log.cc:927] Time spent T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.503s	user 0.000s	sys 0.002s
I20260709 13:07:29.025315 22038 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:29.029189 22144 raft_consensus.cc:3055] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:29.029718 22144 raft_consensus.cc:740] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:29.030725 22144 consensus_queue.cc:260] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:29.032835 22041 raft_consensus.cc:3055] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:29.033293 22041 raft_consensus.cc:740] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:29.034236 22041 consensus_queue.cc:260] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:28.969894 21942 raft_consensus.cc:2804] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:29.036581 21942 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:29.037007 21942 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:29.039510 21588 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:28.977511 22157 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:29.006582 22127 raft_consensus.cc:3055] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:29.016572 15932 rpcz_store.cc:276] 0709 13:07:27.941864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.962574 (+ 20710us) service_pool.cc:224] Handling call
0709 13:07:29.015823 (+1053249us) spinlock_profiling.cc:243] Waited 67.2 ms on lock 0x7b5400328a54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:29.015984 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":154672000}
W20260709 13:07:29.054299 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:29.061569 15827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:28.977694 22163 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:07:29.051590 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 60944.6us
I20260709 13:07:29.035526 22041 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:29.051364 15933 consensus_queue.cc:260] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:29.080204 15933 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:29.011433 22103 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:29.088212 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:29.090472 16010 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:07:29.053431 22126 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:29.093084 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:29.106879 15932 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:46118
I20260709 13:07:29.092192 22048 raft_consensus.cc:2749] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:29.072243 22157 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:29.124401 16010 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:07:29.126127 21588 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:29.128577 22162 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:29.072373 22127 raft_consensus.cc:740] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:29.084029 22144 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:29.081213 22163 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:29.156316 16533 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:29.163556 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 181160us
I20260709 13:07:29.129451 22162 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:29.154838 22163 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:29.171422 22108 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:29.175813 22164 log.cc:927] Time spent T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.334s	user 0.004s	sys 0.000s
I20260709 13:07:29.133155 22122 raft_consensus.cc:2749] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:29.132928 22145 consensus_queue.cc:1048] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 13:07:29.141999 21588 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:29.184316 21588 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:29.185410 22162 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:29.185575 15933 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
W20260709 13:07:29.187418 22130 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
W20260709 13:07:29.191068 15974 leader_election.cc:336] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 2.000s (SENT)
I20260709 13:07:29.191617 15974 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:29.192552 21936 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:07:29.193001 15974 leader_election.cc:336] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 2.077s (SENT)
I20260709 13:07:29.193517 15974 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:29.193837 21588 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:29.194674 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:29.194689 21936 raft_consensus.cc:2749] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:29.197940 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:29.199021 15932 raft_consensus.cc:606] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Received request to transfer leadership to TS 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:29.199468 15932 raft_consensus.cc:614] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:07:29.202237 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:29.202689 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 46991us
I20260709 13:07:29.203465 16533 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:29.246402 15930 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 2435) took 1347 ms (client timeout 1744 ms). Trace:
I20260709 13:07:29.247865 15934 raft_consensus.cc:1240] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:29.253232 21942 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:29.193009 22152 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:29.256757 21942 leader_election.cc:290] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:29.258451 21942 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:29.258996 21942 raft_consensus.cc:697] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:07:29.246884 15930 rpcz_store.cc:269] 0709 13:07:27.898575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.898754 (+   179us) service_pool.cc:224] Handling call
0709 13:07:29.246160 (+1347406us) spinlock_profiling.cc:243] Waited 508 ms on lock 0x7b54002c2754. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:29.246335 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1168037632}
I20260709 13:07:29.206418 22094 raft_consensus.cc:2804] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:29.245697 22127 consensus_queue.cc:260] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:29.259404 22106 log.cc:927] Time spent T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.868s	user 0.006s	sys 0.000s
I20260709 13:07:29.290889 15974 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:29.253549 22088 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:29.300011 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:29.302132 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:29.304422 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:29.306177 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:29.306977 16006 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
I20260709 13:07:29.307966 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:29.247390 22108 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:29.315112 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:29.373168 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 82420.4us
I20260709 13:07:29.373137 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:29.375214 16562 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:29.268978 22147 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:29.376470 22147 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:29.379544 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:29.380204 16006 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 13.
I20260709 13:07:29.380717 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:29.381618 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:29.382836 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:29.382898 16006 raft_consensus.cc:2410] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
W20260709 13:07:29.357971 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:29.384238 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:29.384667 15930 raft_consensus.cc:1275] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 13:07:29.384959 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:29.386406 16011 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 15.
I20260709 13:07:29.379555 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:29.381716 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:29.390455 16007 raft_consensus.cc:2393] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 13: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:29.269867 22094 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:29.337888 15976 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:29.391376 16563 raft_consensus.cc:1240] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:29.395934 16562 raft_consensus.cc:2410] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:07:29.314998 22127 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:29.314333 21927 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:29.400096 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:29.297365 22148 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:29.314532 22123 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:29.414184 15974 leader_election.cc:400] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 election: Vote denied by peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) with higher term. Message: Invalid argument: T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 11. Current term is 12.
I20260709 13:07:29.414637 15974 leader_election.cc:403] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:07:29.318133 22157 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:29.425350 16570 raft_consensus.cc:1240] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:29.399189 22094 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:29.428377 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:29.429046 16563 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:29.317562 22130 consensus_queue.cc:1059] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.087s
I20260709 13:07:28.809478 15937 rpcz_store.cc:276] 0709 13:07:26.968555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:26.968715 (+   160us) service_pool.cc:224] Handling call
0709 13:07:26.989379 (+ 20664us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:26.992696 (+  3317us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:07:27.056513 (+ 63817us) log.cc:844] Serialized 14 byte log entry
0709 13:07:27.411246 (+354733us) spinlock_profiling.cc:243] Waited 954 us on lock 0x7b1800444a00. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f4823a 00007f73ea3b2a2a
0709 13:07:28.693379 (+1282133us) spinlock_profiling.cc:243] Waited 18 ms on lock 0x7b1800444a00. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e7235c0a 00007f73f0975219 00007f73e72bda40 00007f73e72b84e4 00007f73e72b8455 00007f73edda03c7 00007f73ea3b29a7 00007f73ea3c20bf 00007f73ea3c2038
0709 13:07:28.792571 (+ 99192us) log.cc:844] Serialized 14 byte log entry
0709 13:07:28.799936 (+  7365us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:28.800003 (+    67us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:28.800054 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:28.800162 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:28.800320 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11763,"prepare.queue_time_us":1138913,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":34526,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":4322,"spinlock_wait_cycles":43652352,"thread_start_us":343528,"threads_started":2,"wal-append.queue_time_us":743304}
I20260709 13:07:29.469710 21817 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:29.478642 15854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2599) took 1855 ms. Trace:
I20260709 13:07:29.445160 21927 raft_consensus.cc:2804] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:29.474036 22088 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) with higher term. Message: Invalid argument: T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 11. Current term is 12.
I20260709 13:07:29.426806 22123 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:29.481669 15854 rpcz_store.cc:276] 0709 13:07:27.623703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.623874 (+   171us) service_pool.cc:224] Handling call
0709 13:07:27.636096 (+ 12222us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:27.636541 (+   445us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:27.636573 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:27.637101 (+   528us) log.cc:844] Serialized 26 byte log entry
0709 13:07:27.805061 (+167960us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:27.805108 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:27.805181 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:29.256977 (+1451796us) spinlock_profiling.cc:243] Waited 1.69 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:07:29.433289 (+176312us) raft_consensus.cc:1685] finished
0709 13:07:29.433310 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:29.477051 (+ 43741us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1087,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":101,"spinlock_wait_cycles":3880704,"thread_start_us":167456,"threads_started":1,"wal-append.queue_time_us":664455}
W20260709 13:07:29.493916 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 99244.8us
W20260709 13:07:29.495534 22186 log.cc:927] Time spent T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
I20260709 13:07:29.483589 22041 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:07:29.496584 22187 log.cc:927] Time spent T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.229s	user 0.000s	sys 0.001s
I20260709 13:07:29.496943 22041 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:29.485910 21817 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:29.502396 15974 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:29.503655 22195 raft_consensus.cc:2749] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:29.489701 22130 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:29.506637 22130 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:29.507380 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:29.512838 21817 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:29.514966 22041 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:29.495286 15744 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 83a1b79cdcf046c78453695c798a14a5 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 8430db80a39d4868bcf7641c16de35ea is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:07:29.517364 15744 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 109317us
I20260709 13:07:29.481660 22108 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:29.484143 21927 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:29.473122 22163 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:29.471935 22148 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:29.522271 15974 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:29.530632 22195 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:29.471294 21942 consensus_queue.cc:237] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:29.525497 21927 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:29.533031 15974 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:29.526439 22147 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:29.532006 22195 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:29.525249 22108 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:29.567838 15974 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:29.545109 22088 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:29.570156 22123 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:29.570834 15974 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:29.580860 16008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 1182) took 1052 ms. Trace:
I20260709 13:07:29.548542 22157 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:29.589174 22163 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:29.589263 22148 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:29.591503 16008 rpcz_store.cc:276] 0709 13:07:28.528569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.539107 (+ 10538us) service_pool.cc:224] Handling call
0709 13:07:28.539467 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:28.583402 (+ 43935us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:07:28.632165 (+ 48763us) log.cc:844] Serialized 14 byte log entry
0709 13:07:29.264431 (+632266us) spinlock_profiling.cc:243] Waited 16.5 ms on lock 0x7b180037a920. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f4823a 00007f73ea3b2a2a
0709 13:07:29.533409 (+268978us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:29.558070 (+ 24661us) log.cc:844] Serialized 26 byte log entry
0709 13:07:29.558252 (+   182us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:29.558303 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.558398 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:29.580427 (+ 22029us) raft_consensus.cc:1685] finished
0709 13:07:29.580522 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:29.580791 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43580,"prepare.queue_time_us":35878,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":1984,"raft.queue_time_us":78981,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":59857,"spinlock_wait_cycles":37913984,"thread_start_us":268678,"threads_started":2,"wal-append.queue_time_us":887282}
I20260709 13:07:29.566048 22195 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:29.604485 22189 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:29.606087 22156 raft_consensus.cc:3055] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:29.667936 22156 raft_consensus.cc:740] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:29.668990 22156 consensus_queue.cc:260] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:29.670470 22156 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:29.599193 22130 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:29.630014 16004 raft_consensus.cc:1275] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:07:29.632906 22108 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:29.647373 15935 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 2437) took 1708 ms (client timeout 1906 ms). Trace:
W20260709 13:07:29.619395 15827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:29.666146 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 95382.5us
I20260709 13:07:29.590437 22088 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:29.592414 22147 raft_consensus.cc:2749] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:29.603276 22161 raft_consensus.cc:2749] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:29.626026 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:29.680373 15935 rpcz_store.cc:269] 0709 13:07:27.939089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:27.967481 (+ 28392us) service_pool.cc:224] Handling call
0709 13:07:29.647122 (+1679641us) spinlock_profiling.cc:243] Waited 212 ms on lock 0x7b5400327654. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:29.647293 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":487128960}
I20260709 13:07:29.688661 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:29.694260 15931 raft_consensus.cc:2410] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
W20260709 13:07:29.695912 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 358139us
I20260709 13:07:29.700508 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:29.701545 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:29.714489 16533 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.002s	sys 0.013s
W20260709 13:07:29.714993 16533 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.002s	sys 0.013s
I20260709 13:07:29.724764 15862 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
I20260709 13:07:29.726132 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:29.726933 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:29.681751 22088 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:29.733002 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:29.734962 15862 raft_consensus.cc:2393] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 13: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:07:29.725457 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 522006us
I20260709 13:07:29.735638 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:29.737629 16533 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:29.739168 15854 raft_consensus.cc:1240] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:29.763541 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:29.727661 22148 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:29.739666 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:29.771461 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:29.770203 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:29.772472 16562 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:07:29.772789 15935 raft_consensus.cc:2376] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16. Re-sending same reply.
I20260709 13:07:29.771555 15974 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:29.777101 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:29.782457 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:29.783768 16561 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:29.783113 15937 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:29.786999 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:29.677484 22189 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:29.791900 16561 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:29.730321 22163 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:29.793587 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:29.736203 21987 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:29.771701 22197 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:29.800428 22192 log.cc:927] Time spent T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:07:29.801360 16570 raft_consensus.cc:1275] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:07:29.736204 22124 consensus_queue.cc:1059] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.069s
I20260709 13:07:29.799332 22094 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:29.807859 22108 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:29.771371 22122 consensus_queue.cc:1059] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:07:29.807830 21927 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:29.809589 22157 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:29.770713 22171 raft_consensus.cc:2749] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:29.837113 15932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:46118 (request call id 35) took 1210 ms. Trace:
I20260709 13:07:29.810827 22189 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:29.830188 22108 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:29.840145 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:29.840987 22108 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:29.835139 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:29.820518 22197 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:29.843726 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:29.844699 22157 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:29.845149 22197 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:29.846295 21819 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:29.846804 22157 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:29.837600 15932 rpcz_store.cc:276] 0709 13:07:28.629971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.098281 (+468310us) service_pool.cc:224] Handling call
0709 13:07:29.836902 (+738621us) spinlock_profiling.cc:243] Waited 91.3 ms on lock 0x7b540044b6d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a2853 00007f73ef37060a 00007f73e88f67d4 00007f73e88f6762 00007f73e88f6691 00007f73e88f660c 00007f73e88f58c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:29.837010 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":209905536}
I20260709 13:07:29.848721 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:29.811086 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:29.852663 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:29.780485 22162 consensus_queue.cc:1048] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:07:29.785805 22148 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:29.777830 21965 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:29.786271 21842 raft_consensus.cc:2749] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:29.819176 16562 raft_consensus.cc:1275] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:07:29.823303 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:29.824716 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:29.829159 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:29.852825 15932 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 12.
I20260709 13:07:29.838112 16567 raft_consensus.cc:1240] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 13. Current term is 14. Ops: []
W20260709 13:07:29.854869 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 64465.2us
I20260709 13:07:29.862995 15858 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:29.868098 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:29.869951 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:29.871012 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:29.872491 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:29.868974 16561 raft_consensus.cc:1275] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:07:29.837841 21927 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:29.819512 22163 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:29.880446 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:29.881037 15935 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:07:29.884059 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:29.884240 16566 raft_consensus.cc:1240] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:29.856350 22204 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:29.857344 22144 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:29.889624 15929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 2460) took 1148 ms. Trace:
I20260709 13:07:29.889843 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:29.890457 15862 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 17.
I20260709 13:07:29.874521 22083 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:29.890021 15929 rpcz_store.cc:276] 0709 13:07:28.741509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.200270 (+458761us) service_pool.cc:224] Handling call
0709 13:07:29.200662 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:29.200967 (+   305us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:29.201007 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:29.201517 (+   510us) log.cc:844] Serialized 26 byte log entry
0709 13:07:29.472696 (+271179us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:29.689812 (+217116us) log.cc:844] Serialized 14 byte log entry
0709 13:07:29.701876 (+ 12064us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.702002 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:29.889425 (+187423us) raft_consensus.cc:1685] finished
0709 13:07:29.889443 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:29.889572 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6686,"prepare.queue_time_us":411461,"raft.queue_time_us":11895,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":6823,"thread_start_us":467877,"threads_started":2,"wal-append.queue_time_us":464925}
I20260709 13:07:29.900476 16005 raft_consensus.cc:1275] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:07:29.901572 15934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 2451) took 1958 ms. Trace:
I20260709 13:07:29.902001 15934 rpcz_store.cc:276] 0709 13:07:27.942952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.045830 (+102878us) service_pool.cc:224] Handling call
0709 13:07:29.247632 (+1201802us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:29.266601 (+ 18969us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.901389 (+634788us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54002c2750. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:29.901513 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2762909568}
I20260709 13:07:29.903240 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:29.906764 22124 raft_consensus.cc:3055] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:29.907238 22124 raft_consensus.cc:740] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:29.907316 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:29.907984 15935 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:29.908898 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:29.908763 22124 consensus_queue.cc:260] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:29.909492 15935 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 17.
I20260709 13:07:29.910063 22124 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:29.911162 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:29.911463 22083 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:29.912853 15933 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:46118
I20260709 13:07:29.913488 15933 raft_consensus.cc:606] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Received request to transfer leadership to TS 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:29.913796 15933 raft_consensus.cc:614] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:29.910389 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:29.915129 15933 raft_consensus.cc:1275] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
W20260709 13:07:29.916044 22167 log.cc:927] Time spent T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 1.070s	user 0.015s	sys 0.007s
I20260709 13:07:29.916867 15936 raft_consensus.cc:1275] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:07:29.885133 22129 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:29.918504 22129 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:29.919986 21819 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:29.920535 21819 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:29.921415 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:29.921840 15935 raft_consensus.cc:1275] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:07:29.923004 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:29.874666 22090 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:29.927248 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:29.928325 16010 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 12.
I20260709 13:07:29.929366 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:29.930109 15934 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:29.920964 22083 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:29.933979 16535 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:29.876681 22189 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:29.924499 15902 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:29.942152 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:29.943291 15902 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:29.933254 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:29.944388 15929 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:29.870579 21965 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:29.940454 15858 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 18.
I20260709 13:07:29.945128 16562 raft_consensus.cc:1240] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:29.945151 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:29.921095 21413 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:29.963055 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:29.964946 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:29.967531 16563 raft_consensus.cc:1275] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:07:29.963442 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:29.969643 16568 raft_consensus.cc:2368] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 13. Current term is 14.
I20260709 13:07:29.967002 16570 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:29.948508 15827 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:29.987586 16564 raft_consensus.cc:2393] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 13: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:30.001438 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:30.002286 16562 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:30.005939 15929 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
W20260709 13:07:30.023090 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 77788.5us
I20260709 13:07:30.024405 15827 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:29.920990 22129 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:30.032073 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:30.026432 15934 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:07:30.033950 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:29.894297 22204 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:29.895709 22144 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:29.965102 22130 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:30.036583 15827 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:29.963873 21819 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:29.974038 22200 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:29.988098 22048 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:29.986907 22157 raft_consensus.cc:2749] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:30.060197 16002 raft_consensus.cc:1275] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:07:30.029727 22108 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.054311 16568 raft_consensus.cc:1240] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:30.037371 22037 raft_consensus.cc:2764] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
W20260709 13:07:30.045397 22127 consensus_meta.cc:287] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.634s	user 0.009s	sys 0.004s
I20260709 13:07:30.059178 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:30.047221 15900 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:29.942047 22090 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:30.062657 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:30.077567 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:30.059177 16562 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:07:30.020463 21950 consensus_queue.cc:1048] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:07:30.044373 22137 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:30.042054 22134 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:30.000659 21413 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:30.054105 22083 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:30.020709 22189 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:30.086112 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:30.054621 22048 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:30.099695 16562 raft_consensus.cc:1240] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:30.102160 16563 raft_consensus.cc:2393] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 16: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
W20260709 13:07:30.109839 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 141686us
I20260709 13:07:30.112747 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:30.113652 16563 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 15.
I20260709 13:07:30.061414 22200 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:30.103765 22048 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:30.102881 22094 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:30.118578 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:30.077884 22108 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:30.127187 22163 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:30.129122 21413 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:30.129637 22208 log.cc:927] Time spent T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:07:30.082124 22130 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:30.131603 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:30.132340 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:30.135181 16563 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
I20260709 13:07:30.137034 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:30.145282 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:30.133102 22148 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:30.157194 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:30.158825 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:30.161458 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:30.162438 22204 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:30.162906 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:30.163220 21965 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:30.151602 21819 raft_consensus.cc:2804] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:30.164304 21819 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:30.164479 22144 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:30.164794 21819 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:07:30.149188 22207 log.cc:927] Time spent T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:07:30.166186 15932 raft_consensus.cc:1275] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:07:30.167382 15928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 2457) took 1936 ms. Trace:
I20260709 13:07:30.167791 15928 rpcz_store.cc:276] 0709 13:07:28.230671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.395015 (+164344us) service_pool.cc:224] Handling call
0709 13:07:29.647321 (+1252306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:29.647768 (+   447us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:29.647807 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:29.648405 (+   598us) log.cc:844] Serialized 26 byte log entry
0709 13:07:29.947133 (+298728us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:29.947177 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:29.947250 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:30.166171 (+218921us) raft_consensus.cc:1685] finished
0709 13:07:30.166195 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:30.167133 (+   938us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b5400327650. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:30.167328 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2836621568,"thread_start_us":298300,"threads_started":1,"wal-append.queue_time_us":383426}
I20260709 13:07:30.162982 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:30.159614 16568 raft_consensus.cc:2368] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 13. Current term is 14.
I20260709 13:07:30.171067 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:30.172128 16566 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 15.
I20260709 13:07:30.179566 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:30.180090 22088 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:30.157199 22200 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:30.184069 16007 raft_consensus.cc:1240] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:30.197408 22132 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.198201 22132 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:30.200387 22132 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:30.200778 15932 raft_consensus.cc:1275] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:07:30.201483 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:30.202348 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:30.202575 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:30.202967 16566 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:07:30.205510 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:30.208408 22189 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:30.210193 22130 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:30.211058 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:30.211877 22137 raft_consensus.cc:2749] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:30.196827 22088 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:30.216509 22090 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:30.221091 22108 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:30.215329 22048 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:30.185071 22156 consensus_meta.cc:287] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.514s	user 0.008s	sys 0.004s
I20260709 13:07:30.223382 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:30.224390 15930 raft_consensus.cc:2410] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:07:30.226063 15931 raft_consensus.cc:1240] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:30.226886 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:30.227259 21819 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:30.227727 16009 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:07:30.230185 21819 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:30.221277 16566 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 18.
I20260709 13:07:30.150439 15902 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:30.230316 22048 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:30.231560 22048 raft_consensus.cc:2749] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:07:30.232299 22219 log.cc:927] Time spent T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.105s	user 0.002s	sys 0.000s
W20260709 13:07:30.231592 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 81219.2us
I20260709 13:07:30.237099 22156 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:30.237859 22156 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:30.243257 15902 leader_election.cc:400] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 13. Current term is 14.
I20260709 13:07:30.246737 15855 raft_consensus.cc:1240] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:30.246986 15862 raft_consensus.cc:1240] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:30.249725 15935 raft_consensus.cc:1275] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:07:30.251654 15932 raft_consensus.cc:1240] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:30.256028 16562 raft_consensus.cc:1240] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 17. Current term is 18. Ops: [17.13-17.13]
W20260709 13:07:30.257414 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 177898us
I20260709 13:07:30.262189 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:30.265105 15902 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:30.250973 22048 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:30.267213 15902 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:30.169881 22195 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:30.271236 15902 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:30.273825 15902 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:30.284525 15854 raft_consensus.cc:1240] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:30.248581 22148 consensus_queue.cc:1059] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.030s
I20260709 13:07:30.290628 15902 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:30.291111 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:30.297571 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.294409 15827 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:30.300158 15827 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:30.269459 22083 raft_consensus.cc:697] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:30.298626 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:30.304575 15856 raft_consensus.cc:1275] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:07:30.270567 22222 raft_consensus.cc:2749] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:30.304852 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:30.273041 22108 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:07:30.306586 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.307646 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:30.306020 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 258844us
I20260709 13:07:30.310451 15827 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:30.310840 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:30.311776 15935 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:30.287968 22130 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:30.315367 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:30.316555 15928 raft_consensus.cc:2410] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:07:30.271600 22144 consensus_queue.cc:1048] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.035s
I20260709 13:07:30.322264 15900 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:30.319036 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:30.301380 21819 raft_consensus.cc:2804] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:30.307133 21413 raft_consensus.cc:2764] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
W20260709 13:07:30.310880 22145 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.697s	user 0.020s	sys 0.009s
I20260709 13:07:30.284828 22134 consensus_queue.cc:1048] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:07:30.304610 22221 raft_consensus.cc:2749] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:30.273301 22048 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:30.304646 22137 raft_consensus.cc:2804] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:07:30.405985 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 61890.9us
I20260709 13:07:30.413039 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:30.308894 22108 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:07:30.405212 22145 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.792s	user 0.020s	sys 0.009s
I20260709 13:07:30.411993 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:30.416420 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:30.429133 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:30.430912 15976 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:30.438536 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:30.438905 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:30.439805 15861 raft_consensus.cc:1275] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:07:30.391495 21987 raft_consensus.cc:2749] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:07:30.440616 22231 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.441134 22231 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:30.442751 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:30.443327 22231 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:30.404915 21819 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:30.443789 15857 raft_consensus.cc:2368] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 17. Current term is 18.
I20260709 13:07:30.415934 22129 consensus_queue.cc:1048] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 13:07:30.427316 22156 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:30.431180 22108 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:30.409129 22230 consensus_queue.cc:1048] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 13:07:30.446444 15976 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:30.413479 22137 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:30.454200 16569 consensus_queue.cc:237] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:30.451146 15935 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:30.457477 22124 raft_consensus.cc:2804] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:30.476188 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:30.476146 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 149359us
I20260709 13:07:30.425297 22111 consensus_queue.cc:1048] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:07:30.453233 21819 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:30.480439 15900 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:30.481127 15827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.528149 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:30.530882 15854 raft_consensus.cc:1240] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:30.528944 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:30.590353 15856 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:07:30.529779 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:30.478464 22124 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:30.540134 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:30.555903 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:07:30.572057 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.452211 21965 consensus_queue.cc:1048] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 13:07:30.456442 22083 consensus_queue.cc:237] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:30.462697 22234 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:30.543007 22123 raft_consensus.cc:3055] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:07:30.576134 22217 log.cc:927] Time spent T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.447s	user 0.000s	sys 0.004s
I20260709 13:07:30.576572 22161 consensus_queue.cc:1059] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:30.576853 22127 consensus_queue.cc:1059] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:30.581797 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:30.583349 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:30.466465 22148 raft_consensus.cc:2804] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:30.527922 22195 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:30.536887 22048 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:30.467267 22137 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:30.595209 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:30.596052 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:30.602824 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 122429us
I20260709 13:07:30.589335 15861 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:30.479822 22091 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:30.645910 16568 raft_consensus.cc:1240] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:30.658329 15900 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:30.649169 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:30.650470 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:30.651641 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:30.596771 21950 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.606547 22124 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:30.656678 16005 raft_consensus.cc:2393] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 17: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:30.479741 22088 consensus_queue.cc:1059] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.150s
W20260709 13:07:30.655035 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 64562.2us
I20260709 13:07:30.669598 15900 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:30.597049 22123 raft_consensus.cc:740] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:07:30.676234 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 91072.3us
I20260709 13:07:30.651844 22195 consensus_queue.cc:1048] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 13:07:30.675396 22234 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:30.682744 15974 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:30.627298 22228 raft_consensus.cc:2749] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:07:30.684620 15900 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:30.698355 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:30.699455 15854 raft_consensus.cc:2393] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 16: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:30.703246 16562 raft_consensus.cc:1275] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:07:30.705901 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:30.714854 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:30.673779 22091 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:30.707829 22221 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:30.731432 16566 raft_consensus.cc:1240] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:30.696957 22241 raft_consensus.cc:2749] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:07:30.727208 16533 leader_election.cc:336] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.544s (SENT)
I20260709 13:07:30.680056 21950 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:30.740533 16533 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:30.740857 16005 raft_consensus.cc:1240] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:30.747143 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:30.732218 21965 raft_consensus.cc:3055] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:30.755268 16566 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1"
dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
mode: GRACEFUL
new_leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:42622
I20260709 13:07:30.760776 16566 raft_consensus.cc:606] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Received request to transfer leadership to TS 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:30.760108 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 62380.7us
I20260709 13:07:30.761229 16566 raft_consensus.cc:614] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:07:30.761276 16533 leader_election.cc:336] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.625s (SENT)
I20260709 13:07:30.761830 16533 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:30.737362 22248 raft_consensus.cc:2749] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:30.764950 15974 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
W20260709 13:07:30.771201 22229 log.cc:927] Time spent T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
W20260709 13:07:30.772204 22237 log.cc:927] Time spent T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 13:07:30.751550 22200 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:30.751272 22193 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.767824 15825 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:30.752565 22221 raft_consensus.cc:697] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 18 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:30.754735 22145 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:30.757895 22201 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.760587 21965 raft_consensus.cc:740] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:07:30.784554 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 116198us
I20260709 13:07:30.787943 15825 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:30.738618 21819 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:30.802847 15974 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
W20260709 13:07:30.792976 16533 leader_election.cc:336] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.649s (SENT)
I20260709 13:07:30.776121 22189 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:30.807283 16533 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:30.739291 22123 consensus_queue.cc:260] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:30.804111 22240 log.cc:927] Time spent T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.226s	user 0.000s	sys 0.001s
W20260709 13:07:30.842427 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 54603.3us
I20260709 13:07:30.843616 15825 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:30.792133 22232 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:30.848562 21950 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:30.817543 22228 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.833403 22244 raft_consensus.cc:2804] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:30.850747 21965 consensus_queue.cc:260] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:30.810524 22148 raft_consensus.cc:697] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 14 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:30.812460 22234 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.829252 22241 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.829447 22130 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.786196 22193 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:30.864583 22242 log.cc:927] Time spent T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:07:30.856746 22048 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.873705 21819 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:30.874421 22088 raft_consensus.cc:3055] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:30.852443 22111 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:30.858100 22091 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:30.856616 22090 raft_consensus.cc:2749] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:30.876749 22252 consensus_queue.cc:1059] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 13:07:30.875543 22108 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:30.877506 22221 consensus_queue.cc:237] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:30.885896 22244 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:30.840765 22200 raft_consensus.cc:2749] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:30.846424 22123 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:30.889655 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 96713.5us
I20260709 13:07:30.795305 22145 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:30.938829 22231 consensus_queue.cc:1059] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:07:30.804386 22201 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:30.884780 22228 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:30.862938 22248 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:30.895817 22088 raft_consensus.cc:740] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:07:30.957947 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 96632.5us
W20260709 13:07:30.958189 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.932636 22111 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:30.935722 22244 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:30.961534 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:30.894232 22082 consensus_queue.cc:1048] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.067s
I20260709 13:07:30.892860 22195 consensus_queue.cc:1048] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:07:30.895494 22048 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:30.969314 15974 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:30.956079 22252 raft_consensus.cc:3055] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:30.893337 22234 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:30.938349 22255 consensus_queue.cc:1048] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 13:07:30.944939 22248 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:30.973814 15934 raft_consensus.cc:1240] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:30.974227 15929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2703) took 1114 ms. Trace:
I20260709 13:07:30.971642 16533 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:30.985424 15929 rpcz_store.cc:276] 0709 13:07:29.860146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:30.051234 (+191088us) service_pool.cc:224] Handling call
0709 13:07:30.072971 (+ 21737us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:30.295037 (+222066us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:30.295076 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:30.296794 (+  1718us) log.cc:844] Serialized 26 byte log entry
0709 13:07:30.435868 (+139074us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:30.435904 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:30.435981 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:30.974028 (+538047us) raft_consensus.cc:1685] finished
0709 13:07:30.974047 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:30.974172 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":72900,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":31001,"thread_start_us":234859,"threads_started":2,"wal-append.queue_time_us":371886}
I20260709 13:07:30.986711 16533 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:30.978012 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:30.979646 15933 raft_consensus.cc:1275] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:07:30.894682 22130 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:30.963104 22111 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:30.987462 15936 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:30.972892 22091 consensus_queue.cc:1048] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 13:07:30.991752 16533 leader_election.cc:304] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:30.956462 22124 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:30.894436 22241 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:30.993438 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:30.996829 15936 raft_consensus.cc:2393] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 12: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:30.992255 22247 consensus_queue.cc:1048] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.085s
I20260709 13:07:30.991073 22157 consensus_queue.cc:1048] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:07:30.994622 22258 raft_consensus.cc:2749] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:30.969695 22090 consensus_queue.cc:1048] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:07:31.021912 15937 raft_consensus.cc:1275] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:07:31.020294 16002 raft_consensus.cc:1240] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:30.989848 21950 raft_consensus.cc:2749] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:31.023370 15937 pending_rounds.cc:85] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 7
I20260709 13:07:30.987807 22248 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:31.003711 16533 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:31.029048 16533 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:31.026575 15937 pending_rounds.cc:107] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:07:31.024503 15974 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:30.992271 22189 raft_consensus.cc:2749] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:07:31.018615 22216 log.cc:927] Time spent T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.890s	user 0.001s	sys 0.003s
I20260709 13:07:31.011936 22162 raft_consensus.cc:2749] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:07:31.017302 22246 log.cc:927] Time spent T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 13:07:30.981577 22252 raft_consensus.cc:740] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:31.060006 22252 consensus_queue.cc:260] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:31.061064 22252 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:31.061471 15929 raft_consensus.cc:1240] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:31.064651 22108 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:31.066191 22130 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:31.031157 21950 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:31.036725 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:31.068641 15935 raft_consensus.cc:1240] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:07:31.037258 15933 raft_consensus.cc:1275] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:07:31.038020 16533 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:31.050037 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:31.037185 22250 consensus_queue.cc:1059] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:07:31.042654 22127 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:31.057792 22234 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:31.030575 15937 raft_consensus.cc:2889] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:30.968293 22148 consensus_queue.cc:237] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:31.043603 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:31.020117 22124 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:31.072158 15932 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:30.991360 22088 consensus_queue.cc:260] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:07:31.017217 22228 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:31.032616 22157 raft_consensus.cc:2749] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:31.019658 22145 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:31.075104 21950 raft_consensus.cc:2749] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:31.094882 22127 raft_consensus.cc:697] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:31.109459 22193 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:31.088616 22137 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:31.113344 22048 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:31.132248 16008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 1188) took 1316 ms. Trace:
I20260709 13:07:31.132881 16008 rpcz_store.cc:276] 0709 13:07:29.815848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.816034 (+   186us) service_pool.cc:224] Handling call
0709 13:07:29.816379 (+   345us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:07:29.880631 (+ 64252us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:29.880661 (+    30us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:07:29.889466 (+  8805us) log.cc:844] Serialized 78 byte log entry
0709 13:07:30.045931 (+156465us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:30.045971 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:30.274313 (+228342us) spinlock_profiling.cc:243] Waited 63.8 ms on lock 0x7b54002e1dd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:30.274358 (+    45us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:30.983515 (+709157us) spinlock_profiling.cc:243] Waited 5.33 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:07:31.131919 (+148404us) raft_consensus.cc:1685] finished
0709 13:07:31.131948 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:31.132171 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3305,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":1486,"spinlock_wait_cycles":159092224,"thread_start_us":97319,"threads_started":1,"wal-append.queue_time_us":195629}
I20260709 13:07:31.123805 22201 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:31.128713 22088 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:31.126812 22252 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:31.133741 22091 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:31.153527 22252 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:31.163836 16565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 1463) took 1035 ms. Trace:
I20260709 13:07:31.164172 22252 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:31.169067 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:31.169907 15930 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:07:31.169355 22147 raft_consensus.cc:3055] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:31.170409 22147 raft_consensus.cc:740] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:31.171280 22147 consensus_queue.cc:260] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:31.172540 22147 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:31.174845 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:31.185662 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:31.188036 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:31.187772 22127 consensus_queue.cc:237] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:31.188778 16568 raft_consensus.cc:2393] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 18: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:31.187574 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:31.193063 15902 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:31.195935 16535 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:31.198335 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:31.182925 22244 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:31.200450 15902 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:31.164310 16565 rpcz_store.cc:276] 0709 13:07:30.128198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:30.181760 (+ 53562us) service_pool.cc:224] Handling call
0709 13:07:30.182091 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:30.870996 (+688905us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:07:30.871334 (+   338us) log.cc:844] Serialized 14 byte log entry
0709 13:07:30.978503 (+107169us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:30.978588 (+    85us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:30.978658 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:31.163559 (+184901us) spinlock_profiling.cc:243] Waited 689 ms on lock 0x7b5400724554. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:31.163625 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:31.163777 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2280,"prepare.queue_time_us":1319,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":90,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":2953,"spinlock_wait_cycles":1583698688,"thread_start_us":106636,"threads_started":1,"wal-append.queue_time_us":176085}
I20260709 13:07:31.202647 15902 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:31.204702 15902 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:31.210471 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:31.198871 22137 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:31.213871 16569 raft_consensus.cc:2393] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 16: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:31.158217 22091 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:31.203547 22193 raft_consensus.cc:2804] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:31.203526 22201 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.201705 22048 raft_consensus.cc:2749] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:31.209378 22228 raft_consensus.cc:2749] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:31.208333 22130 raft_consensus.cc:2749] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:31.208950 16535 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:31.206005 22263 raft_consensus.cc:2804] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:31.216370 22241 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:31.216410 22193 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:31.224836 22263 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:31.226735 22157 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:31.225332 22111 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:31.240528 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:31.252338 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:31.254103 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:31.255545 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:31.261305 15902 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:31.264586 15902 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:31.266039 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:31.220366 22201 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:31.268220 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:31.237811 22193 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:31.273126 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:31.278367 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:31.279341 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:31.281656 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:31.285141 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:31.286128 16569 raft_consensus.cc:2368] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 13. Current term is 14.
I20260709 13:07:31.286965 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:31.287938 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:31.289080 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:31.289213 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:31.289637 16570 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 18.
I20260709 13:07:31.285162 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:31.241668 22157 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:31.290758 16567 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 13.
I20260709 13:07:31.248371 22263 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:31.288086 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:31.269223 22130 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:31.293185 15902 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:31.304368 22157 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:31.304703 22241 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:31.311926 15902 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:31.329066 15902 leader_election.cc:400] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 election: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 13. Current term is 14.
I20260709 13:07:31.330422 15902 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:31.333140 15902 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:31.313261 22221 raft_consensus.cc:2749] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:31.343600 15902 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:31.344606 22130 raft_consensus.cc:2804] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:31.344987 22130 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:31.345336 22130 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:31.353037 22201 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:31.353449 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:31.339344 22037 raft_consensus.cc:2749] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:31.326097 22127 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.326937 22091 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:31.358968 22162 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:07:31.342445 22266 log.cc:927] Time spent T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 13:07:31.333078 22108 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:31.373148 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:31.380084 16008 raft_consensus.cc:2393] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 16: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:31.374084 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:31.374755 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:31.377342 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:31.380889 16011 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 18.
I20260709 13:07:31.382565 22111 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:31.385262 22112 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.385493 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:31.385775 22112 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:31.386264 15855 raft_consensus.cc:2393] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 20: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:31.387457 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:31.387979 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:31.388665 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:31.389279 15861 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 16.
I20260709 13:07:31.389415 15862 raft_consensus.cc:1275] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:07:31.390599 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:31.390977 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:31.391204 15856 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 16.
I20260709 13:07:31.392270 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:31.392930 15858 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 12.
I20260709 13:07:31.370864 22196 raft_consensus.cc:476] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Not starting pre-election -- already a leader
I20260709 13:07:31.393777 22162 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:31.368677 22241 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:31.395741 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:31.377833 15902 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:31.387496 15855 raft_consensus.cc:1275] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:07:31.399416 16008 raft_consensus.cc:1275] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:07:31.403105 15974 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:07:31.406845 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:31.356748 22161 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:31.408488 16009 raft_consensus.cc:1275] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:07:31.395359 22241 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:31.407019 22244 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:31.410701 22241 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:31.383255 22248 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:31.413362 22161 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:31.415805 22161 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:31.417162 22162 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:31.371657 22127 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:31.420061 21927 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.420687 21927 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:31.421690 22127 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:31.423194 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:31.426908 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:31.413834 22248 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:31.410595 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:31.430727 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:31.432543 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:31.433288 15931 raft_consensus.cc:2393] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 16: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:31.434521 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:31.441277 15936 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 16: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:31.441520 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:31.441675 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:31.442555 16011 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 14.
I20260709 13:07:31.442919 16006 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 16: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:31.444832 15976 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:31.453316 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:31.454223 16011 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 18.
I20260709 13:07:31.455333 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:31.422693 21927 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:31.407843 22221 raft_consensus.cc:2749] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:31.408964 22132 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:31.459124 15928 raft_consensus.cc:1240] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:31.458410 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:31.462427 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:31.472218 15930 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
I20260709 13:07:31.464202 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:31.474582 15933 raft_consensus.cc:2410] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:07:31.460726 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:31.458432 22230 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:31.467716 22241 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:31.468283 15976 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:31.468333 22161 consensus_queue.cc:1048] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:31.463745 22244 raft_consensus.cc:2749] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:31.451076 22112 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:31.480144 15825 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:31.484545 16535 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:31.409009 22193 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:31.471065 22132 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:31.501899 15825 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:31.512989 16535 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:31.477399 22137 consensus_queue.cc:1059] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 13:07:31.514091 16533 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:31.484261 22196 raft_consensus.cc:2749] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:31.486917 22230 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:31.530118 16533 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:31.514421 22132 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:31.482329 21817 consensus_queue.cc:1048] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:07:31.510818 22112 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:31.519456 22083 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:31.523993 22145 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:31.521997 22200 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:31.525720 21950 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:31.539722 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:31.519124 22133 consensus_queue.cc:1059] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:07:31.548580 16004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 1187) took 1756 ms. Trace:
I20260709 13:07:31.549090 16004 rpcz_store.cc:276] 0709 13:07:29.792123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.792320 (+   197us) service_pool.cc:224] Handling call
0709 13:07:29.800204 (+  7884us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:30.005242 (+205038us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:30.005271 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:30.005735 (+   464us) log.cc:844] Serialized 26 byte log entry
0709 13:07:30.243281 (+237546us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:30.243840 (+   559us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:31.548137 (+1304297us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b5400305a54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:31.548194 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:31.548250 (+    56us) raft_consensus.cc:1685] finished
0709 13:07:31.548266 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:31.548512 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":18736,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":6618,"spinlock_wait_cycles":470656256,"thread_start_us":182981,"threads_started":1,"wal-append.queue_time_us":230522,"wal-append.run_cpu_time_us":1825,"wal-append.run_wall_time_us":1007538}
I20260709 13:07:31.523308 22162 consensus_queue.cc:1048] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:07:31.523437 22033 consensus_queue.cc:1048] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 13:07:31.532598 22091 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:31.536593 22112 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:31.540650 22145 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:31.534368 22201 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 13:07:31.545971 22088 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.583384 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:31.584220 15928 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:07:31.568284 22037 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.571599 22129 raft_consensus.cc:3055] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:31.641347 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:31.581172 21950 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:31.569919 22108 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.580260 22145 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:31.573962 22112 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:31.644934 15936 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:07:31.651197 16007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2529) took 2030 ms. Trace:
I20260709 13:07:31.651712 16007 rpcz_store.cc:276] 0709 13:07:29.621118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.623861 (+  2743us) service_pool.cc:224] Handling call
0709 13:07:29.624238 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:30.186853 (+562615us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:31.650952 (+1464099us) spinlock_profiling.cc:243] Waited 560 ms on lock 0x7b54003ee254. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:31.651137 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1287041792}
I20260709 13:07:31.653258 22277 consensus_queue.cc:1048] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:31.662672 22193 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:31.668687 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:31.670086 22130 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:31.670838 22235 log.cc:927] Time spent T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 1.191s	user 0.000s	sys 0.004s
W20260709 13:07:31.673604 22265 log.cc:927] Time spent T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.504s	user 0.000s	sys 0.003s
I20260709 13:07:31.684118 22241 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:31.644671 22216 log.cc:927] Time spent T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:07:31.640237 22088 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:31.687513 22088 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:31.643178 22148 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.688913 22108 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:31.691128 22108 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:31.581559 22091 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:31.700160 22091 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:31.700384 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:31.700961 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:31.701323 15932 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
I20260709 13:07:31.706672 22110 raft_consensus.cc:3055] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:31.707273 22110 raft_consensus.cc:740] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:31.644454 22129 raft_consensus.cc:740] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:31.709010 22129 consensus_queue.cc:260] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:31.709950 22230 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:31.710283 22129 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:31.646592 21950 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:31.713145 21950 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:31.646522 22037 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:31.718725 22037 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:31.724149 15934 raft_consensus.cc:1240] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:31.689075 22148 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:31.726555 15937 raft_consensus.cc:1275] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:07:31.727222 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:31.727514 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:31.728338 15936 raft_consensus.cc:2393] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 19: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:31.730806 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:31.733106 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:31.740944 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:31.741827 16561 raft_consensus.cc:2410] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:07:31.727952 16567 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:07:31.734230 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:31.742671 15862 raft_consensus.cc:1240] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:31.745688 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:31.750267 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:31.750423 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:31.751144 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:31.754495 15858 raft_consensus.cc:1275] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:07:31.754501 15976 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:31.756959 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:31.755609 22263 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:31.798208 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:31.798786 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:31.740048 22157 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:31.805735 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 51293.4us
I20260709 13:07:31.755934 15859 raft_consensus.cc:1240] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:07:31.818203 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 2552) took 1237 ms. Trace:
I20260709 13:07:31.834849 16002 rpcz_store.cc:276] 0709 13:07:30.581258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:30.581365 (+   107us) service_pool.cc:224] Handling call
0709 13:07:30.581708 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:31.026143 (+444435us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:31.806644 (+780501us) spinlock_profiling.cc:243] Waited 438 ms on lock 0x7b5400304b54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:31.818139 (+ 11495us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1008236416}
I20260709 13:07:31.835296 15976 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:31.841244 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:31.842176 15854 raft_consensus.cc:2410] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
W20260709 13:07:31.852903 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:31.862020 15976 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:31.865190 16568 raft_consensus.cc:2393] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 15: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:31.844686 22088 raft_consensus.cc:2804] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:31.889778 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:31.890462 16561 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:31.890529 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:31.891157 16567 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:31.868157 21936 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:07:31.898445 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 364977us
I20260709 13:07:31.876921 21965 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:31.876717 22142 raft_consensus.cc:2749] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:31.899556 16533 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:31.903466 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:31.906629 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:31.912941 16533 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:31.907475 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:31.910187 16569 raft_consensus.cc:1275] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:07:31.920375 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:31.879395 22088 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:31.885617 22033 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:31.929239 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:31.886739 22162 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:31.897285 22137 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.936930 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:31.916589 22130 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:31.947752 16002 raft_consensus.cc:1240] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:31.927110 22088 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:31.862375 22110 consensus_queue.cc:260] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:31.895864 22132 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:31.950690 16561 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:07:31.968082 22189 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:31.968576 22189 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:31.968951 22189 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:31.969359 15928 raft_consensus.cc:1240] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:31.971457 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:31.971982 22290 raft_consensus.cc:2749] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:31.973201 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:31.974352 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:31.974856 15862 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 16.
I20260709 13:07:31.975379 22083 raft_consensus.cc:2749] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:07:31.976281 22158 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:31.979187 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:31.986582 22263 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:31.986450 22158 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:31.983156 15931 raft_consensus.cc:1275] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:07:31.953174 22283 log.cc:927] Time spent T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 13:07:31.990339 15932 raft_consensus.cc:1240] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:31.998344 15859 raft_consensus.cc:1240] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 11. Current term is 12. Ops: []
I20260709 13:07:32.003544 16006 raft_consensus.cc:2393] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 18: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
W20260709 13:07:31.984340 22297 log.cc:927] Time spent T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
W20260709 13:07:32.012435 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 139292us
I20260709 13:07:31.941157 22033 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:31.946973 22137 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:31.943833 22162 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:31.975664 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:31.978014 15825 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:07:32.024859 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 46941.1us
W20260709 13:07:32.026470 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 113625us
I20260709 13:07:31.994180 22292 consensus_queue.cc:1048] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:07:32.030290 15932 raft_consensus.cc:1275] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:07:32.029654 16533 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:32.032925 15854 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 17.
I20260709 13:07:32.026854 22157 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:32.027109 22263 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:32.041648 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:32.032217 22082 raft_consensus.cc:3055] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:32.030640 22083 raft_consensus.cc:2804] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:32.030957 22148 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:32.035859 16564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1352) took 1455 ms. Trace:
I20260709 13:07:32.036518 22302 raft_consensus.cc:2804] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:32.060875 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:32.067679 15827 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:32.050248 22082 raft_consensus.cc:740] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:32.052589 22083 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:32.235896 22083 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:32.236373 22306 log.cc:927] Time spent T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 13:07:32.078027 16533 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:32.240463 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 162531us
I20260709 13:07:32.218811 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:32.106446 15827 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:32.248028 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:32.249845 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:32.251185 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:32.218556 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:32.256805 22247 raft_consensus.cc:3055] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:07:32.257413 22247 raft_consensus.cc:740] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:32.258420 22247 consensus_queue.cc:260] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:32.260818 22162 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:07:32.262781 22282 log.cc:927] Time spent T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.694s	user 0.004s	sys 0.000s
I20260709 13:07:32.218990 22213 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:07:32.219993 22293 log.cc:927] Time spent T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 13:07:32.221468 22111 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:07:32.223501 22301 log.cc:927] Time spent T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
I20260709 13:07:32.225500 15928 raft_consensus.cc:1275] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:07:32.038455 22290 consensus_queue.cc:1048] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.029s
W20260709 13:07:32.228569 22303 log.cc:927] Time spent T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.175s	user 0.000s	sys 0.000s
I20260709 13:07:32.279767 22304 raft_consensus.cc:2749] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:32.248189 22158 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:32.251329 22209 raft_consensus.cc:3055] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:32.256889 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:32.271166 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:32.271075 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:32.274116 22090 consensus_queue.cc:1048] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.006s
I20260709 13:07:32.071760 22302 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:32.271260 22157 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:32.273665 22129 consensus_queue.cc:1059] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.007s
I20260709 13:07:32.278458 15928 raft_consensus.cc:1240] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:32.267938 22213 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:32.279211 22137 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:32.280539 21987 consensus_queue.cc:1059] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 13:07:32.285071 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:32.055984 22263 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:32.292670 15936 raft_consensus.cc:1240] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:32.272519 22247 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:07:32.288235 22209 raft_consensus.cc:740] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:32.296447 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:32.288138 22231 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:32.297950 22157 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:32.307453 22305 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.294242 22302 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:32.316488 22129 raft_consensus.cc:3055] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:32.318344 22129 raft_consensus.cc:740] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:32.318449 15858 raft_consensus.cc:1240] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:32.320024 22305 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:32.319304 22129 consensus_queue.cc:260] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:32.322751 22129 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:32.325130 22157 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:32.301028 22132 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:32.288394 22111 consensus_queue.cc:1048] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 13:07:32.062049 16564 rpcz_store.cc:276] 0709 13:07:30.580699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:30.582953 (+  2254us) service_pool.cc:224] Handling call
0709 13:07:30.596515 (+ 13562us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:30.596818 (+   303us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:07:30.648704 (+ 51886us) log.cc:844] Serialized 14 byte log entry
0709 13:07:31.908893 (+1260189us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b18001a1820. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e9f504bc 00007f73e729e48b 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f4823a
0709 13:07:32.035463 (+126570us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:32.035533 (+    70us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:32.035600 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:32.035706 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:32.035805 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":56632,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":61,"raft.queue_time_us":44885,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":11807,"spinlock_wait_cycles":23178624,"thread_start_us":153432,"threads_started":2,"wal-append.queue_time_us":86753,"wal-append.run_cpu_time_us":1069,"wal-append.run_wall_time_us":1006963}
W20260709 13:07:32.310777 21950 consensus_meta.cc:287] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.597s	user 0.011s	sys 0.000s
I20260709 13:07:32.347939 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:32.351392 15936 raft_consensus.cc:2393] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 13: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:32.347182 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:32.352304 15928 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:07:32.354180 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 247804us
W20260709 13:07:32.355114 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 72890us
I20260709 13:07:32.355579 15827 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:32.357793 22136 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:32.334007 22033 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:32.330595 21950 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:32.358191 21987 consensus_queue.cc:1048] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:07:32.385669 15827 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:32.385841 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 110022us
I20260709 13:07:32.387934 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:32.388578 16533 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
W20260709 13:07:32.375101 22145 consensus_meta.cc:287] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.683s	user 0.009s	sys 0.004s
I20260709 13:07:32.376106 21819 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.399691 22111 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.399816 22318 raft_consensus.cc:2804] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:32.400388 22111 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:32.400535 21819 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:32.402344 16533 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:32.403088 16565 raft_consensus.cc:1240] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:07:32.403501 16533 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:32.400426 22318 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:32.406488 15928 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
I20260709 13:07:32.406548 22318 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:32.408209 16533 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:32.375099 22136 consensus_queue.cc:1059] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 13:07:32.410478 16533 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:32.411911 15928 raft_consensus.cc:1240] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 15. Current term is 16. Ops: []
W20260709 13:07:32.384292 22214 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.282s	user 0.007s	sys 0.014s
I20260709 13:07:32.376300 22033 consensus_queue.cc:1059] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
W20260709 13:07:32.416587 22311 log.cc:927] Time spent T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
W20260709 13:07:32.417697 22112 consensus_meta.cc:287] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.753s	user 0.011s	sys 0.000s
I20260709 13:07:32.406106 22090 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.420035 16010 raft_consensus.cc:1240] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:32.399968 22137 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:32.429875 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:32.408387 22162 raft_consensus.cc:2749] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:32.409192 22157 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:32.407301 22305 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:32.434372 15935 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:32.403926 22145 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:32.450700 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:32.437772 22214 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.330s	user 0.008s	sys 0.017s
I20260709 13:07:32.390865 22209 consensus_queue.cc:260] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:32.420102 22295 consensus_queue.cc:1059] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 12.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.037s
I20260709 13:07:32.435901 22314 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:32.432466 22112 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:32.466773 22305 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:32.438804 22157 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:07:32.480105 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 70301.5us
W20260709 13:07:32.489251 15900 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.530s (SENT) [suppressed 10 similar messages]
I20260709 13:07:32.468632 22231 raft_consensus.cc:2749] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:32.492841 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:32.435122 22090 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:32.493538 15931 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:07:32.495465 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:32.479146 22209 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:32.498651 15857 raft_consensus.cc:1240] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 15. Current term is 16. Ops: []
W20260709 13:07:32.487756 22309 log.cc:927] Time spent T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
I20260709 13:07:32.508949 22158 raft_consensus.cc:3055] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:32.509443 22158 raft_consensus.cc:740] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:32.510452 22158 consensus_queue.cc:260] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:32.513043 16568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45244 (request call id 1493) took 1108 ms. Trace:
I20260709 13:07:32.513614 16568 rpcz_store.cc:276] 0709 13:07:31.404970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:31.405170 (+   200us) service_pool.cc:224] Handling call
0709 13:07:32.512738 (+1107568us) spinlock_profiling.cc:243] Waited 421 ms on lock 0x7b540059bd54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:32.512931 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":967651712}
I20260709 13:07:32.520601 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:32.521549 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:32.521888 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:32.522255 15859 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
I20260709 13:07:32.522500 15858 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:32.522012 22082 consensus_queue.cc:260] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:32.521602 15862 raft_consensus.cc:1275] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:07:32.521574 15854 raft_consensus.cc:2410] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:07:32.492434 22157 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:32.496095 15931 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
W20260709 13:07:32.550776 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:32.497249 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 88942.5us
I20260709 13:07:32.552179 21705 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:32.555742 21705 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:32.561151 22137 raft_consensus.cc:3055] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:32.561652 22137 raft_consensus.cc:740] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:32.562614 22137 consensus_queue.cc:260] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:32.567824 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 157414us
I20260709 13:07:32.568782 16533 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:07:32.573088 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:32.566501 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:32.578568 16568 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:32.584067 22108 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:32.584936 22123 consensus_queue.cc:1059] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:07:32.571079 21950 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:32.585121 22108 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:32.589740 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:32.586057 22148 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:32.615845 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 61658.2us
I20260709 13:07:32.614207 15858 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 18.
I20260709 13:07:32.621205 15974 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:32.621452 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:32.627421 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:32.628041 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:32.571357 22263 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:32.631702 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:32.585811 22123 raft_consensus.cc:3055] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:32.571467 22111 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:32.586259 22314 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:07:32.590816 22319 log.cc:927] Time spent T 00000000000000000000000000000000 P d33cebb189b14a989836214fdbafb898: Append to log took a long time: real 0.202s	user 0.000s	sys 0.001s
I20260709 13:07:32.633888 16006 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:32.636178 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:32.666997 15854 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:32.636497 22195 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:32.620847 22148 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:32.640235 22088 consensus_meta.cc:287] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.648s	user 0.024s	sys 0.002s
I20260709 13:07:32.641180 21705 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:32.641914 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:32.644301 22325 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:07:32.645046 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:32.657372 15827 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:32.660902 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:32.662420 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:32.573886 22323 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.632427 22255 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.637492 15974 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:32.633098 22137 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:32.688050 22323 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:07:32.689604 22318 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:32.690441 21705 consensus_queue.cc:1048] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:32.692550 15857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2620) took 2226 ms. Trace:
W20260709 13:07:32.692709 22320 log.cc:927] Time spent T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 13:07:32.632233 22142 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:07:32.698546 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 209306us
I20260709 13:07:32.656471 22037 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:07:32.739413 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 102046us
I20260709 13:07:32.753062 15974 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:32.657069 22158 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:32.636008 22123 raft_consensus.cc:740] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:32.664757 21819 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:32.671321 22195 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:32.743402 15857 rpcz_store.cc:276] 0709 13:07:30.466515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:30.476214 (+  9699us) service_pool.cc:224] Handling call
0709 13:07:31.554644 (+1078430us) spinlock_profiling.cc:243] Waited 4.89 ms on lock 0x7b5800005798. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef381fc0 00007f73ef36c929 00007f73ef36c3d2 00007f73e88e3bd4
0709 13:07:31.554847 (+   203us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:32.499236 (+944389us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:32.692269 (+193033us) spinlock_profiling.cc:243] Waited 943 ms on lock 0x7b5400219654. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:32.692456 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2181236352}
I20260709 13:07:32.751835 16006 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
W20260709 13:07:32.747562 22200 consensus_meta.cc:287] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 1.200s	user 0.019s	sys 0.003s
I20260709 13:07:32.695176 22230 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:07:32.751821 22083 consensus_meta.cc:287] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.515s	user 0.013s	sys 0.000s
I20260709 13:07:32.633286 22213 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:32.746528 22132 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:32.771317 15855 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "741afa89185645afa8b1c82d31a75748"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:07:32.774190 15855 raft_consensus.cc:606] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Received request to transfer leadership to TS 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:32.684119 22088 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:32.759048 22311 log.cc:927] Time spent T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:07:32.777240 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:32.755278 22295 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.760704 22195 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:32.771871 22312 log.cc:927] Time spent T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:07:32.764598 22090 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:32.741374 22255 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
W20260709 13:07:32.809396 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 120238us
I20260709 13:07:32.773533 22200 raft_consensus.cc:2749] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:32.809926 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:32.764907 22145 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:32.767591 22230 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:32.811699 15937 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 1240) took 1112 ms. Trace:
I20260709 13:07:32.816238 15937 rpcz_store.cc:276] 0709 13:07:31.698961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:31.712071 (+ 13110us) service_pool.cc:224] Handling call
0709 13:07:31.712418 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:31.726926 (+ 14508us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:32.796471 (+1069545us) spinlock_profiling.cc:243] Waited 13.8 ms on lock 0x7b5400198054. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:32.811612 (+ 15141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31749888}
I20260709 13:07:32.696293 22325 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:32.817840 22230 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:32.757005 22037 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:32.775272 22083 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:32.824853 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:32.754905 22142 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:32.831104 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 78167.3us
I20260709 13:07:32.831336 16563 raft_consensus.cc:1240] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:32.831933 15974 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:32.813270 22132 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:32.828297 15937 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
I20260709 13:07:32.839980 15936 raft_consensus.cc:1275] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:07:32.844748 15937 raft_consensus.cc:1275] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:07:32.846747 15862 raft_consensus.cc:2368] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 16. Current term is 17.
I20260709 13:07:32.802060 22295 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:32.839231 22189 consensus_meta.cc:287] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.870s	user 0.021s	sys 0.005s
I20260709 13:07:32.832974 22200 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:32.835390 22132 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:07:32.849906 21965 consensus_meta.cc:287] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.924s	user 0.017s	sys 0.003s
W20260709 13:07:32.856601 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:32.859598 15974 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:32.863782 22248 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.864463 22248 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:32.866132 22108 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:32.866847 22148 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:32.866918 22142 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:32.868046 22332 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:32.868729 22332 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:32.871119 22332 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:32.871703 22037 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:32.858531 22200 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:32.876375 22323 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:32.857477 22132 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:32.824294 22302 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:32.879982 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:32.880980 21705 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.816341 22123 consensus_queue.cc:260] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:32.883208 15854 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 17.
I20260709 13:07:32.881206 22302 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:07:32.896312 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 79481.4us
I20260709 13:07:32.875109 22157 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:32.901772 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:32.905942 22083 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:32.897080 22148 raft_consensus.cc:2749] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:32.907972 22123 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:32.894677 21705 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:32.910245 21705 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:32.912375 22200 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:32.912621 22329 log.cc:927] Time spent T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:07:32.913991 22318 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:32.915722 22325 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:32.849300 22145 raft_consensus.cc:2955] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Committing config change with OpId 13.9: config changed from index 3 to 9, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:07:32.856366 22189 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:32.921399 22247 consensus_meta.cc:287] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.611s	user 0.010s	sys 0.001s
I20260709 13:07:32.921877 22189 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:32.922796 22255 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:32.924326 22157 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:07:32.924487 22129 consensus_meta.cc:287] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.601s	user 0.007s	sys 0.002s
W20260709 13:07:32.904234 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 78005.6us
W20260709 13:07:32.903138 22291 log.cc:927] Time spent T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 1.035s	user 0.004s	sys 0.001s
I20260709 13:07:32.939284 22088 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:32.949019 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 49682.4us
I20260709 13:07:32.904016 21965 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:32.927896 22195 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:32.964138 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:32.968531 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:32.975168 16004 raft_consensus.cc:1275] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:07:32.975482 16008 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 17.
I20260709 13:07:32.972304 16003 raft_consensus.cc:1240] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:32.987514 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:32.989733 16011 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 17.
I20260709 13:07:32.998299 22195 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:32.984261 22137 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.999399 22137 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:33.000025 22346 consensus_queue.cc:1059] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:33.001207 22346 raft_consensus.cc:3055] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:33.001609 22137 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:33.001583 22346 raft_consensus.cc:740] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:33.003379 22088 consensus_queue.cc:1048] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:33.005828 22295 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:32.976320 22263 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:33.011744 22263 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:33.003368 22346 consensus_queue.cc:260] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:33.014361 22346 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:33.016808 16010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2557) took 1084 ms. Trace:
I20260709 13:07:33.017239 16010 rpcz_store.cc:276] 0709 13:07:31.932111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:31.933239 (+  1128us) service_pool.cc:224] Handling call
0709 13:07:32.419849 (+486610us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:32.431598 (+ 11749us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:33.016564 (+584966us) spinlock_profiling.cc:243] Waited 486 ms on lock 0x7b5400382cd0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:33.016734 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1118196608}
I20260709 13:07:32.974977 22037 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:33.012554 22248 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:33.020088 22037 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:07:33.025880 22230 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:33.031067 15974 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:33.032603 22340 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:33.033017 22340 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:33.033349 22340 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:33.036157 22088 consensus_queue.cc:1048] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:33.037937 22088 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.038514 22088 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:33.039386 22321 log.cc:927] Time spent T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.403s	user 0.001s	sys 0.000s
I20260709 13:07:33.041150 22088 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:33.043296 22037 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
I20260709 13:07:33.049813 16563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1365) took 1370 ms. Trace:
I20260709 13:07:33.050227 16563 rpcz_store.cc:276] 0709 13:07:31.679481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:31.722072 (+ 42591us) service_pool.cc:224] Handling call
0709 13:07:31.722461 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:32.835664 (+1113203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:33.049587 (+213923us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54007017d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:33.049755 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2545040128}
I20260709 13:07:33.014266 22263 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:33.054955 15974 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:33.055604 22110 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:33.057020 22248 raft_consensus.cc:2749] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:33.057826 22248 consensus_queue.cc:1059] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:07:33.058657 22195 raft_consensus.cc:3055] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:33.059109 22195 raft_consensus.cc:740] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:33.060092 22195 consensus_queue.cc:260] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:33.063284 16005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 1212) took 1315 ms. Trace:
I20260709 13:07:33.068794 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:33.069675 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:33.070294 15855 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 18.
I20260709 13:07:33.073019 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:33.063733 16005 rpcz_store.cc:276] 0709 13:07:31.747747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:31.807271 (+ 59524us) service_pool.cc:224] Handling call
0709 13:07:31.840292 (+ 33021us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:31.840663 (+   371us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:31.840697 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:31.860481 (+ 19784us) log.cc:844] Serialized 26 byte log entry
0709 13:07:33.062726 (+1202245us) spinlock_profiling.cc:243] Waited 1.4 ms on lock 0x7b1800323740. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e9f504bc 00007f73e729e48b 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f480d1
0709 13:07:33.062883 (+   157us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:33.062919 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:33.062987 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:33.063027 (+    40us) raft_consensus.cc:1685] finished
0709 13:07:33.063038 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:33.063225 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1864,"raft.queue_time_us":3121,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":11927,"spinlock_wait_cycles":3227008,"thread_start_us":135898,"threads_started":1,"wal-append.queue_time_us":160671}
I20260709 13:07:33.075230 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:33.075863 15861 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:33.074740 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:33.028225 22230 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:33.078812 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:33.077685 15857 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:07:33.067837 22090 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.092046 15974 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:07:33.097080 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 427786us
I20260709 13:07:33.102778 15902 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:33.101605 15974 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:33.103216 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:33.104303 15902 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:33.104496 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:33.105175 16561 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:07:33.106138 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 172657us
I20260709 13:07:33.107520 15974 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:33.105934 15902 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:33.110699 16567 raft_consensus.cc:2393] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 19: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:33.113708 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:33.089345 22277 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:33.125344 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:33.116727 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:33.127432 16562 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:33.118176 16563 raft_consensus.cc:1240] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:33.129760 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:33.123070 16565 raft_consensus.cc:1240] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:33.089208 22315 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:33.123881 16009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 1211) took 1376 ms. Trace:
I20260709 13:07:33.135823 15974 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:33.137321 16009 rpcz_store.cc:276] 0709 13:07:31.747410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:31.747571 (+   161us) service_pool.cc:224] Handling call
0709 13:07:31.747853 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:31.748046 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:07:31.799104 (+ 51058us) log.cc:844] Serialized 14 byte log entry
0709 13:07:32.532906 (+733802us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b18001984c0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f4823a 00007f73ea3b2a2a
0709 13:07:33.122893 (+589987us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:33.122948 (+    55us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:33.122998 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:33.123084 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:33.123823 (+   739us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3811,"prepare.queue_time_us":51097,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":44,"raft.queue_time_us":87969,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":4162,"spinlock_wait_cycles":23597824,"thread_start_us":589441,"threads_started":2,"wal-append.queue_time_us":1003363}
I20260709 13:07:33.136381 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:33.143850 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:33.146214 15827 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:33.103297 22088 raft_consensus.cc:2749] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:33.106380 22037 raft_consensus.cc:2764] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:07:33.174919 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:33.174962 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:33.179149 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:33.111148 22221 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:33.110949 22123 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:33.114461 22302 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.177570 16533 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:33.204380 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:33.204694 16533 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:33.124895 22209 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:33.172712 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:33.208225 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:33.179633 16564 raft_consensus.cc:1275] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:07:33.182643 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:33.211762 16561 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 14.
I20260709 13:07:33.104005 22137 consensus_queue.cc:1059] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.070s
I20260709 13:07:33.183063 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:33.137794 22201 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:33.199328 21965 raft_consensus.cc:2749] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:33.199635 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:33.200434 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:33.236261 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:33.124028 22295 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:33.201704 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:33.236946 15928 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:07:33.237119 16568 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:07:33.137948 22318 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.239355 22318 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:33.203662 21819 consensus_queue.cc:1059] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 13:07:33.240968 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:33.241384 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:33.241762 22318 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:33.126713 22263 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:33.205652 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:33.206280 16003 raft_consensus.cc:2410] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:07:33.208225 22162 raft_consensus.cc:3055] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:33.232052 22136 raft_consensus.cc:2749] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:33.243444 22162 raft_consensus.cc:740] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:33.239382 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:07:33.210907 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:33.244956 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:33.245553 16561 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:07:33.246361 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:33.178992 22315 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:33.232265 22189 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:33.248528 22315 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:33.248629 22189 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:33.249136 22189 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:33.249573 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:33.250293 16007 raft_consensus.cc:2393] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 16: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:33.210742 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:33.222364 22325 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:33.251875 16004 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:33.254402 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:33.254719 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:33.213711 16533 leader_election.cc:400] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 16. Current term is 17.
I20260709 13:07:33.256631 16533 leader_election.cc:403] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:33.214336 16563 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 14.
I20260709 13:07:33.187023 22112 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:33.216138 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:33.221189 15825 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:33.265635 16009 raft_consensus.cc:2410] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:07:33.260913 15853 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 16.
I20260709 13:07:33.222066 16570 raft_consensus.cc:1240] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:33.198828 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:33.101558 22090 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:33.223138 15974 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:33.282819 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:33.267608 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:33.284034 15928 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 16.
I20260709 13:07:33.284219 15932 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
W20260709 13:07:33.284247 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 62332us
I20260709 13:07:33.285187 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:33.219588 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 73482.8us
I20260709 13:07:33.230577 16562 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 16.
I20260709 13:07:33.222705 15857 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 16.
I20260709 13:07:33.201584 22195 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:33.235478 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:33.286312 16008 raft_consensus.cc:1240] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:33.287366 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:33.218703 22255 raft_consensus.cc:2749] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:33.244445 22295 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:33.259491 15858 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 16.
I20260709 13:07:33.255349 16002 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 18.
I20260709 13:07:33.264798 15931 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7c0fef632f9a4485838855b553a79163"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:46118
I20260709 13:07:33.211959 22209 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:33.268630 16005 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:07:33.236454 16011 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 17.
I20260709 13:07:33.289656 15931 raft_consensus.cc:606] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Received request to transfer leadership to TS 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:33.295331 15931 raft_consensus.cc:614] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:07:33.237943 22317 log.cc:927] Time spent T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:07:33.283008 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:33.290491 15930 raft_consensus.cc:2393] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 16: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:33.285743 15932 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:33.144235 22277 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:33.306217 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:33.302609 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:33.310024 15930 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
W20260709 13:07:33.291005 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 69922.6us
I20260709 13:07:33.291318 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:33.290078 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:33.312317 15928 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:33.312820 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:33.313967 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:33.314522 16570 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
W20260709 13:07:33.282932 22305 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.012s	sys 0.028s
W20260709 13:07:33.315923 22305 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.012s	sys 0.028s
I20260709 13:07:33.294174 22325 raft_consensus.cc:697] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:33.316744 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:33.292990 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:33.316887 22354 consensus_queue.cc:1059] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:33.317677 16006 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 16: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:33.318061 22354 raft_consensus.cc:3055] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:07:33.317523 22325 consensus_queue.cc:237] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:33.318454 22354 raft_consensus.cc:740] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:33.318740 22145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:32976 (request call id 7) took 3588 ms. Trace:
W20260709 13:07:33.297201 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 91445.3us
I20260709 13:07:33.319231 22145 rpcz_store.cc:276] 0709 13:07:29.730349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.730516 (+   167us) service_pool.cc:224] Handling call
0709 13:07:30.630135 (+899619us) log.cc:844] Serialized 494 byte log entry
0709 13:07:31.162447 (+532312us) spinlock_profiling.cc:243] Waited 674 ms on lock 0x7b5400503f54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ae735 00007f73ea3a69fa 00007f73ef36e4c9 00007f73e88eac54 00007f73e88eabe2 00007f73e88eab11 00007f73e88eaa8c 00007f73e88e9d42 00007f73e86e348c 00007f73e86e28c6
0709 13:07:33.318649 (+2156202us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5267,"raft.run_cpu_time_us":654,"raft.run_wall_time_us":654,"spinlock_wait_cycles":1549086592}
I20260709 13:07:33.297375 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:33.322355 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:33.298231 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:33.323976 15933 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:33.300643 22247 consensus_queue.cc:1048] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 13:07:33.294229 15976 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:33.325330 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:33.293586 22088 consensus_queue.cc:1059] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 13:07:33.283401 22136 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:33.322428 15935 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 18.
I20260709 13:07:33.326701 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:33.337718 15929 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:33.327314 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:33.292882 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:33.338801 16535 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:33.330613 16561 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 16.
I20260709 13:07:33.331774 15902 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:33.339975 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:33.332273 16004 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 19.
I20260709 13:07:33.328917 15827 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:33.301157 22209 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:33.311965 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:33.334625 15928 raft_consensus.cc:2410] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 16 because replica has last-logged OpId of term: 15 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 6.
I20260709 13:07:33.330235 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:33.223323 22302 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:33.341423 22130 raft_consensus.cc:2804] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:33.342613 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:33.343127 22263 consensus_queue.cc:1048] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:07:33.315268 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:33.344655 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:33.317581 15931 raft_consensus.cc:2393] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 19: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:33.355764 16009 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 17.
I20260709 13:07:33.346446 15825 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:33.346909 15976 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:33.304843 22315 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:33.234323 22132 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:33.360260 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:33.355120 15930 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 16: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:33.355654 16570 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 18.
I20260709 13:07:33.361119 15935 raft_consensus.cc:2393] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 18: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:33.340742 22332 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:33.352520 15827 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:33.359498 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:33.362356 15976 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:33.364212 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:33.340960 15902 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:33.373111 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:33.374114 16005 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 18.
I20260709 13:07:33.377640 15974 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:33.379962 16535 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:07:33.332763 22314 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.008s	sys 0.031s
I20260709 13:07:33.282626 22340 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:33.255412 22201 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:33.382359 16011 raft_consensus.cc:1240] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:33.380926 15902 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:33.392416 15902 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:33.395869 16565 raft_consensus.cc:2393] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 17: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:33.395435 16566 raft_consensus.cc:1240] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:33.396914 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:33.244467 22162 consensus_queue.cc:260] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:33.402559 16010 raft_consensus.cc:2393] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 16: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
W20260709 13:07:33.384905 22314 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.008s	sys 0.032s
I20260709 13:07:33.330740 22255 consensus_queue.cc:1048] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:07:33.383050 22332 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:33.362462 21987 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:33.360035 22315 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:33.382339 22248 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:33.412011 22108 raft_consensus.cc:2749] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:33.368902 22158 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:33.335896 22292 consensus_queue.cc:1059] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 13:07:33.413780 16568 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 17.
I20260709 13:07:33.385267 22088 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:33.385643 21965 raft_consensus.cc:2804] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:33.345453 21819 consensus_queue.cc:1059] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
W20260709 13:07:33.418933 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 60535.2us
I20260709 13:07:33.393254 21950 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:33.419932 15929 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 17.
I20260709 13:07:33.386925 22221 raft_consensus.cc:2804] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:33.423884 15930 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:46118
I20260709 13:07:33.470234 15928 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:07:33.390208 22130 raft_consensus.cc:697] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 14 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:33.401691 22037 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:33.423159 22337 consensus_queue.cc:1059] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.091s
I20260709 13:07:33.421800 22110 raft_consensus.cc:2804] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:33.412392 22314 raft_consensus.cc:697] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:33.319370 22354 consensus_queue.cc:260] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:33.424439 22332 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:33.471767 21950 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:33.424240 22349 raft_consensus.cc:2804] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:33.431367 22315 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:33.424368 21705 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:33.352351 22136 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:33.431087 22248 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:33.427976 22247 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:33.477404 16533 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:33.436921 22158 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:33.440191 22318 raft_consensus.cc:2749] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 16. Current term is 17.
I20260709 13:07:33.443825 22088 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:33.426465 22083 consensus_queue.cc:1059] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:07:33.443622 21965 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:33.444744 15825 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:07:33.580078 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 185212us
I20260709 13:07:33.485733 22110 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:33.482820 22200 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:33.523762 22339 raft_consensus.cc:2804] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:07:33.598716 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 154041us
I20260709 13:07:33.600006 15825 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:33.525996 21950 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:07:33.548753 22346 consensus_meta.cc:287] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.534s	user 0.010s	sys 0.000s
W20260709 13:07:33.566440 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 89121.2us
I20260709 13:07:33.605090 16533 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:33.481576 22360 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.089s
I20260709 13:07:33.555984 21705 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:33.559068 22248 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:33.608832 21705 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:33.471042 22221 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:33.611910 22221 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:33.552157 21819 raft_consensus.cc:3055] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:07:33.571792 22356 log.cc:927] Time spent T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.240s	user 0.000s	sys 0.001s
I20260709 13:07:33.619881 16533 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:33.550025 22130 consensus_queue.cc:237] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:33.581511 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 189197us
I20260709 13:07:33.565714 22158 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:33.565442 22247 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:33.575059 22343 raft_consensus.cc:2749] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:33.592970 22110 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:33.622149 15825 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:33.622354 15902 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:33.697461 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 75159.8us
I20260709 13:07:33.624338 22339 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:33.585204 21965 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:33.625880 16533 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:33.705760 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 80010.6us
W20260709 13:07:33.706187 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 122145us
I20260709 13:07:33.707118 16533 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:33.621762 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 197665us
I20260709 13:07:33.653580 22345 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:33.677212 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:33.616463 22292 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:33.603454 22295 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:33.575779 22088 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:33.619244 21819 raft_consensus.cc:740] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:33.698393 15902 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:33.643265 22360 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:33.623811 22090 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:33.703526 15856 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.002s	sys 0.014s
W20260709 13:07:33.852391 15856 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.002s	sys 0.014s
I20260709 13:07:33.709483 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:07:33.853178 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 231103us
I20260709 13:07:33.853495 16568 raft_consensus.cc:2393] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 13: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:33.710114 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:33.660772 22132 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:33.642159 22302 leader_election.cc:290] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:33.602986 22314 consensus_queue.cc:237] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:33.602620 22200 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:33.687110 22201 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:07:33.698585 22363 log.cc:927] Time spent T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.106s	user 0.000s	sys 0.014s
I20260709 13:07:33.743574 22305 raft_consensus.cc:3055] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:33.834532 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:33.813066 22292 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:33.824334 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:33.817391 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:33.830200 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:33.814353 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 107292us
I20260709 13:07:33.817559 15855 raft_consensus.cc:1275] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:07:33.836081 22345 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:33.753306 22193 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:33.831018 16005 raft_consensus.cc:1240] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:33.889070 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:33.775959 15746 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 679229ffe7114facacb8e3a2c4c6d226 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 8)
I20260709 13:07:33.889716 16568 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 20.
I20260709 13:07:33.859012 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:33.693480 22346 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:33.700376 22359 log.cc:927] Time spent T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.275s	user 0.001s	sys 0.000s
I20260709 13:07:33.860540 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:33.722398 22209 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:33.705603 22339 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:07:33.715157 22362 log.cc:927] Time spent T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 13:07:33.885658 22292 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:33.869535 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:07:33.872156 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:33.873442 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:07:33.852070 22189 consensus_meta.cc:287] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.602s	user 0.008s	sys 0.004s
I20260709 13:07:33.727828 22224 raft_consensus.cc:2804] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:07:33.740288 22361 log.cc:927] Time spent T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.179s	user 0.000s	sys 0.001s
I20260709 13:07:33.699949 22037 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:33.877058 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:33.773483 22353 raft_consensus.cc:3055] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:33.780146 22157 raft_consensus.cc:2749] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:07:33.762446 22082 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:33.940196 22037 raft_consensus.cc:697] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 19 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:33.941177 22037 consensus_queue.cc:237] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:33.880342 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:33.944886 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:33.880646 15827 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:33.848351 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 150038us
I20260709 13:07:33.830200 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:33.718116 22277 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:33.768440 22370 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:07:33.698697 22358 log.cc:927] Time spent T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.257s	user 0.000s	sys 0.007s
I20260709 13:07:33.895813 22193 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:33.886214 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:33.886582 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 396016us
I20260709 13:07:33.891842 16009 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 20.
I20260709 13:07:33.893666 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:33.740589 22367 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:33.870245 22231 raft_consensus.cc:2804] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:07:33.895256 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:33.897306 15746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 121366us
I20260709 13:07:33.719907 22033 consensus_queue.cc:1048] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.163s
I20260709 13:07:33.899668 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:33.984922 16565 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:33.904811 15928 raft_consensus.cc:2393] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 13: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:33.866202 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:33.719183 22136 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:33.911646 15858 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 14.
I20260709 13:07:33.990693 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:33.991653 15858 raft_consensus.cc:2376] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 14. Re-sending same reply.
I20260709 13:07:33.882256 22305 raft_consensus.cc:740] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:33.896186 22201 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:07:33.944748 22195 consensus_meta.cc:287] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.649s	user 0.009s	sys 0.002s
W20260709 13:07:33.953526 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 72958.4us
W20260709 13:07:33.872926 22283 log.cc:927] Time spent T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.637s	user 0.004s	sys 0.014s
I20260709 13:07:33.946192 22224 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:33.957414 15862 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:33.958277 15900 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:33.960523 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:33.906108 22248 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:34.001739 16006 raft_consensus.cc:2410] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 16 because replica has last-logged OpId of term: 15 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 6.
I20260709 13:07:33.914031 22346 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:33.954290 22263 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:33.997092 15827 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:33.954123 22353 raft_consensus.cc:740] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:33.905942 21819 consensus_queue.cc:260] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:33.928967 22189 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:33.955029 22108 consensus_queue.cc:1059] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 13:07:33.961501 22327 consensus_queue.cc:1059] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.143s
I20260709 13:07:34.012684 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:07:34.002050 22373 log.cc:927] Time spent T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.299s	user 0.001s	sys 0.001s
I20260709 13:07:34.017918 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:33.941169 22221 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:34.019526 15827 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:33.998111 22136 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:34.036695 22247 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:34.036938 22136 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:34.037312 22247 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.037833 22247 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:34.017191 22263 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:34.040270 22263 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:34.036103 22108 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:07:34.040983 15853 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 2510) took 2122 ms (client timeout 2059 ms). Trace:
W20260709 13:07:34.041440 15853 rpcz_store.cc:269] 0709 13:07:31.919146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:32.218228 (+299082us) service_pool.cc:224] Handling call
0709 13:07:34.040782 (+1822554us) spinlock_profiling.cc:243] Waited 986 ms on lock 0x7b5400217d54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:34.040890 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2268393216}
I20260709 13:07:34.042132 22110 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:34.042974 15853 raft_consensus.cc:1240] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:07:34.045588 22364 raft_consensus.cc:2804] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:34.046160 22364 raft_consensus.cc:697] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:34.046772 22247 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:34.048691 22340 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:34.048889 22248 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:34.000984 22201 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:34.051600 22386 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:34.024984 15902 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:34.005010 22224 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:34.037477 22136 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:34.030251 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:34.038169 22315 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:34.063797 15862 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 13.
I20260709 13:07:34.004101 22231 raft_consensus.cc:697] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 17 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:34.049842 22110 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:34.064778 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 106623us
I20260709 13:07:33.981693 22370 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:34.008522 22375 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.048826 22345 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:34.070461 15900 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:34.071718 15900 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:34.073601 15900 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:34.015664 22369 raft_consensus.cc:2804] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:34.047047 22364 consensus_queue.cc:237] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:33.989033 22367 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:34.053198 22130 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:34.083492 16562 raft_consensus.cc:1240] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:34.063997 22263 raft_consensus.cc:2804] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:34.055331 22157 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:34.087759 22231 consensus_queue.cc:237] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:34.088271 22157 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:34.089282 22370 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:34.090318 22157 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:34.058507 15902 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:34.069756 22305 consensus_queue.cc:260] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:34.110167 21950 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:34.111985 22391 raft_consensus.cc:2749] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:34.112891 22391 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:34.113601 22391 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:34.114945 15854 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 16.
I20260709 13:07:34.117045 22136 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:34.118052 22123 raft_consensus.cc:3055] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:34.118482 22123 raft_consensus.cc:740] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:34.119297 22123 consensus_queue.cc:260] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:34.120607 22123 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:34.121112 22367 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:34.121837 22367 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:34.122566 22200 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:34.123982 16003 raft_consensus.cc:1240] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:34.125686 22354 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:34.085807 22369 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:34.100775 22130 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:34.101218 22353 consensus_queue.cc:260] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:34.128022 15857 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 16.
I20260709 13:07:34.148685 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:34.137775 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:34.136406 16010 raft_consensus.cc:2955] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Committing config change with OpId 13.9: config changed from index 3 to 9, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:07:34.149587 16004 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 17.
I20260709 13:07:34.129065 15859 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 13.
W20260709 13:07:34.127996 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:34.152545 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:34.136535 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:34.139997 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:34.154189 16563 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:34.142520 16564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 1354) took 1022 ms. Trace:
I20260709 13:07:34.161451 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:34.149765 16567 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 18.
I20260709 13:07:34.155714 16564 rpcz_store.cc:276] 0709 13:07:33.120477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:33.360837 (+240360us) service_pool.cc:224] Handling call
0709 13:07:33.393945 (+ 33108us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:33.417472 (+ 23527us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:07:33.432110 (+ 14638us) log.cc:844] Serialized 14 byte log entry
0709 13:07:33.685666 (+253556us) log.cc:844] Serialized 14 byte log entry
0709 13:07:33.688391 (+  2725us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:33.688453 (+    62us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:33.688513 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:34.142307 (+453794us) spinlock_profiling.cc:243] Waited 23.2 ms on lock 0x7b5400550ad4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:34.142355 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:34.142470 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":61865,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":6710,"raft.queue_time_us":2496,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":6852,"spinlock_wait_cycles":60754432,"thread_start_us":207044,"threads_started":1,"wal-append.queue_time_us":443514}
I20260709 13:07:34.168848 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:34.132711 22332 consensus_meta.cc:287] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.582s	user 0.009s	sys 0.000s
W20260709 13:07:34.142714 22376 log.cc:927] Time spent T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.224s	user 0.000s	sys 0.001s
I20260709 13:07:34.144585 22353 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:34.143764 22385 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.166952 16535 leader_election.cc:304] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:34.198493 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:34.139642 22209 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:34.216998 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:34.142445 22391 raft_consensus.cc:2749] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:34.135589 22368 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:34.197926 15859 raft_consensus.cc:1240] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:34.199152 15857 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 18.
I20260709 13:07:34.201013 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:34.201831 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:34.127919 22305 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:34.140645 22221 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:34.212673 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:34.133656 22346 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:34.139434 22315 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:34.217686 15936 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:07:34.218178 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:34.142059 22369 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:34.196318 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:34.238881 16564 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:34.130820 22158 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:34.238293 15858 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 17.
I20260709 13:07:34.222395 15856 raft_consensus.cc:2955] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Committing config change with OpId 13.9: config changed from index 3 to 9, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:07:34.127629 22088 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:34.225296 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 205836us
I20260709 13:07:34.081426 22375 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:34.138293 22292 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:34.236474 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 162934us
I20260709 13:07:34.168326 21965 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:07:34.234997 22200 raft_consensus.cc:2804] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:34.197731 22332 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:34.270713 22332 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:34.272056 21950 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:34.272176 22392 consensus_queue.cc:1059] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.016s
I20260709 13:07:34.273058 22158 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:34.273428 22392 raft_consensus.cc:3055] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:34.273839 22392 raft_consensus.cc:740] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:34.235178 22368 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:34.274595 22392 consensus_queue.cc:260] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:34.275842 22368 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:34.276156 22392 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:34.280321 22201 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:34.282282 22230 raft_consensus.cc:3055] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:34.282749 22230 raft_consensus.cc:740] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:34.285526 22292 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:34.286031 21987 raft_consensus.cc:3055] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:34.286504 21987 raft_consensus.cc:740] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:34.280361 22130 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:34.288691 15853 raft_consensus.cc:1240] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:07:34.227339 22385 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:34.292028 22230 consensus_queue.cc:260] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:34.293330 22384 log.cc:927] Time spent T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.255s	user 0.002s	sys 0.000s
I20260709 13:07:34.294569 22389 consensus_queue.cc:1059] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:07:34.271490 16565 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:34.299786 16003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 1221) took 1054 ms. Trace:
I20260709 13:07:34.300287 16003 rpcz_store.cc:276] 0709 13:07:33.245756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:33.253964 (+  8208us) service_pool.cc:224] Handling call
0709 13:07:34.123771 (+869807us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:34.124419 (+   648us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:34.299535 (+175116us) spinlock_profiling.cc:243] Waited 851 ms on lock 0x7b5400259ed0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:34.299714 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1957829888}
W20260709 13:07:34.306061 22378 log.cc:927] Time spent T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.404s	user 0.000s	sys 0.002s
I20260709 13:07:34.306144 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:34.306572 15936 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 18.
I20260709 13:07:34.307090 16009 raft_consensus.cc:2393] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 17: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
W20260709 13:07:34.307190 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 43532.2us
I20260709 13:07:34.309533 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:34.310436 16006 raft_consensus.cc:2393] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 19: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
W20260709 13:07:34.310595 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:34.261730 22339 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:34.255401 22200 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:34.313433 22200 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:34.313601 22345 consensus_queue.cc:1059] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:34.313932 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:34.265630 22088 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:34.266043 21965 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:34.267031 22375 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:34.311193 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:34.322587 15928 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:34.311793 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:34.324445 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:34.325773 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:34.326541 15935 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 16.
I20260709 13:07:34.312551 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:34.315418 15929 raft_consensus.cc:1240] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:34.331146 16003 raft_consensus.cc:2368] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 18. Current term is 19.
I20260709 13:07:34.315707 22345 raft_consensus.cc:3055] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:34.308866 15827 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:34.317348 16563 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:07:34.318179 15825 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:34.333120 15932 raft_consensus.cc:1240] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 12. Current term is 13. Ops: []
I20260709 13:07:34.339632 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:34.347221 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:34.349825 16564 raft_consensus.cc:1240] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:07:34.351662 15900 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:34.353456 15900 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:34.353180 15937 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
W20260709 13:07:34.361462 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 306554us
I20260709 13:07:34.362358 15902 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:34.364061 16535 leader_election.cc:400] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 election: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 18. Current term is 19.
I20260709 13:07:34.364399 16535 leader_election.cc:403] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:07:34.346849 22392 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:34.336092 22345 raft_consensus.cc:740] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:34.287359 21987 consensus_queue.cc:260] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:34.316642 22315 consensus_queue.cc:1059] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:07:34.428483 22396 raft_consensus.cc:2749] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:34.333514 22349 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:34.370031 16006 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 17.
W20260709 13:07:34.397946 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:34.400228 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:34.448482 16009 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:34.356110 22230 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:34.411842 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:34.240471 22263 raft_consensus.cc:697] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 16 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:07:34.412417 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 59028.6us
I20260709 13:07:34.412415 16565 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 19.
W20260709 13:07:34.416043 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 53753us
I20260709 13:07:34.416779 15928 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 17.
I20260709 13:07:34.363757 22375 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:34.367702 16003 raft_consensus.cc:1240] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:07:34.431305 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:34.433646 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:34.434343 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:34.347190 22090 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:34.470288 16010 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:34.470670 15902 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:34.471781 15853 raft_consensus.cc:2376] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 16. Re-sending same reply.
I20260709 13:07:34.443567 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:34.357537 22354 consensus_queue.cc:1048] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.076s
I20260709 13:07:34.428702 22353 raft_consensus.cc:2804] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:34.447795 15827 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:34.474735 15859 raft_consensus.cc:2393] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 18: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:34.475658 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:34.476759 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:34.436013 22136 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.497371 15928 raft_consensus.cc:1240] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:07:34.499084 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:34.473285 22318 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 18. Current term is 19.
I20260709 13:07:34.500053 15934 raft_consensus.cc:2393] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 13: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
W20260709 13:07:34.507727 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 60027.8us
I20260709 13:07:34.508404 15930 raft_consensus.cc:606] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Received request to transfer leadership to TS 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:34.508814 15930 raft_consensus.cc:614] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:34.509699 15931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2760) took 1202 ms. Trace:
I20260709 13:07:34.479266 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:34.512693 15933 raft_consensus.cc:1240] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:07:34.512739 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:34.515292 16007 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 18.
I20260709 13:07:34.512012 22386 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:34.525897 22394 log.cc:927] Time spent T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.177s	user 0.001s	sys 0.000s
I20260709 13:07:34.528182 22224 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:07:34.529495 15861 raft_consensus.cc:2368] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 17. Current term is 18.
W20260709 13:07:34.531454 22399 log.cc:927] Time spent T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 13:07:34.532665 22345 consensus_queue.cc:260] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:34.534180 22345 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:34.450517 22349 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:34.538478 22386 raft_consensus.cc:2749] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:34.447366 22392 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:34.544020 22392 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:34.509020 22090 raft_consensus.cc:697] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:34.548599 22090 consensus_queue.cc:237] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:34.550083 22263 consensus_queue.cc:237] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:34.477780 22137 consensus_queue.cc:1059] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.073s
I20260709 13:07:34.551781 22137 raft_consensus.cc:3055] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:34.552136 22137 raft_consensus.cc:740] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:34.552999 22137 consensus_queue.cc:260] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:34.501140 22136 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:34.554757 15859 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
I20260709 13:07:34.555843 22136 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:34.495489 22033 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.569408 16010 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 18.
I20260709 13:07:34.506376 22396 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:34.571990 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:34.574069 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:34.574338 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:34.574740 16003 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 12.
I20260709 13:07:34.575150 16006 raft_consensus.cc:2393] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 14: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:34.572642 16007 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 18.
W20260709 13:07:34.578912 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:34.570199 16009 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
W20260709 13:07:34.582265 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:34.583295 15902 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:34.586712 15902 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:34.587838 15902 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:34.566072 22353 raft_consensus.cc:697] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 13 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:34.591627 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:07:34.584801 22400 log.cc:927] Time spent T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 13:07:34.495249 22369 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:07:34.578346 22396 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:34.576073 22088 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.574450 22224 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:34.582518 22354 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:34.590763 22375 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:34.576231 22033 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:34.590497 22402 raft_consensus.cc:2804] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:34.597872 22130 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:34.588692 22200 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:34.621721 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 113269us
I20260709 13:07:34.603273 22396 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:34.622653 15827 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:34.626560 15854 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58214 (request call id 1187) took 1515 ms (client timeout 1971 ms). Trace:
W20260709 13:07:34.639232 15854 rpcz_store.cc:269] 0709 13:07:33.111590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:33.111732 (+   142us) service_pool.cc:224] Handling call
0709 13:07:34.626370 (+1514638us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b5400427cd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:34.626495 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2300961152}
I20260709 13:07:34.612879 22354 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:34.611574 22375 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:34.616909 22130 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:34.637602 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:34.616257 22402 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:34.644999 16010 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:34.639495 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:34.633174 22369 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
I20260709 13:07:34.635880 22200 leader_election.cc:290] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:34.617678 22162 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:34.645056 22375 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:34.653920 22224 raft_consensus.cc:2749] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:34.678542 22189 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:34.646294 16009 raft_consensus.cc:2368] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 17. Current term is 18.
I20260709 13:07:34.651341 15827 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:34.611048 22088 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:34.662184 22033 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:34.686597 15827 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:34.647140 22402 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:34.712853 15827 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:34.714093 15827 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:34.690189 22403 log.cc:927] Time spent T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:07:34.697834 22407 consensus_queue.cc:1059] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:07:34.716256 22364 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:34.716645 22407 raft_consensus.cc:2749] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:34.717067 22322 raft_consensus.cc:3055] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:34.717617 22322 raft_consensus.cc:740] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:34.700150 22375 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:34.718453 22322 consensus_queue.cc:260] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:34.646759 22130 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:34.685293 22353 consensus_queue.cc:237] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:34.729183 15900 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:34.682646 22349 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:34.711190 21819 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:34.711148 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:34.741364 16009 raft_consensus.cc:2368] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 14. Current term is 15.
I20260709 13:07:34.716944 22364 raft_consensus.cc:697] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:34.700004 22396 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:34.727186 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:34.709622 22354 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:34.732815 15900 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:34.722420 22129 raft_consensus.cc:2804] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:34.744949 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:34.745644 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:34.752097 16009 raft_consensus.cc:2393] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 18: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:34.755218 22383 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.755806 22383 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:34.756321 22308 raft_consensus.cc:3055] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:34.756731 22308 raft_consensus.cc:740] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:34.757552 22308 consensus_queue.cc:260] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:34.758524 22308 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:34.742095 22352 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:34.761466 22352 raft_consensus.cc:697] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 16 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:34.762358 22352 consensus_queue.cc:237] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:34.762832 15930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:46118 (request call id 38) took 1339 ms. Trace:
I20260709 13:07:34.764606 16566 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:07:34.764313 22364 consensus_queue.cc:237] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:34.746253 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:34.768307 16006 raft_consensus.cc:2393] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 17: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:34.768455 22402 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:34.749567 16569 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45246 (request call id 1348) took 1631 ms (client timeout 1865 ms). Trace:
W20260709 13:07:34.769264 16569 rpcz_store.cc:269] 0709 13:07:33.118571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:33.118705 (+   134us) service_pool.cc:224] Handling call
0709 13:07:34.727030 (+1608325us) spinlock_profiling.cc:243] Waited 2.19 ms on lock 0x7b5800044798. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:07:34.749505 (+ 22475us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5026560}
I20260709 13:07:34.772261 22129 raft_consensus.cc:697] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 17 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:34.773269 22129 consensus_queue.cc:237] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:34.774441 22385 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:34.728186 22088 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:34.748396 21965 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.779289 21965 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:34.779893 22375 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:34.741681 22230 raft_consensus.cc:2804] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:34.780880 22230 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:34.781287 22158 raft_consensus.cc:3055] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:34.781402 22230 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:34.781772 22158 raft_consensus.cc:740] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:34.782642 22193 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:34.784097 21965 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:34.741067 22392 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:34.747619 16007 raft_consensus.cc:1240] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:34.757959 22383 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:34.787628 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:07:34.795289 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 62565.6us
I20260709 13:07:34.788519 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:34.789057 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:34.750983 16003 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 12.
I20260709 13:07:34.763204 15930 rpcz_store.cc:276] 0709 13:07:33.423600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:33.423761 (+   161us) service_pool.cc:224] Handling call
0709 13:07:34.762639 (+1338878us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b540022cc54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a2853 00007f73ef37060a 00007f73e88f67d4 00007f73e88f6762 00007f73e88f6691 00007f73e88f660c 00007f73e88f58c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:34.762751 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2388258048}
I20260709 13:07:34.744455 16010 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 18.
I20260709 13:07:34.793300 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:34.787889 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:34.799551 16007 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:34.801851 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:34.804189 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:34.804977 15857 raft_consensus.cc:2393] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 18: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:34.812237 22402 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:34.802542 22375 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:34.808357 15827 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:34.820515 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:34.821261 16005 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:34.800864 21965 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.803552 22385 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.819548 22088 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.832881 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:07:34.840003 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:34.836690 22407 raft_consensus.cc:2804] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:34.782684 22158 consensus_queue.cc:260] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:34.828902 21965 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:34.842778 22407 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:34.862648 16006 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 18.
I20260709 13:07:34.832491 22385 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:34.856679 22407 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:34.843228 22088 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:34.859301 22158 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:34.870615 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:34.870420 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:34.889313 16006 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:34.889786 15902 leader_election.cc:400] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 election: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 17. Current term is 18.
I20260709 13:07:34.890230 15902 leader_election.cc:403] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:07:34.879343 16533 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:34.878942 15859 raft_consensus.cc:1240] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:34.878283 16562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 1359) took 1044 ms. Trace:
I20260709 13:07:34.875133 22385 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:34.880867 22353 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:34.882252 22339 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:34.886538 21965 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:34.888159 16535 leader_election.cc:304] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:34.891471 15902 leader_election.cc:400] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 14. Current term is 15.
I20260709 13:07:34.892503 16533 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
W20260709 13:07:34.919294 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:34.894526 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:34.865485 22372 log.cc:927] Time spent T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.943s	user 0.010s	sys 0.000s
I20260709 13:07:34.918885 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:34.895157 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:34.896894 16006 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 21.
I20260709 13:07:34.898702 15825 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:34.875047 22230 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:34.921176 22277 raft_consensus.cc:2804] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:34.920557 16007 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:34.918373 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:34.919282 15902 leader_election.cc:403] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:34.923449 15936 raft_consensus.cc:2393] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 13: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:34.930061 15902 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:34.892009 16562 rpcz_store.cc:276] 0709 13:07:33.834265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:33.835262 (+   997us) service_pool.cc:224] Handling call
0709 13:07:33.835565 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:34.085742 (+250177us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:34.878045 (+792303us) spinlock_profiling.cc:243] Waited 248 ms on lock 0x7b54002b5f54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:34.878227 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":569672960}
I20260709 13:07:34.925025 16533 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:34.917042 22402 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:34.916306 22338 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:34.938616 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:34.942991 16005 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 13.
I20260709 13:07:34.944290 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:34.944973 15936 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 17.
I20260709 13:07:34.945775 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:34.946309 15936 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 16.
I20260709 13:07:34.947067 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:34.948488 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:34.949105 15935 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:34.949930 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:34.951488 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:34.952229 15859 raft_consensus.cc:2393] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 17: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:34.952473 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:34.953083 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:34.953286 15853 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:34.953994 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:34.919852 22263 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:34.956198 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:34.956670 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:34.957259 15936 raft_consensus.cc:2393] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 18: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:34.957214 16533 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:34.961719 15825 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:34.966228 16533 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:34.967557 16533 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:34.968487 15853 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 19.
I20260709 13:07:34.920297 22354 consensus_queue.cc:1059] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 13:07:34.919090 22353 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:34.928100 22130 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:34.918903 22110 consensus_queue.cc:1059] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 13:07:34.933355 22277 raft_consensus.cc:697] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:34.931649 22193 raft_consensus.cc:2764] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:07:34.955813 22345 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:35.075623 15861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 2532) took 1247 ms. Trace:
I20260709 13:07:35.037387 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:35.088845 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:35.088867 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:34.987883 22088 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:34.934631 22369 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:07:35.089021 15861 rpcz_store.cc:276] 0709 13:07:33.828178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:33.944609 (+116431us) service_pool.cc:224] Handling call
0709 13:07:35.075348 (+1130739us) spinlock_profiling.cc:243] Waited 584 ms on lock 0x7b54001009d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:35.075541 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1342482176}
I20260709 13:07:35.019893 22349 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:35.042661 15976 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:35.091567 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:07:35.092222 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 49677.5us
I20260709 13:07:35.003815 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:35.026466 22367 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:34.977298 22112 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.093631 22367 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:35.093881 22112 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:35.095978 22112 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:35.011516 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:35.107714 22322 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:35.039588 22402 raft_consensus.cc:2749] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 17. Current term is 18.
I20260709 13:07:35.021936 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:35.046797 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:07:35.061967 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:35.110342 16565 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:35.097396 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:35.053558 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:35.112586 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:07:35.084915 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 169027us
W20260709 13:07:35.084846 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 117354us
I20260709 13:07:35.115211 16533 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:35.116735 16533 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:35.085899 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 155857us
I20260709 13:07:34.942517 21965 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.122385 21965 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:35.086591 22200 raft_consensus.cc:2749] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:35.071130 22418 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:35.090612 22148 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.137316 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:35.036360 22325 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.970678 22305 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:35.092721 22349 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:34.973138 22327 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:35.146729 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:35.147307 15937 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:35.037781 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:35.148654 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:35.094175 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:35.149539 15859 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:07:35.150508 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:35.012854 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:34.944330 22385 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:35.096447 22037 raft_consensus.cc:3055] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:35.050935 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:35.152813 16561 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:35.153707 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:34.986560 21819 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.153563 15825 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:34.965555 22263 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:35.011868 22346 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:35.039283 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:35.096817 22345 raft_consensus.cc:697] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:35.045421 22318 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.165294 22318 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:35.061908 22090 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:35.166720 22090 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:35.145099 22148 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:35.169385 22148 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:35.170011 22090 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:35.169955 22148 consensus_queue.cc:1059] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:07:35.096832 22367 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:35.097760 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:35.163959 22158 raft_consensus.cc:2804] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:07:35.171872 22158 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:35.172302 22158 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:35.172992 22130 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:35.098719 15855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 2528) took 1271 ms. Trace:
I20260709 13:07:35.098521 22277 consensus_queue.cc:237] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:35.173497 15855 rpcz_store.cc:276] 0709 13:07:33.826983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:33.885984 (+ 59001us) service_pool.cc:224] Handling call
0709 13:07:35.098533 (+1212549us) spinlock_profiling.cc:243] Waited 299 ms on lock 0x7b54000e9cd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:35.098666 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":688512896}
I20260709 13:07:35.173668 21819 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:35.174674 22189 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.174947 15855 raft_consensus.cc:1240] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:35.097488 15976 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:35.175194 22189 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:35.176010 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 78584.1us
I20260709 13:07:35.176625 21819 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:35.176682 15976 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:35.177261 22189 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:35.145011 22325 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:34.510059 15931 rpcz_store.cc:276] 0709 13:07:33.307228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:33.360724 (+ 53496us) service_pool.cc:224] Handling call
0709 13:07:33.396155 (+ 35431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:33.396355 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:07:33.419538 (+ 23183us) log.cc:844] Serialized 14 byte log entry
0709 13:07:34.277405 (+857867us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b18001b4480. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f4823a 00007f73ea3b2a2a
0709 13:07:34.508188 (+230783us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:34.508241 (+    53us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:34.508292 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:34.509516 (+  1224us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:34.509645 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51433,"prepare.queue_time_us":21769,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":59635,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":49734,"spinlock_wait_cycles":25265920,"thread_start_us":171921,"threads_started":1}
I20260709 13:07:35.155069 22037 raft_consensus.cc:740] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:35.055711 22137 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:35.105429 22396 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:35.096958 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:35.178469 22037 consensus_queue.cc:260] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:35.179154 16563 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:35.104446 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:35.114259 15860 raft_consensus.cc:2393] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 18: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:35.179913 22037 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:35.179780 22346 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:34.983848 22392 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:35.121831 22304 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.999059 22375 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:35.124259 16533 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:35.181569 22304 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:35.181980 22346 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:35.182420 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 58225us
I20260709 13:07:35.182548 22346 raft_consensus.cc:3055] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:35.182972 22346 raft_consensus.cc:740] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:35.125779 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:35.183702 22304 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:35.184252 22304 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:35.089704 15853 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:35.184546 21965 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:35.185120 21965 raft_consensus.cc:2804] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:35.185433 21965 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:35.185748 21965 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:35.186201 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:35.187340 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:35.134470 16010 raft_consensus.cc:1275] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:07:35.134533 16565 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:35.188048 22088 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:35.188177 15934 raft_consensus.cc:1275] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:07:35.188432 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:35.136134 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:35.147090 22327 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:35.193045 16567 raft_consensus.cc:2393] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 17: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:35.138075 16570 raft_consensus.cc:2393] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 19: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:35.136963 22418 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:35.141073 15900 leader_election.cc:400] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 17. Current term is 18.
W20260709 13:07:35.198496 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 57532.9us
I20260709 13:07:35.200127 15853 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:07:35.142089 15827 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:35.143913 15930 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 18.
I20260709 13:07:35.123636 22162 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:35.146435 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:35.210584 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 76881.4us
I20260709 13:07:35.210940 15929 raft_consensus.cc:2393] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 13: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:35.146509 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:35.211376 15974 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:35.151029 15859 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:35.145233 22354 consensus_queue.cc:1059] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.044s
I20260709 13:07:34.969496 22338 raft_consensus.cc:2804] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:35.155500 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:35.153278 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:35.215965 15936 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:07:35.216787 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:35.217789 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:35.158248 22385 raft_consensus.cc:697] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 17 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:35.146037 22349 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:35.182726 22318 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:35.139794 22305 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:35.139950 22402 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:35.088951 16562 raft_consensus.cc:2393] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 19: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
W20260709 13:07:35.238811 15900 leader_election.cc:336] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.779s (SENT)
I20260709 13:07:35.172600 22345 consensus_queue.cc:237] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:35.241600 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 64991.6us
I20260709 13:07:34.989624 22407 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:34.954710 22352 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:35.181171 16568 raft_consensus.cc:2393] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 13: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:35.183028 15931 raft_consensus.cc:1240] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:35.128394 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:35.132256 15933 raft_consensus.cc:1240] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:35.188576 22088 raft_consensus.cc:697] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:35.136897 22201 raft_consensus.cc:2804] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:35.092445 15861 raft_consensus.cc:2393] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 19: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:35.250138 16563 raft_consensus.cc:1275] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:07:35.252074 16568 raft_consensus.cc:1275] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:07:35.191020 15934 raft_consensus.cc:1275] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:07:35.194018 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:35.192579 16533 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:35.192543 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:35.254570 15855 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
I20260709 13:07:35.236212 22349 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:35.211468 16003 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:35.154301 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:35.212493 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:35.009585 15928 raft_consensus.cc:1275] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:07:35.220180 22346 consensus_queue.cc:260] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:35.255393 22158 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:35.213128 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 71128.4us
I20260709 13:07:35.213317 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:35.214004 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:35.260007 15853 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:07:35.214592 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:35.260880 16566 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:35.230459 21965 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:35.160123 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:35.224792 15974 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:35.268573 22158 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:35.268312 22088 consensus_queue.cc:237] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:35.269681 15827 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:35.270638 22158 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:07:35.273314 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 48641.9us
I20260709 13:07:35.274516 15974 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:35.276710 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:35.277079 15827 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:35.277995 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:35.278594 15855 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 13.
I20260709 13:07:35.226063 22421 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:07:35.228307 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:35.198740 22327 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:35.189687 22325 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:35.184585 22304 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:35.264770 22318 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:34.972347 22230 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:35.242631 15976 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:35.218580 22365 raft_consensus.cc:2764] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:07:35.249301 16562 raft_consensus.cc:1275] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:07:35.253773 16563 raft_consensus.cc:1275] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:07:35.248309 22402 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:35.204389 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:35.261878 15857 raft_consensus.cc:2393] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 13: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:35.240681 22305 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:35.221433 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:35.310813 15936 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
I20260709 13:07:35.275070 15861 raft_consensus.cc:1275] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:07:35.284806 22263 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:35.293808 22422 raft_consensus.cc:2749] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:35.252544 22420 consensus_queue.cc:1059] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:07:35.250047 22201 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:35.292344 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:35.293006 16566 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 21.
I20260709 13:07:35.286199 15937 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 19.
I20260709 13:07:35.296088 15827 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:35.256482 22352 raft_consensus.cc:697] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 18 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:35.239303 15900 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:35.279860 22346 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:35.255940 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 63441.8us
W20260709 13:07:35.306440 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 63912.2us
I20260709 13:07:35.305784 22339 raft_consensus.cc:2749] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:07:35.313433 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:35.319478 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:35.286818 22407 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:35.299089 22369 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 14. Current term is 15.
I20260709 13:07:35.302075 22304 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:35.300856 22392 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:07:35.333010 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 94231.8us
I20260709 13:07:35.334321 15825 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:35.334427 16533 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:35.330366 22201 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:35.342027 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:35.349591 15827 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:35.355888 15936 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 18.
W20260709 13:07:35.341926 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:35.339720 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:35.357511 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:35.354449 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:35.358146 15932 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:07:35.329195 22338 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:35.351600 16533 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:35.359414 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:35.370427 15934 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:35.368351 15827 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:35.360880 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:35.361912 15936 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:35.362455 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:35.372280 15930 raft_consensus.cc:2393] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 13: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
W20260709 13:07:35.364209 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 89760.4us
I20260709 13:07:35.357637 15937 raft_consensus.cc:2393] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 18: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:35.373731 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:35.375157 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:35.373747 15974 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:35.376888 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:35.377913 15937 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 18.
I20260709 13:07:35.337086 22263 raft_consensus.cc:2749] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:35.334971 22088 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:35.380921 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:35.393347 16563 raft_consensus.cc:2393] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 13: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:35.381348 15902 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:35.397951 15902 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:35.398492 15932 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:35.400822 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:35.401252 15853 raft_consensus.cc:1275] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:07:35.401453 16565 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 13.
I20260709 13:07:35.360635 22325 raft_consensus.cc:2749] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:35.335610 22364 consensus_queue.cc:1048] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:07:35.391498 15929 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:07:35.384117 16006 raft_consensus.cc:1275] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:07:35.353611 22392 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:35.300051 22385 consensus_queue.cc:237] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:35.386032 16533 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:35.407115 15825 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:35.407315 22162 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:35.337656 22090 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:35.379922 22393 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:35.382862 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:35.405583 15902 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:35.380255 22338 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:35.420482 21965 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:35.422123 22316 consensus_queue.cc:1048] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 13:07:35.402920 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:35.403848 21819 raft_consensus.cc:2749] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:35.424183 16010 raft_consensus.cc:2393] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 13: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:35.425117 22263 raft_consensus.cc:2749] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:35.428941 22200 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.429517 22200 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:35.431654 22200 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:35.409097 15900 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:35.381590 22387 raft_consensus.cc:2804] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:35.433131 22387 raft_consensus.cc:697] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:35.434019 22387 consensus_queue.cc:237] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:07:35.434909 22418 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:35.369510 22371 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.401032 15827 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:35.436054 22371 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:35.436604 22402 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:35.437997 22110 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:35.438391 22371 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:35.438541 15827 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:35.438815 22110 consensus_queue.cc:1048] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:35.439548 22323 raft_consensus.cc:2749] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:07:35.441481 22148 raft_consensus.cc:2764] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:07:35.443271 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:35.444481 16006 raft_consensus.cc:2410] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:07:35.403972 22396 raft_consensus.cc:2804] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:35.448220 22396 raft_consensus.cc:697] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 13 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:35.449265 22396 consensus_queue.cc:237] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:35.449920 22308 consensus_queue.cc:1048] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:35.409647 22392 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:35.453974 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:35.455123 15827 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:35.456101 22308 consensus_queue.cc:1059] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:35.456947 21817 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:35.458624 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:35.410378 15974 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:35.385026 22129 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:35.460660 21987 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:35.460780 22129 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:35.460892 22371 consensus_queue.cc:1048] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:35.461997 22148 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:35.408818 16533 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:35.411355 22162 raft_consensus.cc:697] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:07:35.463054 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 52749.6us
W20260709 13:07:35.463454 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 54688.3us
I20260709 13:07:35.463783 15974 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:35.463894 22162 consensus_queue.cc:237] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:35.465858 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 56828.3us
I20260709 13:07:35.415473 22090 raft_consensus.cc:697] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 21 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:35.394932 22189 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:07:35.411309 22367 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:35.461400 22129 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:35.412711 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:35.473253 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:35.460541 15825 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:35.353566 22231 consensus_queue.cc:1048] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.052s
I20260709 13:07:35.469357 15929 raft_consensus.cc:1275] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:07:35.470330 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 126166us
I20260709 13:07:35.474232 15900 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:35.476956 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:35.474728 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:35.475075 16570 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 19.
I20260709 13:07:35.470790 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:35.475556 15974 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:35.404557 22088 raft_consensus.cc:697] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:35.481236 15976 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:35.468137 22365 raft_consensus.cc:2804] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:07:35.501101 15902 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:35.508490 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:35.510547 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:35.510607 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:35.478294 22369 raft_consensus.cc:2749] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:35.530670 16003 raft_consensus.cc:1240] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 13. Current term is 14. Ops: []
W20260709 13:07:35.530541 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 55080.9us
I20260709 13:07:35.538655 15974 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:35.473385 22200 raft_consensus.cc:2804] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:07:35.523977 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:35.529795 16007 raft_consensus.cc:1275] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:07:35.492403 22108 consensus_queue.cc:1048] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 13:07:35.487159 22393 raft_consensus.cc:2749] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:35.478008 22148 raft_consensus.cc:2749] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:35.515652 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:35.560724 22427 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:35.540920 22037 raft_consensus.cc:2804] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:35.540135 22339 raft_consensus.cc:2804] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:35.534495 22365 raft_consensus.cc:697] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:35.500159 22323 consensus_queue.cc:1048] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 13:07:35.540621 22263 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:35.541246 22130 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:35.541806 22200 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:35.542831 22375 consensus_queue.cc:1048] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.037s
I20260709 13:07:35.540863 22429 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:35.512718 21817 consensus_queue.cc:1048] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.057s
I20260709 13:07:35.560252 22418 consensus_queue.cc:1048] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 13:07:35.530007 22402 consensus_queue.cc:1048] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.067s
I20260709 13:07:35.479846 22367 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:35.615026 22367 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:35.600865 22424 log.cc:927] Time spent T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.208s	user 0.001s	sys 0.000s
I20260709 13:07:35.586602 15974 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:35.573989 22037 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:35.472401 22352 consensus_queue.cc:237] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:35.561072 22428 consensus_queue.cc:1059] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 13:07:35.584554 22339 raft_consensus.cc:697] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:35.590878 22200 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:35.573474 22427 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:35.611984 16004 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 18.
I20260709 13:07:35.618317 16007 raft_consensus.cc:1275] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:07:35.619251 15974 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:35.639125 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:35.640033 16008 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 15.
W20260709 13:07:35.626387 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 122673us
I20260709 13:07:35.629426 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:35.632813 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:35.636288 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:35.643499 15857 raft_consensus.cc:1275] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 13 index: 14. (index mismatch)
I20260709 13:07:35.644114 22158 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:07:35.645630 22322 consensus_meta.cc:287] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.537s	user 0.009s	sys 0.000s
I20260709 13:07:35.646446 15862 raft_consensus.cc:2410] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:07:35.586158 22088 consensus_queue.cc:237] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:35.649721 22428 raft_consensus.cc:3055] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:35.650178 22428 raft_consensus.cc:740] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:35.651109 22428 consensus_queue.cc:260] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:35.652395 22428 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:35.627588 22037 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:35.660490 22436 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:35.661087 22436 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:35.663219 22436 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:35.663345 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:35.664201 22339 consensus_queue.cc:237] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:35.664867 22148 raft_consensus.cc:2749] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:07:35.665928 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:35.666568 15857 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:35.669402 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:35.670209 15937 raft_consensus.cc:2410] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 16 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:07:35.591341 22130 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:35.673352 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:35.674125 15857 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:35.677456 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:35.678164 15928 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:35.681674 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:35.684011 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:35.574133 22090 consensus_queue.cc:237] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:35.630479 22429 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:35.709312 22429 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:35.709188 22365 consensus_queue.cc:237] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:35.710028 22429 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:35.709376 22201 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:35.711390 22338 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:35.712495 22201 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:35.670264 22392 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:35.713585 22110 raft_consensus.cc:3055] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:35.714015 22110 raft_consensus.cc:740] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:35.714802 22110 consensus_queue.cc:260] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:35.715159 22392 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:35.715842 22323 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:35.716360 22323 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:35.718583 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:35.718803 22323 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:35.718842 15902 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:35.720723 22110 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:35.721670 22422 raft_consensus.cc:3055] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:35.722066 22422 raft_consensus.cc:740] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:35.722414 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:35.722853 22422 consensus_queue.cc:260] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:35.744846 22422 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:35.724145 15858 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 18.
I20260709 13:07:35.747025 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:35.732673 15974 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:35.749274 15974 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:35.737136 22130 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:35.704247 15928 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 18.
I20260709 13:07:35.706347 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:35.728904 22037 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:35.750281 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:35.750519 15929 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:35.707024 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:35.663427 22305 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:35.708385 22304 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:35.752892 16007 raft_consensus.cc:2368] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 17. Current term is 18.
I20260709 13:07:35.753123 16568 raft_consensus.cc:1240] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:07:35.708551 22419 raft_consensus.cc:3055] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:35.751947 15857 raft_consensus.cc:1240] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:35.732152 22346 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:35.659189 22349 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:35.755556 15974 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:35.758896 15974 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:35.759593 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:35.760462 15902 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:35.761971 16535 leader_election.cc:400] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 17. Current term is 18.
I20260709 13:07:35.762333 16535 leader_election.cc:403] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
W20260709 13:07:35.757117 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 51841.2us
I20260709 13:07:35.762333 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:35.763547 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:35.763548 16565 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 18.
I20260709 13:07:35.771040 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:35.771767 16007 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 14.
I20260709 13:07:35.773952 16564 raft_consensus.cc:1275] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 13 index: 14. (index mismatch)
I20260709 13:07:35.676852 22427 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:35.839020 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:35.839665 22189 consensus_queue.cc:1048] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:35.844326 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:35.846145 16565 raft_consensus.cc:1275] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:07:35.747020 22308 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.849926 16005 raft_consensus.cc:1240] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:35.857398 22308 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:35.762481 22345 consensus_queue.cc:1059] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.096s
I20260709 13:07:35.859447 22345 raft_consensus.cc:3055] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:35.859882 22345 raft_consensus.cc:740] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:35.860800 22345 consensus_queue.cc:260] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:35.854944 22429 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:35.756583 22436 raft_consensus.cc:2749] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:35.851310 22435 consensus_queue.cc:1059] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:07:35.762024 22428 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:35.756577 22437 raft_consensus.cc:2749] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:07:35.862542 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 100715us
W20260709 13:07:35.862914 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 102530us
I20260709 13:07:35.854830 22129 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:35.886510 15902 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:35.890069 22437 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:35.890683 22428 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:35.891821 22352 consensus_queue.cc:1048] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:35.892340 16563 raft_consensus.cc:1240] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:35.862026 22427 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:35.863453 22315 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.892632 16010 raft_consensus.cc:1275] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 13:07:35.924868 16010 pending_rounds.cc:85] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting all ops after (but not including) 12
I20260709 13:07:35.925127 16010 pending_rounds.cc:107] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:07:35.925351 16010 raft_consensus.cc:2889] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:35.894727 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:35.872320 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:35.914235 15902 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:35.756385 22421 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:07:35.759071 22420 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.719341 22327 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:35.756666 22419 raft_consensus.cc:740] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:35.927521 16011 raft_consensus.cc:1240] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:35.864140 22292 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:35.863942 22112 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.863163 22263 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:35.874155 22162 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:35.862140 22345 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:35.775427 22367 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:35.864581 22392 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:35.921382 22428 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:35.958657 22305 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:35.959213 22429 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:35.924180 22435 raft_consensus.cc:2749] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:07:35.959844 22427 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:35.865417 22304 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:35.870371 22353 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:35.968250 16009 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
I20260709 13:07:35.951790 15902 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:35.912540 22437 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:35.775159 22418 consensus_queue.cc:1048] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.162s
W20260709 13:07:35.972301 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 45731.8us
I20260709 13:07:35.973628 16533 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:36.024020 15827 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:36.024988 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 51450.2us
I20260709 13:07:36.025772 16533 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:07:35.960270 22318 consensus_meta.cc:287] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.653s	user 0.010s	sys 0.000s
I20260709 13:07:35.796732 22346 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:35.968981 22193 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.027s	sys 0.036s
I20260709 13:07:35.981685 15854 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 17: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:35.962642 22396 raft_consensus.cc:2749] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:35.930722 22315 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:35.972435 22263 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:35.976312 22440 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.981997 22367 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:36.046573 16002 raft_consensus.cc:1240] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:35.972100 22292 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:35.975373 22112 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:36.022876 22375 raft_consensus.cc:2804] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:36.064271 16006 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 19.
I20260709 13:07:36.066115 22446 raft_consensus.cc:2749] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:36.048555 22419 consensus_queue.cc:260] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:36.068425 22375 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:36.073905 22375 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:36.024020 22353 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:36.076457 22353 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:36.046985 22090 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:36.078473 22090 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:36.078891 22090 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:36.045599 22325 raft_consensus.cc:2749] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:07:36.081059 22442 log.cc:927] Time spent T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:07:35.968786 22420 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:35.986485 22392 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:36.046446 22318 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:36.085722 22392 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:36.086323 22315 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:36.088469 22037 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:36.089704 22354 consensus_queue.cc:1048] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:36.049187 22435 consensus_queue.cc:1048] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 13:07:36.071760 22440 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:36.095230 22440 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:36.061383 22367 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:36.097069 22432 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:36.097744 22432 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:36.020442 22437 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:36.064987 22263 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:36.055109 22193 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.027s	sys 0.037s
I20260709 13:07:36.062897 22130 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:07:36.131129 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 372289us
W20260709 13:07:36.130050 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 209808us
I20260709 13:07:36.137255 22292 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:36.138672 22129 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:36.067224 22200 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:36.140547 22130 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:36.104921 22353 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:36.156433 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:36.157177 15974 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:36.143189 22367 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:36.200682 15974 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:36.212313 16565 raft_consensus.cc:1240] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 13:07:36.206813 15974 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:36.219599 15974 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:36.198289 22304 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.203816 22349 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:36.203544 22432 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:36.228907 15974 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:36.236289 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 188780us
I20260709 13:07:36.138718 22437 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:36.215873 22455 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.239589 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:36.205291 22459 raft_consensus.cc:2749] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:36.240391 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:36.246347 15861 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:36.228484 22436 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:07:36.248857 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:36.241540 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:36.242244 15859 raft_consensus.cc:1275] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:07:36.237241 16533 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:36.211519 22457 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:36.246721 16569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1398) took 1209 ms. Trace:
I20260709 13:07:36.225832 22450 raft_consensus.cc:2749] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:36.257463 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:36.260197 15856 raft_consensus.cc:1275] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:07:36.254762 16569 rpcz_store.cc:276] 0709 13:07:35.037663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:35.039813 (+  2150us) service_pool.cc:224] Handling call
0709 13:07:35.040219 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:35.215817 (+175598us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:07:35.294628 (+ 78811us) log.cc:844] Serialized 14 byte log entry
0709 13:07:35.328993 (+ 34365us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:35.329047 (+    54us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:35.329090 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:35.930374 (+601284us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b1800396460. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72892a6 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419
0709 13:07:36.246488 (+316114us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b540041d554. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:36.246539 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:36.246667 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":108442,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":83,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":4006,"spinlock_wait_cycles":514608128,"thread_start_us":83491,"threads_started":2,"wal-append.queue_time_us":108249}
I20260709 13:07:36.261032 16568 raft_consensus.cc:1240] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:36.240756 22349 raft_consensus.cc:2804] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:36.264758 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:36.265290 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:36.265952 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:36.270579 15937 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
I20260709 13:07:36.266470 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:36.271545 15928 raft_consensus.cc:2368] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 17. Current term is 18.
I20260709 13:07:36.266520 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:36.267897 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:36.272136 15936 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 16.
I20260709 13:07:36.272279 15858 raft_consensus.cc:2393] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 16: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:36.273566 16533 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:36.274812 16533 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:36.276458 15974 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:36.275084 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:36.279306 16533 leader_election.cc:400] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Vote denied by peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) with higher term. Message: Invalid argument: T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 17. Current term is 18.
I20260709 13:07:36.281515 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:36.282091 15856 raft_consensus.cc:3055] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:36.282299 15932 raft_consensus.cc:2393] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 17: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:36.282495 15856 raft_consensus.cc:740] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:36.280262 16533 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:36.258040 22457 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:36.253024 22436 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:36.287150 22448 log.cc:927] Time spent T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 13:07:36.262146 22456 raft_consensus.cc:2804] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:36.235082 22304 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:36.268872 22459 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.274837 22440 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:36.278244 22443 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.282714 22318 raft_consensus.cc:2804] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:07:36.293884 22422 consensus_meta.cc:287] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.549s	user 0.008s	sys 0.003s
I20260709 13:07:36.291013 22450 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.291275 22436 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:36.290179 22456 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:36.289867 22308 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:36.291164 22457 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:36.298391 22432 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:36.307219 22200 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:36.249996 22455 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:36.300961 22318 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:36.205431 22451 consensus_queue.cc:1048] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.109s
I20260709 13:07:36.314249 22318 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:36.316985 22349 raft_consensus.cc:697] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 18 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:07:36.319865 22444 log.cc:927] Time spent T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 13:07:36.320325 22349 consensus_queue.cc:237] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:36.309075 22456 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:36.324539 22130 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:36.326016 22304 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:36.334022 22428 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:36.373306 15856 consensus_queue.cc:260] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:36.325300 22375 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:36.329800 22338 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:36.325682 22263 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:36.326026 22367 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:36.403040 15931 raft_consensus.cc:1240] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:36.405089 22263 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:36.337622 22419 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:36.399320 22308 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.406949 15854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 2666) took 1396 ms. Trace:
I20260709 13:07:36.407896 22308 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:36.412279 22308 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:36.417245 22318 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:36.425993 22318 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:36.301570 15974 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:36.434057 22375 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:36.436717 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 135060us
I20260709 13:07:36.438537 15974 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:36.440330 22338 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.427439 16005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2591) took 1306 ms. Trace:
I20260709 13:07:36.427587 16008 raft_consensus.cc:1240] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:36.310930 22432 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:36.441450 22338 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:36.443486 22432 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:36.446467 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:36.329113 22437 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:36.442548 16005 rpcz_store.cc:276] 0709 13:07:35.121171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:35.121311 (+   140us) service_pool.cc:224] Handling call
0709 13:07:35.121586 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:35.857448 (+735862us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:36.426829 (+569381us) spinlock_profiling.cc:243] Waited 728 ms on lock 0x7b54000aafd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:36.427197 (+   368us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1674594176}
I20260709 13:07:36.449054 22437 raft_consensus.cc:2804] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:36.449617 22437 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:36.450058 22437 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:36.451782 22338 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:36.304301 22443 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:36.299177 22459 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:36.460002 22463 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:36.460345 22443 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:36.460963 22088 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:36.462459 16006 raft_consensus.cc:1275] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:07:36.463507 22459 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:36.432282 22418 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:36.427994 15854 rpcz_store.cc:276] 0709 13:07:35.010241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:35.104169 (+ 93928us) service_pool.cc:224] Handling call
0709 13:07:36.406305 (+1302136us) spinlock_profiling.cc:243] Waited 800 ms on lock 0x7b5400427cd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:36.406657 (+   352us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1839987072}
I20260709 13:07:36.464638 22418 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:36.466188 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:36.467048 16002 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 16.
I20260709 13:07:36.469415 16005 raft_consensus.cc:1275] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:07:36.470332 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:07:36.316001 22449 log.cc:927] Time spent T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:07:36.471067 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:36.467449 22418 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:36.455363 22162 raft_consensus.cc:2749] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 17. Current term is 18.
W20260709 13:07:36.317854 22452 log.cc:927] Time spent T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 13:07:36.484915 15853 raft_consensus.cc:1240] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:36.486881 22463 consensus_queue.cc:1059] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:36.488969 22425 raft_consensus.cc:3055] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:36.489507 22425 raft_consensus.cc:740] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:36.341862 22455 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:36.431856 22428 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:36.476089 22137 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.310614 22450 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:36.491528 22112 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:36.491467 22425 consensus_queue.cc:260] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:36.498292 22464 consensus_queue.cc:1048] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:36.499262 22425 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:36.500708 22450 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:36.513267 22420 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:36.506592 22137 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:36.513348 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:36.515159 15853 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 17: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:07:36.513520 22426 log.cc:927] Time spent T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 1.075s	user 0.000s	sys 0.004s
I20260709 13:07:36.518257 22327 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:36.520684 22346 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:36.520764 22327 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.534226 16009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60156 (request call id 1251) took 1026 ms. Trace:
I20260709 13:07:36.534943 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:36.536062 15854 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:07:36.537573 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:36.538287 15974 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:36.534693 16009 rpcz_store.cc:276] 0709 13:07:35.508131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:35.508258 (+   127us) service_pool.cc:224] Handling call
0709 13:07:36.533886 (+1025628us) spinlock_profiling.cc:243] Waited 459 ms on lock 0x7b54000ee2d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:36.534046 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1055749888}
I20260709 13:07:36.554407 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:36.521610 22327 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:36.555871 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:36.556960 16569 raft_consensus.cc:1240] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:07:36.557492 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:36.557758 22327 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:36.555474 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:36.554874 22432 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:36.567778 22305 consensus_queue.cc:1059] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:07:36.570796 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:36.571182 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:36.572063 16535 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:36.572162 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:36.573005 16561 raft_consensus.cc:2393] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 15: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:36.558380 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:36.574258 22443 consensus_queue.cc:1048] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:07:36.577572 22443 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:36.577975 22443 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:36.578572 22418 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.572954 16566 raft_consensus.cc:1275] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:07:36.574259 16569 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:07:36.569410 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:36.578748 22325 consensus_queue.cc:1048] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 13:07:36.588255 22418 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:36.569427 22305 raft_consensus.cc:3055] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:36.590444 22305 raft_consensus.cc:740] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:36.578377 22443 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:36.579209 22346 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:36.599555 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:36.600199 16004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60156 (request call id 1253) took 1091 ms. Trace:
I20260709 13:07:36.599785 22305 consensus_queue.cc:260] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:36.600606 16004 rpcz_store.cc:276] 0709 13:07:35.508805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:35.515337 (+  6532us) service_pool.cc:224] Handling call
0709 13:07:36.599832 (+1084495us) spinlock_profiling.cc:243] Waited 54.6 ms on lock 0x7b54002e1dd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:36.600009 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":125496064}
I20260709 13:07:36.599501 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:36.602075 16561 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:36.601353 22346 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:36.603217 22346 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:36.605737 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:36.606567 16563 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
W20260709 13:07:36.607016 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:36.607300 15861 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 18.
I20260709 13:07:36.601353 22305 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:07:36.614950 22387 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.148s	user 0.006s	sys 0.016s
W20260709 13:07:36.615501 22387 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.169s	user 0.006s	sys 0.016s
I20260709 13:07:36.603446 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:36.630312 16561 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:07:36.635776 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:07:36.641398 22385 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.127s	user 0.012s	sys 0.006s
W20260709 13:07:36.641906 22385 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.208s	user 0.012s	sys 0.006s
I20260709 13:07:36.636229 22325 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.652243 22325 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:36.653812 22456 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:36.654661 22325 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:36.635816 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:36.641333 15930 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:07:36.664084 16562 raft_consensus.cc:2393] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 14: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:36.667418 16569 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 20.
I20260709 13:07:36.668231 21965 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:07:36.669178 22090 consensus_meta.cc:287] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.590s	user 0.006s	sys 0.016s
I20260709 13:07:36.668955 21965 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:36.669656 22090 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:36.671387 21965 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:36.672418 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:36.674149 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:36.677408 22418 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:36.677865 22396 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:36.678231 15937 raft_consensus.cc:1240] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:36.707019 22437 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:36.709242 22456 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:36.710253 22396 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:36.712431 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:36.712632 22396 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:36.693982 22090 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:36.714004 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:36.714344 16002 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
I20260709 13:07:36.714787 15937 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:07:36.716123 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:36.716382 22437 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:36.718159 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:07:36.719956 22472 log.cc:927] Time spent T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 13:07:36.720256 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:36.641333 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 276136us
I20260709 13:07:36.721244 15935 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 18.
I20260709 13:07:36.723613 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:36.723784 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:36.724314 16009 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:36.724498 15857 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:36.717525 16011 raft_consensus.cc:1275] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:07:36.727893 15930 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 18.
I20260709 13:07:36.728248 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:36.772567 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:36.773313 15861 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 16.
I20260709 13:07:36.774691 22418 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:36.675843 22436 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:36.776223 15934 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:07:36.707118 22457 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:36.775301 22418 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:36.797119 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:36.797781 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:36.797924 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:36.814350 16006 raft_consensus.cc:2368] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 16. Current term is 17.
I20260709 13:07:36.821077 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:36.822544 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:36.823453 15853 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
W20260709 13:07:36.834439 22445 log.cc:927] Time spent T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.776s	user 0.001s	sys 0.005s
I20260709 13:07:36.837150 22457 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:36.837396 16003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2605) took 1167 ms. Trace:
I20260709 13:07:36.837742 16003 rpcz_store.cc:276] 0709 13:07:35.669590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:35.669733 (+   143us) service_pool.cc:224] Handling call
0709 13:07:35.670040 (+   307us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:35.670357 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:35.670384 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:35.670861 (+   477us) log.cc:844] Serialized 52 byte log entry
0709 13:07:35.912782 (+241921us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:35.912826 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:35.912901 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:36.833537 (+920636us) spinlock_profiling.cc:243] Waited 4.74 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:07:36.836981 (+  3444us) raft_consensus.cc:1685] finished
0709 13:07:36.837001 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:36.837232 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26989,"raft.queue_time_us":953,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":15520,"spinlock_wait_cycles":10903936,"thread_start_us":241367,"threads_started":1,"wal-append.queue_time_us":312007}
I20260709 13:07:36.797746 22456 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.850348 16009 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 14.
I20260709 13:07:36.852851 15936 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:36.865155 16535 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:36.866175 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:36.867060 15930 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:36.868166 22349 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.855607 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:36.896099 15857 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
I20260709 13:07:36.903340 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:36.905334 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:36.920004 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:36.921011 15931 raft_consensus.cc:2393] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 15: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
W20260709 13:07:36.923389 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:36.927496 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:07:36.928349 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:36.928841 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:36.929075 22277 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.929724 16002 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 14: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:36.929608 22277 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:36.932684 22446 consensus_queue.cc:1048] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:36.935040 22432 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:36.935189 16533 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:36.936165 22432 raft_consensus.cc:2749] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:36.938190 22436 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:36.941483 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:36.942231 16006 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:07:36.944283 15930 raft_consensus.cc:1240] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:36.947115 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:36.948022 15858 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:36.947410 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:36.949076 15859 raft_consensus.cc:2393] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 16: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:36.950026 22425 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:36.950776 22425 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:36.952472 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:36.953558 22418 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:36.954856 22425 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:36.956456 22432 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:36.957472 22304 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:36.957793 16003 raft_consensus.cc:2393] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 14: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
W20260709 13:07:36.964370 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 99373us
I20260709 13:07:36.966394 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:36.966751 15858 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:07:36.964612 15825 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:36.968880 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:36.943336 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:36.972332 22441 log.cc:927] Time spent T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.945s	user 0.003s	sys 0.003s
I20260709 13:07:36.859763 22456 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:36.973745 22466 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:36.868896 22349 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:36.978132 22443 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:36.984758 22304 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:36.982877 15935 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:36.986573 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:36.991106 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:36.991957 16535 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:36.995570 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:37.023506 15931 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:36.981614 22418 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:37.032049 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:37.029042 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:36.993697 22466 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.032873 15933 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:07:37.037701 15937 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:37.044251 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:07:37.045042 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:37.046752 22148 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:37.047813 22158 raft_consensus.cc:2749] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:37.048115 22427 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.023319 22459 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:37.048763 22427 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:37.049186 22459 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:37.053395 16569 raft_consensus.cc:1275] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:07:37.057534 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 93173.3us
I20260709 13:07:37.059116 15825 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:37.060012 15825 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:37.060626 22325 raft_consensus.cc:2804] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:37.060946 22349 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:37.067232 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:37.034936 22456 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:37.037310 22466 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:37.080309 15936 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 17.
I20260709 13:07:37.037358 22277 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:37.047391 22148 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:37.051628 22427 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:37.051656 22459 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:37.033875 22432 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:37.085669 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 54751.8us
I20260709 13:07:37.085899 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:37.038053 22162 consensus_queue.cc:1059] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 13:07:37.071233 22325 raft_consensus.cc:697] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:37.094202 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:37.090648 15935 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 19.
I20260709 13:07:37.102860 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:37.073580 22387 consensus_queue.cc:1059] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.014s
I20260709 13:07:37.107337 15936 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:37.110782 15825 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:37.106555 22162 raft_consensus.cc:3055] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:37.111820 22304 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:37.111914 22162 raft_consensus.cc:740] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:37.112917 22162 consensus_queue.cc:260] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:37.073762 22418 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:37.120098 15936 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 19.
I20260709 13:07:37.088888 22395 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:37.105540 22346 consensus_meta.cc:287] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.502s	user 0.010s	sys 0.001s
I20260709 13:07:37.116402 16533 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:37.134346 16533 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:37.122632 15937 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:07:37.103711 22443 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:37.123519 22230 raft_consensus.cc:2804] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:37.123637 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:07:37.128904 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:37.095927 22327 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:37.141425 22432 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:37.143461 15935 raft_consensus.cc:2393] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 14: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:37.151167 22418 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:37.152123 22387 raft_consensus.cc:3055] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:37.155534 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:37.115347 22148 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:07:37.162951 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 68308.1us
I20260709 13:07:37.165290 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:37.168296 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:37.168159 16533 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:37.164882 22108 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:37.171972 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:37.172224 22421 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.172937 22421 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:37.173264 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:37.174417 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:37.175168 15935 raft_consensus.cc:2393] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 19: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:37.175282 22421 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:37.156661 22230 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.181263 16533 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:37.176275 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:37.158516 22304 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:37.158706 22162 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:37.159822 22327 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.179827 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:37.176848 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:37.177297 15974 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:37.184437 16565 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 19.
I20260709 13:07:37.188691 16006 raft_consensus.cc:1275] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:07:37.190663 15858 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:37.190989 15936 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:37.165136 22387 raft_consensus.cc:740] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:37.191916 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:37.194732 15932 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 16.
I20260709 13:07:37.196656 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:37.192997 15825 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:37.137353 22346 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:37.184114 22230 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:37.201061 16004 raft_consensus.cc:1275] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:07:37.187218 22327 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:37.185228 22304 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.189587 22432 raft_consensus.cc:2804] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:37.208799 16569 raft_consensus.cc:1275] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:07:36.486385 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:37.216060 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:37.201740 22443 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:37.219600 22201 raft_consensus.cc:2804] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:37.220075 22201 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:37.172227 22108 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:37.236768 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:37.222335 22316 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:37.222746 22453 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:37.212726 22418 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:37.222363 15753 catalog_manager.cc:5697] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 10 to 14, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 14 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:37.183414 22395 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:37.233136 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:37.246294 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:37.210211 22304 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:37.237509 15935 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
I20260709 13:07:37.223515 22148 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:37.258374 16003 raft_consensus.cc:2393] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 19: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:37.247277 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:37.238716 22201 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:37.251914 15861 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:37.252084 22387 consensus_queue.cc:260] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:37.219070 22432 raft_consensus.cc:697] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:37.258198 15857 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 19.
I20260709 13:07:37.259080 16569 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:07:37.247799 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:37.257568 15933 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:07:37.267668 15858 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:07:37.261243 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:37.261963 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:37.265010 22395 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:37.265594 15931 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 19.
I20260709 13:07:37.265456 22418 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:37.276803 15935 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:07:37.277076 15936 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:07:37.255751 16004 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:37.241829 22316 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.244616 15937 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
I20260709 13:07:37.269889 22148 raft_consensus.cc:697] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:37.243166 22453 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.273988 22108 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:37.276571 16567 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 21.
I20260709 13:07:37.283809 15934 raft_consensus.cc:1240] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:37.279052 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:37.257521 22346 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:37.282841 15931 raft_consensus.cc:1275] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:07:37.286267 22365 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:37.287770 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:37.269220 22427 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.294979 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:37.281941 22316 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:37.307981 15825 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:37.237879 22325 consensus_queue.cc:237] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:37.285276 22453 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:37.396756 22492 consensus_queue.cc:1059] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:07:37.397771 16533 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:37.408509 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:37.410451 15858 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 18.
I20260709 13:07:37.296090 22108 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:37.412894 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:37.413794 16533 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:37.417292 22148 consensus_queue.cc:237] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:37.418186 22432 consensus_queue.cc:237] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:37.408464 22346 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.421866 22346 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:37.424175 15936 raft_consensus.cc:1275] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:07:37.425546 21987 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:37.425969 21987 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.426271 21987 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:37.428572 22478 log.cc:927] Time spent T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
I20260709 13:07:37.430629 15862 raft_consensus.cc:1275] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:07:37.417726 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:37.433902 15858 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:37.438611 15861 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:07:37.413765 15934 raft_consensus.cc:1275] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:07:37.442247 16533 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:37.459005 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:37.459882 15857 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:37.392005 22427 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:37.416275 22387 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:37.427973 22108 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:37.437992 22305 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:37.427574 22230 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:37.474578 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 166723us
I20260709 13:07:37.406574 22365 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:37.492195 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:37.492887 15858 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:07:37.480437 22277 raft_consensus.cc:2804] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:37.496479 15974 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:37.501955 15974 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:37.506750 15974 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:37.508785 15853 raft_consensus.cc:2393] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 16: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:07:37.510512 22488 log.cc:927] Time spent T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:07:37.511133 22108 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:37.512212 22108 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.512813 22108 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:37.514750 22440 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:37.515190 22440 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.515518 22440 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:37.517935 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:37.518556 16010 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:37.520027 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:37.520748 16002 raft_consensus.cc:2393] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 18: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
W20260709 13:07:37.522276 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:37.524690 15857 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 18.
I20260709 13:07:37.525547 22491 raft_consensus.cc:3055] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:37.526007 22491 raft_consensus.cc:740] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:07:37.526532 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 245587us
I20260709 13:07:37.527535 15902 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:37.528997 22137 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:37.530383 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:37.526957 22491 consensus_queue.cc:260] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:37.534662 22491 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:37.536165 15856 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:37.509688 15974 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:07:37.539105 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 125326us
I20260709 13:07:37.487185 22305 raft_consensus.cc:697] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:37.523825 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:37.605755 15974 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:07:37.612514 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 88295.8us
I20260709 13:07:37.503417 22277 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.523113 22088 raft_consensus.cc:2804] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:37.523378 22421 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:37.612215 22443 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:07:37.671664 22490 log.cc:927] Time spent T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 13:07:37.672691 22305 consensus_queue.cc:237] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:37.674518 22418 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:37.675721 22489 log.cc:927] Time spent T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.177s	user 0.000s	sys 0.000s
I20260709 13:07:37.676673 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:37.678123 16533 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:37.679416 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:37.605121 22425 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:37.529008 22108 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:37.608179 22148 raft_consensus.cc:2804] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:37.604555 22346 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:37.687834 16002 raft_consensus.cc:1275] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:07:37.688397 15929 raft_consensus.cc:1240] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:37.695406 16003 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 17: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:37.669737 22277 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:37.674551 22137 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.699553 16565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45232 (request call id 1417) took 1105 ms. Trace:
I20260709 13:07:37.681082 22421 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:37.704563 16565 rpcz_store.cc:276] 0709 13:07:36.589429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:36.603009 (+ 13580us) service_pool.cc:224] Handling call
0709 13:07:37.694490 (+1091481us) spinlock_profiling.cc:243] Waited 408 ms on lock 0x7b54000ce754. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:37.694589 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":937340288}
I20260709 13:07:37.694903 22425 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.701640 22148 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.677225 22304 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:37.677744 22327 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:37.701498 22446 raft_consensus.cc:2804] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:37.713434 22421 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:37.694962 22443 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:37.718024 22425 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:37.723632 22148 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:37.727825 22137 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:37.736497 22230 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:37.723122 22466 consensus_meta.cc:287] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.634s	user 0.011s	sys 0.000s
I20260709 13:07:37.713220 22499 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:37.711916 22346 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:37.727841 22304 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:37.740579 15854 raft_consensus.cc:1240] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:37.735178 22440 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:37.743136 22501 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.715442 22108 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.772311 15859 raft_consensus.cc:2393] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 19: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:37.783811 15856 raft_consensus.cc:1275] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 14. (term mismatch)
I20260709 13:07:37.786724 22108 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:37.790766 22201 consensus_meta.cc:287] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.512s	user 0.009s	sys 0.000s
I20260709 13:07:37.784349 15856 pending_rounds.cc:85] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8: Aborting all ops after (but not including) 13
I20260709 13:07:37.760490 22497 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.792572 15856 pending_rounds.cc:107] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8: Aborting uncommitted NO_OP operation due to leader change: 17.14
I20260709 13:07:37.809835 15856 raft_consensus.cc:2889] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:37.782771 22346 raft_consensus.cc:697] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 17 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:07:37.815341 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 209651us
I20260709 13:07:37.753237 22396 consensus_queue.cc:1059] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:07:37.791266 22201 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:37.787184 22499 raft_consensus.cc:697] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 20 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:07:37.819262 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 211907us
I20260709 13:07:37.776064 22466 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:37.816377 15974 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:37.809569 22304 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:37.808847 22108 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:37.826148 15974 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:37.811921 22427 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:37.813431 22463 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:37.817205 22158 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.838299 15856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 1212) took 1570 ms. Trace:
I20260709 13:07:37.783598 22501 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:37.841670 15856 rpcz_store.cc:276] 0709 13:07:36.264029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:36.267202 (+  3173us) service_pool.cc:224] Handling call
0709 13:07:36.281970 (+ 14768us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:37.535912 (+1253942us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b5800669bd0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3040aa 00007f73ea386447 00007f73ea39efe7 00007f73ea3a07de 00007f73ea3aab49 00007f73ea3ab416 00007f73ea3a7d8f 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4
0709 13:07:37.815906 (+279994us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:37.834315 (+ 18409us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":241323264}
W20260709 13:07:37.848946 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:37.852092 16562 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45244 (request call id 1527) took 1301 ms (client timeout 1501 ms). Trace:
W20260709 13:07:37.859808 16562 rpcz_store.cc:269] 0709 13:07:36.550558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:36.555060 (+  4502us) service_pool.cc:224] Handling call
0709 13:07:37.851822 (+1296762us) spinlock_profiling.cc:243] Waited 86.8 ms on lock 0x7b540048a8d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:37.852015 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":199547648}
I20260709 13:07:37.862394 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:37.863780 15854 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 18.
I20260709 13:07:37.841127 22365 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:37.844624 22427 raft_consensus.cc:2804] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:37.839218 22088 raft_consensus.cc:697] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 17 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:07:37.869107 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 145921us
I20260709 13:07:37.878010 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:37.870072 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:37.870954 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:37.877173 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:37.817673 22497 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:37.880064 16562 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:07:37.842163 22304 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:37.890098 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:37.894524 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:37.874691 22427 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.901947 16567 raft_consensus.cc:2393] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 17: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:37.848076 22158 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:37.858354 22463 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:37.916395 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:37.918090 16563 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
W20260709 13:07:37.921319 15827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:37.885507 22446 raft_consensus.cc:697] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:37.887590 22509 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:37.920643 15902 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:37.930509 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:37.931174 15928 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:37.935031 22201 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:07:37.935639 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:37.935760 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:37.936519 16564 raft_consensus.cc:2368] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 16. Current term is 17.
I20260709 13:07:37.937436 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:37.938169 16564 raft_consensus.cc:2368] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 18. Current term is 19.
I20260709 13:07:37.938433 22088 consensus_queue.cc:237] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:37.939052 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:37.939926 22137 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:37.940533 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:37.915515 22427 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:37.914036 22327 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:37.946406 16563 raft_consensus.cc:1275] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:07:37.946940 16564 raft_consensus.cc:2393] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 17: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:37.939950 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:07:37.930878 22507 log.cc:927] Time spent T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 13:07:37.953660 16566 raft_consensus.cc:1240] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:37.954357 16562 raft_consensus.cc:1240] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:37.954775 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:37.955587 16570 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 17: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:37.922456 22443 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:37.947861 22316 consensus_meta.cc:287] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.537s	user 0.005s	sys 0.006s
I20260709 13:07:37.976297 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:37.979177 15854 raft_consensus.cc:2393] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 14: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:37.926980 22516 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:37.976472 15928 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 20.
I20260709 13:07:37.981842 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:37.950814 22511 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.895012 22346 consensus_queue.cc:237] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:37.958884 22440 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:37.942569 22517 consensus_queue.cc:1048] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.045s
I20260709 13:07:38.010663 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:37.984912 22500 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:38.032601 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:38.003927 22466 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:38.047181 15861 raft_consensus.cc:1240] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 13:07:37.983983 22316 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:38.049870 22446 consensus_queue.cc:237] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.025295 22500 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:07:38.055994 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:38.059744 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 162078us
W20260709 13:07:38.059917 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 160765us
I20260709 13:07:37.947094 22499 consensus_queue.cc:237] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.084911 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:38.085659 15861 raft_consensus.cc:2368] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 16. Current term is 17.
I20260709 13:07:38.071247 15856 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 20.
I20260709 13:07:38.091938 15860 raft_consensus.cc:1240] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:07:38.063252 22500 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:37.988270 22516 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.072974 22443 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.077333 22513 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:38.077064 22501 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:38.022446 22511 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:38.125511 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:38.164198 15857 raft_consensus.cc:1275] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:07:38.131927 15902 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:38.137303 15860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 1226) took 1189 ms. Trace:
I20260709 13:07:38.126839 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:38.113703 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:38.133662 15853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58214 (request call id 1219) took 1253 ms. Trace:
I20260709 13:07:38.102651 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:38.168329 15853 rpcz_store.cc:276] 0709 13:07:36.880410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:36.900464 (+ 20054us) service_pool.cc:224] Handling call
0709 13:07:37.278905 (+378441us) spinlock_profiling.cc:243] Waited 32.1 ms on lock 0x7b5800005798. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:07:38.133412 (+854507us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b540008dc54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:38.133590 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":588557056}
I20260709 13:07:38.166389 15860 rpcz_store.cc:276] 0709 13:07:36.947575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:36.948163 (+   588us) service_pool.cc:224] Handling call
0709 13:07:38.039299 (+1091136us) spinlock_profiling.cc:243] Waited 32.7 ms on lock 0x7b5800005798. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef381fc0 00007f73ef36c929 00007f73ef36c3d2 00007f73e88e3bd4
0709 13:07:38.051815 (+ 12516us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:38.132130 (+ 80315us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:38.137226 (+  5096us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":75115392}
W20260709 13:07:38.133994 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 219431us
W20260709 13:07:38.120160 22510 log.cc:927] Time spent T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
W20260709 13:07:38.118152 22453 consensus_meta.cc:287] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.702s	user 0.008s	sys 0.001s
W20260709 13:07:38.120539 21987 consensus_meta.cc:287] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.694s	user 0.012s	sys 0.000s
W20260709 13:07:38.114086 22504 log.cc:927] Time spent T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.229s	user 0.003s	sys 0.000s
I20260709 13:07:38.180300 22501 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:38.204253 22501 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:38.124025 22497 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:38.206624 22501 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:38.117255 22515 log.cc:927] Time spent T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
W20260709 13:07:38.131156 22491 consensus_meta.cc:287] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.596s	user 0.008s	sys 0.004s
I20260709 13:07:38.177424 15827 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:38.113200 22277 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.114537 22463 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:07:38.228099 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 51423.4us
I20260709 13:07:38.110116 22516 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:38.183781 22518 log.cc:927] Time spent T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 13:07:38.184026 15974 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:38.131445 22158 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:38.107504 22323 consensus_queue.cc:1059] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.077s
W20260709 13:07:38.185001 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 53257.3us
I20260709 13:07:38.185617 22148 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.186666 22527 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:38.191650 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:38.112106 22304 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:38.193734 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:38.237092 15858 raft_consensus.cc:1813] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Handling vote request from an unknown peer 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:38.114001 22521 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.163411 22137 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 13:07:38.244393 16568 raft_consensus.cc:2393] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 16: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:38.209832 21987 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.209141 22453 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.211280 22511 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:38.211885 22466 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.248158 22466 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.250398 22466 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:38.125144 22513 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.252187 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:38.252682 22513 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:38.213840 22316 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:38.190855 22440 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:38.253669 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:38.254264 15861 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:38.254098 22440 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.256062 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:38.256155 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:38.256394 22440 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:38.124433 22443 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.257263 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:38.257407 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:38.258460 22443 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:38.258527 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:38.114192 22519 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.259325 22519 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.259734 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:38.261587 22519 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:38.228892 15827 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:38.099315 22509 raft_consensus.cc:697] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:38.113897 22514 consensus_queue.cc:1048] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.145s
I20260709 13:07:38.198379 22512 raft_consensus.cc:2749] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:38.240496 15902 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:38.241871 15857 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 19.
I20260709 13:07:38.245358 15858 raft_consensus.cc:2410] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:07:38.233769 22463 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.255764 22521 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.256647 15862 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:38.114688 22418 consensus_queue.cc:1059] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 13:07:38.232167 22230 raft_consensus.cc:2804] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:07:38.231755 22162 consensus_meta.cc:287] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 1.045s	user 0.013s	sys 0.000s
I20260709 13:07:38.271441 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:38.276120 15902 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:38.276607 15932 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:07:38.277467 15855 raft_consensus.cc:2393] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 17: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:38.273502 22230 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:38.278710 22230 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:38.279353 22148 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:38.271000 22425 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:38.274367 22513 raft_consensus.cc:2749] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:38.284498 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:38.284986 22425 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:38.271166 22421 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:38.286604 15902 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:38.287613 22421 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:38.287915 15902 leader_election.cc:400] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 16. Current term is 17.
I20260709 13:07:38.288278 15902 leader_election.cc:403] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:38.288810 22492 raft_consensus.cc:2749] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:38.289429 22492 consensus_queue.cc:1048] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:38.290693 22492 raft_consensus.cc:2749] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:38.291276 22492 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 16. Current term is 17.
I20260709 13:07:38.291381 22500 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.275131 22427 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.293637 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:38.294242 16569 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 19.
I20260709 13:07:38.295253 22427 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:38.278605 22491 raft_consensus.cc:2749] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:38.296783 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:38.297365 15931 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:38.299445 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:38.300055 16569 raft_consensus.cc:1813] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Handling vote request from an unknown peer 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:38.300621 16569 raft_consensus.cc:2410] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:07:38.302858 22365 consensus_queue.cc:1059] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:38.304634 22500 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:38.306041 22453 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:38.306897 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:38.308144 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:38.308982 15937 raft_consensus.cc:2393] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 19: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:38.310009 15861 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 19.
I20260709 13:07:38.278849 22509 consensus_queue.cc:237] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.281157 22277 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:38.282015 15857 raft_consensus.cc:3055] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:38.282086 22521 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:38.320883 15857 raft_consensus.cc:740] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:38.330125 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:38.331010 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:38.331066 15858 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 20.
W20260709 13:07:38.331535 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 147616us
I20260709 13:07:38.331233 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:38.333887 15929 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:38.330253 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:38.335929 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:38.336565 16565 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 17.
I20260709 13:07:38.336445 22514 consensus_queue.cc:1048] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:38.339205 15902 leader_election.cc:400] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 16. Current term is 17.
I20260709 13:07:38.339569 15902 leader_election.cc:403] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:38.341252 16533 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:38.341380 15932 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:38.342482 15931 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:38.342599 16533 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:07:38.295315 15900 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.501s (SENT) [suppressed 8 similar messages]
I20260709 13:07:38.345502 16533 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:38.349915 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:38.348479 15902 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:38.401052 16563 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 18.
W20260709 13:07:38.403102 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 54708.1us
I20260709 13:07:38.404114 15902 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:07:38.405351 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 59934.1us
I20260709 13:07:38.410738 16533 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:38.276295 22463 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.402096 15857 consensus_queue.cc:260] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.411834 22418 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:38.415997 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:38.326090 22519 consensus_queue.cc:1059] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.079s
I20260709 13:07:38.398350 22162 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:38.412348 16533 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:38.422456 15857 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:07:38.427987 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 150328us
I20260709 13:07:38.404683 22521 raft_consensus.cc:2749] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:38.407591 15974 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:38.267180 22527 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.405432 22500 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:07:38.407946 22492 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:38.416256 22277 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:38.399909 22443 consensus_queue.cc:1059] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.084s
I20260709 13:07:38.428164 22418 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:38.450291 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:38.428997 22466 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:38.452615 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:38.458333 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 163034us
W20260709 13:07:38.461232 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 53736us
I20260709 13:07:38.446004 22492 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:38.462975 15853 raft_consensus.cc:2410] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 20 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 13:07:38.466953 15857 raft_consensus.cc:1275] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:07:38.467252 15857 pending_rounds.cc:85] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8: Aborting all ops after (but not including) 10
I20260709 13:07:38.467463 15857 pending_rounds.cc:107] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:07:38.467658 15857 raft_consensus.cc:2889] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:38.468952 15974 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:38.470597 22532 log.cc:927] Time spent T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:07:38.472620 22534 log.cc:927] Time spent T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:07:38.476902 22527 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:38.479823 22137 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 13:07:38.481351 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:38.482712 15825 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:38.483258 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:38.483737 15825 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:38.484002 15861 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 17.
I20260709 13:07:38.454581 22305 raft_consensus.cc:3055] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:38.486040 22305 raft_consensus.cc:740] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:38.463260 15827 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:38.489313 15929 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 17.
I20260709 13:07:38.454319 22427 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:38.458001 22418 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:38.457545 22277 raft_consensus.cc:697] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:38.458032 22516 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:38.461100 22346 raft_consensus.cc:2749] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:38.468772 22492 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:38.504552 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:38.494266 22158 raft_consensus.cc:2804] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:38.494087 22387 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:38.468611 22466 raft_consensus.cc:697] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:07:38.509802 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:38.511487 22277 consensus_queue.cc:237] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.513599 22322 raft_consensus.cc:3055] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:38.514097 22322 raft_consensus.cc:740] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:38.515961 15825 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:38.479564 22230 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.496179 22527 raft_consensus.cc:2749] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:38.497999 22516 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:38.492606 22443 consensus_queue.cc:1048] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:07:38.525966 15859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 2684) took 1370 ms. Trace:
I20260709 13:07:38.526854 15859 rpcz_store.cc:276] 0709 13:07:37.155647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:37.160506 (+  4859us) service_pool.cc:224] Handling call
0709 13:07:38.525758 (+1365252us) spinlock_profiling.cc:243] Waited 578 ms on lock 0x7b54000b5c54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:38.525899 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1330108160}
I20260709 13:07:38.496423 22427 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:07:38.520308 22422 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:07:38.530411 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 67230.6us
I20260709 13:07:38.533030 15827 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:38.521917 21987 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:38.534439 15856 raft_consensus.cc:2368] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 15. Current term is 16.
I20260709 13:07:38.536952 22503 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:38.539492 22463 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:38.540200 22322 consensus_queue.cc:260] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.526654 22516 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:38.549294 15862 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 17.
I20260709 13:07:38.551538 22322 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:38.549970 22547 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:38.557473 22500 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 16. Current term is 17.
I20260709 13:07:38.565168 22230 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:38.565326 22466 consensus_queue.cc:237] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.550525 22316 raft_consensus.cc:2764] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:07:38.567888 22305 consensus_queue.cc:260] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:38.568945 22538 log.cc:927] Time spent T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:07:38.569159 22305 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:38.570405 22540 log.cc:927] Time spent T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:07:38.570703 22427 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:38.573609 22514 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.574100 22514 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.576175 22514 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:38.577070 22422 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.579185 22422 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:38.580034 15974 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:38.581024 15974 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:38.583933 15928 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 17.
I20260709 13:07:38.589284 22392 raft_consensus.cc:2804] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:38.589752 22392 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:38.590096 22392 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:38.591523 15974 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:07:38.595065 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 62108.1us
I20260709 13:07:38.595943 15827 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:38.596316 22387 raft_consensus.cc:697] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:38.597541 22387 consensus_queue.cc:237] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:38.609898 15974 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:38.614496 15974 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:38.622373 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:38.624137 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:38.624822 16569 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:07:38.591954 22422 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:38.593456 22427 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:38.630264 15827 leader_election.cc:400] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 election: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 18. Current term is 19.
I20260709 13:07:38.634635 15827 leader_election.cc:403] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:07:38.631991 22551 raft_consensus.cc:2749] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:38.632911 22418 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.616902 22547 raft_consensus.cc:697] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 17 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:38.635146 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:38.643319 22547 consensus_queue.cc:237] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.644100 16009 raft_consensus.cc:2368] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 15. Current term is 16.
I20260709 13:07:38.645076 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:38.645669 16010 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:38.651868 22349 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.652036 22459 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:38.652566 22349 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.652853 22459 raft_consensus.cc:697] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:38.654589 15855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58214 (request call id 1225) took 1707 ms. Trace:
I20260709 13:07:38.654224 22459 consensus_queue.cc:237] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.654983 15855 rpcz_store.cc:276] 0709 13:07:36.947233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:36.947844 (+   611us) service_pool.cc:224] Handling call
0709 13:07:38.010603 (+1062759us) spinlock_profiling.cc:243] Waited 31.3 ms on lock 0x7b5800005798. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:07:38.654391 (+643788us) spinlock_profiling.cc:243] Waited 238 ms on lock 0x7b54003b5954. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:38.654532 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":618655232}
I20260709 13:07:38.618249 22463 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:38.656019 22501 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:38.656687 22519 raft_consensus.cc:3055] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:38.656605 22501 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.657094 22519 raft_consensus.cc:740] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:38.657182 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:38.621222 22088 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:38.657963 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:38.658455 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:38.659241 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:38.659518 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:38.657845 15936 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
I20260709 13:07:38.660145 16533 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:38.660320 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:38.660609 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:38.661274 15936 raft_consensus.cc:2393] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 16: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:38.661386 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:38.661924 15929 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
I20260709 13:07:38.662184 15936 raft_consensus.cc:1240] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:38.664312 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:38.665257 15931 raft_consensus.cc:1275] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:07:38.665757 15934 raft_consensus.cc:1240] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:38.666942 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:38.668948 15935 raft_consensus.cc:1240] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:38.612232 22158 raft_consensus.cc:697] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:38.675073 15932 raft_consensus.cc:1275] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:07:38.656044 22463 raft_consensus.cc:697] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:38.675120 22322 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 18. Current term is 19.
I20260709 13:07:38.677552 15902 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:38.678369 15827 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:38.683208 16533 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:38.689311 16009 raft_consensus.cc:1240] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:38.690935 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:38.692818 16533 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:38.694603 16533 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:38.679927 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:38.696302 15974 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:38.697139 16570 raft_consensus.cc:3060] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:07:38.595534 22492 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.698957 15902 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:38.669461 22440 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:38.703325 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:38.705289 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:38.706691 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:38.718688 16569 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:38.719166 15862 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:38.708038 22514 raft_consensus.cc:2804] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:38.706288 22201 consensus_queue.cc:1048] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 13:07:38.711197 22421 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.711328 22110 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
I20260709 13:07:38.711216 22304 consensus_queue.cc:1059] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:07:38.712072 22542 log.cc:927] Time spent T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 13:07:38.712579 22443 consensus_queue.cc:1059] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:07:38.714237 22501 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:38.715837 22549 log.cc:927] Time spent T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 13:07:38.721542 16570 raft_consensus.cc:2468] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:38.725513 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:38.775292 15853 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:38.677026 22418 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:38.726795 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:38.697264 22493 consensus_queue.cc:1048] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:07:38.752231 22513 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:38.781325 22162 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.781252 22513 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:38.781885 22162 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.724243 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:38.731442 15933 raft_consensus.cc:2393] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 20: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:38.731798 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:38.787114 15853 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 18.
I20260709 13:07:38.733321 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:38.723914 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:38.697018 22325 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:38.708686 22497 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:38.792012 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:38.792397 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:38.792734 16006 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:38.717226 22500 raft_consensus.cc:2804] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:38.793586 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:38.793759 22500 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:38.794204 22500 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:38.794569 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:38.796416 16003 raft_consensus.cc:1275] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:07:38.796993 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:38.717228 22422 raft_consensus.cc:2804] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:38.796186 15976 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:38.803649 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:38.748338 15974 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:07:38.806049 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 57807.5us
I20260709 13:07:38.750979 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:38.755730 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:38.756528 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:38.757788 16563 raft_consensus.cc:1240] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:38.758620 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:38.760125 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:38.760583 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:38.760856 22545 log.cc:927] Time spent T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 13:07:38.658031 22519 consensus_queue.cc:260] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.727259 22551 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:38.691851 22427 consensus_queue.cc:1048] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:07:38.716806 22521 consensus_queue.cc:1059] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 13:07:38.762739 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:38.764816 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 86551.9us
I20260709 13:07:38.699944 22520 consensus_queue.cc:1048] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 13:07:38.696909 22509 consensus_queue.cc:1048] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
W20260709 13:07:38.831303 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 315424us
I20260709 13:07:38.766299 22440 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:38.767714 22528 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:38.768368 22148 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:38.679869 22499 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.722831 22514 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:38.843817 22162 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:38.843642 22148 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:38.844064 22514 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:38.845768 22148 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:38.847263 22520 raft_consensus.cc:2749] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:38.847761 16004 raft_consensus.cc:2393] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 19: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:38.751907 22108 raft_consensus.cc:3055] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:38.848369 22108 raft_consensus.cc:740] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:38.849416 22108 consensus_queue.cc:260] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:38.849702 22392 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.850498 22108 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:38.850749 15860 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:07:38.825657 22530 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.789836 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:38.852255 15928 raft_consensus.cc:2393] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 19: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:38.851971 22530 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.790122 15753 catalog_manager.cc:5697] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 12 to 13, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 13 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:07:38.794152 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:38.710016 22466 raft_consensus.cc:2804] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:38.797735 16567 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:07:38.854079 22466 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:38.798403 15856 raft_consensus.cc:2393] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 16: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:38.854537 22466 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:38.802552 15862 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 19.
I20260709 13:07:38.794222 16005 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 19.
I20260709 13:07:38.799408 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:38.856743 15936 raft_consensus.cc:2393] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 16: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:38.856454 22499 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.810467 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:38.857625 16562 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:38.857785 16567 raft_consensus.cc:1240] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:38.811709 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:38.837860 22440 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.813490 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:38.813565 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:38.723574 22432 consensus_queue.cc:1048] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.075s
I20260709 13:07:38.677037 22463 consensus_queue.cc:237] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:38.768414 22523 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:38.874928 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:38.875048 22523 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:38.745036 22421 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.876169 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:38.876932 16562 raft_consensus.cc:2393] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 17: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:38.807986 22425 consensus_queue.cc:1059] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 13:07:38.827541 16569 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 18.
I20260709 13:07:38.877938 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:38.878203 22425 raft_consensus.cc:3055] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:38.878707 22425 raft_consensus.cc:740] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:38.873034 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:38.879690 16002 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 13.
I20260709 13:07:38.879619 22425 consensus_queue.cc:260] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.880661 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:38.877976 22421 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:38.881043 22425 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:38.881311 15930 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 17.
I20260709 13:07:38.878062 22392 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:38.808398 22422 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:38.871480 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:38.877338 22523 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:38.878808 16567 raft_consensus.cc:2393] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 18: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:38.835814 15932 raft_consensus.cc:1275] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 13. (term mismatch)
I20260709 13:07:38.836720 22519 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:38.871836 16008 raft_consensus.cc:2393] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 18: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:38.871526 16006 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 19.
I20260709 13:07:38.869227 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:38.883661 15900 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:38.870635 15859 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 19.
I20260709 13:07:38.889567 15932 pending_rounds.cc:85] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 12
I20260709 13:07:38.896700 15932 pending_rounds.cc:107] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 14.13
I20260709 13:07:38.897065 15932 raft_consensus.cc:2889] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:38.890473 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:38.812538 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:38.899034 15931 raft_consensus.cc:2393] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 18: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:38.899976 16566 raft_consensus.cc:2393] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 16: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:38.901996 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:38.903609 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 107495us
I20260709 13:07:38.904543 15976 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:38.904991 15862 raft_consensus.cc:1240] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:07:38.906340 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:38.913439 16010 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 17.
I20260709 13:07:38.915097 15855 raft_consensus.cc:1240] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 17 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:38.916446 16004 pending_rounds.cc:85] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting all ops after (but not including) 9
I20260709 13:07:38.916923 16004 pending_rounds.cc:107] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:07:38.917210 16004 raft_consensus.cc:2889] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:38.918030 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:38.889683 22422 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:38.886894 22511 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:38.812171 22158 consensus_queue.cc:237] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.923480 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:38.924366 15859 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 17.
I20260709 13:07:38.923192 15853 raft_consensus.cc:1275] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 13:07:38.931851 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:38.937565 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:38.900039 22349 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:38.938616 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 131906us
I20260709 13:07:38.900204 22459 raft_consensus.cc:2764] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:07:38.900000 22418 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.941154 16566 raft_consensus.cc:2393] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 17: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:38.942291 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:38.916436 22501 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:07:38.943291 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 59703.4us
I20260709 13:07:38.947580 16562 raft_consensus.cc:2393] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 16: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:38.947741 15974 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:38.953033 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:38.863871 22528 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.953660 16006 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:38.955134 15900 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:38.958305 15900 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:38.951359 15974 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:38.959827 15974 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:38.939831 22513 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:07:38.963061 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 58620.4us
I20260709 13:07:38.963840 15976 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:38.945266 22365 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.945905 22346 raft_consensus.cc:2749] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:07:38.970224 15859 raft_consensus.cc:1240] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:38.971493 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:38.950932 22418 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.974884 22530 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:38.967995 22492 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:38.941581 22511 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:38.966308 22523 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:38.978890 16006 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:39.003109 15936 raft_consensus.cc:1240] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 19. Current term is 20. Ops: [17.14-19.15]
I20260709 13:07:39.002449 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:39.016203 15827 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:38.957641 22463 raft_consensus.cc:2804] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:38.997051 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:39.018607 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:39.019294 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:39.020573 15827 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:39.020620 15862 raft_consensus.cc:1240] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 13:07:39.024087 15861 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 19.
I20260709 13:07:39.025363 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:39.029836 16007 raft_consensus.cc:2393] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 16: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:39.030786 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:38.967209 22322 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.034616 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:39.035264 15974 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:39.030395 22491 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:38.977289 22528 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:38.980311 22492 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:07:39.040396 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 76625.7us
I20260709 13:07:39.042809 22500 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:39.043485 22550 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.008034 22547 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:38.989604 22511 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:39.018594 22421 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:07:39.030750 22488 log.cc:927] Time spent T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:07:38.996022 22560 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.986434 22514 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:39.025650 22523 raft_consensus.cc:697] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:38.975653 22501 consensus_queue.cc:1048] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 13:07:39.024873 22551 raft_consensus.cc:2749] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:07:39.062665 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 104409us
I20260709 13:07:39.068102 15974 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:39.082525 15974 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:39.068295 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:39.083248 15931 raft_consensus.cc:1240] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:07:39.028316 22516 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:39.064429 22440 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:39.048424 22346 raft_consensus.cc:2804] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:39.033645 22463 raft_consensus.cc:697] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:07:39.059187 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:38.986585 22365 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:39.031066 22387 raft_consensus.cc:2804] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:39.045356 22322 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:39.071779 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:39.076855 22088 raft_consensus.cc:2749] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:07:39.092940 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 258116us
I20260709 13:07:39.075799 22466 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:39.132176 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:39.127456 15900 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:39.136729 16533 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:39.089782 22421 raft_consensus.cc:697] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:39.093746 22346 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.092373 22349 raft_consensus.cc:2749] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:39.143100 16533 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:07:39.179122 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 51708.7us
I20260709 13:07:39.180223 16533 leader_election.cc:400] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 15. Current term is 16.
I20260709 13:07:39.180595 16533 leader_election.cc:403] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:39.091066 22520 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.177223 15859 raft_consensus.cc:2393] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 14: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:39.076494 22547 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:39.131187 22443 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.191946 16568 raft_consensus.cc:1240] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:39.138842 22511 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.075248 22550 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:39.132678 22387 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.223337 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:39.223433 16533 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:39.093255 22509 consensus_queue.cc:1048] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.089s
I20260709 13:07:39.240249 15932 raft_consensus.cc:1275] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:07:39.241776 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:39.242736 15933 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 19.
I20260709 13:07:39.183290 22346 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:39.080379 22560 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:39.185544 22304 raft_consensus.cc:2764] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:07:39.155175 22440 raft_consensus.cc:2749] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:39.246145 15932 raft_consensus.cc:1240] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:39.216701 22162 raft_consensus.cc:2749] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 15. Current term is 16.
I20260709 13:07:39.238662 22387 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:39.135136 22492 consensus_queue.cc:1059] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.202s
I20260709 13:07:39.255962 22277 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:39.355269 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:39.196058 22577 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:39.215616 22568 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:39.196576 22520 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:39.357601 22497 consensus_meta.cc:287] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.566s	user 0.008s	sys 0.004s
I20260709 13:07:39.408347 15933 raft_consensus.cc:1275] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 13:07:39.258606 22511 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:39.255525 22088 consensus_queue.cc:1048] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.111s
I20260709 13:07:39.416391 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:39.424832 15862 raft_consensus.cc:2368] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 16. Current term is 17.
I20260709 13:07:39.425704 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:39.427099 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:39.362639 22577 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.428485 22577 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:39.429435 15862 raft_consensus.cc:2376] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 20. Re-sending same reply.
I20260709 13:07:39.434521 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:39.438647 22108 consensus_meta.cc:287] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.588s	user 0.010s	sys 0.000s
I20260709 13:07:39.418566 22565 consensus_queue.cc:1048] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.241s
W20260709 13:07:39.420205 22574 log.cc:927] Time spent T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
W20260709 13:07:39.420852 22425 consensus_meta.cc:287] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.539s	user 0.008s	sys 0.002s
I20260709 13:07:39.422737 22511 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:39.423087 22497 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:39.439260 15862 raft_consensus.cc:2368] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 16. Current term is 17.
I20260709 13:07:39.443357 22466 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:39.343544 22463 consensus_queue.cc:237] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:39.332052 22523 consensus_queue.cc:237] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:39.447141 22305 raft_consensus.cc:3055] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:39.451367 22499 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:39.474714 22492 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.456722 22365 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:39.460731 22560 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:39.463812 22446 raft_consensus.cc:3055] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:39.465517 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:39.544098 15853 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:07:39.465852 22422 consensus_meta.cc:287] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.517s	user 0.007s	sys 0.000s
I20260709 13:07:39.466594 22577 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:39.471272 15900 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:39.550627 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 79443.1us
I20260709 13:07:39.550806 22577 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:39.551469 15900 leader_election.cc:400] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 16. Current term is 17.
I20260709 13:07:39.552320 22547 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:39.552979 15900 leader_election.cc:400] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 16. Current term is 17.
I20260709 13:07:39.473057 22550 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:39.237947 22443 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:39.554901 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:39.555236 22322 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:39.556169 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:39.556501 22443 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:39.557278 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:39.558792 15860 raft_consensus.cc:1275] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 13:07:39.560882 22519 consensus_meta.cc:287] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.660s	user 0.009s	sys 0.000s
W20260709 13:07:39.501515 16008 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.011s	sys 0.051s
W20260709 13:07:39.562520 22491 consensus_meta.cc:287] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.522s	user 0.002s	sys 0.006s
I20260709 13:07:39.563244 22491 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:39.532466 22583 consensus_queue.cc:1059] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:07:39.505014 15974 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:39.566190 22325 raft_consensus.cc:3055] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:39.566663 22325 raft_consensus.cc:740] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:39.567581 22325 consensus_queue.cc:260] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:39.541283 22514 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:39.568897 22325 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:39.570400 22418 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:39.547861 22492 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:39.512454 16562 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:07:39.514274 22387 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:39.562580 16008 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.011s	sys 0.051s
I20260709 13:07:39.454960 22520 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:39.547458 22446 raft_consensus.cc:740] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:39.428177 22516 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:39.541222 15937 raft_consensus.cc:1240] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:39.575661 22387 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:39.577880 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:39.578680 16562 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:07:39.579201 15853 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 20.
I20260709 13:07:39.566211 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:39.580472 15853 raft_consensus.cc:1240] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:07:39.583745 15900 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:39.480264 22568 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:39.545406 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:39.585217 15900 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:39.585863 15930 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:07:39.576793 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:39.588052 22422 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:39.605438 15860 raft_consensus.cc:1275] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:07:39.588049 22497 raft_consensus.cc:2749] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:39.581380 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:39.535226 22305 raft_consensus.cc:740] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:39.606261 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:39.606577 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:39.607393 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:39.622310 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 492043us
I20260709 13:07:39.622666 16011 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:07:39.606014 22559 consensus_queue.cc:1048] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.018s
I20260709 13:07:39.609309 22583 raft_consensus.cc:3055] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:07:39.610097 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 105386us
I20260709 13:07:39.625541 15974 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:39.610747 22520 raft_consensus.cc:2749] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:39.608536 22346 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:39.612056 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:39.612198 22501 raft_consensus.cc:3055] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:39.612490 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:39.634837 15860 raft_consensus.cc:1275] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 13:07:39.612736 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:39.574640 22446 consensus_queue.cc:260] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:39.613202 22557 log.cc:927] Time spent T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.757s	user 0.000s	sys 0.005s
I20260709 13:07:39.613190 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:39.606264 22568 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:39.615082 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:39.644765 15976 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:39.648065 16567 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:39.616078 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:39.616595 16562 raft_consensus.cc:1240] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:39.616770 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:39.648944 16565 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 17.
I20260709 13:07:39.502864 22421 consensus_queue.cc:237] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:39.537384 22500 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:39.551404 22577 raft_consensus.cc:2749] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:39.620085 22418 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:39.629488 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:07:39.633188 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:39.640071 16005 raft_consensus.cc:1240] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:39.641901 15862 raft_consensus.cc:1275] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:07:39.645602 16003 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:39.648325 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:39.667433 16006 raft_consensus.cc:2368] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 14. Current term is 15.
I20260709 13:07:39.634128 22501 raft_consensus.cc:740] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:39.628446 22583 raft_consensus.cc:740] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:39.628759 22499 consensus_queue.cc:1059] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:07:39.655186 22158 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:39.665622 22446 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:07:39.700798 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 44437us
I20260709 13:07:39.649909 22425 consensus_queue.cc:1048] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 13:07:39.701951 15974 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:39.690192 22418 raft_consensus.cc:697] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 20 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:39.707953 15974 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:39.675310 22520 consensus_queue.cc:1048] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:07:39.710553 15827 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:39.714955 15827 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:39.718199 15827 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:39.702426 22492 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:39.732862 15827 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:39.704653 22523 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:39.721495 22387 raft_consensus.cc:2749] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:39.721057 22508 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:39.700736 22158 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:39.712626 22592 raft_consensus.cc:2764] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:07:39.745105 16563 raft_consensus.cc:1275] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 14. (term mismatch)
I20260709 13:07:39.747735 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:39.620585 15753 catalog_manager.cc:5697] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 8 to 12, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 12 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:07:39.753165 16567 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 19.
I20260709 13:07:39.763084 16563 pending_rounds.cc:85] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting all ops after (but not including) 13
I20260709 13:07:39.763499 16563 pending_rounds.cc:107] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting uncommitted NO_OP operation due to leader change: 16.14
I20260709 13:07:39.763805 16563 raft_consensus.cc:2889] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:39.812747 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:39.813478 15854 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:39.813544 22158 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:39.743773 22322 raft_consensus.cc:2749] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:39.818549 22501 consensus_queue.cc:260] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:39.819600 22422 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:39.819828 22501 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:07:39.823222 22581 log.cc:927] Time spent T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 13:07:39.824452 15827 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:39.825521 15827 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:39.826622 15827 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:39.828609 22387 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:39.828898 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:39.829057 22387 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:39.829489 22387 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:39.829675 16567 raft_consensus.cc:2393] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 14: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:39.828429 22418 consensus_queue.cc:237] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:39.707049 22509 consensus_queue.cc:1059] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 13:07:39.732664 22584 consensus_queue.cc:1048] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.110s
W20260709 13:07:39.740983 22586 log.cc:927] Time spent T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 13:07:39.741235 22551 raft_consensus.cc:2749] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:39.846383 22110 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:39.846936 22110 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:39.827392 22323 raft_consensus.cc:2804] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:39.848232 22323 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.848677 22323 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:39.743970 22585 consensus_queue.cc:1048] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.107s
I20260709 13:07:39.756009 22583 consensus_queue.cc:260] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:39.699045 22305 consensus_queue.cc:260] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:39.860224 22305 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:39.865890 22148 consensus_queue.cc:1048] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:39.902444 16003 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:07:39.902897 15854 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:39.723858 22346 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:39.907626 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:39.934710 15935 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 17.
W20260709 13:07:39.913633 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:39.917641 22508 raft_consensus.cc:2764] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election decision vote started in defunct term 14: won
I20260709 13:07:39.919034 22110 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:39.919760 22158 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.921672 15825 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:39.924413 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:39.924880 22551 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.926276 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:39.928159 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:39.932722 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:39.940183 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:39.942370 15928 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 18.
I20260709 13:07:39.909626 15974 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:39.942139 15930 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:39.942119 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:39.949235 15936 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:07:39.743005 22523 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:39.951990 22110 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:07:39.976783 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 67289.7us
I20260709 13:07:39.904225 22584 raft_consensus.cc:3055] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:40.175992 15928 raft_consensus.cc:1275] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:07:39.718647 22463 raft_consensus.cc:2749] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:07:40.197810 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:40.194411 15976 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.050s	sys 0.072s
W20260709 13:07:40.221725 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:39.958221 22523 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:07:40.230759 16008 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.018s	sys 0.043s
I20260709 13:07:40.179008 22592 raft_consensus.cc:2804] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:07:40.233582 16008 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.018s	sys 0.043s
W20260709 13:07:40.216394 15976 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.050s	sys 0.072s
W20260709 13:07:40.235425 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 592791us
I20260709 13:07:40.237582 15976 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:40.133149 22492 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.244887 15976 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
W20260709 13:07:40.246040 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:40.252420 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:40.209790 22584 raft_consensus.cc:740] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:39.953635 22158 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:40.248382 16008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2637) took 1257 ms. Trace:
I20260709 13:07:40.243487 22569 raft_consensus.cc:2749] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:07:40.266366 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 311604us
I20260709 13:07:40.251835 22463 raft_consensus.cc:2749] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:39.957932 22551 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:40.265007 15976 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:40.273787 15976 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
W20260709 13:07:40.276073 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 322008us
I20260709 13:07:40.262679 22592 raft_consensus.cc:697] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:07:40.278085 15900 leader_election.cc:336] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.501s (SENT)
I20260709 13:07:40.278625 15900 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:40.267215 22568 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:40.254235 22492 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:40.269729 22459 raft_consensus.cc:2804] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:40.285284 15825 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:40.295162 15825 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:40.276662 22349 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:40.298398 15825 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:40.264621 16008 rpcz_store.cc:276] 0709 13:07:38.991261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:38.998065 (+  6804us) service_pool.cc:224] Handling call
0709 13:07:38.998350 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:38.998553 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:07:39.573762 (+575209us) log.cc:844] Serialized 14 byte log entry
0709 13:07:40.234601 (+660839us) log.cc:844] Serialized 14 byte log entry
0709 13:07:40.239192 (+  4591us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:40.239259 (+    67us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:40.239326 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:40.239511 (+   185us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:40.248188 (+  8677us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4971,"prepare.queue_time_us":94055,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":269,"raft.queue_time_us":18872,"raft.run_cpu_time_us":292,"raft.run_wall_time_us":8335,"spinlock_wait_cycles":5696512,"thread_start_us":1234916,"threads_started":3,"wal-append.queue_time_us":132427}
I20260709 13:07:40.283125 22346 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:40.301872 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:40.302120 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:40.305536 16010 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 15.
I20260709 13:07:40.306789 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:40.304199 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:40.307646 16011 raft_consensus.cc:2410] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
W20260709 13:07:40.311939 22578 log.cc:927] Time spent T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.943s	user 0.005s	sys 0.008s
I20260709 13:07:40.316547 22365 raft_consensus.cc:2804] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:40.316980 22365 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:40.317338 22365 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:40.284435 22584 consensus_queue.cc:260] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:40.324158 22584 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:40.324429 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:40.324712 16005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2624) took 1465 ms. Trace:
I20260709 13:07:40.325213 16007 raft_consensus.cc:2393] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 14: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:40.304641 22346 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:40.326242 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:40.326669 22346 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:40.328549 22492 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:40.332007 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:40.332481 15859 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 2575) took 1363 ms (client timeout 1501 ms). Trace:
W20260709 13:07:40.332944 15859 rpcz_store.cc:269] 0709 13:07:38.968956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:38.971247 (+  2291us) service_pool.cc:224] Handling call
0709 13:07:40.332284 (+1361037us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b540032d554. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:40.332426 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":471229184}
I20260709 13:07:40.325227 16005 rpcz_store.cc:276] 0709 13:07:38.859386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:38.859864 (+   478us) service_pool.cc:224] Handling call
0709 13:07:38.871541 (+ 11677us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:39.676218 (+804677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:40.324392 (+648174us) spinlock_profiling.cc:243] Waited 768 ms on lock 0x7b54001b9754. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:40.324645 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1766950528}
I20260709 13:07:40.338243 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:40.300252 22512 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.348889 16566 raft_consensus.cc:1275] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:07:40.299980 22349 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:40.343251 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:40.352816 16567 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:07:40.353793 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:40.354102 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:40.354475 16567 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:07:40.342474 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:40.302187 22459 raft_consensus.cc:697] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:40.293504 22463 consensus_queue.cc:1059] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.011s
I20260709 13:07:40.347895 16562 raft_consensus.cc:1240] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:40.357971 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:40.302322 22418 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:07:40.358670 22463 raft_consensus.cc:3055] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:40.349520 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:40.359107 22463 raft_consensus.cc:740] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:40.359557 16569 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:40.360355 22463 consensus_queue.cc:260] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:40.361604 22463 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:40.363955 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:40.365156 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:40.327347 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:40.365693 16010 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:40.366360 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:40.349915 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:40.356930 16567 raft_consensus.cc:1240] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 18. Current term is 19. Ops: [18.13-18.13]
W20260709 13:07:40.352262 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:40.358577 16565 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:07:40.371084 16569 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:40.369314 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:40.373023 16005 raft_consensus.cc:1240] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:07:40.374773 22592 consensus_queue.cc:237] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:40.377681 22551 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:40.339206 22277 consensus_meta.cc:287] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.976s	user 0.009s	sys 0.004s
I20260709 13:07:40.394148 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:40.357061 22349 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:40.378805 22594 raft_consensus.cc:2804] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:40.355643 22550 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:07:40.389425 22387 consensus_meta.cc:287] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.559s	user 0.017s	sys 0.000s
W20260709 13:07:40.425765 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:40.425563 15976 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:40.395107 22277 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:40.432386 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:40.412503 22550 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:40.412905 22594 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:40.356462 22512 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:40.439107 16561 raft_consensus.cc:1275] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:07:40.419934 22110 raft_consensus.cc:697] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 17 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:40.434242 22550 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:40.433817 22421 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:40.441581 22594 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:40.416302 22387 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:40.454607 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:40.456358 22158 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:40.449626 22421 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:40.459342 22418 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:40.437286 16010 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 20.
I20260709 13:07:40.463780 22421 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:40.452116 22459 consensus_queue.cc:237] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:40.467942 22418 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:40.470376 22418 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:40.471935 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:40.472628 16003 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:07:40.473690 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:40.474328 16003 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 17.
I20260709 13:07:40.480641 22564 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:40.481249 22564 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:07:40.488497 16006 raft_consensus.cc:2410] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
W20260709 13:07:40.490098 22608 log.cc:927] Time spent T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:07:40.491361 16006 raft_consensus.cc:1240] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:07:40.494035 16006 raft_consensus.cc:1275] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:07:40.494320 22446 consensus_meta.cc:287] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.794s	user 0.015s	sys 0.000s
W20260709 13:07:40.496326 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:40.498061 22349 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:40.499152 16008 raft_consensus.cc:2393] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 18: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:40.500039 22523 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:40.501432 22158 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:40.502033 22158 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:40.503238 22583 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:40.504544 15854 raft_consensus.cc:1275] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 13:07:40.508623 16564 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 20.
I20260709 13:07:40.510125 22550 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:40.510967 22110 consensus_queue.cc:237] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:40.510054 22365 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:40.513628 22523 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:40.514760 22550 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:40.515232 22349 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:40.515507 22365 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:40.517540 22593 log.cc:927] Time spent T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.777s	user 0.005s	sys 0.005s
I20260709 13:07:40.523116 22493 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.523725 22493 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:40.525878 22493 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:40.527298 15937 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 2564) took 1527 ms. Trace:
I20260709 13:07:40.528357 22509 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:40.528941 22509 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:40.529546 22425 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:40.530071 22425 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:40.531083 22509 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:40.534536 15827 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:40.536281 22551 raft_consensus.cc:2804] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:40.536842 22551 raft_consensus.cc:697] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:40.537779 22551 consensus_queue.cc:237] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:40.539052 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:40.539525 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:40.541215 22497 raft_consensus.cc:3055] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:40.541656 22497 raft_consensus.cc:740] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:40.542519 15902 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:40.542474 22497 consensus_queue.cc:260] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:40.543393 22492 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:40.492066 22606 log.cc:927] Time spent T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 13:07:40.549520 15937 rpcz_store.cc:276] 0709 13:07:38.999603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:39.003632 (+  4029us) service_pool.cc:224] Handling call
0709 13:07:39.025346 (+ 21714us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:39.574492 (+549146us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:40.527036 (+952544us) spinlock_profiling.cc:243] Waited 516 ms on lock 0x7b540019c3d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:40.527205 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1185935616}
I20260709 13:07:40.563529 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:40.483418 22564 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
I20260709 13:07:40.560868 16003 raft_consensus.cc:1240] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 13:07:40.552747 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:40.568933 16561 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 17.
I20260709 13:07:40.544281 15827 leader_election.cc:400] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 election: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 14. Current term is 15.
W20260709 13:07:40.541249 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:40.572064 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:40.573192 16533 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:40.565835 15902 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:40.574939 15902 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:40.575901 16564 raft_consensus.cc:1275] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:07:40.577558 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:40.576567 22427 consensus_queue.cc:1048] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 13:07:40.587571 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:40.588279 16002 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 19.
W20260709 13:07:40.562831 22598 log.cc:927] Time spent T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.856s	user 0.013s	sys 0.025s
W20260709 13:07:40.590041 15857 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58214 (request call id 1241) took 2166 ms (client timeout 2620 ms). Trace:
I20260709 13:07:40.592190 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:40.544381 22493 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.543618 22497 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:40.593693 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:40.609498 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:40.611272 15902 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:40.531626 22509 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:07:40.596168 15857 rpcz_store.cc:269] 0709 13:07:38.409647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:38.491163 (+ 81516us) service_pool.cc:224] Handling call
0709 13:07:40.575977 (+2084814us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b540008dc54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:40.576104 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2581842304}
I20260709 13:07:40.543956 22501 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:40.576248 22418 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:40.617841 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:40.615701 16009 raft_consensus.cc:1275] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:07:40.639050 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:40.642818 16004 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 17.
I20260709 13:07:40.598927 22508 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.634334 15854 raft_consensus.cc:1240] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:40.590747 22564 raft_consensus.cc:2749] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:40.636185 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:40.581327 22511 raft_consensus.cc:2749] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:40.587854 22547 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.589802 22277 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:40.577054 22446 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:40.566941 22492 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:40.652684 15858 raft_consensus.cc:1275] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:07:40.660571 15854 raft_consensus.cc:1240] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:40.599196 22530 consensus_queue.cc:1048] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.056s
I20260709 13:07:40.659492 15857 raft_consensus.cc:1275] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:07:40.613935 22493 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:40.670575 16006 raft_consensus.cc:1275] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 15. (term mismatch)
I20260709 13:07:40.667644 22446 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:40.672850 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:40.671212 22511 consensus_queue.cc:1059] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:07:40.677847 16006 pending_rounds.cc:85] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting all ops after (but not including) 14
W20260709 13:07:40.679509 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 381161us
I20260709 13:07:40.633564 22509 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:40.697697 16006 pending_rounds.cc:107] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 13:07:40.715658 16006 raft_consensus.cc:2889] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:07:40.708102 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 430015us
I20260709 13:07:40.662427 22277 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:40.642056 22501 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:40.721021 16007 raft_consensus.cc:2410] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:07:40.668547 22425 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:40.679687 22610 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.678922 22564 raft_consensus.cc:2749] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:40.726687 16002 raft_consensus.cc:1240] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:40.731148 15857 raft_consensus.cc:1275] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:07:40.668154 22547 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:40.718752 22277 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:40.657837 22508 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:40.704722 22427 consensus_queue.cc:1059] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:07:40.715167 22584 consensus_queue.cc:1048] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
W20260709 13:07:40.768136 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:40.716467 22523 consensus_queue.cc:1048] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:07:40.719077 22527 consensus_queue.cc:1059] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:07:40.739620 22512 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:40.705495 22611 consensus_queue.cc:1059] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.084s
I20260709 13:07:40.776444 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:40.768714 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:40.785295 15862 raft_consensus.cc:2393] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 17: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:40.788041 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:40.783826 22158 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:40.790419 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:40.791064 15928 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:07:40.761528 22387 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:40.765784 22564 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:40.736078 22610 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:40.793993 22387 consensus_queue.cc:1048] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:07:40.794350 22508 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:40.796285 22611 raft_consensus.cc:3055] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:40.796720 22463 raft_consensus.cc:3055] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:40.797732 22493 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:40.801910 22277 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:40.804423 15932 raft_consensus.cc:1240] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 19. Current term is 20. Ops: []
W20260709 13:07:40.813608 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:40.792759 22512 raft_consensus.cc:3055] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:40.850744 22512 raft_consensus.cc:740] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:40.782017 22421 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:40.851828 22512 consensus_queue.cc:260] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 13.14, Last appended by leader: 14, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:40.853377 22512 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:40.853499 22421 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:40.854733 22446 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:40.854990 15859 raft_consensus.cc:1240] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:40.807700 22564 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:40.858134 22564 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:40.858839 22564 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.859190 22584 consensus_queue.cc:1059] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:40.858048 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:40.861222 22594 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:40.865783 22594 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:40.763473 22349 consensus_queue.cc:1048] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 13:07:40.880281 22493 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:40.881063 22493 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:40.839488 22463 raft_consensus.cc:740] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:40.889526 22463 consensus_queue.cc:260] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:40.891117 22463 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:40.892995 22530 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.893522 22530 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:40.843050 22610 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:40.844830 22501 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:40.845759 22305 consensus_meta.cc:287] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.985s	user 0.013s	sys 0.001s
I20260709 13:07:40.845992 22547 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:40.847146 22323 consensus_meta.cc:287] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.998s	user 0.010s	sys 0.003s
I20260709 13:07:40.847851 22492 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:40.819468 22611 raft_consensus.cc:740] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
W20260709 13:07:40.909650 16570 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.032s	sys 0.048s
W20260709 13:07:40.925998 16570 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.032s	sys 0.048s
W20260709 13:07:40.917944 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:40.850194 22346 consensus_meta.cc:287] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.523s	user 0.010s	sys 0.000s
I20260709 13:07:40.859381 22564 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:40.920920 22323 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:40.921671 22530 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:40.930574 22346 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:40.946010 22418 raft_consensus.cc:2749] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:07:40.957828 15976 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.015s	sys 0.019s
W20260709 13:07:40.963737 15976 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.015s	sys 0.019s
W20260709 13:07:40.964399 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 538885us
W20260709 13:07:40.958146 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 461256us
W20260709 13:07:40.972472 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:40.981703 22617 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.982224 22617 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:40.984575 22617 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:40.955225 22493 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:40.977483 22346 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:40.978168 16005 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.006s	sys 0.014s
W20260709 13:07:40.997157 16005 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.006s	sys 0.014s
I20260709 13:07:40.966327 22509 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:40.978987 22564 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:41.007290 16535 leader_election.cc:400] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 15. Current term is 16.
I20260709 13:07:40.988354 22617 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.009636 22463 consensus_queue.cc:1059] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:41.010545 22564 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.948900 22611 consensus_queue.cc:260] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:41.007539 22425 consensus_queue.cc:1048] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.011s
I20260709 13:07:41.025306 15862 raft_consensus.cc:1240] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 17. Current term is 18. Ops: []
W20260709 13:07:41.020989 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 398639us
I20260709 13:07:41.030916 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:41.032145 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:41.022584 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 409989us
I20260709 13:07:41.034462 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:41.035348 16009 raft_consensus.cc:2410] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:07:41.014705 22277 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:41.036000 15862 raft_consensus.cc:1275] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:07:41.036310 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:41.037031 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 312410us
I20260709 13:07:41.032974 16004 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 17.
I20260709 13:07:41.038029 15825 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:41.023519 22459 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.005s	sys 0.012s
I20260709 13:07:41.041047 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:41.041394 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:41.042209 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:41.059288 16006 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 18.
I20260709 13:07:41.060602 16569 raft_consensus.cc:2393] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 21: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:41.042456 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:41.003947 15976 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:41.035043 15902 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:41.048228 22611 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:41.062253 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:41.063176 16565 raft_consensus.cc:2410] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:07:41.047866 15753 catalog_manager.cc:5697] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 12 to 14, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 14 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:07:41.067579 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.067015 15902 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:41.065989 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 51688.5us
I20260709 13:07:41.069674 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:41.072980 16535 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:41.073812 15902 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:07:41.078744 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 74912.5us
W20260709 13:07:41.060359 22459 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.005s	sys 0.012s
I20260709 13:07:41.082356 15932 raft_consensus.cc:1275] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:07:41.080232 15976 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:41.083479 16535 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:41.086674 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:41.021492 22617 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:41.071379 22609 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:41.091140 15976 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:41.025382 22564 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:41.069368 22530 consensus_queue.cc:1048] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 13:07:41.073017 22511 consensus_queue.cc:1059] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.068s
I20260709 13:07:41.136039 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:41.140643 15932 raft_consensus.cc:1275] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:07:41.163247 15901 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.006s	sys 0.007s
I20260709 13:07:41.175292 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:41.139664 16568 raft_consensus.cc:1275] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:07:41.175992 15858 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:41.102952 15935 raft_consensus.cc:1275] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 13:07:41.175365 15901 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.007s	sys 0.007s
I20260709 13:07:41.140386 15929 raft_consensus.cc:1240] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 19. Current term is 20. Ops: [17.14-19.15]
I20260709 13:07:41.077071 22497 raft_consensus.cc:2804] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:41.069130 22627 raft_consensus.cc:2749] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:41.094115 16535 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:41.178881 15936 raft_consensus.cc:2393] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 18: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:41.084327 22523 raft_consensus.cc:2804] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:41.090480 22387 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.083447 22613 raft_consensus.cc:2804] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:41.185775 22493 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:41.092482 22418 consensus_queue.cc:1048] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.097s
I20260709 13:07:41.183494 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:41.179289 22610 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.192590 22509 raft_consensus.cc:3055] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:07:41.156764 22616 log.cc:927] Time spent T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:07:41.187482 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:41.090094 22277 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:07:41.184722 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 90714.6us
I20260709 13:07:41.199728 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:07:41.200284 16533 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.010s	sys 0.041s
W20260709 13:07:41.173563 22592 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.700s	user 0.009s	sys 0.004s
W20260709 13:07:41.205891 16008 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.015s	sys 0.044s
I20260709 13:07:41.215973 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:41.186873 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.223400 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:41.187471 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:41.238827 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:41.240795 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:41.242738 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:41.244947 15858 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 21.
I20260709 13:07:41.246380 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:41.246727 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:41.247438 15929 raft_consensus.cc:1275] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 12. (term mismatch)
I20260709 13:07:41.251442 15932 raft_consensus.cc:2393] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 15: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:41.254473 15853 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:07:41.253786 16533 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.010s	sys 0.042s
I20260709 13:07:41.207983 22523 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:41.093130 22603 consensus_queue.cc:1059] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.141s
I20260709 13:07:41.183317 22631 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:07:41.303498 16008 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.018s	sys 0.050s
I20260709 13:07:41.196591 22323 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:41.303378 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:41.307252 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:41.308254 16564 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 16.
I20260709 13:07:41.186625 22497 raft_consensus.cc:697] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 19 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:41.305460 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:41.303398 15929 pending_rounds.cc:85] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 11
I20260709 13:07:41.305927 15931 raft_consensus.cc:1240] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:41.309823 16565 raft_consensus.cc:2410] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:07:41.305488 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:07:41.302855 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 711403us
I20260709 13:07:41.310724 15929 pending_rounds.cc:107] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 18.12
I20260709 13:07:41.311378 15929 raft_consensus.cc:2889] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:41.311585 16569 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 17.
I20260709 13:07:41.308889 15858 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 17.
I20260709 13:07:41.312629 15861 raft_consensus.cc:2393] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 15: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:41.313285 15935 raft_consensus.cc:2393] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 16: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
W20260709 13:07:41.313717 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:41.314034 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:41.314338 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.315765 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:41.318758 16533 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:41.318535 22497 consensus_queue.cc:237] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:41.321224 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:41.321846 15855 raft_consensus.cc:2376] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 17. Re-sending same reply.
I20260709 13:07:41.327112 15853 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 20.
I20260709 13:07:41.320019 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:41.228803 22613 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:41.332091 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:41.332408 15934 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 17.
I20260709 13:07:41.340554 15858 raft_consensus.cc:2393] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 16: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:41.340879 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:41.303211 22592 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.830s	user 0.014s	sys 0.009s
I20260709 13:07:41.338369 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:41.346819 15929 raft_consensus.cc:1275] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:07:41.348510 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:41.349273 15935 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:41.250730 22509 raft_consensus.cc:740] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:41.352010 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:41.374264 15931 raft_consensus.cc:2410] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 18 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 7.
I20260709 13:07:41.374275 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:41.351713 15932 raft_consensus.cc:1275] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:07:41.310153 22305 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:41.305447 22523 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:07:41.359062 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.371037 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:41.385824 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:41.339516 22550 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.338567 22629 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:41.379128 15827 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:41.380132 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:41.251956 22493 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:41.383976 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:41.393297 15929 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:41.261417 22610 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:41.395574 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:41.376428 15934 raft_consensus.cc:2393] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 20: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
W20260709 13:07:41.387890 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.229615 22387 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:41.347586 22613 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:41.389873 15932 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:41.384750 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:41.394434 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:41.381995 22305 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:41.395164 15827 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:41.403733 15861 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 17.
W20260709 13:07:41.397224 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 141558us
I20260709 13:07:41.410617 15936 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:41.412385 22648 consensus_queue.cc:1048] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:41.413851 22648 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:41.414196 22648 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:41.414510 22648 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:41.415769 22614 raft_consensus.cc:3055] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:07:41.416195 22614 raft_consensus.cc:740] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:41.417014 22614 consensus_queue.cc:260] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:41.418243 22614 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:41.418460 22617 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:41.398216 22110 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:41.420181 22110 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:41.420563 22110 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:41.411437 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:41.423491 15858 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 14.
I20260709 13:07:41.426826 15929 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 18.
I20260709 13:07:41.410696 22305 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:41.439868 22560 consensus_queue.cc:1048] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 13:07:41.395620 22629 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:41.449746 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 131088us
I20260709 13:07:41.396682 22550 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:41.455372 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:41.436040 22387 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:41.456691 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:41.434090 22594 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:41.464993 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:41.523197 15932 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
W20260709 13:07:41.523572 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 268557us
I20260709 13:07:41.463330 22564 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:41.557488 22418 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.558239 22418 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:41.525252 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.507071 22631 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:41.441877 22509 consensus_queue.cc:260] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:41.569121 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 131085us
I20260709 13:07:41.506996 22492 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:41.462670 22594 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:41.563524 15902 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:41.564427 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:41.540827 22629 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:41.572772 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 49750.2us
I20260709 13:07:41.587782 16569 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 15.
I20260709 13:07:41.589129 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:41.592382 15825 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:41.594923 15827 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:41.596508 15827 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:07:41.611845 22635 log.cc:927] Time spent T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 13:07:41.612399 22632 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:41.612838 22632 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:41.613042 22346 raft_consensus.cc:2749] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:41.589641 22492 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:41.617659 22492 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:41.618938 22636 log.cc:927] Time spent T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
W20260709 13:07:41.619546 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 534706us
W20260709 13:07:41.620446 22637 log.cc:927] Time spent T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:07:41.621989 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:41.622423 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.604502 22560 consensus_queue.cc:1059] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.008s
I20260709 13:07:41.597177 22564 raft_consensus.cc:2804] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:41.590025 22594 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:41.629418 15935 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 19.
I20260709 13:07:41.626026 15902 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:41.638890 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:41.636469 15902 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:41.639734 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:41.640342 15928 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 20.
I20260709 13:07:41.605753 22509 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:41.608424 22277 raft_consensus.cc:2749] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:41.623003 22634 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:41.647003 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:41.647208 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:41.598855 22387 consensus_queue.cc:1048] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.043s
W20260709 13:07:41.668605 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 577511us
I20260709 13:07:41.568347 22648 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:41.633831 22632 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:41.633353 22564 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:41.672402 15857 raft_consensus.cc:1275] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:07:41.670275 15932 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 16.
I20260709 13:07:41.670639 15928 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:41.675102 16533 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:41.571139 22631 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:41.643404 22582 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:41.682139 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 89874.8us
I20260709 13:07:41.644766 22629 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:41.681066 16533 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:41.684118 16008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2646) took 1161 ms. Trace:
I20260709 13:07:41.684638 16008 rpcz_store.cc:276] 0709 13:07:40.523112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:40.567298 (+ 44186us) service_pool.cc:224] Handling call
0709 13:07:40.594489 (+ 27191us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:40.594683 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:07:40.654644 (+ 59961us) log.cc:844] Serialized 14 byte log entry
0709 13:07:41.683796 (+1029152us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:41.683845 (+    49us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:41.683896 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:41.683963 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:41.684069 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":375607,"prepare.queue_time_us":37053,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":3389,"thread_start_us":653130,"threads_started":1,"wal-append.queue_time_us":663785}
I20260709 13:07:41.674154 22560 raft_consensus.cc:3055] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:07:41.634640 22513 consensus_queue.cc:1048] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:07:41.609135 22110 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:41.669132 22583 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.673997 22425 raft_consensus.cc:2749] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:41.671996 22512 raft_consensus.cc:2804] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:41.670197 22655 raft_consensus.cc:2749] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:41.680482 22564 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:41.749768 15853 raft_consensus.cc:1240] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:41.688315 22617 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:41.705673 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:41.751690 16569 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:07:41.734619 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:41.689574 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 93150.6us
I20260709 13:07:41.679466 22550 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:41.688602 22560 raft_consensus.cc:740] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:41.679105 22634 raft_consensus.cc:697] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:41.683620 22277 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:41.681154 22501 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.742901 15928 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 18.
I20260709 13:07:41.684458 22629 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:41.683884 22610 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:41.745230 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:41.795174 16562 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:41.747560 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:41.747768 16568 raft_consensus.cc:1240] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 19. Current term is 20. Ops: [19.15-19.15]
W20260709 13:07:41.748227 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 61957.9us
I20260709 13:07:41.689447 22493 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:41.757045 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:41.688659 22603 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.758124 16567 raft_consensus.cc:1275] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:07:41.815994 22603 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:41.765054 22349 consensus_queue.cc:1048] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.052s
I20260709 13:07:41.689994 16565 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 17.
W20260709 13:07:41.756876 22646 log.cc:927] Time spent T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.250s	user 0.000s	sys 0.001s
I20260709 13:07:41.818728 22603 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:41.744603 22512 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:41.798503 16533 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:07:41.867465 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 69145.4us
I20260709 13:07:41.868939 16533 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:41.799392 16564 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:41.797151 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:41.871832 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:07:41.803076 22652 log.cc:927] Time spent T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:07:41.802709 22560 consensus_queue.cc:260] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:41.690657 22582 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:41.807818 22638 log.cc:927] Time spent T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.394s	user 0.000s	sys 0.023s
I20260709 13:07:41.744158 22583 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:41.689216 22523 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:41.814412 15853 raft_consensus.cc:1240] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 13:07:41.817188 16567 raft_consensus.cc:1240] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:07:41.768703 15902 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:41.897954 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 129325us
I20260709 13:07:41.798979 22387 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:41.827713 15861 raft_consensus.cc:1240] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 13. Current term is 14. Ops: [13.9-13.9]
W20260709 13:07:41.844931 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.847838 22527 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.848958 22346 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.849707 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:41.875617 22657 log.cc:927] Time spent T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.132s	user 0.001s	sys 0.010s
I20260709 13:07:41.851186 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:07:41.920104 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 58084.3us
I20260709 13:07:41.852589 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:07:41.853161 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:41.921934 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.853549 22422 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:41.854851 15758 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58434 (request call id 345) took 29499 ms (client timeout 15000 ms). Trace:
I20260709 13:07:41.769820 22528 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.876113 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:41.894886 22656 log.cc:927] Time spent T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.129s	user 0.003s	sys 0.001s
I20260709 13:07:41.928617 16003 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 16.
I20260709 13:07:41.929363 16564 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 18.
I20260709 13:07:41.929996 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:41.931285 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:41.796732 22629 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:41.869940 16533 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:41.855528 15753 catalog_manager.cc:5697] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 11 to 12, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 12 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:07:41.764124 16569 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 22.
I20260709 13:07:41.839774 22421 raft_consensus.cc:3055] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:41.825973 22648 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:41.905723 16562 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
W20260709 13:07:41.914302 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:41.907449 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:41.943020 16002 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 15.
I20260709 13:07:41.850499 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:41.916548 16008 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 20.
W20260709 13:07:41.895076 22658 log.cc:927] Time spent T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.153s	user 0.000s	sys 0.004s
I20260709 13:07:41.920372 16006 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:41.916452 22492 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:41.925619 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:07:41.900388 22621 log.cc:927] Time spent T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.473s	user 0.000s	sys 0.003s
I20260709 13:07:41.932230 16003 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 17.
I20260709 13:07:41.961513 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:41.962615 16003 raft_consensus.cc:2410] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:07:41.878118 22512 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:41.878378 22493 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:41.942608 22613 consensus_meta.cc:287] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.537s	user 0.005s	sys 0.004s
I20260709 13:07:41.920949 15976 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:41.800421 22501 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:41.936229 16004 raft_consensus.cc:2376] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 19. Re-sending same reply.
I20260709 13:07:41.899044 22349 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:41.942013 15974 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:41.851887 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:07:41.949714 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 266250us
I20260709 13:07:41.915876 22627 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.958814 16566 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:07:41.964221 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:07:41.937230 22497 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.007s	sys 0.005s
I20260709 13:07:41.965421 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:41.936653 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 66770.6us
I20260709 13:07:42.003990 16006 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 18.
I20260709 13:07:41.970487 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:42.000129 16003 raft_consensus.cc:2376] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 20. Re-sending same reply.
I20260709 13:07:41.934976 22560 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:41.938647 22530 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:07:41.971591 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 50715.9us
W20260709 13:07:42.001508 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 59580.7us
I20260709 13:07:41.937493 22421 raft_consensus.cc:740] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:42.004062 16008 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 17.
I20260709 13:07:41.969271 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:42.027169 15928 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:42.006188 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:41.933661 22346 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:42.007190 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:42.032411 16006 raft_consensus.cc:2410] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:07:41.939927 22446 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:41.921548 22387 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:42.016390 15825 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:42.002941 22425 raft_consensus.cc:2749] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:07:42.029160 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 402873us
I20260709 13:07:41.999267 22648 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:07:42.003823 22497 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.007s	sys 0.005s
I20260709 13:07:41.942299 22603 consensus_queue.cc:1048] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.070s
I20260709 13:07:41.926061 22527 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:42.034694 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:41.940923 22422 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:42.044718 15854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2730) took 1370 ms. Trace:
I20260709 13:07:42.045559 15902 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:42.028085 16003 raft_consensus.cc:2410] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 21 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:07:41.942040 22528 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:42.048041 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:42.005731 22663 raft_consensus.cc:2804] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:42.007026 22349 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:42.060132 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 43813.1us
I20260709 13:07:42.063349 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:42.022290 22659 consensus_queue.cc:1048] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 13:07:42.117903 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:42.044363 22446 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:41.999799 22493 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:42.120362 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 74495.3us
I20260709 13:07:42.020143 22627 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:41.969560 22634 consensus_queue.cc:237] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:42.006719 22613 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:42.026432 22632 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:42.022426 15753 catalog_manager.cc:5697] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 12 to 13, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 13 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:42.205369 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:42.205736 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:42.207363 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:42.206115 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:42.230078 15974 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:42.071507 15928 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 20.
I20260709 13:07:42.206486 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:42.236626 16562 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
W20260709 13:07:42.232221 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 186722us
I20260709 13:07:42.137249 16009 raft_consensus.cc:2376] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 17. Re-sending same reply.
I20260709 13:07:42.234056 16565 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 19.
W20260709 13:07:42.235028 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 184670us
W20260709 13:07:42.235154 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:42.240365 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:42.230091 15825 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:42.121541 15854 rpcz_store.cc:276] 0709 13:07:40.674689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:40.674813 (+   124us) service_pool.cc:224] Handling call
0709 13:07:40.675127 (+   314us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:41.072659 (+397532us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:41.072689 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:41.393733 (+321044us) spinlock_profiling.cc:243] Waited 20 ms on lock 0x7b2c00000f48. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e8ca1a11 00007f73e8cb1372 00007f73ea3e3a01 00007f73ea3a843a 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2
0709 13:07:41.397936 (+  4203us) log.cc:844] Serialized 52 byte log entry
0709 13:07:41.467129 (+ 69193us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:41.467183 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:41.467278 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:42.029329 (+562051us) raft_consensus.cc:1685] finished
0709 13:07:42.029350 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:42.044646 (+ 15296us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31884,"raft.queue_time_us":59321,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":9663,"spinlock_wait_cycles":45915520,"thread_start_us":413734,"threads_started":2,"wal-append.queue_time_us":196584}
I20260709 13:07:42.122996 16008 raft_consensus.cc:2393] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 18: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
W20260709 13:07:42.118585 22638 log.cc:927] Time spent T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.054s	user 0.000s	sys 0.002s
I20260709 13:07:42.033667 22584 consensus_queue.cc:1048] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.158s
I20260709 13:07:42.256454 15825 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:42.060369 22648 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:42.257294 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:42.268465 16562 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:07:42.238021 22594 consensus_meta.cc:287] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.595s	user 0.005s	sys 0.005s
W20260709 13:07:42.183096 22670 log.cc:927] Time spent T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 13:07:42.232842 22663 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:07:42.120069 22609 consensus_meta.cc:287] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.978s	user 0.013s	sys 0.000s
I20260709 13:07:42.263581 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:42.264714 16570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1439) took 1902 ms. Trace:
I20260709 13:07:42.265677 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:42.351253 16566 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
W20260709 13:07:42.135010 22305 consensus_meta.cc:287] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.687s	user 0.009s	sys 0.001s
I20260709 13:07:42.361311 15860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2734) took 1507 ms. Trace:
I20260709 13:07:42.056614 22459 consensus_queue.cc:1059] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.254s
I20260709 13:07:42.247835 22425 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:42.258019 15902 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:42.363291 15857 raft_consensus.cc:1217] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Deduplicated request from leader. Original: 16.10->[17.11-18.12]   Dedup: 17.11->[18.12-18.12]
I20260709 13:07:42.363855 15857 raft_consensus.cc:1240] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 18. Current term is 19. Ops: [17.11-18.12]
I20260709 13:07:42.125649 22661 consensus_queue.cc:1048] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.217s
I20260709 13:07:42.268538 22610 raft_consensus.cc:2804] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:42.234220 22446 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:42.204178 22523 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:42.363876 22614 consensus_meta.cc:287] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.945s	user 0.011s	sys 0.002s
I20260709 13:07:42.206106 22501 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:42.202530 22346 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:42.183053 22527 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:42.145146 22583 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:42.352344 22365 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:42.355350 22663 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:42.059664 22513 consensus_queue.cc:1048] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.215s
I20260709 13:07:42.181720 22631 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:42.237412 22662 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.353291 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:42.408010 15853 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 19.
I20260709 13:07:42.408077 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:42.409204 16565 raft_consensus.cc:2393] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 20: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:42.249033 22497 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:07:42.354898 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 98493.5us
I20260709 13:07:42.356159 15934 raft_consensus.cc:1240] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:42.358580 16562 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
W20260709 13:07:42.359086 15827 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.006s	sys 0.009s
W20260709 13:07:42.418183 15827 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.006s	sys 0.009s
W20260709 13:07:42.418555 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 662589us
I20260709 13:07:42.264145 22509 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:07:42.369371 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 111408us
I20260709 13:07:42.117172 22569 consensus_queue.cc:1048] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.226s
W20260709 13:07:42.377635 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:42.361725 15860 rpcz_store.cc:276] 0709 13:07:40.854088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:40.855935 (+  1847us) service_pool.cc:224] Handling call
0709 13:07:40.856303 (+   368us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:40.856705 (+   402us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:40.856738 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:40.857301 (+   563us) log.cc:844] Serialized 52 byte log entry
0709 13:07:41.162975 (+305674us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:41.201580 (+ 38605us) log.cc:844] Serialized 14 byte log entry
0709 13:07:41.236523 (+ 34943us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:41.236623 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:41.413391 (+176768us) spinlock_profiling.cc:243] Waited 34.6 ms on lock 0x7b1800486460. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb3f3 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd 000000000038f354 000000000038f199 00007f73e729e83a
0709 13:07:41.807221 (+393830us) spinlock_profiling.cc:243] Waited 11.5 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:07:42.361037 (+553816us) raft_consensus.cc:1685] finished
0709 13:07:42.361086 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:42.361249 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69957,"prepare.queue_time_us":34076,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":3155,"raft.queue_time_us":47704,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":12071,"spinlock_wait_cycles":105956864,"thread_start_us":305216,"threads_started":1,"wal-append.queue_time_us":460070}
I20260709 13:07:42.313920 22648 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:42.311281 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:42.386917 16010 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.051s	sys 0.064s
W20260709 13:07:42.425117 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:42.425316 16010 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.051s	sys 0.064s
I20260709 13:07:42.386586 22610 raft_consensus.cc:697] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:42.122299 22665 raft_consensus.cc:2749] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:42.064339 22110 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:42.364415 22609 raft_consensus.cc:2749] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:42.388310 22508 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:42.346710 22594 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:42.420511 16566 raft_consensus.cc:1275] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 13:07:42.425539 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 171029us
I20260709 13:07:42.308970 22530 raft_consensus.cc:697] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:42.425146 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:42.388805 22619 raft_consensus.cc:3055] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:42.426827 15932 raft_consensus.cc:1275] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:07:42.478242 15974 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:07:42.429951 15930 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.013s	sys 0.016s
W20260709 13:07:42.479105 15930 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.013s	sys 0.016s
W20260709 13:07:41.927155 15758 rpcz_store.cc:269] 0709 13:07:12.355575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.499160 (+143585us) service_pool.cc:224] Handling call
0709 13:07:15.961705 (+3462545us) spinlock_profiling.cc:243] Waited 8.18 ms on lock 0x7b5400030890. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7284640 00007f73f015bb76 00007f73f0265c3e 00007f73ee7262a4 00007f73ee726232 00007f73ee726161 00007f73ee7260dc 00007f73ee725392 00007f73e86e348c 00007f73e86e28c6
0709 13:07:41.854603 (+25892898us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:28.700782 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:28.885649 (+184867us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:28.885702 (+    53us) write_op.cc:435] Acquired schema lock
0709 13:07:28.885720 (+    18us) tablet.cc:662] Decoding operations
0709 13:07:28.910149 (+ 24429us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:28.910250 (+   101us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:28.910282 (+    32us) tablet.cc:685] Acquiring locks for 22 operations
0709 13:07:29.127424 (+217142us) tablet.cc:701] Row locks acquired
0709 13:07:29.127503 (+    79us) write_op.cc:260] PREPARE: finished
0709 13:07:29.127757 (+   254us) write_op.cc:270] Start()
0709 13:07:29.127943 (+   186us) write_op.cc:276] Timestamp: P: 1783602449127661 usec, L: 0
0709 13:07:29.127997 (+    54us) op_driver.cc:348] REPLICATION: starting
0709 13:07:29.129042 (+  1045us) log.cc:844] Serialized 8659 byte log entry
0709 13:07:29.849096 (+720054us) spinlock_profiling.cc:243] Waited 5.1 ms on lock 0x7b1800119740. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb522 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd 000000000038f354 000000000038f199 00007f73e729e83a
0709 13:07:29.912329 (+ 63233us) op_driver.cc:464] REPLICATION: finished
0709 13:07:30.147150 (+234821us) write_op.cc:301] APPLY: starting
0709 13:07:30.170851 (+ 23701us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:30.882151 (+711300us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:30.882194 (+    43us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:31.101018 (+218824us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:31.101565 (+   547us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=44,key_file_lookups=44,delta_file_lookups=44,mrs_lookups=0
0709 13:07:31.101646 (+    81us) write_op.cc:312] APPLY: finished
0709 13:07:31.102240 (+   594us) log.cc:844] Serialized 196 byte log entry
0709 13:07:32.330677 (+1228437us) spinlock_profiling.cc:243] Waited 3.65 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e729e0ac 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b
0709 13:07:32.478865 (+148188us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:32.493058 (+ 14193us) write_op.cc:454] Released schema lock
0709 13:07:32.504228 (+ 11170us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:32.504833 (+   605us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:32.504887 (+    54us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":165687,"reported_tablets":47,"spinlock_wait_cycles":18806656,"child_traces":[["op",{"apply.queue_time_us":206538,"apply.run_cpu_time_us":61727,"apply.run_wall_time_us":2371467,"cfile_cache_hit":90,"cfile_cache_hit_bytes":96307,"num_ops":22,"prepare.queue_time_us":314736,"prepare.run_cpu_time_us":37465,"prepare.run_wall_time_us":742040,"raft.queue_time_us":380203,"raft.run_cpu_time_us":31080,"raft.run_wall_time_us":164599,"replication_time_us":784120,"spinlock_wait_cycles":20128896,"thread_start_us":2003313,"threads_started":5,"wal-append.queue_time_us":1555906,"wal-append.run_cpu_time_us":39377,"wal-append.run_wall_time_us":2643036}]]}
I20260709 13:07:42.432585 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:42.372704 22305 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:42.373826 22425 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:42.416831 22497 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:42.419622 22631 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.476070 15860 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 19.
I20260709 13:07:42.420826 15902 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:42.479715 15854 raft_consensus.cc:1240] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:07:42.411089 22365 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:42.483017 15853 raft_consensus.cc:2393] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 15: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:07:42.481608 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:42.499286 15859 raft_consensus.cc:1240] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:42.391398 22676 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:07:42.469187 22673 log.cc:927] Time spent T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.322s	user 0.001s	sys 0.000s
I20260709 13:07:42.430471 22681 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:42.504254 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:42.500984 22346 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.508486 22349 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:42.508599 22632 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:42.484184 22619 raft_consensus.cc:740] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:42.432973 15753 catalog_manager.cc:5697] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 9 to 13, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 13 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:07:42.510320 22671 log.cc:927] Time spent T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
W20260709 13:07:42.511559 22678 log.cc:927] Time spent T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.158s	user 0.000s	sys 0.000s
I20260709 13:07:42.421061 22662 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:42.517484 16561 raft_consensus.cc:1240] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:42.501654 22387 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:42.500155 22611 consensus_meta.cc:287] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 1.438s	user 0.009s	sys 0.004s
I20260709 13:07:42.523842 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:42.544818 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:07:42.503616 22670 log.cc:927] Time spent T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:07:42.529976 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 110787us
I20260709 13:07:42.500813 22497 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:42.498884 22508 raft_consensus.cc:697] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 19 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:42.504328 22527 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:42.532320 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:42.564361 22527 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:42.564749 22527 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:42.565148 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:42.552660 15853 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:42.531621 22346 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:42.555824 22530 consensus_queue.cc:237] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:42.617686 22629 consensus_meta.cc:287] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.670s	user 0.011s	sys 0.000s
I20260709 13:07:42.557955 16533 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:42.618405 22346 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:42.618732 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 60841.1us
I20260709 13:07:42.554991 15827 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:42.514187 22631 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:42.611891 22690 consensus_queue.cc:1059] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:07:42.584652 22493 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:42.535379 22564 consensus_meta.cc:287] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.767s	user 0.011s	sys 0.000s
I20260709 13:07:42.499140 22665 consensus_queue.cc:1048] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 13:07:42.535917 22610 consensus_queue.cc:237] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:42.525880 22681 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:42.568302 15753 catalog_manager.cc:5697] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:42.612756 15860 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
W20260709 13:07:42.570956 22687 log.cc:927] Time spent T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:07:42.614753 22365 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:42.612603 22659 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.617345 22387 consensus_queue.cc:1048] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:42.576185 22528 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:42.622138 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 67283.6us
I20260709 13:07:42.618824 22686 raft_consensus.cc:2749] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:07:42.613432 22560 consensus_meta.cc:287] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.591s	user 0.006s	sys 0.005s
I20260709 13:07:42.626497 22679 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:07:42.636376 15936 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 2585) took 1429 ms (client timeout 1536 ms). Trace:
I20260709 13:07:42.623492 22611 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:42.637867 15827 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:42.640632 15936 rpcz_store.cc:269] 0709 13:07:41.200960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:41.202987 (+  2027us) service_pool.cc:224] Handling call
0709 13:07:42.630152 (+1427165us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b5400099cd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:42.630259 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":247357184}
I20260709 13:07:42.622520 22613 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:42.645828 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:42.646894 15827 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:42.642006 22679 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:42.625319 22527 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:42.647925 22508 consensus_queue.cc:237] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:42.637056 22690 raft_consensus.cc:3055] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:42.559801 22680 consensus_queue.cc:1048] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.077s
I20260709 13:07:42.626955 22681 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:42.633599 22693 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.673672 15853 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 18.
I20260709 13:07:42.675311 16562 raft_consensus.cc:1275] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:07:42.675209 16565 raft_consensus.cc:1240] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 21. Current term is 22. Ops: []
W20260709 13:07:42.654006 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:42.624087 22629 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:42.652256 15860 raft_consensus.cc:1240] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:07:42.649353 22527 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:42.692816 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:42.689317 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:42.690086 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:42.694533 16565 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 19.
I20260709 13:07:42.691375 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:42.695711 16564 raft_consensus.cc:2393] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 15: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:42.627439 22564 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:42.696847 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:42.700517 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:42.701102 15936 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:42.638387 22627 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:42.652129 22628 raft_consensus.cc:2749] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:42.555480 22676 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:42.651746 22690 raft_consensus.cc:740] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:42.690398 22528 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.687251 22609 raft_consensus.cc:2749] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:42.709092 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:42.709932 15853 raft_consensus.cc:1240] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:42.689201 22305 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:42.692088 22611 consensus_queue.cc:1048] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 13:07:42.719306 15857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 2588) took 1517 ms. Trace:
I20260709 13:07:42.719702 15857 rpcz_store.cc:276] 0709 13:07:41.201695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:41.677793 (+476098us) service_pool.cc:224] Handling call
0709 13:07:42.363130 (+685337us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:42.364516 (+  1386us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:42.719139 (+354623us) spinlock_profiling.cc:243] Waited 674 ms on lock 0x7b54002490d0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:42.719254 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1550919424}
I20260709 13:07:42.691967 22619 consensus_queue.cc:260] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:42.713990 22692 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:07:42.720314 22691 log.cc:927] Time spent T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
I20260709 13:07:42.727421 22692 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:42.727887 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:42.645892 22659 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:42.728526 15853 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:42.729821 22692 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:42.720980 22619 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:42.722154 15974 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:42.736824 15853 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 20.
I20260709 13:07:42.734961 15974 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:42.740801 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:42.766745 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:42.715034 22365 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:42.768395 22662 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:42.761353 22676 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:42.774187 22631 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:42.773957 22365 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:42.760931 22681 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:42.776221 22365 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:42.777668 22681 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:42.740814 22661 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:42.744285 15936 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 18.
I20260709 13:07:42.784188 15929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 1336) took 1137 ms. Trace:
I20260709 13:07:42.717136 15753 catalog_manager.cc:5697] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:42.779313 22594 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:42.779948 22690 consensus_queue.cc:260] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:42.765908 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:42.787809 16533 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:07:42.765233 22512 consensus_meta.cc:287] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.762s	user 0.012s	sys 0.006s
W20260709 13:07:42.764467 22492 consensus_meta.cc:287] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.798s	user 0.010s	sys 0.000s
I20260709 13:07:42.784765 15929 rpcz_store.cc:276] 0709 13:07:41.646575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:41.647635 (+  1060us) service_pool.cc:224] Handling call
0709 13:07:41.648071 (+   436us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:41.669350 (+ 21279us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:07:41.741784 (+ 72434us) log.cc:844] Serialized 14 byte log entry
0709 13:07:42.055628 (+313844us) log.cc:844] Serialized 14 byte log entry
0709 13:07:42.345017 (+289389us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:42.354126 (+  9109us) log.cc:844] Serialized 52 byte log entry
0709 13:07:42.354452 (+   326us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:42.360817 (+  6365us) log.cc:844] Serialized 14 byte log entry
0709 13:07:42.361122 (+   305us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:42.361460 (+   338us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:42.781083 (+419623us) spinlock_profiling.cc:243] Waited 16.4 ms on lock 0x7b24001146f8. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71419c4 00007f73e713facb 00007f73e72fadc9 00007f73e72faa55 00007f73e72f9fb0 00007f73e9f70488 00007f73e9f70be8 00007f73e9f48398 00007f73e9f4376a 00007f73e9f42bb6
0709 13:07:42.783914 (+  2831us) raft_consensus.cc:1685] finished
0709 13:07:42.783938 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:42.784113 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20767,"prepare.queue_time_us":151834,"prepare.run_cpu_time_us":133,"prepare.run_wall_time_us":139,"raft.queue_time_us":360051,"raft.run_cpu_time_us":447,"raft.run_wall_time_us":67511,"spinlock_wait_cycles":108164224,"thread_start_us":579646,"threads_started":4,"wal-append.queue_time_us":273059}
I20260709 13:07:42.781991 22661 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:42.689831 22693 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:42.769734 22305 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.794219 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:42.770532 22527 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:42.798529 15929 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:42.769481 22629 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:42.794922 22661 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:42.762913 22425 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:42.732642 22528 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:42.817550 15859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 2584) took 1634 ms. Trace:
I20260709 13:07:42.799180 22512 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:42.765362 22564 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:42.815640 15827 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:07:42.828188 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 332575us
I20260709 13:07:42.828395 15862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2742) took 1204 ms. Trace:
I20260709 13:07:42.877197 15862 rpcz_store.cc:276] 0709 13:07:41.624942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:41.849665 (+224723us) service_pool.cc:224] Handling call
0709 13:07:41.849973 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:42.121274 (+271301us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:42.121305 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:42.121346 (+    41us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:42.121377 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:42.828143 (+706766us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b5400247cd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:42.828189 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:42.828329 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":619091200}
I20260709 13:07:42.878600 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:42.829038 16565 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 18.
I20260709 13:07:42.841974 15859 rpcz_store.cc:276] 0709 13:07:41.183543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:41.183892 (+   349us) service_pool.cc:224] Handling call
0709 13:07:41.229298 (+ 45406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:42.504883 (+1275585us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:42.817275 (+312392us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b54002d0ad4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:42.817488 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2920215424}
W20260709 13:07:42.802194 22645 log.cc:927] Time spent T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 1.379s	user 0.000s	sys 0.003s
I20260709 13:07:42.879643 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:42.880343 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:42.879866 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:42.880955 22551 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:42.880976 15937 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 2577) took 1795 ms. Trace:
W20260709 13:07:42.799831 22617 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.969s	user 0.001s	sys 0.011s
I20260709 13:07:42.881532 22551 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:42.881881 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 66323.4us
W20260709 13:07:42.820052 22639 log.cc:927] Time spent T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 1.400s	user 0.003s	sys 0.000s
I20260709 13:07:42.881590 15937 rpcz_store.cc:276] 0709 13:07:41.085718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:41.085877 (+   159us) service_pool.cc:224] Handling call
0709 13:07:41.086183 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:41.086522 (+   339us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:07:41.086553 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:41.139686 (+ 53133us) log.cc:844] Serialized 26 byte log entry
0709 13:07:41.332776 (+193090us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:07:41.332976 (+   200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:41.333484 (+   508us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:42.801531 (+1468047us) spinlock_profiling.cc:243] Waited 8.86 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:07:42.880630 (+ 79099us) raft_consensus.cc:1685] finished
0709 13:07:42.880650 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:42.880922 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3937,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":63697,"spinlock_wait_cycles":20373760,"thread_start_us":147344,"threads_started":1,"wal-append.queue_time_us":214612}
W20260709 13:07:42.821079 22642 log.cc:927] Time spent T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 1.404s	user 0.000s	sys 0.003s
I20260709 13:07:42.886279 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:42.762657 22497 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:42.819993 22690 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:42.926640 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:42.802279 22527 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:42.800122 22650 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:42.797430 22492 raft_consensus.cc:2749] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:42.931135 15929 raft_consensus.cc:2410] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 18 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 7.
W20260709 13:07:42.884042 22617 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.084s	user 0.002s	sys 0.014s
I20260709 13:07:42.818298 22418 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:42.933712 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:42.934594 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:42.813443 22693 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:42.814879 22631 consensus_queue.cc:1059] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.110s
I20260709 13:07:42.814940 22615 consensus_queue.cc:1048] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.114s
I20260709 13:07:42.888062 22551 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:42.815024 22305 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:42.942137 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:42.969453 15929 raft_consensus.cc:1240] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 13:07:42.967000 15931 raft_consensus.cc:2393] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 18: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:42.967351 16005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 2611) took 2623 ms. Trace:
I20260709 13:07:42.931694 22527 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:42.940562 22568 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:42.970595 16005 rpcz_store.cc:276] 0709 13:07:40.316769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:40.373937 (+ 57168us) service_pool.cc:224] Handling call
0709 13:07:40.374182 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:40.374398 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:07:40.374964 (+   566us) log.cc:844] Serialized 14 byte log entry
0709 13:07:41.066596 (+691632us) log.cc:844] Serialized 14 byte log entry
0709 13:07:41.069778 (+  3182us) log.cc:844] Serialized 14 byte log entry
0709 13:07:41.195973 (+126195us) spinlock_profiling.cc:243] Waited 4.68 ms on lock 0x7b18004b43c0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb522 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd 000000000038f354 000000000038f199 00007f73e729e83a
0709 13:07:41.521538 (+325565us) spinlock_profiling.cc:243] Waited 44.9 ms on lock 0x7b18004b43c0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72892a6 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb3f3 00007f73e72bd7c1
0709 13:07:41.522023 (+   485us) spinlock_profiling.cc:243] Waited 6.37 ms on lock 0x7b18004b43c0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb3f3 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd 000000000038f354 000000000038f199 00007f73e729e83a
0709 13:07:41.612834 (+ 90811us) spinlock_profiling.cc:243] Waited 2.71 ms on lock 0x7b18004b43c0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e7235c0a 00007f73f0975219 00007f73ee27c58f 00007f73e72bb571 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd
0709 13:07:42.616618 (+1003784us) spinlock_profiling.cc:243] Waited 2.37 ms on lock 0x7b480021c258. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ef2d1aa1 00007f73ef2c8b37 00007f73ef2c89ca 00007f73ef3c026f 00007f73ef3bff37 00007f73ef3d455c 00007f73ef3d4508 00007f73ef3d4488 00007f73ef3d4443 00007f73ef3d36ec 00007f73ee2fda96
0709 13:07:42.940041 (+323423us) spinlock_profiling.cc:243] Waited 8.64 ms on lock 0x7b18004b43c0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e9f504bc 00007f73e729e48b 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72b8455 00007f73ea3a0060 00007f73ea3b2716 00007f73ea3c3287
0709 13:07:42.940140 (+    99us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:42.940195 (+    55us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:42.940269 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:42.940357 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:42.940475 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1146796,"prepare.queue_time_us":131162,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":90,"raft.queue_time_us":673869,"raft.run_cpu_time_us":3339,"raft.run_wall_time_us":1018387,"spinlock_wait_cycles":160244608,"thread_start_us":674357,"threads_started":2,"wal-append.queue_time_us":708912,"wal-append.run_cpu_time_us":1632,"wal-append.run_wall_time_us":1031434}
I20260709 13:07:42.976774 22568 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:42.977443 22305 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:42.977617 22512 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:42.349515 16570 rpcz_store.cc:276] 0709 13:07:40.346336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:40.346507 (+   171us) service_pool.cc:224] Handling call
0709 13:07:40.358091 (+ 11584us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:40.926453 (+568362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:40.926503 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:40.936904 (+ 10401us) log.cc:844] Serialized 26 byte log entry
0709 13:07:41.073613 (+136709us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:41.235109 (+161496us) log.cc:844] Serialized 14 byte log entry
0709 13:07:42.231109 (+996000us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b18004c1c20. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e9f504bc 00007f73e729e48b 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72b8455 00007f73ea3a0060 00007f73ea3b2716 00007f73ea3c3287
0709 13:07:42.231242 (+   133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:42.231309 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:42.231354 (+    45us) raft_consensus.cc:1685] finished
0709 13:07:42.248214 (+ 16860us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:42.248420 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":312810,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":93955,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":19204,"spinlock_wait_cycles":28842496,"thread_start_us":952176,"threads_started":4,"wal-append.queue_time_us":308681,"wal-append.run_cpu_time_us":2222,"wal-append.run_wall_time_us":1087872}
I20260709 13:07:42.985626 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:42.987073 22528 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:42.989498 22422 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:42.935297 22617 raft_consensus.cc:697] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:42.992425 22617 consensus_queue.cc:237] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:42.994748 15856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2736) took 2102 ms. Trace:
I20260709 13:07:42.995157 15856 rpcz_store.cc:276] 0709 13:07:40.892522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:40.893086 (+   564us) service_pool.cc:224] Handling call
0709 13:07:40.893420 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:40.893771 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:40.893796 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:40.894332 (+   536us) log.cc:844] Serialized 26 byte log entry
0709 13:07:41.176028 (+281696us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:41.176066 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:41.176126 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:42.819294 (+1643168us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:07:42.994421 (+175127us) raft_consensus.cc:1685] finished
0709 13:07:42.994443 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:42.994650 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":94,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":80,"spinlock_wait_cycles":28006784,"thread_start_us":281266,"threads_started":1,"wal-append.queue_time_us":525491}
I20260709 13:07:42.936831 22650 raft_consensus.cc:697] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:42.997788 22650 consensus_queue.cc:237] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:43.012663 22582 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:43.013527 22497 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:43.014894 22663 consensus_meta.cc:287] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.581s	user 0.009s	sys 0.000s
W20260709 13:07:43.015618 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:43.015388 22663 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:43.016404 22446 consensus_meta.cc:287] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.627s	user 0.010s	sys 0.000s
I20260709 13:07:43.016469 16002 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 15.
I20260709 13:07:43.018205 15932 raft_consensus.cc:1240] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:43.017040 22446 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:43.018378 22663 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:43.019030 22663 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:07:43.022414 22636 log.cc:927] Time spent T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 1.083s	user 0.001s	sys 0.000s
I20260709 13:07:42.815706 15753 catalog_manager.cc:5697] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 10 to 15, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 15 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:07:43.023255 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:43.023885 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:43.027055 16561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 1568) took 1463 ms. Trace:
I20260709 13:07:43.027341 22603 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:43.027473 16561 rpcz_store.cc:276] 0709 13:07:41.563787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:41.564524 (+   737us) service_pool.cc:224] Handling call
0709 13:07:41.564808 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:42.533615 (+968807us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:43.026831 (+493216us) spinlock_profiling.cc:243] Waited 952 ms on lock 0x7b54000cdad4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:43.026997 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2190518528}
I20260709 13:07:43.027931 22603 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:43.024019 16005 raft_consensus.cc:2393] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 15: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:43.030162 22603 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:07:43.030709 22688 log.cc:927] Time spent T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.535s	user 0.001s	sys 0.003s
I20260709 13:07:43.033293 15974 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:43.035298 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:43.036890 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:42.803112 22648 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:42.991077 22551 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:43.043115 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:43.045146 22582 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:43.045571 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:43.046373 22446 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:43.036536 22528 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:43.048069 22528 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:43.048743 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41662 (request call id 2629) took 1189 ms. Trace:
I20260709 13:07:43.049160 16002 rpcz_store.cc:276] 0709 13:07:41.859780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:42.033900 (+174120us) service_pool.cc:224] Handling call
0709 13:07:43.048569 (+1014669us) spinlock_profiling.cc:243] Waited 783 ms on lock 0x7b54000aa5d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:43.048686 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1801365888}
I20260709 13:07:43.049690 22702 raft_consensus.cc:3055] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:43.048066 22463 raft_consensus.cc:3055] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:43.050279 22528 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:43.050161 22702 raft_consensus.cc:740] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:43.051720 22679 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:43.051800 22702 consensus_queue.cc:260] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:43.055910 15933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2875) took 1684 ms. Trace:
I20260709 13:07:43.056092 15928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53636 (request call id 1339) took 1184 ms. Trace:
I20260709 13:07:43.064870 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:43.066079 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:43.066708 16562 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 19.
I20260709 13:07:43.068871 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:43.072000 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:43.072675 15854 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:07:43.056361 15933 rpcz_store.cc:276] 0709 13:07:41.371358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:41.371619 (+   261us) service_pool.cc:224] Handling call
0709 13:07:42.630398 (+1258779us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:42.630565 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:42.630590 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:42.630630 (+    40us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:42.630661 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:42.630714 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:43.055725 (+425011us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b5400099cd0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:43.055852 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2871958784}
I20260709 13:07:43.073875 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:43.056541 15928 rpcz_store.cc:276] 0709 13:07:41.871434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:41.871571 (+   137us) service_pool.cc:224] Handling call
0709 13:07:43.055918 (+1184347us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b540044cfd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:43.056035 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":352368896}
I20260709 13:07:43.077330 16533 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:43.068471 15976 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:43.081633 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:43.019490 22663 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:07:43.088303 15853 raft_consensus.cc:1240] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:07:43.091409 15753 catalog_manager.cc:5697] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 3 to 13, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 13 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:07:43.092542 22551 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:43.093250 15753 catalog_manager.cc:5697] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:43.041301 22422 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.096010 22422 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:43.098206 22422 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:43.099521 22661 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:43.100049 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:43.040841 22530 consensus_queue.cc:1059] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 13:07:43.101680 22679 raft_consensus.cc:2804] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:43.101852 22661 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:43.102147 22679 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:43.102294 22530 raft_consensus.cc:3055] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:43.102593 22679 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:43.102885 22530 raft_consensus.cc:740] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:43.104152 22530 consensus_queue.cc:260] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:43.105712 22530 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:43.091449 22527 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:43.110451 22527 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:43.110801 22421 consensus_queue.cc:260] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:43.114075 22663 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:43.114969 16008 raft_consensus.cc:2393] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 20: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:43.114773 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:07:43.121140 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:43.126179 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:43.127127 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 706346us
I20260709 13:07:43.119815 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:43.128979 15854 raft_consensus.cc:2410] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 19 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 13:07:43.130479 16561 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:43.127338 15930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 1337) took 1480 ms. Trace:
I20260709 13:07:43.050652 22463 raft_consensus.cc:740] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:43.132622 15902 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:43.133544 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 51743.2us
W20260709 13:07:43.133774 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 65374.5us
I20260709 13:07:43.134725 15827 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:43.150350 15976 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:43.131531 15930 rpcz_store.cc:276] 0709 13:07:41.646874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:41.646988 (+   114us) service_pool.cc:224] Handling call
0709 13:07:41.672093 (+ 25105us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:41.672327 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:07:41.918015 (+245688us) log.cc:844] Serialized 14 byte log entry
0709 13:07:43.127027 (+1209012us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:43.127077 (+    50us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:43.127124 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:43.127186 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:43.127286 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":647420,"prepare.queue_time_us":146357,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"thread_start_us":806608,"threads_started":2,"wal-append.queue_time_us":598174}
I20260709 13:07:43.151234 15855 raft_consensus.cc:1240] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 19 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:43.153640 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:43.155422 15861 raft_consensus.cc:1240] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 17. Current term is 18. Ops: []
W20260709 13:07:43.156008 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:43.156510 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:43.156891 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:43.153219 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:43.159143 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:43.159761 16006 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:43.114606 22582 raft_consensus.cc:697] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:43.151531 22421 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:43.166069 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:07:43.168828 16009 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60156 (request call id 1306) took 1321 ms (client timeout 1516 ms). Trace:
I20260709 13:07:43.152261 22667 raft_consensus.cc:2749] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:43.119238 22528 consensus_queue.cc:1059] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.044s
W20260709 13:07:43.169338 16009 rpcz_store.cc:269] 0709 13:07:41.847127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:41.851631 (+  4504us) service_pool.cc:224] Handling call
0709 13:07:43.168652 (+1317021us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b5400259ed4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:43.168758 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":313706368}
I20260709 13:07:43.170156 15858 raft_consensus.cc:2393] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 20: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:43.173131 15976 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:43.151602 22613 raft_consensus.cc:2764] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:07:43.155063 22662 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:43.169705 15930 raft_consensus.cc:1240] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:43.160071 22564 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.174871 15902 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:43.177632 15976 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:43.180758 15900 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:43.181895 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:43.183250 22425 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:43.183840 22425 raft_consensus.cc:697] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:43.184728 22425 consensus_queue.cc:237] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:43.188400 22603 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:07:43.189038 22603 raft_consensus.cc:697] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 22 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:43.189426 22492 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:43.189992 22492 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:43.189927 22603 consensus_queue.cc:237] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:43.192207 22677 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.192775 22663 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.192917 22582 consensus_queue.cc:237] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:43.192991 22677 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:43.197753 22662 raft_consensus.cc:697] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:43.197742 15860 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:07:43.198997 15860 raft_consensus.cc:606] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Received request to transfer leadership to TS 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:43.199268 22662 consensus_queue.cc:237] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:43.192212 22528 raft_consensus.cc:3055] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:43.200580 22528 raft_consensus.cc:740] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:43.201561 22528 consensus_queue.cc:260] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:43.151971 22690 consensus_queue.cc:1059] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.022s
I20260709 13:07:43.202864 22528 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:43.203418 22617 raft_consensus.cc:3055] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:43.201561 22675 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:43.203869 22617 raft_consensus.cc:740] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:43.204308 22675 raft_consensus.cc:697] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 20 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:43.204804 22617 consensus_queue.cc:260] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:43.205230 22675 consensus_queue.cc:237] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:43.206125 22617 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:43.164311 22648 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:43.163156 22681 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:07:43.208976 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 52830.9us
I20260709 13:07:43.210048 15902 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:43.210513 15900 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:43.212566 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:43.224653 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 47080.2us
I20260709 13:07:43.230259 15900 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:43.233186 15900 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:43.233672 16006 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 18.
I20260709 13:07:43.235821 15827 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:43.227246 16009 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 18.
I20260709 13:07:43.228688 15976 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:07:43.274261 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:43.095276 15753 catalog_manager.cc:5697] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 6 to 13, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 13 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:43.276643 15827 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:07:43.309075 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:43.283279 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:43.237293 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:43.322775 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 46237.8us
W20260709 13:07:43.329101 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 100500us
I20260709 13:07:43.349146 15976 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:43.240983 15900 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:43.276362 22676 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:43.356313 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:43.358071 15858 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:07:43.358704 15858 raft_consensus.cc:606] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Received request to transfer leadership to TS 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:43.237442 22418 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:43.282172 22512 raft_consensus.cc:2804] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:43.227861 22629 raft_consensus.cc:2749] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:43.238740 22346 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:43.279827 22648 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.276218 22613 raft_consensus.cc:2749] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:43.237233 22501 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:43.227473 22663 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:43.351763 22422 raft_consensus.cc:2804] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:43.289009 22492 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:43.361742 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:43.308236 22615 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:43.326776 22493 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.342311 22508 raft_consensus.cc:2804] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:43.276610 22679 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:43.225523 22564 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:43.238099 22681 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:43.374305 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:43.380951 16006 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:43.356346 22619 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:43.358174 22614 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.223445 22463 consensus_queue.cc:260] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:43.362035 22677 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:43.387604 16009 raft_consensus.cc:1275] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:07:43.368191 22422 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:43.364426 22512 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:43.380244 22501 raft_consensus.cc:697] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 20 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:43.368635 22346 raft_consensus.cc:697] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 17 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:43.367452 22609 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:07:43.401235 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:43.380414 22629 raft_consensus.cc:2764] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:07:43.380167 22508 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:43.401489 16006 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 18.
W20260709 13:07:43.414219 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 62930.3us
I20260709 13:07:43.422194 22609 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:43.424692 22609 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:43.392007 22422 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:43.411816 22463 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:43.383299 22619 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:43.432082 22619 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:43.412362 22512 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:43.422401 15976 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:43.440212 22421 raft_consensus.cc:2804] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:43.440764 22421 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:43.441210 22421 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:07:43.444187 22697 log.cc:927] Time spent T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.717s	user 0.000s	sys 0.007s
I20260709 13:07:43.427038 22346 consensus_queue.cc:237] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:43.446918 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:43.447788 16011 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 19.
I20260709 13:07:43.448874 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:43.449520 16011 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:43.452709 16535 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:43.453708 22651 raft_consensus.cc:2764] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:07:43.461477 16569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45232 (request call id 1462) took 1256 ms. Trace:
I20260709 13:07:43.462060 16569 rpcz_store.cc:276] 0709 13:07:42.204985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:42.205103 (+   118us) service_pool.cc:224] Handling call
0709 13:07:43.461214 (+1256111us) spinlock_profiling.cc:243] Waited 703 ms on lock 0x7b54005654d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:43.461396 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1617259776}
W20260709 13:07:43.465699 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:43.465891 15861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2747) took 1195 ms. Trace:
I20260709 13:07:43.467048 15861 rpcz_store.cc:276] 0709 13:07:42.270249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:42.270393 (+   144us) service_pool.cc:224] Handling call
0709 13:07:42.354353 (+ 83960us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:43.156002 (+801649us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:43.465624 (+309622us) spinlock_profiling.cc:243] Waited 801 ms on lock 0x7b540032da54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:43.465827 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1841752320}
I20260709 13:07:43.472314 22501 consensus_queue.cc:237] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:43.473104 15855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2746) took 1207 ms. Trace:
I20260709 13:07:43.473547 15855 rpcz_store.cc:276] 0709 13:07:42.265270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:42.265421 (+   151us) service_pool.cc:224] Handling call
0709 13:07:42.344953 (+ 79532us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:43.151710 (+806757us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:43.472851 (+321141us) spinlock_profiling.cc:243] Waited 794 ms on lock 0x7b540008e654. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:43.473041 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1827275520}
I20260709 13:07:43.476068 15934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2881) took 1955 ms. Trace:
I20260709 13:07:43.476567 15934 rpcz_store.cc:276] 0709 13:07:41.520199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:41.520454 (+   255us) service_pool.cc:224] Handling call
0709 13:07:41.520767 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:42.417419 (+896652us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:43.475783 (+1058364us) spinlock_profiling.cc:243] Waited 757 ms on lock 0x7b5400197b54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:43.475980 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1741019904}
I20260709 13:07:43.480130 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:43.367993 22648 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:43.386543 22615 raft_consensus.cc:697] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:43.481439 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:43.488605 16009 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:43.480180 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 58064.4us
I20260709 13:07:43.490056 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:43.488134 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:43.490947 16569 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:43.498857 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:43.499739 16567 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:43.494334 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:43.379908 22493 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:43.506392 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:43.507071 22679 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:43.507582 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:43.508324 16565 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 17.
I20260709 13:07:43.508757 22615 consensus_queue.cc:237] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:43.513223 22663 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:43.455518 16535 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:43.507107 16564 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:43.519244 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:43.519160 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:43.520274 16562 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:43.530174 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:43.532544 16569 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 18.
I20260709 13:07:43.534041 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:43.535693 16569 raft_consensus.cc:1240] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:43.484695 22508 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:43.543592 16006 raft_consensus.cc:1275] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 13:07:43.539085 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 83663.7us
I20260709 13:07:43.547142 16535 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:43.389596 22614 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:43.482501 22648 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:43.553287 16564 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 17.
I20260709 13:07:43.564371 15934 raft_consensus.cc:1275] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:07:43.565069 16561 raft_consensus.cc:1240] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 13:07:43.565548 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:43.554342 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:43.555496 16562 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 16.
I20260709 13:07:43.560734 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:43.574075 16563 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:43.561486 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:43.492481 22653 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:43.562252 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:43.626024 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:43.569061 22651 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:43.580699 22648 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:43.572335 22551 consensus_queue.cc:1059] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.009s
I20260709 13:07:43.628011 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:07:43.584197 22710 log.cc:927] Time spent T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.209s	user 0.003s	sys 0.000s
I20260709 13:07:43.583799 22422 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:43.562283 22679 consensus_queue.cc:1048] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.023s
I20260709 13:07:43.585728 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:43.585862 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:43.646498 16003 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:43.587931 22659 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:43.587808 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:43.588902 16005 raft_consensus.cc:2410] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:07:43.591275 22681 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:43.593551 22568 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:43.595283 22614 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:43.598696 22713 log.cc:927] Time spent T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:07:43.603895 16567 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 18.
W20260709 13:07:43.608433 22708 log.cc:927] Time spent T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.370s	user 0.001s	sys 0.000s
I20260709 13:07:43.612912 15827 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:43.613987 15853 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:43.614444 22512 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:43.725987 22512 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:43.543385 22609 consensus_queue.cc:1059] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.075s
I20260709 13:07:43.617265 15976 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:43.618590 22702 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:43.484535 22610 consensus_queue.cc:1059] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.070s
I20260709 13:07:43.619613 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:43.628791 16562 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:07:43.539582 22690 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.640465 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:43.552426 22617 raft_consensus.cc:2749] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:07:43.656147 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 415250us
I20260709 13:07:43.750797 15900 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:43.578549 16009 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 16.
I20260709 13:07:43.751999 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:43.752547 16009 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:07:43.691717 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:43.753836 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:43.754187 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:43.712289 16570 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 18.
I20260709 13:07:43.756155 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:43.757746 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:43.765974 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:43.766412 15900 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:43.596694 15753 catalog_manager.cc:5697] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 7 to 13, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 13 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:07:43.768667 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 155846us
I20260709 13:07:43.770569 16002 raft_consensus.cc:1275] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:07:43.772297 16002 raft_consensus.cc:1275] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 13:07:43.775295 16533 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 1.529s (SENT) [suppressed 19 similar messages]
I20260709 13:07:43.775547 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:43.644099 22653 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:43.717075 16003 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 20.
I20260709 13:07:43.726994 16568 raft_consensus.cc:1240] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:07:43.648396 22648 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:43.736946 15853 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 18.
I20260709 13:07:43.738435 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:43.739615 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 122414us
I20260709 13:07:43.706314 22421 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:43.787930 15976 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:43.741979 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:43.792776 16535 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:43.759438 22663 raft_consensus.cc:2764] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election decision vote started in defunct term 15: won
I20260709 13:07:43.727885 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:43.784255 15900 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:43.794130 16535 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:43.563657 22550 consensus_queue.cc:1059] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.155s
I20260709 13:07:43.796563 16011 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 16.
I20260709 13:07:43.800818 16006 raft_consensus.cc:1240] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:43.801023 16565 raft_consensus.cc:2393] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 20: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:43.796826 16535 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:43.797348 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 454069us
I20260709 13:07:43.781055 22614 raft_consensus.cc:2749] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:43.781468 22497 raft_consensus.cc:2749] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:43.785470 22322 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:43.767750 22603 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.804261 16564 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:43.803967 16009 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 21.
I20260709 13:07:43.759637 22690 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:43.797627 15976 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:43.806421 16010 raft_consensus.cc:2393] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 20: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:43.815918 15900 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:43.816459 16535 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:43.751434 22619 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:43.819080 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:43.801659 22617 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:43.817268 16009 raft_consensus.cc:1240] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:43.818550 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:43.819005 15937 raft_consensus.cc:1240] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 17. Current term is 18. Ops: []
W20260709 13:07:43.797704 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:43.821343 15934 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:07:43.823515 15902 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:43.802903 22650 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:43.834734 15902 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:43.802616 22662 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:43.845633 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:43.810398 22594 raft_consensus.cc:3055] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:43.847872 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:43.862557 16004 raft_consensus.cc:2393] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 18: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:43.848841 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:43.810293 22651 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:43.851310 16564 raft_consensus.cc:1275] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:07:43.863619 15935 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:43.862794 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:43.857282 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:43.865857 15930 raft_consensus.cc:2393] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 18: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:43.820690 22523 raft_consensus.cc:2749] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:07:43.879747 15935 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 17.
I20260709 13:07:43.880362 22523 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.880867 22523 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:43.832907 22683 consensus_queue.cc:1048] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 13:07:43.882477 22508 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:43.884943 22523 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:43.867910 15902 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:43.870159 22422 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:43.871079 22718 log.cc:927] Time spent T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 13:07:43.887715 22717 log.cc:927] Time spent T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:07:43.871217 22421 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:43.850186 22662 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:43.888829 22681 raft_consensus.cc:2804] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:43.889498 22681 raft_consensus.cc:697] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 17 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:43.890589 22662 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:43.890439 22681 consensus_queue.cc:237] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:43.893603 22619 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:43.828248 22603 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:43.897816 22648 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:43.832543 22677 consensus_queue.cc:1059] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.066s
I20260709 13:07:43.899575 22690 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:43.900218 22677 raft_consensus.cc:3055] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:43.900745 22677 raft_consensus.cc:740] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:43.902098 15928 raft_consensus.cc:1275] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:07:43.901918 22677 consensus_queue.cc:260] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:43.861930 22493 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:43.904556 22512 raft_consensus.cc:2804] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:07:43.873265 22715 log.cc:927] Time spent T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 13:07:43.872344 15753 catalog_manager.cc:5697] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 10 to 14, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 14 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:43.829097 22629 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:07:43.874006 22716 log.cc:927] Time spent T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:07:43.875888 15937 raft_consensus.cc:1275] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 15. (term mismatch)
I20260709 13:07:43.939575 15937 pending_rounds.cc:85] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 14
I20260709 13:07:43.939881 15937 pending_rounds.cc:107] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 20.15
I20260709 13:07:43.940142 15937 raft_consensus.cc:2889] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:43.940704 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:07:43.876773 22676 consensus_meta.cc:287] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.519s	user 0.008s	sys 0.000s
I20260709 13:07:43.877923 22653 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:43.814554 22721 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:43.870169 15932 raft_consensus.cc:1240] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:43.886543 15929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2895) took 1097 ms. Trace:
I20260709 13:07:43.888316 15902 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:43.872581 16009 raft_consensus.cc:1240] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 17. Current term is 18. Ops: []
W20260709 13:07:43.915998 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:43.921631 16008 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:07:43.961642 15929 rpcz_store.cc:276] 0709 13:07:42.789043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:42.980567 (+191524us) service_pool.cc:224] Handling call
0709 13:07:42.980925 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:43.166380 (+185455us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:43.166410 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:43.180510 (+ 14100us) log.cc:844] Serialized 26 byte log entry
0709 13:07:43.609836 (+429326us) spinlock_profiling.cc:243] Waited 1.97 ms on lock 0x7b4800002dd8. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ef2d1aa1 00007f73ef2c8b37 00007f73ef2c89ca 00007f73ef3c026f 00007f73ef3bff37 00007f73ef3d455c 00007f73ef3d4508 00007f73ef3d4488 00007f73ef3d4443 00007f73ef3d36ec 00007f73ee2fda96
0709 13:07:43.726357 (+116521us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:43.726439 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:43.726584 (+   145us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:43.886161 (+159577us) raft_consensus.cc:1685] finished
0709 13:07:43.886201 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:43.886465 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":185077,"raft.queue_time_us":186762,"raft.run_cpu_time_us":2886,"raft.run_wall_time_us":439395,"spinlock_wait_cycles":4534656,"thread_start_us":293794,"threads_started":1,"wal-append.queue_time_us":602455}
I20260709 13:07:43.862766 22594 raft_consensus.cc:740] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:43.941357 15933 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:43.855633 22497 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:43.936352 16563 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 20.
I20260709 13:07:43.905545 22512 raft_consensus.cc:697] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 18 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:43.897972 22603 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:07:43.985991 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 97772.2us
I20260709 13:07:43.999881 15902 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:44.019050 15932 raft_consensus.cc:1275] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:07:44.019413 15937 raft_consensus.cc:1240] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:07:44.020205 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:44.020208 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:44.021804 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:44.031615 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:43.961333 22676 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:44.050891 15937 raft_consensus.cc:1240] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:07:43.965497 22528 raft_consensus.cc:2804] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:44.059336 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:44.062994 15928 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:44.063055 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:43.964030 22550 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:43.991058 22497 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:44.076756 16002 raft_consensus.cc:1240] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 19. Current term is 20. Ops: []
W20260709 13:07:44.064751 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 295163us
I20260709 13:07:44.072144 15931 raft_consensus.cc:2393] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 15: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:07:44.023554 22639 log.cc:927] Time spent T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:07:44.096174 15827 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:43.985615 22629 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:44.159664 15928 raft_consensus.cc:1240] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:07:44.034034 22721 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:44.194494 15933 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
I20260709 13:07:44.024036 22501 raft_consensus.cc:2804] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:07:44.156073 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:44.169231 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 393920us
I20260709 13:07:44.196084 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:44.204576 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:44.206228 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:44.208545 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:44.209920 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:44.153754 22497 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:44.210650 15857 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:44.212936 22731 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.213443 22731 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:44.215278 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:44.216051 22508 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:44.216589 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:44.216758 22508 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:44.217384 22508 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:44.217803 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:44.218447 15859 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 22.
I20260709 13:07:44.154321 22722 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.225678 15857 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
I20260709 13:07:44.228914 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:44.215378 22731 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:44.229794 15931 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:44.248136 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:44.248392 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:44.250128 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:44.251186 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:44.252015 16010 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:07:44.248898 15929 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 13.
I20260709 13:07:44.256846 15902 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:44.257269 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:44.257982 15929 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:44.258191 15902 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:44.260087 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:44.260805 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:44.157697 22550 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:44.220041 22721 raft_consensus.cc:697] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 18 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:44.233556 22501 raft_consensus.cc:697] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 16 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:44.262989 15902 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:44.238600 22110 raft_consensus.cc:2749] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:07:44.271620 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 175514us
I20260709 13:07:44.271725 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:44.194108 22629 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:44.278270 15827 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:44.277490 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:44.281188 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:44.157021 22594 consensus_queue.cc:260] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:44.285271 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:44.285903 15933 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:07:44.266920 22493 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:44.262312 22508 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:44.288527 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:44.289824 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:44.219587 22722 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:44.277071 22346 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.295126 15935 raft_consensus.cc:2410] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:07:44.312160 15929 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:07:44.293932 15827 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:44.322641 15928 raft_consensus.cc:1240] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:44.329025 22739 consensus_queue.cc:1048] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:44.328886 22346 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:44.332273 22722 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:44.332829 22346 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:44.333182 22501 consensus_queue.cc:237] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:44.333899 22739 raft_consensus.cc:2804] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:44.334406 22739 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:44.334796 22739 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:07:44.336349 22730 log.cc:927] Time spent T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 13:07:44.233716 22512 consensus_queue.cc:237] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:44.340006 22721 consensus_queue.cc:237] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:44.340425 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 46708.1us
I20260709 13:07:44.343223 22735 consensus_queue.cc:1059] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:07:44.343684 22594 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:44.346321 22735 consensus_queue.cc:1059] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:07:44.348120 15827 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:44.350591 22725 log.cc:927] Time spent T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 13:07:44.351284 22729 raft_consensus.cc:2749] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:07:44.352176 22723 log.cc:927] Time spent T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 13:07:44.290253 22110 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:44.361297 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:44.365430 22629 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:44.291113 22493 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:44.293882 22683 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:44.381259 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:44.384622 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:44.385671 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 122756us
I20260709 13:07:44.385983 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:44.386818 15902 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:44.387023 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:44.388002 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:44.388525 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:44.389320 15854 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 21.
I20260709 13:07:44.392628 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:44.393930 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:44.388785 15856 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 14.
I20260709 13:07:44.396903 22729 consensus_queue.cc:1048] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:44.398777 22739 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:44.400382 22702 consensus_meta.cc:287] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.652s	user 0.010s	sys 0.000s
I20260709 13:07:44.383875 22629 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.401397 22629 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:44.402877 15827 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:44.419615 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:44.420754 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:44.422745 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:44.423652 15857 raft_consensus.cc:2393] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 15: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:44.382926 22493 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:07:44.415199 22723 log.cc:927] Time spent T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:07:44.380488 22676 raft_consensus.cc:2749] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:44.421476 15862 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 18.
I20260709 13:07:44.416431 22729 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:44.415062 22675 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:44.464112 22564 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:44.469226 15929 pending_rounds.cc:85] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 8
I20260709 13:07:44.469587 15929 pending_rounds.cc:107] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:07:44.469836 15929 raft_consensus.cc:2889] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:07:44.469915 22742 log.cc:927] Time spent T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:07:44.472286 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:44.473088 15935 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:07:44.474377 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:44.475618 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:44.477311 15825 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:44.476460 15935 raft_consensus.cc:2393] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 15: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:44.481349 15936 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2886) took 2054 ms. Trace:
I20260709 13:07:44.481886 15935 raft_consensus.cc:1240] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 18 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:44.483830 15935 raft_consensus.cc:1240] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:44.481822 15936 rpcz_store.cc:276] 0709 13:07:42.427116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:42.886342 (+459226us) service_pool.cc:224] Handling call
0709 13:07:42.932913 (+ 46571us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:43.166614 (+233701us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:07:43.358604 (+191990us) log.cc:844] Serialized 14 byte log entry
0709 13:07:43.944491 (+585887us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:43.945947 (+  1456us) log.cc:844] Serialized 26 byte log entry
0709 13:07:43.947288 (+  1341us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:07:43.947341 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:43.948178 (+   837us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:44.481092 (+532914us) raft_consensus.cc:1685] finished
0709 13:07:44.481119 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:44.481286 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":242188,"prepare.queue_time_us":463649,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":262242,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":2426,"thread_start_us":614189,"threads_started":2,"wal-append.queue_time_us":606617}
I20260709 13:07:44.447520 22675 raft_consensus.cc:697] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 16 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:44.497097 15928 raft_consensus.cc:1275] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 13:07:44.487965 22743 log.cc:927] Time spent T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:07:44.440099 22748 consensus_queue.cc:1048] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.056s
I20260709 13:07:44.480643 22110 consensus_queue.cc:1048] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:07:44.450399 22497 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:44.480552 22564 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:44.538887 16004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41662 (request call id 2637) took 1101 ms. Trace:
I20260709 13:07:44.502279 22659 raft_consensus.cc:3055] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:44.543552 22659 raft_consensus.cc:740] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:44.543054 15855 raft_consensus.cc:2410] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:07:44.538950 22564 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:44.547577 16004 rpcz_store.cc:276] 0709 13:07:43.437811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:43.487826 (+ 50015us) service_pool.cc:224] Handling call
0709 13:07:44.534528 (+1046702us) spinlock_profiling.cc:243] Waited 360 ms on lock 0x7b5400304654. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:44.538782 (+  4254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":828565888}
I20260709 13:07:44.535145 22550 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:44.545287 22727 log.cc:927] Time spent T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
I20260709 13:07:44.550616 15930 raft_consensus.cc:2393] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 18: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:44.545220 22745 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.546411 22594 raft_consensus.cc:2804] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:44.530177 22508 consensus_queue.cc:1048] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:07:44.575359 16566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1466) took 1514 ms. Trace:
W20260709 13:07:44.574559 22747 log.cc:927] Time spent T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
W20260709 13:07:44.584764 22754 log.cc:927] Time spent T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:07:44.582024 22564 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:44.585853 22739 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:44.587952 16005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41662 (request call id 2636) took 1150 ms. Trace:
I20260709 13:07:44.588392 16005 rpcz_store.cc:276] 0709 13:07:43.437537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:43.493989 (+ 56452us) service_pool.cc:224] Handling call
0709 13:07:44.587624 (+1093635us) spinlock_profiling.cc:243] Waited 85.6 ms on lock 0x7b5400305a54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:44.587885 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":196974336}
I20260709 13:07:44.590034 22629 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:44.552170 22675 consensus_queue.cc:237] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:44.574973 22745 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:44.592849 22744 log.cc:927] Time spent T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.158s	user 0.000s	sys 0.002s
I20260709 13:07:44.599974 22528 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:44.600417 22528 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:44.575788 16566 rpcz_store.cc:276] 0709 13:07:43.061136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:43.066150 (+  5014us) service_pool.cc:224] Handling call
0709 13:07:43.113314 (+ 47164us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:43.113562 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:43.113608 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:43.114073 (+   465us) log.cc:844] Serialized 26 byte log entry
0709 13:07:43.352925 (+238852us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:43.352980 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:43.353074 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:43.598360 (+245286us) spinlock_profiling.cc:243] Waited 20.7 ms on lock 0x7b24002f66d8. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71419c4 00007f73e713facb 00007f73e72fadc9 00007f73e72faa55 00007f73e72fa09d 00007f73e9f2c10e 00007f73e9f4388c 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221
0709 13:07:44.573774 (+975414us) spinlock_profiling.cc:243] Waited 4.18 ms on lock 0x7b24002f6648. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71419c4 00007f73e713facb 00007f73e72fadc9 00007f73e72faa55 00007f73e72f9fb0 00007f73e9f70488 00007f73e9f70be8 00007f73e9f48398 00007f73e9f4376a 00007f73e9f42bb6
0709 13:07:44.575099 (+  1325us) raft_consensus.cc:1685] finished
0709 13:07:44.575117 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:44.575303 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":57240064,"thread_start_us":238349,"threads_started":1,"wal-append.queue_time_us":299851}
I20260709 13:07:44.603056 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 2635) took 1166 ms. Trace:
I20260709 13:07:44.603442 16002 rpcz_store.cc:276] 0709 13:07:43.437185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:43.782998 (+345813us) service_pool.cc:224] Handling call
0709 13:07:44.076523 (+293525us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:44.153221 (+ 76698us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:44.602883 (+449662us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b5400304b50. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:44.603003 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":674353920}
I20260709 13:07:44.604674 22497 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:44.557896 22659 consensus_queue.cc:260] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:44.606520 22659 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:44.611617 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:44.614048 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:44.615912 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:44.617976 15928 raft_consensus.cc:1275] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:07:44.615041 15931 raft_consensus.cc:2376] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17. Re-sending same reply.
W20260709 13:07:44.619133 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:44.620841 15974 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:44.628038 15974 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:44.630749 15974 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:44.633212 15974 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:44.558126 22594 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:44.634903 22663 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:44.636409 22513 raft_consensus.cc:2749] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:44.636171 22661 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:44.638751 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:44.713953 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:44.769104 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:44.635324 22493 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:44.769791 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 136691us
W20260709 13:07:44.787626 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 310411us
I20260709 13:07:44.639468 22528 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:44.792294 15974 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:07:44.769274 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 334156us
I20260709 13:07:44.811196 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:44.812156 15934 raft_consensus.cc:2410] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:07:44.645386 22594 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:44.786952 22738 raft_consensus.cc:2749] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:44.731293 22569 consensus_queue.cc:1048] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.170s
I20260709 13:07:44.823752 22547 raft_consensus.cc:3055] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:44.825726 22528 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:44.832791 22734 log.cc:927] Time spent T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.287s	user 0.003s	sys 0.003s
I20260709 13:07:44.928320 22425 raft_consensus.cc:2749] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:07:44.835041 15853 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 2766) took 1678 ms (client timeout 1570 ms). Trace:
I20260709 13:07:44.815030 22745 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:44.929517 15853 rpcz_store.cc:269] 0709 13:07:43.152833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:43.153000 (+   167us) service_pool.cc:224] Handling call
0709 13:07:44.831415 (+1678415us) spinlock_profiling.cc:243] Waited 455 ms on lock 0x7b54002481d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:44.831579 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1047287040}
W20260709 13:07:44.836467 22753 log.cc:927] Time spent T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.195s	user 0.000s	sys 0.000s
I20260709 13:07:44.790433 22749 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:44.922528 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 130330us
W20260709 13:07:44.923780 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:44.921936 22547 raft_consensus.cc:740] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:44.815495 22729 raft_consensus.cc:697] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:44.927418 22493 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:44.947783 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 206608us
I20260709 13:07:44.948235 15974 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:44.958551 15974 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:44.967244 22592 raft_consensus.cc:2749] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:44.959355 22509 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:44.968667 15974 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:44.969189 22509 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:44.969622 22509 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:44.970304 15974 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:44.971825 22527 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:44.968191 22583 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:44.972312 22569 raft_consensus.cc:2749] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:44.972353 15974 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:44.972898 22583 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:44.973444 22569 raft_consensus.cc:2804] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:44.974009 22569 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:44.974455 22569 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:44.975083 22583 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:44.978072 22547 consensus_queue.cc:260] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:44.975077 22677 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:07:44.984105 16570 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45246 (request call id 1447) took 1460 ms (client timeout 1663 ms). Trace:
W20260709 13:07:44.984510 16570 rpcz_store.cc:269] 0709 13:07:43.523838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:43.524736 (+   898us) service_pool.cc:224] Handling call
0709 13:07:44.983913 (+1459177us) spinlock_profiling.cc:243] Waited 86.7 ms on lock 0x7b54006a22d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:44.984047 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":199473920}
I20260709 13:07:44.988299 15974 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:44.933550 22749 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:44.992697 22088 raft_consensus.cc:2749] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:44.990222 22547 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:45.000222 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:45.011471 16010 raft_consensus.cc:1275] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 13:07:45.011898 16010 pending_rounds.cc:85] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting all ops after (but not including) 10
I20260709 13:07:45.011735 16002 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 18.
I20260709 13:07:45.012272 16010 pending_rounds.cc:107] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 13:07:45.012620 16010 raft_consensus.cc:2889] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:45.001426 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:45.014536 16006 raft_consensus.cc:2368] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 14. Current term is 15.
I20260709 13:07:45.002379 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:45.016266 16005 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 18.
I20260709 13:07:44.996726 22527 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.004140 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:45.007972 16004 raft_consensus.cc:1240] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:44.991869 15974 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:45.010571 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:44.994427 22493 consensus_queue.cc:1059] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.014s
I20260709 13:07:45.021668 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:45.022492 16004 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 17.
I20260709 13:07:45.021301 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:45.025386 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:45.026458 16004 raft_consensus.cc:2393] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 18: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:45.027727 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:45.029973 16005 raft_consensus.cc:1240] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:45.031013 16008 raft_consensus.cc:1275] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:07:45.032213 15902 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:45.034451 15902 leader_election.cc:400] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 14. Current term is 15.
I20260709 13:07:45.034814 15902 leader_election.cc:403] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:45.034627 15974 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:45.035920 22613 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:45.035982 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:45.036419 22613 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:45.036809 22613 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:45.038484 22551 raft_consensus.cc:3055] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:45.038923 22551 raft_consensus.cc:740] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:45.040038 22501 consensus_queue.cc:1048] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:45.026613 22592 consensus_queue.cc:1059] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.014s
I20260709 13:07:45.041924 15902 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:45.023617 22088 raft_consensus.cc:2804] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:45.002092 22729 consensus_queue.cc:237] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:45.032644 22493 consensus_queue.cc:1243] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer 85e997ba5003439781ec4f1f8db0c3ee log is divergent from this leader: its last log entry 18.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
W20260709 13:07:45.046265 22681 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.801s	user 0.013s	sys 0.036s
I20260709 13:07:45.049285 22749 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:45.050451 22564 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:45.052286 22738 raft_consensus.cc:2749] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:45.051674 22501 consensus_queue.cc:1048] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:45.058107 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:45.170794 16008 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 18.
I20260709 13:07:45.044979 22726 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:07:45.024209 22527 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:45.066071 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:45.056432 22681 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.811s	user 0.013s	sys 0.036s
I20260709 13:07:45.071312 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:45.204099 15933 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 18.
I20260709 13:07:45.172518 15902 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:45.057200 22603 raft_consensus.cc:3055] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:45.015664 22569 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:45.206049 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:45.211127 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:45.211329 15974 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:45.205950 22603 raft_consensus.cc:740] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:45.208122 15937 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 20.
I20260709 13:07:45.044835 22676 consensus_queue.cc:1059] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.124s
I20260709 13:07:45.057404 22088 raft_consensus.cc:697] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:07:45.066928 22615 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.001s	sys 0.025s
I20260709 13:07:45.060355 22493 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:45.067696 22564 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:45.219249 16008 raft_consensus.cc:2410] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:07:45.219826 15932 raft_consensus.cc:1240] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 17 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 13:07:45.213651 15974 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:45.041810 22551 consensus_queue.cc:260] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:45.223074 22551 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:45.233788 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:45.233703 22603 consensus_queue.cc:260] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:45.241565 22527 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:45.242583 22738 raft_consensus.cc:2804] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:45.243150 22738 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:45.243624 22738 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:07:45.249006 22659 consensus_meta.cc:287] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.642s	user 0.011s	sys 0.001s
I20260709 13:07:45.205998 22418 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:45.250770 22418 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:07:45.251750 22493 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:45.199954 22513 consensus_queue.cc:1048] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.086s
I20260709 13:07:45.254707 22493 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:45.256884 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:45.266011 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:45.269248 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:45.269862 15931 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 20.
I20260709 13:07:45.273346 22418 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
W20260709 13:07:45.216565 22615 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.817s	user 0.001s	sys 0.025s
I20260709 13:07:45.219094 22675 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:45.281667 22675 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:45.282217 22675 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:45.284113 22676 raft_consensus.cc:3055] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:07:45.284694 22676 raft_consensus.cc:740] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:45.285220 22613 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:45.285686 22676 consensus_queue.cc:260] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:45.256675 22564 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:45.215997 22509 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:45.294335 22509 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:45.256217 22088 consensus_queue.cc:237] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:45.304072 22613 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:45.305974 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:45.307632 22726 raft_consensus.cc:2749] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 14. Current term is 15.
W20260709 13:07:45.311177 15930 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 2593) took 1291 ms (client timeout 1663 ms). Trace:
W20260709 13:07:45.316926 15930 rpcz_store.cc:269] 0709 13:07:44.019288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:44.019919 (+   631us) service_pool.cc:224] Handling call
0709 13:07:45.310916 (+1290997us) spinlock_profiling.cc:243] Waited 528 ms on lock 0x7b540019c3d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:45.311099 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1214404992}
I20260709 13:07:45.313218 22629 raft_consensus.cc:3055] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:45.272620 15974 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:07:45.318969 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 46474.7us
I20260709 13:07:45.323024 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:45.323017 15974 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:45.324014 16002 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:45.314731 22594 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:45.315835 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 1.4993e+06us
I20260709 13:07:45.235440 22603 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:45.331468 16005 raft_consensus.cc:1240] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:07:45.287358 22676 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:45.337296 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:45.337716 16011 raft_consensus.cc:2393] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 15: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:45.330315 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:45.352188 15930 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:45.370625 16010 raft_consensus.cc:1275] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:07:45.339354 22617 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:45.330698 22582 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:45.383617 22582 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:45.385910 22702 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.386406 22702 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:45.388125 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:45.389277 22594 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:45.390803 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:45.388291 22702 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:45.391636 16561 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 20.
I20260709 13:07:45.392340 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:45.392577 16567 raft_consensus.cc:1275] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 15. (term mismatch)
I20260709 13:07:45.332401 22629 raft_consensus.cc:740] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:45.395309 16002 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 19.
I20260709 13:07:45.392890 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:45.393513 16563 raft_consensus.cc:1275] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:07:45.401521 16563 raft_consensus.cc:1275] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:07:45.389286 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:45.404752 16003 raft_consensus.cc:2393] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 18: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:45.397612 16570 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:45.399019 16568 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:45.397570 16567 pending_rounds.cc:85] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting all ops after (but not including) 14
I20260709 13:07:45.335033 22513 raft_consensus.cc:2749] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:45.337919 22663 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:45.338634 22651 raft_consensus.cc:697] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:45.392628 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:45.412823 16566 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:45.412910 16561 raft_consensus.cc:1240] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:45.406529 16567 pending_rounds.cc:107] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 13:07:45.413566 16567 raft_consensus.cc:2889] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:45.398489 15936 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:45.380352 22418 consensus_queue.cc:1059] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.014s
I20260709 13:07:45.417086 15930 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:45.421675 15974 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:45.421806 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:45.422261 16564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 1453) took 1878 ms. Trace:
I20260709 13:07:45.422442 16561 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:45.413581 16533 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:45.369474 22613 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:45.342489 22592 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:45.380000 22690 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:45.422698 16564 rpcz_store.cc:276] 0709 13:07:43.544145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:43.565055 (+ 20910us) service_pool.cc:224] Handling call
0709 13:07:43.663664 (+ 98609us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:43.863724 (+200060us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:45.422052 (+1558328us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b5400102554. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:45.422211 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":322991872}
I20260709 13:07:45.396893 22726 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.433261 16561 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
W20260709 13:07:45.416354 22730 log.cc:927] Time spent T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:07:45.435811 16570 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:07:45.398952 22653 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.393832 22648 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.440498 16533 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:45.416040 22663 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:45.441310 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:45.450080 16561 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 20.
I20260709 13:07:45.416922 22617 raft_consensus.cc:697] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:45.427138 22418 consensus_queue.cc:1243] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer 85e997ba5003439781ec4f1f8db0c3ee log is divergent from this leader: its last log entry 18.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:07:45.483001 16569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 1451) took 1956 ms. Trace:
W20260709 13:07:45.487241 16007 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41674 (request call id 2664) took 2448 ms (client timeout 2513 ms). Trace:
W20260709 13:07:45.487715 16007 rpcz_store.cc:269] 0709 13:07:43.038400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:43.038521 (+   121us) service_pool.cc:224] Handling call
0709 13:07:45.486990 (+2448469us) spinlock_profiling.cc:243] Waited 2.13 s on lock 0x7b54002e1dd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:45.487138 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4893091712}
I20260709 13:07:45.490481 15932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2912) took 1020 ms. Trace:
I20260709 13:07:45.490931 15932 rpcz_store.cc:276] 0709 13:07:44.470126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:44.582822 (+112696us) service_pool.cc:224] Handling call
0709 13:07:44.583155 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:45.258174 (+675019us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:45.490192 (+232018us) spinlock_profiling.cc:243] Waited 631 ms on lock 0x7b54003b9054. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:45.490418 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1451839872}
I20260709 13:07:45.529582 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:45.495656 15861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 2622) took 1106 ms. Trace:
I20260709 13:07:45.496419 15859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 2613) took 1111 ms. Trace:
W20260709 13:07:45.496662 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:45.531848 15860 raft_consensus.cc:1240] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 18. Current term is 19. Ops: []
W20260709 13:07:45.497066 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 685436us
I20260709 13:07:45.527983 15976 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:45.477936 22648 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:45.507500 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:45.508337 15855 raft_consensus.cc:1275] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:07:45.508935 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:45.483470 16569 rpcz_store.cc:276] 0709 13:07:43.526872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:43.537594 (+ 10722us) service_pool.cc:224] Handling call
0709 13:07:43.537925 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:43.560826 (+ 22901us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:07:43.579072 (+ 18246us) log.cc:844] Serialized 14 byte log entry
0709 13:07:43.812047 (+232975us) log.cc:844] Serialized 14 byte log entry
0709 13:07:45.314022 (+1501975us) spinlock_profiling.cc:243] Waited 7.94 ms on lock 0x7b18004ab6c0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e7235c0a 00007f73f0975219 00007f73e72bda40 00007f73e72b84e4 00007f73e72b8455 00007f73ea3a0060 00007f73ea3b2716 00007f73ea3c3287 00007f73ea3c3208
0709 13:07:45.314226 (+   204us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:45.314821 (+   595us) log.cc:844] Serialized 26 byte log entry
0709 13:07:45.315014 (+   193us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:45.315063 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:45.315156 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:45.482759 (+167603us) raft_consensus.cc:1685] finished
0709 13:07:45.482786 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:45.482940 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1594907,"prepare.queue_time_us":20273,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":68,"raft.queue_time_us":1532242,"raft.run_cpu_time_us":386,"raft.run_wall_time_us":62564,"spinlock_wait_cycles":18262784,"thread_start_us":223500,"threads_started":1,"wal-append.queue_time_us":445018}
I20260709 13:07:45.553026 16565 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1475) took 1959 ms. Trace:
I20260709 13:07:45.553555 16565 rpcz_store.cc:276] 0709 13:07:43.594021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:43.805878 (+211857us) service_pool.cc:224] Handling call
0709 13:07:44.991550 (+1185672us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:44.991849 (+   299us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:44.991889 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:44.991950 (+    61us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:44.992002 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:44.992078 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:45.552855 (+560777us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54006a22d0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:45.552969 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2714359552}
I20260709 13:07:45.510317 15858 raft_consensus.cc:1275] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:07:45.512085 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:45.555569 15857 raft_consensus.cc:2393] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 20: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:07:45.556761 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:45.513015 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:45.513921 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:45.560070 15854 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 20.
W20260709 13:07:45.560882 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:45.530297 15861 rpcz_store.cc:276] 0709 13:07:44.389498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:44.390961 (+  1463us) service_pool.cc:224] Handling call
0709 13:07:44.391228 (+   267us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:44.391444 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:07:44.481445 (+ 90001us) log.cc:844] Serialized 14 byte log entry
0709 13:07:44.639747 (+158302us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:44.788883 (+149136us) log.cc:844] Serialized 26 byte log entry
0709 13:07:44.789091 (+   208us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:07:44.789824 (+   733us) log.cc:844] Serialized 14 byte log entry
0709 13:07:45.494710 (+704886us) spinlock_profiling.cc:243] Waited 2.26 ms on lock 0x7b18003b6da0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e7235c0a 00007f73f0975219 00007f73ee27c58f 00007f73e72bb571 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd
0709 13:07:45.495224 (+   514us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:45.495304 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:45.495426 (+   122us) raft_consensus.cc:1685] finished
0709 13:07:45.495444 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:45.495595 (+   151us) inbound_call.cc:177] Queueing success response
0709 13:07:45.509198 (+ 13603us) spinlock_profiling.cc:243] Waited 1.66 ms on lock 0x7b18003b6da0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb522 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd 000000000038f354 000000000038f199 00007f73e729e83a
Metrics: {"mutex_wait_us":727471,"prepare.queue_time_us":72191,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":79,"raft.queue_time_us":709360,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":88339,"spinlock_wait_cycles":9019648,"thread_start_us":166190,"threads_started":2,"wal-append.queue_time_us":289588}
W20260709 13:07:45.561822 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:45.514343 22609 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:45.515974 22702 consensus_queue.cc:1048] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:07:45.439172 22527 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:07:45.589370 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 53787.7us
I20260709 13:07:45.518390 22650 consensus_queue.cc:1048] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:45.520699 22569 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:45.480475 22564 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:45.452437 22663 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:07:45.524457 22547 consensus_meta.cc:287] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.511s	user 0.010s	sys 0.000s
I20260709 13:07:45.478276 22653 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:45.614594 15937 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58402 (request call id 2610) took 1368 ms (client timeout 1525 ms). Trace:
I20260709 13:07:45.525974 22668 consensus_queue.cc:1059] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:07:45.530756 15859 rpcz_store.cc:276] 0709 13:07:44.384571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:44.385651 (+  1080us) service_pool.cc:224] Handling call
0709 13:07:45.496152 (+1110501us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400217d54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:45.496344 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2336721152}
I20260709 13:07:45.500085 22651 consensus_queue.cc:237] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:45.481717 22679 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:45.543891 22619 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:45.652644 22619 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:45.653066 22619 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:45.544541 22509 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:07:45.546397 22731 consensus_queue.cc:1048] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:45.547252 15902 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:07:45.658898 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 111749us
I20260709 13:07:45.551028 15827 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:45.660413 22653 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:45.562251 15862 raft_consensus.cc:2393] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 15: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:45.564762 16566 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 20.
I20260709 13:07:45.565737 16568 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 18.
I20260709 13:07:45.551227 15976 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:07:45.665186 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 114007us
I20260709 13:07:45.439946 22613 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:45.516988 22617 consensus_queue.cc:237] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:45.450302 22446 raft_consensus.cc:2749] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:45.607100 16567 raft_consensus.cc:1275] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:07:45.660213 22497 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:45.447329 22690 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:45.584905 22749 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.633451 15930 raft_consensus.cc:1240] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:45.633495 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:45.634701 15932 raft_consensus.cc:1240] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:45.626008 22527 raft_consensus.cc:697] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:45.587868 22609 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:45.562728 22418 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.066s
I20260709 13:07:45.695544 22659 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.696199 22659 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:45.640774 22648 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:45.605196 22675 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:45.643993 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:45.706548 15856 raft_consensus.cc:2376] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 18. Re-sending same reply.
I20260709 13:07:45.644816 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:45.645516 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:45.649398 22564 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:45.657720 22629 consensus_queue.cc:260] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:45.663271 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:45.713819 15854 raft_consensus.cc:2393] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 19: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:45.559285 22582 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:45.717309 22446 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.669255 15935 raft_consensus.cc:1240] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:45.659614 15902 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:45.718107 22446 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:45.719638 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 60091us
W20260709 13:07:45.720602 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:45.721221 22418 raft_consensus.cc:2749] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:45.721426 22690 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:07:45.722028 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:45.722326 22690 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:07:45.615077 15937 rpcz_store.cc:269] 0709 13:07:44.245757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:44.249809 (+  4052us) service_pool.cc:224] Handling call
0709 13:07:45.614417 (+1364608us) spinlock_profiling.cc:243] Waited 957 ms on lock 0x7b540008a7d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:45.614534 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2201450368}
I20260709 13:07:45.723939 22702 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.724196 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:45.724884 22702 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:45.726256 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:45.625207 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:45.731649 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:45.732844 16005 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:45.683218 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 132256us
I20260709 13:07:45.740643 22613 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:45.451843 22726 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:45.687861 15933 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:45.582623 22513 consensus_queue.cc:1048] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.082s
I20260709 13:07:45.688946 22667 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:45.584194 22592 consensus_queue.cc:1048] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.089s
W20260709 13:07:45.682493 22747 log.cc:927] Time spent T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
W20260709 13:07:45.764525 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 133150us
I20260709 13:07:45.755162 22702 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:45.681150 22088 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.770221 22088 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:45.705636 15900 leader_election.cc:336] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.771s (SENT)
I20260709 13:07:45.771971 15900 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:45.682222 22738 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:45.684026 22569 raft_consensus.cc:2804] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:45.682049 22610 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.691176 22547 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:45.689940 22497 raft_consensus.cc:697] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 19 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:45.698307 22609 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:45.698271 22659 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:45.723016 22690 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:45.767782 22745 consensus_queue.cc:1059] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:07:45.769290 22765 consensus_queue.cc:1059] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:07:45.772850 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 67263.9us
W20260709 13:07:45.849459 15900 leader_election.cc:336] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.784s (SENT)
I20260709 13:07:45.850138 15900 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:45.708254 15857 raft_consensus.cc:2393] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 15: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:45.773218 15855 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:07:45.701951 16535 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:45.696226 22527 consensus_queue.cc:237] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:45.711604 22648 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:45.813030 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:45.822310 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:45.861851 16568 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 22.
I20260709 13:07:45.807597 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:45.829511 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:45.864253 16561 raft_consensus.cc:2410] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 21 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
I20260709 13:07:45.833192 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:45.834576 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:45.763517 22492 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.838920 15933 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:45.701922 22749 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:45.874142 22780 log.cc:927] Time spent T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:07:45.847213 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:45.876145 22513 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:45.876849 22513 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:45.877864 22749 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:45.806424 16562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1474) took 2212 ms. Trace:
I20260709 13:07:45.852042 15855 raft_consensus.cc:606] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Received request to transfer leadership to TS eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:45.851382 15900 leader_election.cc:336] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.855s (SENT)
I20260709 13:07:45.762396 22667 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:45.872980 22492 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:45.851257 22702 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:45.856905 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 155055us
I20260709 13:07:45.885005 15825 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:45.881215 15931 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:07:45.896747 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:45.906849 16008 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 18.
I20260709 13:07:45.890141 15855 raft_consensus.cc:614] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:45.842463 22653 consensus_queue.cc:1059] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 13:07:45.908995 22653 raft_consensus.cc:2764] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader election decision vote started in defunct term 14: won
I20260709 13:07:45.909683 22653 raft_consensus.cc:3055] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:45.909920 22675 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:45.896960 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 443875us
I20260709 13:07:45.843564 22569 raft_consensus.cc:697] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 15 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:45.910848 22759 raft_consensus.cc:2804] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:45.856305 22564 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:45.911409 22569 consensus_queue.cc:237] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:45.850395 22786 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.913355 22786 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:45.915381 22786 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:45.910094 22653 raft_consensus.cc:740] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:45.915784 22513 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:45.916643 22653 consensus_queue.cc:260] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:45.917913 22653 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:45.920172 16533 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:45.924185 22786 raft_consensus.cc:2804] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:45.927497 16565 raft_consensus.cc:1240] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:45.879901 16562 rpcz_store.cc:276] 0709 13:07:43.593798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:43.805374 (+211576us) service_pool.cc:224] Handling call
0709 13:07:43.827718 (+ 22344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:43.827936 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:43.827968 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:43.828339 (+   371us) log.cc:844] Serialized 26 byte log entry
0709 13:07:44.262194 (+433855us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:44.324636 (+ 62442us) log.cc:844] Serialized 14 byte log entry
0709 13:07:45.806059 (+1481423us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:45.806139 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:45.806180 (+    41us) raft_consensus.cc:1685] finished
0709 13:07:45.806192 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:45.806356 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1065029,"prepare.queue_time_us":62144,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":1069856,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":15295,"thread_start_us":779260,"threads_started":2,"wal-append.queue_time_us":501686,"wal-append.run_cpu_time_us":2893,"wal-append.run_wall_time_us":1119095}
I20260709 13:07:45.838084 22610 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:45.932641 16005 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 18.
I20260709 13:07:45.964933 16011 raft_consensus.cc:1240] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:45.965668 16535 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:45.934484 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:45.966512 16008 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:45.883136 22594 raft_consensus.cc:3055] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:45.967947 15861 raft_consensus.cc:1240] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:45.975631 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:45.976464 16002 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 18.
I20260709 13:07:45.977602 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:45.978812 15976 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:45.979440 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:45.984248 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:45.984480 15856 raft_consensus.cc:1275] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:07:45.986662 15902 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:45.924708 22786 raft_consensus.cc:697] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:45.931243 22759 raft_consensus.cc:697] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:45.996320 22786 consensus_queue.cc:237] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.14, Last appended by leader: 14, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:45.997359 22509 raft_consensus.cc:2804] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:45.998041 22509 raft_consensus.cc:697] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 20 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:45.997664 22759 consensus_queue.cc:237] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:45.996587 15976 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:45.924736 22731 raft_consensus.cc:3055] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:45.905383 22659 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:46.004019 15857 raft_consensus.cc:1275] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:07:46.004549 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:46.004885 16566 raft_consensus.cc:1275] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:07:45.879956 22663 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:45.963832 22550 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:46.011746 15859 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 17.
I20260709 13:07:46.014809 16003 raft_consensus.cc:1275] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:07:46.026332 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:45.887724 22648 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:46.025944 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:46.029699 15859 raft_consensus.cc:1240] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:46.032886 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:46.041342 16008 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 19.
I20260709 13:07:46.045678 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:45.970431 22594 raft_consensus.cc:740] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:46.003732 22564 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:46.017594 22568 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:45.900643 22729 consensus_queue.cc:1059] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.105s
I20260709 13:07:46.053966 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:45.979236 22765 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:46.009522 22690 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:46.059355 15933 raft_consensus.cc:1240] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:46.027168 15855 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 20.
I20260709 13:07:45.927029 22702 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:46.006585 22731 raft_consensus.cc:740] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:46.067801 15857 raft_consensus.cc:2410] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 18 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:07:46.056716 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:45.996726 22619 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:46.067611 16566 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:07:45.999191 22509 consensus_queue.cc:237] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:46.086164 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:46.104259 16568 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 18.
I20260709 13:07:46.042506 22550 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:46.118700 16562 raft_consensus.cc:2410] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:07:46.062601 22446 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:46.088394 22668 raft_consensus.cc:2749] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:46.090337 22650 raft_consensus.cc:3055] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:46.093263 15861 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 18.
I20260709 13:07:46.108318 22610 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:46.109328 22667 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:46.130007 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:46.130931 16569 raft_consensus.cc:2368] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 14. Current term is 15.
W20260709 13:07:46.110574 22603 consensus_meta.cc:287] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.771s	user 0.008s	sys 0.000s
I20260709 13:07:46.111318 22619 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:46.112316 22648 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:46.009056 22497 consensus_queue.cc:237] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:46.112504 22726 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:46.136365 15902 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:46.112775 22592 consensus_queue.cc:1048] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:46.137004 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:46.137835 16565 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:46.138949 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:46.113924 22663 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:46.079797 22675 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:46.140086 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:46.140383 22675 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:46.140920 16568 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 18.
I20260709 13:07:46.142364 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:46.091188 22568 raft_consensus.cc:697] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 18 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:46.114809 22492 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:46.092402 22765 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:46.117349 22088 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:46.120699 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:46.165020 16569 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 20.
I20260709 13:07:46.139611 16008 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 15.
I20260709 13:07:46.086908 22564 raft_consensus.cc:697] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 18 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:46.035732 22659 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:46.062938 22463 consensus_queue.cc:1048] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 13:07:46.177852 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:07:46.144007 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:46.127564 22676 consensus_meta.cc:287] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.785s	user 0.013s	sys 0.000s
I20260709 13:07:46.103215 22749 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:46.127077 22690 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:46.170011 16565 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 21.
I20260709 13:07:46.127751 22650 raft_consensus.cc:740] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:46.144732 22594 consensus_queue.cc:260] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:46.174619 22564 consensus_queue.cc:237] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:46.142942 22790 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:46.162251 22726 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:46.156862 22614 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:46.185850 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:46.199318 16008 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 14.
W20260709 13:07:46.200273 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 434193us
I20260709 13:07:46.118834 22793 consensus_queue.cc:1059] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.082s
I20260709 13:07:46.194366 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:46.203243 15902 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:46.181313 22738 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:46.185196 22610 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:46.220470 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:46.186695 22675 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:46.148125 22731 consensus_queue.cc:260] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:46.216289 22676 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:46.257509 22676 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:46.258165 22676 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:46.206475 22614 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:46.227300 22610 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:46.261974 22614 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:46.263067 22610 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:46.264991 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:46.265949 16008 raft_consensus.cc:2410] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 18 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:07:46.248603 16568 raft_consensus.cc:1240] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:46.222889 22568 consensus_queue.cc:237] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:46.253328 22765 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:46.202240 22790 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:46.209913 22726 raft_consensus.cc:697] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 19 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:46.195724 22690 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:46.187379 22749 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:46.269878 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:46.245922 22702 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:46.268855 22594 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:07:46.277204 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 459133us
I20260709 13:07:46.306829 15827 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:46.291494 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:07:46.297887 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:46.304883 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:46.312093 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:46.374394 15859 raft_consensus.cc:1275] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:07:46.311339 15854 raft_consensus.cc:1275] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:07:46.310329 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:46.311760 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:46.418368 16003 raft_consensus.cc:1240] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 18. Current term is 19. Ops: []
W20260709 13:07:46.418663 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 143860us
I20260709 13:07:46.419885 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:46.421103 15862 raft_consensus.cc:1275] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:07:46.421629 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:46.420665 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:46.422511 16008 raft_consensus.cc:2393] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 15: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:46.430183 16564 raft_consensus.cc:1240] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:07:46.384231 22765 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:46.430460 22603 consensus_queue.cc:1059] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:07:46.430666 22463 consensus_queue.cc:1048] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:46.431058 22765 raft_consensus.cc:697] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 20 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
W20260709 13:07:46.430250 22795 log.cc:927] Time spent T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 13:07:46.430536 16007 raft_consensus.cc:1240] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:07:46.307498 22592 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:46.302613 22663 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:46.432660 15974 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:46.439424 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:46.433113 15900 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.010s	sys 0.025s
W20260709 13:07:46.456120 15900 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.011s	sys 0.025s
W20260709 13:07:46.456563 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 566904us
I20260709 13:07:46.459069 15900 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:46.461148 15974 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:07:46.413831 22679 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.726s	user 0.020s	sys 0.013s
I20260709 13:07:46.431694 22603 raft_consensus.cc:3055] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:46.469169 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:46.474090 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:46.474783 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:46.469509 22603 raft_consensus.cc:740] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:46.488478 22603 consensus_queue.cc:260] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:46.490370 22603 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:07:46.467638 22679 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.786s	user 0.020s	sys 0.013s
I20260709 13:07:46.466917 22614 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:46.481778 22690 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:46.463043 22592 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:46.486948 22463 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:46.481084 22550 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:46.501772 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:46.518884 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:07:46.527503 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 67310.2us
I20260709 13:07:46.529306 15900 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:46.463804 22651 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.008s	sys 0.019s
I20260709 13:07:46.504568 22614 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:46.517449 22492 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:46.498320 22679 raft_consensus.cc:697] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:46.517904 22463 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:07:46.533939 16535 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.010s	sys 0.014s
I20260709 13:07:46.464354 22663 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:46.515239 22793 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:46.541136 22614 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:07:46.546834 16561 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.001s	sys 0.026s
I20260709 13:07:46.505193 22513 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.034s
W20260709 13:07:46.572402 16561 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.001s	sys 0.026s
I20260709 13:07:46.575642 16002 raft_consensus.cc:1240] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 15. Current term is 16. Ops: []
W20260709 13:07:46.537636 22651 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.009s	sys 0.019s
W20260709 13:07:46.583549 15929 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.027s	sys 0.068s
W20260709 13:07:46.584702 15929 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.027s	sys 0.068s
I20260709 13:07:46.467931 22726 consensus_queue.cc:237] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:46.543387 22675 raft_consensus.cc:2764] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:07:46.571178 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:46.553786 22463 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:46.607236 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:07:46.549115 16535 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.011s	sys 0.015s
I20260709 13:07:46.601773 16565 raft_consensus.cc:1813] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Handling vote request from an unknown peer 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:46.608381 16568 raft_consensus.cc:2393] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 18: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:46.608719 16565 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 15: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
W20260709 13:07:46.608584 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 632705us
W20260709 13:07:46.611032 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:46.601558 15928 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:46118
W20260709 13:07:46.555475 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:46.543977 15937 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.002s	sys 0.024s
W20260709 13:07:46.613422 15937 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.770s	user 0.002s	sys 0.024s
I20260709 13:07:46.613420 15928 raft_consensus.cc:606] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Received request to transfer leadership to TS 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:46.620426 15928 raft_consensus.cc:614] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:07:46.625733 16533 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.005s	sys 0.021s
W20260709 13:07:46.631819 16533 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.006s	sys 0.021s
I20260709 13:07:46.528712 22765 consensus_queue.cc:237] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:46.642416 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:46.645589 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 179958us
I20260709 13:07:46.644263 16533 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:46.548655 22492 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:46.660713 15974 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:46.662879 22834 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:46.620992 22629 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:07:46.692129 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 47967.7us
I20260709 13:07:46.693082 16533 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:46.695638 22446 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:46.697749 16533 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:46.698451 22446 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:07:46.700213 22808 log.cc:927] Time spent T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
W20260709 13:07:46.701231 22811 log.cc:927] Time spent T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:07:46.691870 22790 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:46.704387 16566 raft_consensus.cc:1275] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:07:46.704370 22446 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:46.705780 22446 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:46.706218 22446 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:46.706346 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:46.707103 16566 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 16.
I20260709 13:07:46.708133 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:46.709324 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:46.709913 16566 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 18.
I20260709 13:07:46.710762 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:46.711161 22592 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:46.691340 22738 raft_consensus.cc:2749] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:46.712638 16568 raft_consensus.cc:1275] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:07:46.713666 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:46.574416 22793 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:46.663785 22834 raft_consensus.cc:697] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:46.622076 22667 consensus_queue.cc:1059] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.072s
W20260709 13:07:46.754283 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 60505.8us
I20260709 13:07:46.762818 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:46.626364 22729 consensus_queue.cc:1059] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.089s
I20260709 13:07:46.665530 22749 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:46.754241 16533 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:46.695123 15974 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:46.789280 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:47.181955 15929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2924) took 1491 ms. Trace:
I20260709 13:07:46.784536 16568 raft_consensus.cc:1275] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:07:47.328081 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:47.329787 15928 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
mode: GRACEFUL
new_leader_uuid: "31882606e87447fb9a36bbf625129fe8"
 from {username='slave'} at 127.0.0.1:46118
I20260709 13:07:47.328212 15929 rpcz_store.cc:276] 0709 13:07:45.690303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:45.690533 (+   230us) service_pool.cc:224] Handling call
0709 13:07:45.690809 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:45.691024 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:07:46.025622 (+334598us) log.cc:844] Serialized 14 byte log entry
0709 13:07:46.819845 (+794223us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:47.180795 (+360950us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:47.181079 (+   284us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:47.181269 (+   190us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:47.181623 (+   354us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":158540,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":8502,"raft.queue_time_us":76946,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"thread_start_us":1115585,"threads_started":3}
I20260709 13:07:47.330780 15928 raft_consensus.cc:606] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 16 LEADER]: Received request to transfer leadership to TS 31882606e87447fb9a36bbf625129fe8
W20260709 13:07:47.180332 15900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.638s	user 0.007s	sys 0.007s
W20260709 13:07:47.336215 15900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.794s	user 0.007s	sys 0.008s
I20260709 13:07:47.197562 15856 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 20.
I20260709 13:07:47.201527 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:47.204550 15827 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:47.209924 15937 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2926) took 1513 ms. Trace:
I20260709 13:07:47.215911 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:47.303272 22088 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:47.219137 22322 consensus_queue.cc:1048] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:47.220813 22418 raft_consensus.cc:2749] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:46.769836 22492 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:47.373289 16561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1494) took 1391 ms. Trace:
I20260709 13:07:47.239425 22509 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:47.240883 22527 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:47.376339 22527 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:47.376331 22509 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:47.238808 22508 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:46.785389 22610 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:47.384927 22755 raft_consensus.cc:3055] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:47.385594 22755 raft_consensus.cc:740] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:47.385872 22610 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:47.386739 22527 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:46.683082 22679 consensus_queue.cc:237] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:47.395159 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:07:47.395210 22508 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:47.398583 22508 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:46.664269 22702 consensus_queue.cc:1048] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.126s
W20260709 13:07:47.271538 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:46.761452 22663 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:46.786497 22676 consensus_meta.cc:287] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.528s	user 0.015s	sys 0.000s
I20260709 13:07:47.272490 22726 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:47.272935 22749 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:07:47.281777 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 527446us
W20260709 13:07:47.280861 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:47.283666 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:47.352231 22497 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:47.289536 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:47.293469 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:47.297492 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:46.764945 22793 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:47.462227 16569 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 20.
I20260709 13:07:47.463363 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:07:46.785801 22799 log.cc:927] Time spent T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.332s	user 0.000s	sys 0.001s
I20260709 13:07:47.465327 16010 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:47.466140 16562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1502) took 1286 ms. Trace:
W20260709 13:07:46.767293 22801 log.cc:927] Time spent T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.166s	user 0.000s	sys 0.000s
I20260709 13:07:47.467108 16010 raft_consensus.cc:1240] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:47.467887 16565 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 18.
W20260709 13:07:46.774541 22810 log.cc:927] Time spent T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.232s	user 0.000s	sys 0.001s
I20260709 13:07:47.469040 16008 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 20.
I20260709 13:07:47.469825 16009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 2666) took 1741 ms. Trace:
I20260709 13:07:47.470618 22793 consensus_queue.cc:1059] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:07:46.974959 22834 consensus_queue.cc:237] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:47.470920 22791 consensus_queue.cc:1048] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 13:07:47.480342 22610 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:47.470263 16009 rpcz_store.cc:276] 0709 13:07:45.728585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:45.735806 (+  7221us) service_pool.cc:224] Handling call
0709 13:07:45.736327 (+   521us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:45.736940 (+   613us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:45.736982 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:45.737622 (+   640us) log.cc:844] Serialized 26 byte log entry
0709 13:07:46.194674 (+457052us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:46.194757 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:46.203354 (+  8597us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:47.469386 (+1266032us) raft_consensus.cc:1685] finished
0709 13:07:47.469422 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:47.469614 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15817,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":6110,"thread_start_us":456476,"threads_started":1,"wal-append.queue_time_us":800927}
W20260709 13:07:47.324332 22817 log.cc:927] Time spent T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.538s	user 0.010s	sys 0.000s
W20260709 13:07:47.485180 22629 consensus_meta.cc:287] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.781s	user 0.012s	sys 0.016s
I20260709 13:07:47.486081 22564 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:47.217877 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:47.329048 16568 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
W20260709 13:07:47.352100 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 148053us
W20260709 13:07:47.351655 22463 consensus_meta.cc:287] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.724s	user 0.010s	sys 0.020s
I20260709 13:07:47.352093 22825 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:47.363233 22088 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:47.495865 22088 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:47.340526 22668 raft_consensus.cc:2749] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:07:47.340420 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 800362us
I20260709 13:07:47.497732 15900 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:47.466570 16562 rpcz_store.cc:276] 0709 13:07:46.180043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:46.180912 (+   869us) service_pool.cc:224] Handling call
0709 13:07:46.190978 (+ 10066us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:46.191473 (+   495us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:46.191522 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:46.196851 (+  5329us) log.cc:844] Serialized 26 byte log entry
0709 13:07:46.581790 (+384939us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:46.581854 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:46.581955 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:47.465733 (+883778us) raft_consensus.cc:1685] finished
0709 13:07:47.465772 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:47.465970 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14774,"raft.queue_time_us":1178,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":5480,"thread_start_us":345579,"threads_started":1,"wal-append.queue_time_us":1177900}
I20260709 13:07:47.253540 22594 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:47.500159 15827 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:47.500715 22802 log.cc:927] Time spent T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.956s	user 0.003s	sys 0.000s
I20260709 13:07:47.500075 22594 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:47.373858 16561 rpcz_store.cc:276] 0709 13:07:45.982109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:45.992714 (+ 10605us) service_pool.cc:224] Handling call
0709 13:07:45.993213 (+   499us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:46.572868 (+579655us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:07:46.572903 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:46.573498 (+   595us) log.cc:844] Serialized 26 byte log entry
0709 13:07:46.676349 (+102851us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:46.676409 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:46.676515 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:47.372797 (+696282us) raft_consensus.cc:1685] finished
0709 13:07:47.372841 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:47.373066 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23671,"raft.queue_time_us":155078,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":7416,"thread_start_us":592270,"threads_started":2,"wal-append.queue_time_us":769268}
I20260709 13:07:47.426270 22650 consensus_queue.cc:260] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:47.432442 22755 consensus_queue.cc:260] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:47.470270 16533 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:47.436834 22659 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:47.286114 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:47.339915 15937 rpcz_store.cc:276] 0709 13:07:45.696836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:45.782184 (+ 85348us) service_pool.cc:224] Handling call
0709 13:07:45.806524 (+ 24340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:45.806946 (+   422us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:07:46.619164 (+812218us) log.cc:844] Serialized 14 byte log entry
0709 13:07:47.208814 (+589650us) log.cc:844] Serialized 14 byte log entry
0709 13:07:47.209198 (+   384us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:47.209269 (+    71us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:47.209389 (+   120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:47.209517 (+   128us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:47.209746 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":346,"prepare.queue_time_us":325866,"prepare.run_cpu_time_us":123,"prepare.run_wall_time_us":10745,"raft.queue_time_us":445598,"raft.run_cpu_time_us":474,"raft.run_wall_time_us":475,"thread_start_us":1336298,"threads_started":3}
I20260709 13:07:47.446441 22749 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:47.443109 22676 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:46.766460 22796 log.cc:927] Time spent T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.262s	user 0.001s	sys 0.000s
I20260709 13:07:47.476022 22790 consensus_queue.cc:1048] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:07:47.491178 22663 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:47.491310 22463 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:47.529991 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:47.551461 22446 consensus_meta.cc:287] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.747s	user 0.024s	sys 0.010s
I20260709 13:07:47.552114 22446 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:47.552639 22755 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:47.554730 22446 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:47.556725 16006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2691) took 1814 ms. Trace:
I20260709 13:07:47.557423 16567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1493) took 1575 ms. Trace:
W20260709 13:07:47.558055 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 88012.3us
I20260709 13:07:47.557930 16567 rpcz_store.cc:276] 0709 13:07:45.981742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:45.981894 (+   152us) service_pool.cc:224] Handling call
0709 13:07:45.982213 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:46.498038 (+515825us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:46.498075 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:46.498743 (+   668us) log.cc:844] Serialized 26 byte log entry
0709 13:07:46.624870 (+126127us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:46.624918 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:46.625016 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:47.556953 (+931937us) raft_consensus.cc:1685] finished
0709 13:07:47.557004 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:47.557235 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34657,"raft.queue_time_us":161216,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":13557,"thread_start_us":606371,"threads_started":2,"wal-append.queue_time_us":1049793}
I20260709 13:07:47.443056 22726 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:47.561714 16004 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 2660) took 2257 ms. Trace:
I20260709 13:07:47.568652 22668 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:47.569387 22668 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:47.569947 22668 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:47.568882 16004 rpcz_store.cc:276] 0709 13:07:45.304362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:45.304907 (+   545us) service_pool.cc:224] Handling call
0709 13:07:45.305314 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:45.316877 (+ 11563us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:45.316921 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:45.317427 (+   506us) log.cc:844] Serialized 26 byte log entry
0709 13:07:45.549685 (+232258us) spinlock_profiling.cc:243] Waited 1.22 ms on lock 0x7b1800435280. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f480d1 00007f73ea363a0f
0709 13:07:46.014796 (+465111us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:46.014885 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:46.015028 (+   143us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:47.561028 (+1546000us) raft_consensus.cc:1685] finished
0709 13:07:47.561076 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:47.561449 (+   373us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11065,"raft.queue_time_us":12765,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":4761,"spinlock_wait_cycles":2801536,"thread_start_us":463777,"threads_started":1,"wal-append.queue_time_us":1088903}
I20260709 13:07:47.572249 22492 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:47.572897 22492 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:47.573325 16008 raft_consensus.cc:1275] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:07:47.573763 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2698) took 1256 ms. Trace:
I20260709 13:07:47.574184 16002 rpcz_store.cc:276] 0709 13:07:46.317112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:46.317247 (+   135us) service_pool.cc:224] Handling call
0709 13:07:46.317539 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:46.576245 (+258706us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:47.573180 (+996935us) spinlock_profiling.cc:243] Waited 258 ms on lock 0x7b540010d6d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:47.573543 (+   363us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":592899968}
W20260709 13:07:47.576584 22614 consensus_meta.cc:287] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.974s	user 0.014s	sys 0.013s
I20260709 13:07:47.576699 22663 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:47.577052 22614 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:47.576361 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 76469us
I20260709 13:07:47.583166 22663 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:47.573387 22492 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:47.587340 22594 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:47.588526 22731 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:47.589393 22676 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:47.555485 22547 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:07:47.590627 15900 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:47.583324 22509 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:07:47.583344 22806 log.cc:927] Time spent T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 1.035s	user 0.002s	sys 0.000s
I20260709 13:07:47.585011 15856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 2629) took 1273 ms. Trace:
I20260709 13:07:47.585650 22463 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:47.586010 22749 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:47.592547 15856 rpcz_store.cc:276] 0709 13:07:46.311366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:46.311802 (+   436us) service_pool.cc:224] Handling call
0709 13:07:47.584736 (+1272934us) spinlock_profiling.cc:243] Waited 386 ms on lock 0x7b5400217854. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:47.584872 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":886833920}
I20260709 13:07:47.522778 22564 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:47.529424 22497 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:47.557175 16006 rpcz_store.cc:276] 0709 13:07:45.742399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:45.743828 (+  1429us) service_pool.cc:224] Handling call
0709 13:07:45.744333 (+   505us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:45.744838 (+   505us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:45.744881 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:45.745546 (+   665us) log.cc:844] Serialized 26 byte log entry
0709 13:07:46.209895 (+464349us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:46.209957 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:46.210056 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:47.499775 (+1289719us) spinlock_profiling.cc:243] Waited 5.29 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:07:47.556236 (+ 56461us) raft_consensus.cc:1685] finished
0709 13:07:47.556255 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:47.556525 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":551,"raft.queue_time_us":15595,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":9547,"spinlock_wait_cycles":12159360,"thread_start_us":463812,"threads_started":1,"wal-append.queue_time_us":724409}
I20260709 13:07:47.605139 22675 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:47.606221 22675 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:47.607457 15936 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2925) took 1910 ms. Trace:
I20260709 13:07:47.608700 22551 raft_consensus.cc:3055] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:47.609109 22551 raft_consensus.cc:740] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:47.607941 15936 rpcz_store.cc:276] 0709 13:07:45.696541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:45.702298 (+  5757us) service_pool.cc:224] Handling call
0709 13:07:45.753401 (+ 51103us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:45.773774 (+ 20373us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:45.806032 (+ 32258us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:45.822922 (+ 16890us) log.cc:844] Serialized 52 byte log entry
0709 13:07:46.272096 (+449174us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:46.272148 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:46.272239 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:47.582490 (+1310251us) spinlock_profiling.cc:243] Waited 5.14 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:07:47.606850 (+ 24360us) raft_consensus.cc:1685] finished
0709 13:07:47.606875 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:47.607209 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8827,"raft.queue_time_us":7390,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":48179,"spinlock_wait_cycles":11833088,"thread_start_us":408124,"threads_started":1,"wal-append.queue_time_us":668969}
I20260709 13:07:47.609526 22497 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:47.610324 22551 consensus_queue.cc:260] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:47.611685 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:47.611788 15827 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:47.613891 22825 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:47.523118 22629 consensus_queue.cc:1048] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.079s
I20260709 13:07:47.616001 22614 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:47.618100 15827 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:47.619333 22508 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:47.620640 22726 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:47.625768 16570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1501) took 1485 ms. Trace:
I20260709 13:07:47.629613 22790 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:47.630318 22790 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:47.630565 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:47.634590 22689 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:47.632849 22790 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:47.626410 16570 rpcz_store.cc:276] 0709 13:07:46.140580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:46.142816 (+  2236us) service_pool.cc:224] Handling call
0709 13:07:46.143160 (+   344us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:46.161132 (+ 17972us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:46.161178 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:46.182087 (+ 20909us) log.cc:844] Serialized 52 byte log entry
0709 13:07:46.573614 (+391527us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:46.573665 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:46.573740 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:47.625210 (+1051470us) raft_consensus.cc:1685] finished
0709 13:07:47.625261 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:47.625523 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5218,"raft.queue_time_us":20333,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":12654,"thread_start_us":364075,"threads_started":1,"wal-append.queue_time_us":1380572}
I20260709 13:07:47.643131 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:47.644372 22825 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:47.644863 22825 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:47.645246 22825 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:47.635494 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:47.651099 16005 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2696) took 1336 ms. Trace:
I20260709 13:07:47.651620 16005 rpcz_store.cc:276] 0709 13:07:46.314645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:46.314780 (+   135us) service_pool.cc:224] Handling call
0709 13:07:46.416946 (+102166us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:46.425117 (+  8171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:46.425162 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:46.425733 (+   571us) log.cc:844] Serialized 26 byte log entry
0709 13:07:46.663187 (+237454us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:46.663233 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:46.663316 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:47.650659 (+987343us) raft_consensus.cc:1685] finished
0709 13:07:47.650715 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:47.650899 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7787,"raft.queue_time_us":12225,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95,"thread_start_us":237118,"threads_started":1,"wal-append.queue_time_us":1222432}
I20260709 13:07:47.657617 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:47.657828 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:47.633009 22675 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:47.635067 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 939834us
I20260709 13:07:47.661916 22568 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:47.662627 22650 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:47.662781 22568 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:47.667063 22564 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:47.669770 15937 raft_consensus.cc:1240] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 19 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:47.671010 22791 raft_consensus.cc:3055] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:47.672940 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:47.673750 15928 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
W20260709 13:07:47.673877 22844 log.cc:927] Time spent T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.300s	user 0.001s	sys 0.001s
I20260709 13:07:47.675760 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:47.676680 16564 raft_consensus.cc:2368] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 14. Current term is 15.
I20260709 13:07:47.677433 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:47.677774 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:47.678186 16567 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 20.
I20260709 13:07:47.678630 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:47.671514 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:47.680038 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:47.671481 22791 raft_consensus.cc:740] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:47.683497 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:47.683418 22791 consensus_queue.cc:260] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:47.671799 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:47.575461 22659 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:47.686699 22659 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:47.690500 22551 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:47.685093 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:47.685946 15936 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 19.
I20260709 13:07:47.695441 16005 raft_consensus.cc:1275] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:07:47.696849 22568 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:47.685961 22791 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:47.695494 15976 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:47.710705 22663 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:47.711328 22663 raft_consensus.cc:697] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:47.712494 22663 consensus_queue.cc:237] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:47.682088 15937 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
W20260709 13:07:47.723862 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 189973us
I20260709 13:07:47.724427 15976 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:47.730770 22676 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:47.731602 22676 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:47.734283 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:47.735286 15929 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 18.
I20260709 13:07:47.735453 16011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2695) took 1429 ms. Trace:
I20260709 13:07:47.736711 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:47.735953 16011 rpcz_store.cc:276] 0709 13:07:46.314168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:46.314313 (+   145us) service_pool.cc:224] Handling call
0709 13:07:46.375259 (+ 60946us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:46.375734 (+   475us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:46.375787 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:46.410921 (+ 35134us) log.cc:844] Serialized 26 byte log entry
0709 13:07:46.663486 (+252565us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:46.663532 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:46.663608 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:47.734991 (+1071383us) raft_consensus.cc:1685] finished
0709 13:07:47.735034 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:47.735244 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3103,"raft.queue_time_us":8021,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":6675,"thread_start_us":193089,"threads_started":1,"wal-append.queue_time_us":1309158}
I20260709 13:07:47.734416 22676 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:47.707370 22659 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:47.752485 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:47.755657 22610 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:47.756063 15936 raft_consensus.cc:1240] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:47.756202 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:47.756569 22610 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:47.744905 15974 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:47.757933 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:47.762006 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:47.766269 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:47.766400 22820 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:47.767017 15862 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 20.
I20260709 13:07:47.768164 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:47.769039 15858 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
I20260709 13:07:47.770787 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:47.771644 15854 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 16.
I20260709 13:07:47.772114 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:47.772368 22819 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:47.772949 22819 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:47.773494 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:47.746186 22614 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:47.775272 22819 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:47.780583 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:47.781404 16564 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 17.
W20260709 13:07:47.783824 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 55897.8us
I20260709 13:07:47.796510 15937 raft_consensus.cc:1240] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:47.795658 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:47.797485 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:47.797425 22738 raft_consensus.cc:2749] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:07:47.795715 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 51194.1us
I20260709 13:07:47.785548 22497 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:47.767277 22820 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:47.800840 22497 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:47.802597 22820 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:47.803207 22497 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:47.803936 22675 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:47.803936 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:47.804327 16566 raft_consensus.cc:2410] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
W20260709 13:07:47.806102 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 61869.7us
I20260709 13:07:47.808645 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:47.812973 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:47.816731 16533 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:47.805299 22568 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:47.809341 22564 consensus_queue.cc:1048] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 13:07:47.835151 22568 raft_consensus.cc:697] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:47.835316 22610 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:47.836351 22568 consensus_queue.cc:237] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:47.837430 15854 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 18.
I20260709 13:07:47.830416 22689 raft_consensus.cc:2749] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:47.838550 15858 raft_consensus.cc:1275] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 13 index: 14. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
I20260709 13:07:47.843003 22793 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:47.843571 22793 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:47.844058 22793 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:47.847025 22610 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:47.843328 22731 consensus_queue.cc:1059] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:07:47.851114 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:47.850656 22731 consensus_queue.cc:1243] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer 8430db80a39d4868bcf7641c16de35ea log is divergent from this leader: its last log entry 16.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:07:47.851984 15857 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 18.
I20260709 13:07:47.852906 22667 raft_consensus.cc:3055] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:47.856221 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:47.856966 16011 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 18.
I20260709 13:07:47.853402 22667 raft_consensus.cc:740] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:47.879130 15976 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:47.880754 22527 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:47.881199 22527 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:47.881548 22527 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:47.882653 16006 raft_consensus.cc:1240] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:47.882719 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:47.884088 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:47.884703 16007 raft_consensus.cc:1275] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:07:47.884856 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:47.882871 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:47.886555 16003 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
I20260709 13:07:47.887082 16002 raft_consensus.cc:1275] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:07:47.890748 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:47.891093 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:47.892318 16008 raft_consensus.cc:1275] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 14. (term mismatch)
I20260709 13:07:47.892683 16008 pending_rounds.cc:85] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee: Aborting all ops after (but not including) 13
I20260709 13:07:47.892935 16008 pending_rounds.cc:107] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 16.14
I20260709 13:07:47.893203 16008 raft_consensus.cc:2889] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:47.893422 16566 raft_consensus.cc:1275] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:07:47.896185 16564 raft_consensus.cc:1275] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:07:47.895884 16562 raft_consensus.cc:1240] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 22. Current term is 23. Ops: []
I20260709 13:07:47.897382 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:47.904573 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 44885.2us
I20260709 13:07:47.829423 22668 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:47.912217 15902 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:47.912470 22848 consensus_queue.cc:1048] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.054s
I20260709 13:07:47.913690 22842 raft_consensus.cc:2749] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:07:47.914496 22836 log.cc:927] Time spent T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 13:07:47.915922 15858 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:47.917263 16563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1497) took 1781 ms. Trace:
I20260709 13:07:47.917640 16563 rpcz_store.cc:276] 0709 13:07:46.135267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:46.136115 (+   848us) service_pool.cc:224] Handling call
0709 13:07:46.136508 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:46.385531 (+249023us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:46.385585 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:46.410942 (+ 25357us) log.cc:844] Serialized 26 byte log entry
0709 13:07:47.482219 (+1071277us) spinlock_profiling.cc:243] Waited 2.92 ms on lock 0x7b18000330a0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f480d1 00007f73ea363a0f
0709 13:07:47.599562 (+117343us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:47.602290 (+  2728us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:47.743797 (+141507us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b5400695ad4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:47.743875 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:47.916944 (+173069us) raft_consensus.cc:1685] finished
0709 13:07:47.916967 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:47.917099 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":78038,"raft.queue_time_us":110667,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":8310,"spinlock_wait_cycles":398739968,"thread_start_us":41329,"threads_started":1,"wal-append.queue_time_us":1503860}
W20260709 13:07:47.924182 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 67874.1us
I20260709 13:07:47.924831 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:47.925043 22702 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:47.925742 16563 raft_consensus.cc:2410] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 21 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:07:47.925634 22702 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:47.926067 15825 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:47.927976 22679 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:47.928589 22679 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:47.930068 22821 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:47.930486 22821 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:47.930846 22821 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:47.931102 22679 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:07:47.933985 22843 log.cc:927] Time spent T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:07:47.834142 22675 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:47.937552 16533 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:47.927541 15825 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:07:47.948835 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 93366.3us
I20260709 13:07:47.950002 15900 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:47.951887 22564 raft_consensus.cc:2749] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:47.956033 16569 raft_consensus.cc:1275] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
W20260709 13:07:47.951988 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:47.963114 15900 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:47.964160 22551 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:47.965456 16569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 1598) took 1264 ms. Trace:
I20260709 13:07:47.974494 22825 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:47.977438 15858 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 23.
I20260709 13:07:47.978156 22675 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:47.978866 16570 raft_consensus.cc:1240] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:47.976401 22527 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:47.980211 22668 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:47.981639 22603 consensus_queue.cc:1048] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:47.983592 22663 consensus_queue.cc:1048] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:07:47.984740 22834 consensus_queue.cc:1059] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:07:47.991349 16569 rpcz_store.cc:276] 0709 13:07:46.701259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:47.464429 (+763170us) service_pool.cc:224] Handling call
0709 13:07:47.464915 (+   486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:47.956467 (+491552us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:47.965178 (+  8711us) spinlock_profiling.cc:243] Waited 483 ms on lock 0x7b54007157d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:47.965376 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1109831040}
W20260709 13:07:48.013010 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 112991us
W20260709 13:07:48.015941 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 88524.4us
I20260709 13:07:48.020272 15825 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:48.007588 22702 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:48.025625 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:48.030485 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 226817us
I20260709 13:07:48.030615 22816 consensus_queue.cc:1048] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.021s
I20260709 13:07:48.031565 22825 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:48.036046 22550 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:48.036849 22550 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:48.037706 15827 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:48.039113 22550 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:48.031497 15902 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:48.024005 22675 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:48.051934 22675 raft_consensus.cc:697] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 20 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:48.052902 22675 consensus_queue.cc:237] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:48.040391 22508 consensus_queue.cc:1059] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 13:07:48.043542 22842 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:48.015456 22551 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:48.039659 22550 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:07:48.057197 16533 leader_election.cc:336] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 1.529s (SENT)
I20260709 13:07:48.056969 22550 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:48.057709 16533 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:48.058077 22551 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:48.059520 22550 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:48.060009 22527 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:48.041141 22610 raft_consensus.cc:2749] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:48.062597 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:48.063504 15857 raft_consensus.cc:2393] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 20: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:48.063725 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:48.022305 22820 raft_consensus.cc:2804] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:48.069746 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:48.044387 22650 raft_consensus.cc:2804] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:48.102139 22650 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:48.102011 22839 consensus_queue.cc:1048] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:48.103019 22650 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:48.081818 15902 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:07:48.087989 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 124959us
I20260709 13:07:48.091701 22821 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:48.095131 15860 raft_consensus.cc:1275] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:07:48.095893 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:48.097563 15859 raft_consensus.cc:1240] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 13:07:48.098446 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:48.065476 22825 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:48.110944 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:48.112572 22821 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:48.114930 22819 consensus_queue.cc:1048] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:07:48.117786 22819 consensus_queue.cc:1048] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:07:48.121114 15902 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:48.122560 22492 consensus_meta.cc:287] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.535s	user 0.011s	sys 0.001s
I20260709 13:07:48.122829 15902 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:48.067073 22816 raft_consensus.cc:2749] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:07:48.127743 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:48.071283 22820 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:48.141587 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:48.164661 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:48.108911 22825 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:48.172583 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:48.149096 22564 raft_consensus.cc:2749] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:48.149633 22820 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:48.179394 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:48.185916 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:48.186674 16568 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:48.189817 15859 raft_consensus.cc:2393] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 20: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:48.046797 22667 consensus_queue.cc:260] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 18.8, Last appended by leader: 8, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:48.192582 15862 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:48.123154 22492 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:48.171554 22825 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:48.176918 22848 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:48.198778 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:48.193994 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:48.196763 15902 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:48.190057 22842 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:48.187472 22839 raft_consensus.cc:2804] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:48.223471 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:48.223949 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 165009us
I20260709 13:07:48.202165 22848 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:48.222209 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 182205us
I20260709 13:07:48.236518 15937 raft_consensus.cc:2393] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 18: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:48.237017 15929 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 19: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:48.259152 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:48.228399 22839 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:48.236384 22848 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:07:48.269727 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 101919us
W20260709 13:07:48.269904 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 232300us
I20260709 13:07:48.270387 22702 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:48.271960 15928 raft_consensus.cc:2376] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 20. Re-sending same reply.
W20260709 13:07:48.272148 16533 leader_election.cc:336] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 2.074s (SENT)
I20260709 13:07:48.280656 16533 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:48.254739 22676 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:48.283231 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:48.295190 16002 raft_consensus.cc:1240] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:48.326018 15936 raft_consensus.cc:1240] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 18 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:48.269024 22839 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:48.272640 22791 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:48.331025 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:48.272532 22508 consensus_queue.cc:1059] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 13:07:48.371776 15928 raft_consensus.cc:2393] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 22: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:48.398343 15902 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:48.423880 15937 raft_consensus.cc:2410] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
W20260709 13:07:48.426676 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:48.397846 22816 raft_consensus.cc:2749] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:07:48.374320 22874 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:07:48.442163 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 170040us
W20260709 13:07:48.449963 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 119902us
I20260709 13:07:48.451309 16564 raft_consensus.cc:1275] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:07:48.451833 15827 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:48.332860 22702 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:48.452874 15827 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:48.454787 22551 raft_consensus.cc:2804] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:48.455214 22551 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:48.455566 22551 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:48.455871 22731 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:48.456317 22731 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:48.456665 22731 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:48.457238 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:48.460032 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:07:48.461360 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 63198.1us
I20260709 13:07:48.461731 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:48.462500 15934 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 17.
I20260709 13:07:48.268147 22842 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:48.438221 22675 raft_consensus.cc:2749] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:48.396389 22676 consensus_queue.cc:1048] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.044s
I20260709 13:07:48.480773 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:48.488881 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:48.489355 15928 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:48.494100 15825 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:48.501809 15862 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 21.
W20260709 13:07:48.503261 22793 consensus_meta.cc:287] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.659s	user 0.012s	sys 0.001s
W20260709 13:07:48.567878 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 87048.3us
W20260709 13:07:48.569476 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 66859.1us
I20260709 13:07:48.502609 22594 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:48.474402 22791 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:48.496053 22659 consensus_queue.cc:1048] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.077s
I20260709 13:07:48.503865 22463 consensus_queue.cc:1048] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.076s
W20260709 13:07:48.583750 22885 log.cc:927] Time spent T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:07:48.590976 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:48.591730 22679 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:48.592203 15930 raft_consensus.cc:1240] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 14. Current term is 15. Ops: []
W20260709 13:07:48.593693 22875 log.cc:927] Time spent T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
W20260709 13:07:48.621114 22854 log.cc:927] Time spent T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.653s	user 0.009s	sys 0.012s
I20260709 13:07:48.637835 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:48.638114 15931 raft_consensus.cc:1240] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 13:07:48.641808 22842 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:48.642056 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:48.647485 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 55943.6us
I20260709 13:07:48.653820 16568 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 21.
I20260709 13:07:48.663237 15853 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 17.
W20260709 13:07:48.659535 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 197071us
I20260709 13:07:48.664358 16561 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 19.
I20260709 13:07:48.673130 16533 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:48.504824 22874 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:48.643476 22594 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:48.574487 22793 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:48.665668 22551 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:48.715407 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:48.689847 22877 log.cc:927] Time spent T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.233s	user 0.000s	sys 0.001s
I20260709 13:07:48.722862 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:48.690369 22679 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:48.722334 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 49331.4us
I20260709 13:07:48.717828 15862 raft_consensus.cc:1275] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:07:48.716614 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:48.741901 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:48.743476 16533 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:48.744895 16568 raft_consensus.cc:1275] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 13. (term mismatch)
W20260709 13:07:48.697360 22650 consensus_meta.cc:287] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.594s	user 0.009s	sys 0.001s
I20260709 13:07:48.741703 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:48.745339 16568 pending_rounds.cc:85] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting all ops after (but not including) 12
I20260709 13:07:48.716871 22886 raft_consensus.cc:3055] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:07:48.751307 16568 pending_rounds.cc:107] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 13:07:48.707614 22594 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:48.742471 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:48.714962 22834 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:48.755230 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:48.747355 22875 log.cc:927] Time spent T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:07:48.717514 22492 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:48.758272 15976 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:48.756434 16568 raft_consensus.cc:2889] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:48.739559 22791 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:48.770265 16564 raft_consensus.cc:1240] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 22. Current term is 23. Ops: []
I20260709 13:07:48.764235 22803 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:48.776268 22892 consensus_queue.cc:1048] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:48.715649 22582 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:48.769683 22890 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:48.777865 22890 raft_consensus.cc:697] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:48.777727 22582 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:48.778918 22890 consensus_queue.cc:237] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:48.715101 22509 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:48.776379 22803 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:48.766674 16567 raft_consensus.cc:1275] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:07:48.789280 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:48.800001 16563 raft_consensus.cc:1275] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:07:48.800920 15928 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 20.
I20260709 13:07:48.803589 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:48.756157 22886 raft_consensus.cc:740] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:48.807041 15902 leader_election.cc:400] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 14. Current term is 15.
I20260709 13:07:48.808772 15936 raft_consensus.cc:2410] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:07:48.809960 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:48.779250 22848 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:48.808620 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:48.754803 22650 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:48.812916 15937 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
W20260709 13:07:48.831034 16008 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.009s	sys 0.013s
W20260709 13:07:48.832647 16008 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.010s	sys 0.013s
W20260709 13:07:48.831521 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 105846us
I20260709 13:07:48.837620 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:48.800212 22663 consensus_queue.cc:1048] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 13:07:48.781634 22731 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:48.802388 22882 consensus_queue.cc:1059] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 14, Time since last communication: 0.027s
I20260709 13:07:48.847165 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:48.844131 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:48.843235 22668 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:07:48.841650 22825 consensus_meta.cc:287] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.638s	user 0.009s	sys 0.002s
I20260709 13:07:48.849874 22594 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:48.850709 15933 raft_consensus.cc:1240] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:07:48.851269 22679 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:48.823827 22582 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:48.852877 22650 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:48.853041 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:48.854470 22803 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:48.859128 22874 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:48.879103 22865 log.cc:927] Time spent T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.600s	user 0.001s	sys 0.000s
I20260709 13:07:48.880330 16561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45246 (request call id 1472) took 1140 ms. Trace:
I20260709 13:07:48.886416 22551 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:48.890287 22842 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:48.896181 15937 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
W20260709 13:07:48.896360 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 159147us
I20260709 13:07:48.803221 22834 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:48.900943 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:48.907907 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 67084.6us
I20260709 13:07:48.823148 22509 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:07:48.846083 22816 raft_consensus.cc:2804] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:48.934198 16561 rpcz_store.cc:276] 0709 13:07:47.739791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:47.743339 (+  3548us) service_pool.cc:224] Handling call
0709 13:07:48.880100 (+1136761us) spinlock_profiling.cc:243] Waited 427 ms on lock 0x7b54004790d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:48.880245 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":982290304}
W20260709 13:07:48.935021 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 166406us
W20260709 13:07:48.925455 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 206546us
I20260709 13:07:48.902114 22825 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:48.952596 16011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 1344) took 1062 ms. Trace:
I20260709 13:07:48.926551 22582 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:48.953202 22825 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:48.940956 16005 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 19.
W20260709 13:07:48.937312 22839 consensus_meta.cc:287] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.535s	user 0.011s	sys 0.000s
I20260709 13:07:48.953814 22582 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:48.943009 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:48.941910 22834 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:48.955227 15928 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:48.945724 22842 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:48.948973 15902 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:48.949213 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:48.953086 22900 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:48.952979 16011 rpcz_store.cc:276] 0709 13:07:47.890054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:47.890591 (+   537us) service_pool.cc:224] Handling call
0709 13:07:47.890910 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:47.891364 (+   454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:47.891392 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:47.894694 (+  3302us) log.cc:844] Serialized 26 byte log entry
0709 13:07:48.107192 (+212498us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:48.107240 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:48.107318 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:48.878307 (+770989us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:07:48.952309 (+ 74002us) raft_consensus.cc:1685] finished
0709 13:07:48.952330 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:48.952532 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3058,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":101,"spinlock_wait_cycles":26739968,"thread_start_us":211669,"threads_started":1,"wal-append.queue_time_us":302012}
I20260709 13:07:48.958173 16533 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:48.954289 22839 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:48.736115 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:48.957798 15934 raft_consensus.cc:1275] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 13 index: 14. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
I20260709 13:07:48.954095 22825 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:48.968029 16533 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:48.971983 22650 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:48.949911 22816 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:48.978230 15937 raft_consensus.cc:1275] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:07:48.972458 22650 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:48.985266 16533 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:48.987615 15931 raft_consensus.cc:2393] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 17: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:48.981570 22816 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:49.010519 15937 raft_consensus.cc:1240] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:48.981485 22492 raft_consensus.cc:2749] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:48.988160 22650 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:48.987913 22509 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:48.971547 22825 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:49.172504 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 223583us
I20260709 13:07:48.976994 22882 consensus_queue.cc:1059] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.035s
W20260709 13:07:49.178936 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 250573us
I20260709 13:07:48.973702 22900 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:49.133900 22820 consensus_meta.cc:287] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.936s	user 0.009s	sys 0.003s
I20260709 13:07:49.186161 15937 raft_consensus.cc:1275] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 15. (term mismatch)
I20260709 13:07:49.188489 15856 raft_consensus.cc:1240] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:49.188565 15937 pending_rounds.cc:85] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 14
I20260709 13:07:49.192549 15937 pending_rounds.cc:107] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 20.15
I20260709 13:07:49.192826 15937 raft_consensus.cc:2889] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:49.187217 15827 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:49.197366 15934 raft_consensus.cc:1275] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:07:49.205498 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:49.192037 22820 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:49.217970 22880 consensus_queue.cc:1048] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.237s
I20260709 13:07:49.218173 22902 raft_consensus.cc:2804] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:49.218775 22902 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:49.219137 22902 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:49.219779 22820 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:49.135282 22904 consensus_queue.cc:1048] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.211s
I20260709 13:07:49.223587 22904 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:07:49.224282 22895 log.cc:927] Time spent T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.288s	user 0.000s	sys 0.001s
I20260709 13:07:49.224120 22904 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:49.224664 22569 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:49.225270 22569 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:49.194868 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:49.227444 16569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1517) took 1049 ms. Trace:
I20260709 13:07:49.230667 15860 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 19.
W20260709 13:07:49.217434 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 230710us
I20260709 13:07:49.238281 16533 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:49.189673 22527 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:49.241843 22088 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.242194 22889 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:49.241876 22527 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:49.242527 22088 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:49.242816 22889 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:49.244308 22582 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:49.228003 16569 rpcz_store.cc:276] 0709 13:07:48.178127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:48.189731 (+ 11604us) service_pool.cc:224] Handling call
0709 13:07:48.198136 (+  8405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:48.204106 (+  5970us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:48.204144 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:48.253339 (+ 49195us) log.cc:844] Serialized 26 byte log entry
0709 13:07:48.750074 (+496735us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:48.750128 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:48.750225 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:49.227198 (+476973us) raft_consensus.cc:1685] finished
0709 13:07:49.227222 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:49.227375 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":90,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":19116,"thread_start_us":410353,"threads_started":1,"wal-append.queue_time_us":549968}
I20260709 13:07:49.244958 15855 raft_consensus.cc:1240] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:49.244663 22088 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:07:49.274552 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 94418.5us
I20260709 13:07:49.288030 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:49.290081 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 110934us
I20260709 13:07:49.288918 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:49.298501 15928 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:49.293277 15825 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:07:49.306710 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 68549.2us
I20260709 13:07:49.230333 22793 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:49.313684 22905 raft_consensus.cc:2749] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:49.314105 22908 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:49.226282 22904 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:49.314697 22910 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:49.314706 22899 consensus_queue.cc:1059] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 13:07:49.320011 15857 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 19.
I20260709 13:07:49.310107 22731 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:49.289593 22848 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:49.329528 22569 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:49.329890 22914 log.cc:927] Time spent T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:07:49.244912 22889 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:49.330442 15827 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:49.331156 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 134378us
I20260709 13:07:49.417429 15902 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:49.418488 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 88172.7us
I20260709 13:07:49.421419 22873 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:49.336360 22902 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:49.292392 22492 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:49.378470 22793 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:07:49.382073 15929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 2944) took 1638 ms. Trace:
I20260709 13:07:49.382238 15935 raft_consensus.cc:1240] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:07:49.399232 22650 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:49.400388 22906 log.cc:927] Time spent T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:07:49.404472 22702 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:49.449350 22793 raft_consensus.cc:697] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:49.409147 22886 consensus_queue.cc:260] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:49.421335 22915 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.450117 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:49.450455 22915 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:49.392091 16533 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:49.451910 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:49.449383 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:49.452520 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 60525.8us
I20260709 13:07:49.452718 15857 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 18.
I20260709 13:07:49.453626 16533 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:07:49.425773 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 73507.9us
I20260709 13:07:49.355223 22908 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:49.427388 15929 rpcz_store.cc:276] 0709 13:07:47.746107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:47.746261 (+   154us) service_pool.cc:224] Handling call
0709 13:07:49.381802 (+1635541us) spinlock_profiling.cc:243] Waited 454 ms on lock 0x7b54001a81d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:49.381982 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1043142400}
I20260709 13:07:49.391287 22547 raft_consensus.cc:3055] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:49.436960 22676 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:07:49.384075 22839 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:49.426580 22842 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:49.440178 22492 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:49.424120 22564 raft_consensus.cc:2749] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:49.442031 22900 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:49.443775 22088 consensus_queue.cc:1048] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:49.389631 22825 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:49.470088 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:49.446689 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:49.471179 16006 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:49.446405 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:49.447286 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:49.421976 22873 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:49.447993 22603 consensus_queue.cc:1048] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:07:49.406625 22668 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:49.450412 22793 consensus_queue.cc:237] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:49.463323 22547 raft_consensus.cc:740] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:49.465955 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:49.476732 15934 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:49.466938 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:49.477394 16011 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 15.
I20260709 13:07:49.470227 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:49.478863 16002 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 14.
I20260709 13:07:49.444927 22689 consensus_queue.cc:1059] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:49.467763 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:49.456655 22702 raft_consensus.cc:2749] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:49.446365 22816 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:49.244717 22527 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:49.464439 22910 raft_consensus.cc:697] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 20 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:49.463353 22908 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:49.473927 22873 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:49.482089 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:49.483124 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:49.486122 15929 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:49.493093 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:49.475814 22668 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:49.477681 22825 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.508075 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:49.510468 15862 raft_consensus.cc:2393] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 19: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:49.500010 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:49.477500 22918 raft_consensus.cc:2804] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:49.516104 16003 raft_consensus.cc:1275] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 13:07:49.497325 22824 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.034s	sys 0.061s
I20260709 13:07:49.521595 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:49.519232 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:49.522832 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:49.522945 15862 raft_consensus.cc:1240] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 15. Current term is 16. Ops: [15.15-15.15]
I20260709 13:07:49.524170 16006 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 15.
I20260709 13:07:49.502230 22915 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:49.524817 15857 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 16.
I20260709 13:07:49.524906 15854 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:49.525362 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:49.526566 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:49.528738 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:49.530454 16005 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:49.531490 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:49.536367 15827 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:49.537246 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:49.516700 22564 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:49.540009 15853 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:07:49.524356 22824 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.785s	user 0.035s	sys 0.062s
I20260709 13:07:49.540784 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:49.517086 22918 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:49.500052 22825 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:49.546311 16568 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 17.
I20260709 13:07:49.549216 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:49.550021 16570 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 18.
I20260709 13:07:49.555943 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:49.556725 16004 raft_consensus.cc:1275] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 13:07:49.558992 16005 raft_consensus.cc:1275] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:07:49.559616 16561 raft_consensus.cc:1240] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:49.565090 15934 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 18.
I20260709 13:07:49.571498 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:49.572755 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:49.501258 22547 consensus_queue.cc:260] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:49.576722 22547 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:49.580292 15858 raft_consensus.cc:3055] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:07:49.580698 22910 consensus_queue.cc:237] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:49.581702 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:49.548427 22564 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:49.582878 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:49.584128 22564 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:49.583972 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:49.585328 15935 raft_consensus.cc:2393] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 15: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:49.581844 16562 raft_consensus.cc:1240] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 18. Current term is 19. Ops: [18.16-18.16]
I20260709 13:07:49.583904 16002 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:49.580701 15858 raft_consensus.cc:740] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:49.587498 15974 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:07:49.590752 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 52275.2us
I20260709 13:07:49.588932 15932 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:49.593219 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:49.594874 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:49.597276 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:49.597880 16567 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:49.607667 16567 raft_consensus.cc:2410] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:07:49.608925 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:49.609772 16567 raft_consensus.cc:2393] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 21: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:49.590364 22650 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:49.611534 15854 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 21.
I20260709 13:07:49.551164 22915 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:49.611909 16569 raft_consensus.cc:1240] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:49.613694 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:49.591454 16535 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:49.613829 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:49.547780 22918 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:49.558786 22900 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:49.619901 16567 raft_consensus.cc:2393] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 17: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:49.625164 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:49.627475 22803 consensus_queue.cc:1059] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:07:49.628098 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 107070us
I20260709 13:07:49.628409 22803 raft_consensus.cc:3055] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:07:49.629451 22803 raft_consensus.cc:740] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:49.630313 22803 consensus_queue.cc:260] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.17, Last appended by leader: 17, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:49.630914 15900 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.506s (SENT) [suppressed 22 similar messages]
W20260709 13:07:49.633764 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:49.635501 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 44120.8us
I20260709 13:07:49.640661 22904 raft_consensus.cc:2804] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:49.641237 22904 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:49.641786 22904 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:49.620438 15858 consensus_queue.cc:260] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:49.644220 15858 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:49.631543 22803 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:49.651247 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:49.652611 15976 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:49.652817 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:49.619968 22898 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:49.628966 15827 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:49.666512 15853 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 19.
I20260709 13:07:49.669929 15827 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:49.677134 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:49.634100 22824 raft_consensus.cc:3055] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:49.625664 22882 consensus_queue.cc:1048] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 13:07:49.623031 22915 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:49.668831 22890 raft_consensus.cc:2804] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:07:49.715276 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:49.668452 22702 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:49.719437 15929 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 19.
W20260709 13:07:49.714027 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 83133us
I20260709 13:07:49.695500 22663 consensus_queue.cc:1059] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.014s
I20260709 13:07:49.640043 22509 consensus_queue.cc:1059] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.064s
I20260709 13:07:49.696614 22492 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:49.685642 22898 raft_consensus.cc:697] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:49.699249 22912 raft_consensus.cc:2749] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:49.709857 22755 raft_consensus.cc:2749] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:49.715214 22890 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:49.687692 22824 raft_consensus.cc:740] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:49.729075 16002 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 19.
I20260709 13:07:49.732403 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:49.741034 16569 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:49.737320 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:49.742305 15929 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:49.737983 16562 raft_consensus.cc:1240] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 22. Current term is 23. Ops: []
I20260709 13:07:49.731240 16567 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 18.
W20260709 13:07:49.746421 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:49.749162 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 86056us
I20260709 13:07:49.750967 15974 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:49.740506 22890 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:49.725956 22702 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:49.761178 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 91343.1us
I20260709 13:07:49.761945 15827 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:49.770581 15827 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:49.772847 15827 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:49.762699 22825 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:49.778901 22874 raft_consensus.cc:2749] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:49.788750 15827 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:49.779348 22755 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:49.784118 22882 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:49.806221 15929 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 21.
I20260709 13:07:49.813349 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:49.814531 22874 raft_consensus.cc:2804] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:07:49.814831 22924 log.cc:927] Time spent T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
W20260709 13:07:49.815259 22923 log.cc:927] Time spent T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 13:07:49.816473 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:49.817616 22925 log.cc:927] Time spent T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:07:49.819747 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:49.820580 15929 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 21.
I20260709 13:07:49.821704 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:49.823321 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:49.824051 15929 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:07:49.825176 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:49.827503 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:49.829011 15929 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 20.
I20260709 13:07:49.830650 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:49.832482 15929 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:49.834440 22668 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:49.796274 22898 consensus_queue.cc:237] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:49.843943 16007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 1352) took 1669 ms. Trace:
I20260709 13:07:49.847543 22915 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:49.847851 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:49.810833 22824 consensus_queue.cc:260] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:49.850517 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:49.854141 22702 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:49.855366 15935 raft_consensus.cc:1275] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:07:49.811393 22882 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:49.857427 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:49.857790 16007 rpcz_store.cc:276] 0709 13:07:48.174280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:48.174414 (+   134us) service_pool.cc:224] Handling call
0709 13:07:48.185637 (+ 11223us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:48.216349 (+ 30712us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:48.216377 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:48.868141 (+651764us) spinlock_profiling.cc:243] Waited 3.41 ms on lock 0x7b480021c258. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ef2d1aa1 00007f73ef2c8b37 00007f73ef2c89ca 00007f73ef3c026f 00007f73ef3bff37 00007f73ef3d455c 00007f73ef3d4508 00007f73ef3d4488 00007f73ef3d4443 00007f73ef3d36ec 00007f73ee2fda96
0709 13:07:49.541391 (+673250us) spinlock_profiling.cc:243] Waited 4.76 ms on lock 0x7b2c000e5768. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e8ca1a11 00007f73e8cb1372 00007f73ea3e3a01 00007f73ea3a843a 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2
0709 13:07:49.558333 (+ 16942us) log.cc:844] Serialized 26 byte log entry
0709 13:07:49.754053 (+195720us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:49.754102 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:49.754163 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:49.843735 (+ 89572us) raft_consensus.cc:1685] finished
0709 13:07:49.843755 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:49.843885 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19133,"raft.queue_time_us":26066,"raft.run_cpu_time_us":2229,"raft.run_wall_time_us":641328,"spinlock_wait_cycles":18790528,"thread_start_us":187068,"threads_started":1,"wal-append.queue_time_us":273148}
I20260709 13:07:49.873452 16570 raft_consensus.cc:1240] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:49.874751 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:49.862504 22874 raft_consensus.cc:697] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 19 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:49.876022 15937 raft_consensus.cc:1275] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:07:49.870972 22882 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:07:49.879724 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 226117us
I20260709 13:07:49.883668 15976 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:49.886909 15929 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:49.878221 22824 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:49.896261 22874 consensus_queue.cc:237] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:49.858448 22873 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:49.888203 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 99534.7us
I20260709 13:07:49.910979 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:49.913079 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:49.918704 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:49.919309 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:49.919992 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:49.919931 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:49.920651 16004 raft_consensus.cc:1240] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:49.889183 22938 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:49.921949 22938 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:49.922366 22938 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:49.936964 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:49.940214 15858 raft_consensus.cc:1240] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:49.888445 22910 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:49.944788 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:49.941378 22873 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:49.950448 22908 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:49.952690 15929 raft_consensus.cc:1275] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:07:49.952776 16007 raft_consensus.cc:1240] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
W20260709 13:07:49.954051 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 198809us
I20260709 13:07:49.954819 16566 raft_consensus.cc:1240] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:07:49.963547 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:49.967078 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:49.969985 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:49.970676 16002 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:49.972920 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:49.973802 16565 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:07:49.953938 22945 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:49.979786 15900 leader_election.cc:336] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.862s (SENT)
I20260709 13:07:49.980634 15900 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:49.987234 15937 raft_consensus.cc:1240] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:49.976773 22904 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:50.012223 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:50.013708 22492 raft_consensus.cc:2749] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:50.013990 22904 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:50.015586 22876 log.cc:927] Time spent T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 1.760s	user 0.007s	sys 0.007s
I20260709 13:07:50.017701 22668 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:49.955996 22918 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:50.020357 22918 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:50.021068 16561 raft_consensus.cc:1240] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:50.023536 22882 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:50.035262 22755 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:50.041672 22908 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:50.043313 22900 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:49.979663 22945 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:50.116739 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 86401.8us
I20260709 13:07:50.156368 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:07:50.160893 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:50.166626 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:50.189204 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:50.190171 15937 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 18.
I20260709 13:07:50.191180 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:50.191861 15937 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 14.
I20260709 13:07:50.192765 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:50.197000 16565 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 19.
I20260709 13:07:50.208242 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:50.169567 15902 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:50.226303 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:50.244305 15859 raft_consensus.cc:1240] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 14. Current term is 15. Ops: []
W20260709 13:07:50.252697 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 277493us
I20260709 13:07:50.259774 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:50.259586 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:50.264359 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:50.265898 16563 raft_consensus.cc:2393] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 16: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
W20260709 13:07:50.260618 15900 leader_election.cc:336] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.907s (SENT)
I20260709 13:07:50.267869 15900 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:50.282500 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:50.283361 15861 raft_consensus.cc:2393] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 16: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:50.284456 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:50.285066 15853 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 16.
I20260709 13:07:50.285856 16002 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:50.169121 22938 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:50.292136 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:50.292586 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:50.293865 22945 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:50.294376 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:50.314002 15937 raft_consensus.cc:1240] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:50.297314 15860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 2818) took 1114 ms. Trace:
I20260709 13:07:50.314416 15860 rpcz_store.cc:276] 0709 13:07:49.183396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.183584 (+   188us) service_pool.cc:224] Handling call
0709 13:07:50.297167 (+1113583us) spinlock_profiling.cc:243] Waited 2.53 ms on lock 0x7b54004263d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:50.297261 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5822720}
I20260709 13:07:50.280683 22803 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:50.316056 16565 raft_consensus.cc:1240] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:50.316489 15936 raft_consensus.cc:1275] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:07:50.315232 15937 raft_consensus.cc:1275] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:07:50.317212 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:50.317852 15861 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 15.
I20260709 13:07:50.319135 22803 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:50.322184 22803 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:50.324728 22946 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:50.324891 22492 raft_consensus.cc:2749] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:50.325292 22946 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:50.328190 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:07:50.329274 22890 consensus_meta.cc:287] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.544s	user 0.009s	sys 0.000s
I20260709 13:07:50.308295 22898 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:50.338162 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:50.339800 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:50.340750 22938 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:50.342559 22689 raft_consensus.cc:3055] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:07:50.349179 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:50.329717 22890 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:50.360558 15853 raft_consensus.cc:1275] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:07:50.356794 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:50.380081 16002 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 19.
I20260709 13:07:50.357808 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:50.360735 15862 raft_consensus.cc:1275] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:07:50.397500 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:50.399616 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:50.410315 15859 raft_consensus.cc:2368] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 18. Current term is 19.
I20260709 13:07:50.410506 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:50.411124 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:50.354568 22816 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:50.351908 22882 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:50.414294 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:50.413725 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:07:50.415136 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 154549us
I20260709 13:07:50.379834 22689 raft_consensus.cc:740] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:50.349546 22898 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:50.416234 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:50.418298 15853 raft_consensus.cc:1240] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:50.419744 15974 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:50.420655 22689 consensus_queue.cc:260] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:50.421371 22943 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:50.421658 15853 pending_rounds.cc:85] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8: Aborting all ops after (but not including) 14
I20260709 13:07:50.422093 15853 pending_rounds.cc:107] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8: Aborting uncommitted NO_OP operation due to leader change: 20.15
I20260709 13:07:50.420698 15859 raft_consensus.cc:1240] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:07:50.422374 15853 raft_consensus.cc:2889] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:50.421722 22689 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:50.425802 22943 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:50.426179 22943 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:50.413233 22946 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:50.429083 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:50.430918 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:50.437695 15861 raft_consensus.cc:1240] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 17. Current term is 18. Ops: []
W20260709 13:07:50.446139 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 276706us
W20260709 13:07:50.448065 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:50.448546 15862 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 15.
I20260709 13:07:50.449303 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:50.449632 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:50.450191 15862 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:50.450248 16002 raft_consensus.cc:2368] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 19. Current term is 20.
I20260709 13:07:50.451386 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:50.448150 15902 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:50.455334 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:50.461818 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 164292us
W20260709 13:07:50.446852 22868 log.cc:927] Time spent T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 2.049s	user 0.001s	sys 0.020s
I20260709 13:07:50.456179 16007 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 18.
I20260709 13:07:50.466216 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:50.467919 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:50.471323 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:50.473817 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:50.476055 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:50.477545 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:50.478142 15859 raft_consensus.cc:1240] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 20 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:50.479959 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:50.481030 16002 raft_consensus.cc:2410] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 21 because replica has last-logged OpId of term: 20 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 16.
I20260709 13:07:50.468736 16009 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 20.
I20260709 13:07:50.481561 15860 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 15.
I20260709 13:07:50.484875 15859 raft_consensus.cc:1240] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:07:50.486909 15859 pending_rounds.cc:85] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8: Aborting all ops after (but not including) 13
I20260709 13:07:50.487167 15859 pending_rounds.cc:107] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8: Aborting uncommitted NO_OP operation due to leader change: 16.14
I20260709 13:07:50.487411 15859 raft_consensus.cc:2889] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:50.488759 16009 raft_consensus.cc:1240] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:07:50.495769 15860 raft_consensus.cc:1240] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 18. Current term is 19. Ops: [18.16-18.16]
I20260709 13:07:50.502214 15862 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 19.
I20260709 13:07:50.468400 22953 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:50.505787 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:50.509420 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:50.513322 22953 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:50.513749 22953 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:50.514114 22953 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:50.511626 22888 log.cc:927] Time spent T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 1.801s	user 0.013s	sys 0.025s
I20260709 13:07:50.522141 22953 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:50.507853 22955 consensus_queue.cc:1048] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 13:07:50.500805 22689 consensus_queue.cc:1059] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 13:07:50.525532 22890 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:50.525872 22689 raft_consensus.cc:3055] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:50.526324 22689 raft_consensus.cc:740] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:50.527148 22689 consensus_queue.cc:260] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:50.532482 22689 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:50.532892 15900 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:50.534090 22955 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:07:50.540446 15931 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53636 (request call id 1368) took 1873 ms (client timeout 2114 ms). Trace:
I20260709 13:07:50.538033 22492 consensus_queue.cc:1048] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 13:07:50.535661 22943 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:50.543900 15931 rpcz_store.cc:269] 0709 13:07:48.666772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:48.723061 (+ 56289us) service_pool.cc:224] Handling call
0709 13:07:50.540247 (+1817186us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b54001a7cd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:50.540383 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":513313536}
I20260709 13:07:50.535806 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:50.545542 22943 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:50.545632 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:50.547855 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:50.537165 22955 consensus_queue.cc:1048] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 13:07:50.549281 22582 raft_consensus.cc:3055] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:50.549662 22582 raft_consensus.cc:740] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:50.550482 22582 consensus_queue.cc:260] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:50.550940 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:50.551478 22582 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:50.551456 15862 raft_consensus.cc:2376] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 15. Re-sending same reply.
I20260709 13:07:50.560024 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:50.560691 15859 raft_consensus.cc:2376] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 15. Re-sending same reply.
I20260709 13:07:50.561445 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:50.564078 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:50.564076 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:50.566203 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:50.566752 15862 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 16.
W20260709 13:07:50.567147 15827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:50.567447 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:50.562188 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 114092us
I20260709 13:07:50.568341 15902 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:50.568879 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:50.569490 15860 raft_consensus.cc:3060] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:50.572692 22825 consensus_queue.cc:1059] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:07:50.573776 22874 raft_consensus.cc:3055] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:50.574172 22874 raft_consensus.cc:740] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:50.575049 22874 consensus_queue.cc:260] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:50.576686 22874 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:50.580694 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:50.582566 15827 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:50.585386 15827 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:50.540975 22953 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:50.588923 15827 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:50.570524 22955 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:50.592780 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:50.589610 22550 raft_consensus.cc:2804] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:50.571815 22492 consensus_queue.cc:1059] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.021s
I20260709 13:07:50.617548 22550 raft_consensus.cc:697] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:50.590875 22564 consensus_queue.cc:1059] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:07:50.587770 22965 consensus_queue.cc:1059] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.035s
I20260709 13:07:50.618885 22550 consensus_queue.cc:237] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:50.619771 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:50.620054 22965 raft_consensus.cc:3055] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:07:50.620442 22965 raft_consensus.cc:740] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:50.620675 16010 raft_consensus.cc:2393] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 16: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:50.618969 22765 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:50.621312 22965 consensus_queue.cc:260] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:50.622016 16010 raft_consensus.cc:1275] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:07:50.622013 22765 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:50.618575 22492 raft_consensus.cc:3055] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:07:50.594352 22913 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:50.625852 22913 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:50.624013 22492 raft_consensus.cc:740] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:50.626214 22913 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:50.623099 22955 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:50.626878 22492 consensus_queue.cc:260] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:50.628203 22492 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:50.628638 22955 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:50.629215 22955 raft_consensus.cc:3055] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:50.629542 22955 raft_consensus.cc:740] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:50.623266 22953 raft_consensus.cc:2804] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:50.630239 22953 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:50.630599 22953 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:50.638561 22902 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:50.638635 22955 consensus_queue.cc:260] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:50.641152 22955 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:50.644526 15853 raft_consensus.cc:2410] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:07:50.646971 15827 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:50.648276 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 66012.7us
I20260709 13:07:50.651513 15860 raft_consensus.cc:2468] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 17.
I20260709 13:07:50.622727 22765 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:50.660279 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:50.662674 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:50.662858 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:50.664706 15858 raft_consensus.cc:1240] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:07:50.666445 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:50.666947 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:50.660300 22874 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:50.667917 22508 consensus_queue.cc:1048] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:07:50.669500 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:50.664036 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 95743.1us
I20260709 13:07:50.675539 15902 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:50.680374 22594 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:50.689530 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:50.694442 22953 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:50.698084 22913 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:50.702991 22881 raft_consensus.cc:3055] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:50.703601 22881 raft_consensus.cc:740] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:50.708276 15902 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:50.708921 22971 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:50.714138 22971 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:50.716400 22971 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:50.709087 22970 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:07:50.700935 22972 consensus_queue.cc:1059] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.041s
I20260709 13:07:50.719602 16004 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 19.
I20260709 13:07:50.720424 16566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1531) took 1174 ms. Trace:
I20260709 13:07:50.720806 16566 rpcz_store.cc:276] 0709 13:07:49.545982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.546768 (+   786us) service_pool.cc:224] Handling call
0709 13:07:49.547078 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:49.955188 (+408110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:50.720166 (+764978us) spinlock_profiling.cc:243] Waited 407 ms on lock 0x7b54005654d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:50.720365 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":937195264}
I20260709 13:07:50.722195 16570 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1544) took 1075 ms. Trace:
I20260709 13:07:50.722601 16570 rpcz_store.cc:276] 0709 13:07:49.647034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.817639 (+170605us) service_pool.cc:224] Handling call
0709 13:07:49.817925 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:49.877006 (+ 59081us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:50.721989 (+844983us) spinlock_profiling.cc:243] Waited 54.4 ms on lock 0x7b5400695ad4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:50.722142 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":125140736}
I20260709 13:07:50.723958 22881 consensus_queue.cc:260] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:50.724622 15932 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 2970) took 1264 ms (client timeout 1622 ms). Trace:
I20260709 13:07:50.725274 22881 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:50.727500 15935 raft_consensus.cc:1275] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:07:50.725000 15932 rpcz_store.cc:269] 0709 13:07:49.460546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.572508 (+111962us) service_pool.cc:224] Handling call
0709 13:07:50.724449 (+1151941us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b5400099cd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:50.724555 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35923328}
W20260709 13:07:50.730319 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 64573.5us
I20260709 13:07:50.731112 15827 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:50.736317 22551 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:50.740825 22492 consensus_queue.cc:1048] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:50.742398 22970 consensus_queue.cc:1048] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:07:50.743144 22874 raft_consensus.cc:2749] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:50.744258 22886 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:50.745694 22551 raft_consensus.cc:697] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 19 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:50.749931 15974 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:50.751613 15974 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:50.751989 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:50.733096 22971 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:50.756829 22582 raft_consensus.cc:2749] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:50.759702 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:50.760324 16011 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:07:50.767292 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:50.767899 16002 raft_consensus.cc:1240] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:50.765019 15900 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:50.768518 15974 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:50.725129 22765 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:50.755291 22881 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:50.770354 15974 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:50.774804 15900 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:50.775944 16002 raft_consensus.cc:1240] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:50.755182 22913 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:50.777456 15974 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:50.778342 15974 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:50.779340 16533 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:50.782582 16007 raft_consensus.cc:1240] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 18. Current term is 19. Ops: [18.11-18.11]
W20260709 13:07:50.785804 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:50.786238 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:50.787395 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:50.787985 15853 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:50.791986 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:50.792605 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:50.793337 15856 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 2827) took 1348 ms (client timeout 1754 ms). Trace:
W20260709 13:07:50.793684 15856 rpcz_store.cc:269] 0709 13:07:49.445379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.449472 (+  4093us) service_pool.cc:224] Handling call
0709 13:07:50.792458 (+1342986us) spinlock_profiling.cc:243] Waited 2.48 ms on lock 0x7b5800005798. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e7283a22 00007f73f019cc09 00007f73ef2f6ee0 00007f73ef3c00ab 00007f73ef3c0131 00007f73ef36e0a0 00007f73ef36d984 00007f73e88e86d4 00007f73e88e8662
0709 13:07:50.793286 (+   828us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5699968}
I20260709 13:07:50.794282 15827 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:50.795856 22755 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:50.796715 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:50.795857 15861 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 19.
I20260709 13:07:50.797977 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:50.798496 16566 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:50.805681 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:50.806061 16566 raft_consensus.cc:2410] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 21 because replica has last-logged OpId of term: 20 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 16.
I20260709 13:07:50.806435 16569 raft_consensus.cc:2393] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 19: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:50.807610 15827 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:50.808876 15827 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:50.809547 22508 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:50.809984 15827 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:50.810297 22508 raft_consensus.cc:2749] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:50.812428 22679 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:50.812948 22679 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:50.814072 15827 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:50.814797 22755 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:50.815217 22679 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:50.815315 22755 raft_consensus.cc:697] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 19 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:50.815742 22508 raft_consensus.cc:2749] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:07:50.813279 15860 raft_consensus.cc:1275] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:07:50.824916 15827 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:50.833067 15853 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 20.
I20260709 13:07:50.762079 22663 consensus_queue.cc:1048] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 13:07:50.836531 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:50.766762 22582 consensus_queue.cc:1048] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.070s
I20260709 13:07:50.780682 22955 raft_consensus.cc:2749] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:07:50.837674 22976 log.cc:927] Time spent T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
W20260709 13:07:50.838747 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 59468.9us
I20260709 13:07:50.839496 16533 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:50.839895 22953 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:50.837198 22582 consensus_queue.cc:1243] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer 8430db80a39d4868bcf7641c16de35ea log is divergent from this leader: its last log entry 19.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:07:50.836156 22921 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:50.772107 22803 raft_consensus.cc:2749] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:50.841109 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:50.851395 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:50.841415 15857 raft_consensus.cc:1240] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:07:50.767576 22551 consensus_queue.cc:237] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:50.842849 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:50.767372 22902 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:50.846951 16533 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:50.847110 22921 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:50.773290 22881 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:50.857416 15902 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:50.848841 22088 consensus_queue.cc:1048] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.012s
I20260709 13:07:50.764353 22889 consensus_queue.cc:1048] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.076s
I20260709 13:07:50.861673 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:50.818908 22603 raft_consensus.cc:2804] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:50.845048 22689 raft_consensus.cc:2749] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:50.867821 22881 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:50.867359 16007 raft_consensus.cc:1240] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:07:50.872675 15902 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:50.874882 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 100118us
I20260709 13:07:50.830152 22943 raft_consensus.cc:2804] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:50.830545 22971 raft_consensus.cc:697] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 15 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:50.852936 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:50.878624 16563 raft_consensus.cc:1240] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:50.819119 22791 consensus_queue.cc:1059] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.054s
W20260709 13:07:50.892606 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 67761us
I20260709 13:07:50.892877 15902 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:50.885003 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 60227us
I20260709 13:07:50.892745 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:50.887305 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:50.845705 22880 consensus_queue.cc:1048] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:07:50.895439 15902 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:50.897403 16535 leader_election.cc:400] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 election: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 19. Current term is 20.
I20260709 13:07:50.897855 16535 leader_election.cc:403] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:07:50.866493 22980 raft_consensus.cc:2749] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:50.893605 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:50.874995 22603 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:50.900122 16010 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 21.
I20260709 13:07:50.867221 22957 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:07:50.902612 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:50.903815 15902 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:50.881729 22921 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:50.907888 22957 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:50.881786 22943 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:50.909883 22965 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:50.910468 22603 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:50.895845 22953 raft_consensus.cc:2749] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:07:50.903069 22955 raft_consensus.cc:2749] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:50.913800 22564 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:50.945640 22564 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:50.946048 22564 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:50.946933 22881 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:50.919811 22972 raft_consensus.cc:2749] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:50.920140 22676 raft_consensus.cc:3055] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:50.924245 22679 raft_consensus.cc:3055] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:07:50.949195 22676 raft_consensus.cc:740] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:50.924477 22971 consensus_queue.cc:237] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:50.929144 16562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1536) took 1337 ms. Trace:
I20260709 13:07:50.958904 16562 rpcz_store.cc:276] 0709 13:07:49.592050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.592163 (+   113us) service_pool.cc:224] Handling call
0709 13:07:49.592449 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:49.746352 (+153903us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:50.928812 (+1182460us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b54007017d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:50.929063 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":328850816}
I20260709 13:07:50.903993 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:50.932621 16570 raft_consensus.cc:1240] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:50.909309 15933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2949) took 2620 ms. Trace:
W20260709 13:07:50.936683 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 55651.5us
I20260709 13:07:50.892081 22492 raft_consensus.cc:2749] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:50.866822 22902 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:50.960637 15933 rpcz_store.cc:276] 0709 13:07:48.288783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:48.411920 (+123137us) service_pool.cc:224] Handling call
0709 13:07:48.460805 (+ 48885us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:48.902005 (+441200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:50.909045 (+2007040us) spinlock_profiling.cc:243] Waited 376 ms on lock 0x7b540034a654. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:50.909230 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":864383872}
I20260709 13:07:50.961184 15900 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:50.933544 16535 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:50.965796 16565 raft_consensus.cc:1275] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:07:50.937984 22791 raft_consensus.cc:3060] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:50.968508 15900 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:50.983341 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:50.986306 16567 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45232 (request call id 1539) took 1366 ms (client timeout 1754 ms). Trace:
I20260709 13:07:50.990002 22791 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 19. Current term is 20.
W20260709 13:07:50.986996 16567 rpcz_store.cc:269] 0709 13:07:49.619852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.624869 (+  5017us) service_pool.cc:224] Handling call
0709 13:07:50.986021 (+1361152us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b5400102554. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:50.986224 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":241856256}
I20260709 13:07:50.949713 22943 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:07:50.995210 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 61777.7us
W20260709 13:07:51.000334 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 106377us
I20260709 13:07:51.001541 15825 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:51.007140 15827 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:50.990085 22898 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:50.950270 22679 raft_consensus.cc:740] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:51.004690 22957 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:51.010174 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 163284us
I20260709 13:07:51.010334 22667 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:51.011940 22881 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:50.904569 22755 consensus_queue.cc:237] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:51.044597 22902 leader_election.cc:290] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:51.045550 22874 raft_consensus.cc:2749] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:07:51.046203 22874 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.046140 15825 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:51.046775 22874 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:51.047441 15825 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:51.048292 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:51.050246 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:50.980144 22980 raft_consensus.cc:2804] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:51.052240 22980 raft_consensus.cc:697] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 17 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:51.052788 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:51.053417 15861 raft_consensus.cc:1240] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:51.053231 22980 consensus_queue.cc:237] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:51.016821 16011 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 21.
I20260709 13:07:51.025146 22679 consensus_queue.cc:260] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:50.993815 22867 consensus_queue.cc:1059] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 13:07:51.009344 22955 raft_consensus.cc:2749] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:51.008134 22953 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:51.005349 22913 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:51.031561 22603 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:07:51.027112 22943 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:51.047070 22899 raft_consensus.cc:2804] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:07:51.097579 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 77750.9us
I20260709 13:07:50.994266 22972 consensus_queue.cc:1059] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.076s
I20260709 13:07:51.098402 16533 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:51.058400 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:51.098714 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 61802.5us
I20260709 13:07:50.951399 22676 consensus_queue.cc:260] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.16, Last appended by leader: 16, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:51.100004 15976 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:51.100488 22676 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:51.100862 22582 raft_consensus.cc:2749] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:51.102058 15976 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:51.102415 22803 consensus_queue.cc:1048] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:07:51.023643 22765 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:51.103643 22943 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:51.105319 15854 raft_consensus.cc:2410] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 19 because replica has last-logged OpId of term: 18 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 13:07:51.015707 22659 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:51.104056 22985 raft_consensus.cc:2804] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:51.106292 22985 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:51.021157 22492 consensus_queue.cc:1059] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.085s
I20260709 13:07:51.106724 22985 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:51.107067 22492 raft_consensus.cc:3055] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:51.107476 22492 raft_consensus.cc:740] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:51.104203 22953 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:51.108350 22492 consensus_queue.cc:260] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:51.109488 22492 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:51.109840 22953 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:51.104744 22603 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:51.104974 22509 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:51.112720 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:51.112937 22509 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:51.113359 22509 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:51.113500 16004 raft_consensus.cc:2393] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 20: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:51.064800 16563 raft_consensus.cc:2468] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 20.
I20260709 13:07:51.011658 22689 consensus_queue.cc:1059] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.103s
I20260709 13:07:51.115783 22689 raft_consensus.cc:3055] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:07:51.116186 22689 raft_consensus.cc:740] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:51.016970 22663 consensus_queue.cc:1059] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 13:07:51.090508 22957 consensus_queue.cc:1059] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.028s
I20260709 13:07:51.118193 22663 raft_consensus.cc:3055] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:51.118635 22663 raft_consensus.cc:740] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:51.118661 22953 raft_consensus.cc:3055] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:07:51.119289 22953 raft_consensus.cc:740] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:51.119656 22663 consensus_queue.cc:260] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:51.121069 22663 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:51.120184 22953 consensus_queue.cc:260] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:51.122170 22953 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:51.124238 22867 raft_consensus.cc:3055] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:51.124678 22867 raft_consensus.cc:740] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:51.098824 22972 raft_consensus.cc:3055] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:51.125387 22972 raft_consensus.cc:740] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:51.125500 22867 consensus_queue.cc:260] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:51.126236 22972 consensus_queue.cc:260] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:51.126765 22867 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:51.127851 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:51.127468 22972 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:07:51.141849 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:51.091559 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:51.091969 22945 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.093053 22874 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:51.096112 22679 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:51.098109 16005 raft_consensus.cc:1240] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:51.053750 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:51.099604 16533 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:51.117043 22689 consensus_queue.cc:260] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:51.152457 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 248698us
I20260709 13:07:51.166720 15856 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 19.
W20260709 13:07:51.170140 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 70603.3us
I20260709 13:07:51.171111 16533 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:51.188105 15902 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:51.152846 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:51.189308 15902 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:51.190510 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:51.146497 15976 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:07:51.192595 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 46184.5us
W20260709 13:07:51.187373 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 178955us
I20260709 13:07:51.201442 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:51.205538 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:51.201778 15974 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:51.197518 16535 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:51.169370 22594 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:51.192011 22899 raft_consensus.cc:697] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:51.205152 22985 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:51.231899 22997 raft_consensus.cc:2749] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:51.232568 22899 consensus_queue.cc:237] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:51.233892 22985 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:51.234519 22985 raft_consensus.cc:2749] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:51.211326 15855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43524 (request call id 2823) took 1850 ms. Trace:
W20260709 13:07:51.236159 22993 log.cc:927] Time spent T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:07:51.237335 22791 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:51.236366 15855 rpcz_store.cc:276] 0709 13:07:49.349251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:49.349545 (+   294us) service_pool.cc:224] Handling call
0709 13:07:49.352061 (+  2516us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:49.385846 (+ 33785us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:49.385896 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:49.397604 (+ 11708us) log.cc:844] Serialized 26 byte log entry
0709 13:07:50.810827 (+1413223us) spinlock_profiling.cc:243] Waited 24.3 ms on lock 0x7b180040eac0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f480d1 00007f73ea363a0f
0709 13:07:51.153548 (+342721us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:51.153602 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:51.153693 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:51.153751 (+    58us) raft_consensus.cc:1685] finished
0709 13:07:51.153768 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:51.199468 (+ 45700us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32522,"raft.queue_time_us":32682,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":3183,"spinlock_wait_cycles":55891200,"thread_start_us":342490,"threads_started":1,"wal-append.queue_time_us":1730336}
I20260709 13:07:51.212667 22955 raft_consensus.cc:2749] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:51.238397 16533 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:51.207775 22582 raft_consensus.cc:2804] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:51.239601 22582 raft_consensus.cc:697] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:51.241063 22902 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:51.240797 22582 consensus_queue.cc:237] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:51.212687 15902 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:51.228114 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:51.243273 16562 raft_consensus.cc:1240] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:51.214407 15934 raft_consensus.cc:1275] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:07:51.243177 15902 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:51.213794 15931 raft_consensus.cc:1240] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 14. Current term is 15. Ops: []
W20260709 13:07:51.246263 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:51.246402 15902 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:51.249847 15854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 2637) took 1065 ms. Trace:
I20260709 13:07:51.250079 15859 raft_consensus.cc:1275] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 18 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 13:07:51.250207 15854 rpcz_store.cc:276] 0709 13:07:50.184522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:50.184699 (+   177us) service_pool.cc:224] Handling call
0709 13:07:51.249650 (+1064951us) spinlock_profiling.cc:243] Waited 896 ms on lock 0x7b5400217d54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:51.249796 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2060684032}
I20260709 13:07:51.248837 22955 raft_consensus.cc:2749] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:51.245790 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:51.241777 22902 consensus_queue.cc:1048] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.019s
I20260709 13:07:51.263024 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:07:51.268882 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 71478us
I20260709 13:07:51.204933 22988 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:07:51.228778 22943 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:51.193594 22945 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:51.246697 22689 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:51.223763 22659 raft_consensus.cc:697] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:51.277859 16005 raft_consensus.cc:1240] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:51.282753 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:51.283433 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:51.304045 15936 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 20.
I20260709 13:07:51.288281 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:51.267359 22970 raft_consensus.cc:2749] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:51.296833 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:51.306699 15937 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 21.
I20260709 13:07:51.298848 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:51.303561 15928 raft_consensus.cc:2410] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:07:51.309252 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:51.309288 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:51.274919 22955 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:51.309991 15934 raft_consensus.cc:2376] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 21. Re-sending same reply.
I20260709 13:07:51.318315 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:51.321951 15900 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:51.322214 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:51.322610 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:51.255177 22995 consensus_queue.cc:1048] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.042s
I20260709 13:07:51.323539 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:51.326336 15935 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 19: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:51.249091 22816 consensus_queue.cc:1059] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 13:07:51.333949 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:51.328759 16009 raft_consensus.cc:1275] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:07:51.332209 15827 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:51.329017 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:51.270746 22527 consensus_queue.cc:1048] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:07:51.338872 15936 raft_consensus.cc:2410] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 22 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:07:51.309408 22551 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.328729 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:51.343147 15934 raft_consensus.cc:2393] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 19: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:51.343079 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:07:51.340934 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 100754us
I20260709 13:07:51.309916 22943 raft_consensus.cc:697] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:51.344868 22943 consensus_queue.cc:237] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:51.351869 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:51.319511 22824 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:51.365643 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:51.390187 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:51.334542 22970 raft_consensus.cc:2804] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:51.391023 15936 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
W20260709 13:07:51.391264 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 46829.9us
I20260709 13:07:51.391470 15932 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:51.340315 22921 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:51.341462 22874 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:07:51.393076 22964 log.cc:927] Time spent T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:07:51.400130 15936 raft_consensus.cc:1275] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:07:51.354763 22873 consensus_queue.cc:1059] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 13:07:51.357584 22551 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:51.406401 22551 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:51.408854 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:51.409029 16533 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:51.333451 22971 consensus_queue.cc:1048] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 13:07:51.424090 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:51.432551 15936 raft_consensus.cc:2393] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 16: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:51.341668 22509 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:51.398509 22921 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:51.438127 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:51.439198 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:51.440111 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:51.409478 22955 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:51.443181 16008 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 19.
I20260709 13:07:51.434424 22971 consensus_queue.cc:1243] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer 31882606e87447fb9a36bbf625129fe8 log is divergent from this leader: its last log entry 18.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:07:51.450310 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:51.450963 15931 raft_consensus.cc:2410] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:07:51.452337 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:51.454007 15934 raft_consensus.cc:2410] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 19 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:07:51.453944 15928 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 17.
W20260709 13:07:51.440285 22998 log.cc:927] Time spent T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:07:51.453028 16002 raft_consensus.cc:1240] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:51.456679 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:51.459142 16003 raft_consensus.cc:1240] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:51.409789 22824 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:51.444878 22921 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:51.419639 22564 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:51.506474 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:51.452598 22955 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:51.531921 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:07:51.544950 23011 log.cc:927] Time spent T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 13:07:51.453347 22816 consensus_queue.cc:1048] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.062s
I20260709 13:07:51.448252 22995 consensus_queue.cc:1048] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.065s
I20260709 13:07:51.633150 15930 raft_consensus.cc:2393] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 16: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:51.500033 23006 raft_consensus.cc:2749] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:51.433350 22659 consensus_queue.cc:237] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:51.660254 15933 raft_consensus.cc:1240] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:51.664860 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:51.630479 22765 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:51.639137 23007 log.cc:927] Time spent T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.195s	user 0.003s	sys 0.002s
W20260709 13:07:51.653836 23008 log.cc:927] Time spent T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.126s	user 0.000s	sys 0.004s
W20260709 13:07:51.653456 22994 log.cc:927] Time spent T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.242s	user 0.000s	sys 0.005s
I20260709 13:07:51.630704 22676 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:07:51.669071 22663 consensus_meta.cc:287] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.548s	user 0.008s	sys 0.008s
W20260709 13:07:51.698235 22996 log.cc:927] Time spent T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.304s	user 0.001s	sys 0.005s
W20260709 13:07:51.697932 23002 log.cc:927] Time spent T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.250s	user 0.005s	sys 0.000s
I20260709 13:07:51.656373 22965 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.721094 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:51.659302 22957 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:51.694814 22874 raft_consensus.cc:697] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 21 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
W20260709 13:07:51.733577 23001 log.cc:927] Time spent T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.292s	user 0.002s	sys 0.007s
I20260709 13:07:51.709847 22676 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:51.776666 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:51.730041 23013 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.783747 15937 raft_consensus.cc:2410] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:07:51.784641 22676 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:51.769217 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:51.754016 22955 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:51.762035 23014 consensus_queue.cc:1059] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 13.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.024s
W20260709 13:07:51.781937 22994 log.cc:927] Time spent T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:07:51.717628 23006 consensus_queue.cc:1059] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.069s
I20260709 13:07:51.710132 22765 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:51.786960 22965 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:51.778476 22564 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:51.789544 22765 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:51.782099 23007 log.cc:927] Time spent T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:07:51.790802 22965 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:51.804425 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:51.784217 23013 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:51.791517 15930 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:51.807566 16003 raft_consensus.cc:2393] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 15: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:51.762732 15936 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 21.
I20260709 13:07:51.808624 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:51.791576 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:51.809330 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:51.809716 16004 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 18.
I20260709 13:07:51.791580 15902 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:51.811662 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:51.813232 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:07:51.794651 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 156668us
I20260709 13:07:51.816504 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:51.795187 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:51.796700 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:51.817749 16566 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 19.
I20260709 13:07:51.802929 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:51.761194 22953 consensus_meta.cc:287] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.639s	user 0.012s	sys 0.000s
W20260709 13:07:51.805676 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 394085us
I20260709 13:07:51.820942 16008 raft_consensus.cc:2393] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 19: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:51.734683 22824 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:51.815999 15936 raft_consensus.cc:2393] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 15: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
W20260709 13:07:51.809924 22679 consensus_meta.cc:287] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.616s	user 0.002s	sys 0.008s
I20260709 13:07:51.828486 22667 consensus_queue.cc:1059] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:51.829562 22667 raft_consensus.cc:3055] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:51.829941 22667 raft_consensus.cc:740] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:51.830888 22667 consensus_queue.cc:260] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:51.832516 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:51.821313 16566 raft_consensus.cc:1275] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:07:51.806978 15932 raft_consensus.cc:2410] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:07:51.760490 23005 consensus_queue.cc:1059] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.321s
I20260709 13:07:51.833122 15936 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:51.823815 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:51.823498 15929 raft_consensus.cc:2410] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 21 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 13:07:51.824429 15827 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:51.825481 16533 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:51.837258 16008 raft_consensus.cc:1240] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:51.827162 16561 raft_consensus.cc:2393] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 18: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:51.784147 22874 consensus_queue.cc:237] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:51.822309 15825 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:51.841126 16533 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:51.825969 15902 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:51.842567 16533 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:51.858777 15902 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:51.861977 15936 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 21.
I20260709 13:07:51.826810 22564 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:51.860756 15902 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:51.867025 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:51.834548 22970 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:51.868788 15902 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:51.868208 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:51.872630 15931 raft_consensus.cc:2410] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 20 because replica has last-logged OpId of term: 18 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 15.
I20260709 13:07:51.843523 22873 raft_consensus.cc:2804] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:51.821660 22965 consensus_queue.cc:1059] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 16.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.040s
I20260709 13:07:51.876757 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:51.877020 15936 raft_consensus.cc:1240] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 15. Current term is 16. Ops: [15.15-15.15]
W20260709 13:07:51.877112 22999 log.cc:927] Time spent T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:07:51.862229 22676 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:51.871529 22667 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:51.871120 22970 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:51.879222 22945 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:51.879490 22921 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:51.871467 22971 raft_consensus.cc:2749] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:51.881259 15935 raft_consensus.cc:1240] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 13:07:51.857293 22953 raft_consensus.cc:2749] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:51.856657 23000 raft_consensus.cc:2749] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:51.865196 23005 raft_consensus.cc:2804] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:51.884543 16566 raft_consensus.cc:1275] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:07:51.826562 22957 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:51.886310 15936 pending_rounds.cc:85] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 12
I20260709 13:07:51.916254 15936 pending_rounds.cc:107] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 13:07:51.916585 15936 raft_consensus.cc:2889] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:51.880347 23021 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:51.875370 22955 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:51.882702 22873 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:51.873284 22509 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:51.912182 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 469466us
I20260709 13:07:51.918695 15935 raft_consensus.cc:1275] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:07:51.923386 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:51.910446 22988 raft_consensus.cc:3055] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:51.931720 23012 raft_consensus.cc:3055] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:51.932062 23013 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:51.932194 23012 raft_consensus.cc:740] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:51.912994 23005 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:51.943734 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:51.945124 15930 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 16.
I20260709 13:07:51.922479 22873 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:51.949965 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:51.980329 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:51.928632 22988 raft_consensus.cc:740] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:51.833900 23017 consensus_queue.cc:1048] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.115s
I20260709 13:07:51.982424 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:51.993975 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:51.995150 15858 raft_consensus.cc:2368] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 19. Current term is 20.
I20260709 13:07:51.982641 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:51.996917 15856 raft_consensus.cc:2393] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 19: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:51.982424 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:07:51.950066 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 107575us
I20260709 13:07:51.878351 23018 consensus_queue.cc:1048] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.071s
I20260709 13:07:51.990285 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:51.990931 15861 raft_consensus.cc:1240] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:51.991827 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:51.948035 23005 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:52.019134 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:51.993010 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:51.997880 16533 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:52.015998 15862 raft_consensus.cc:3060] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:52.016829 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:52.017086 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:51.934068 23004 consensus_queue.cc:1048] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.114s
I20260709 13:07:52.018301 15855 raft_consensus.cc:2393] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 17: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:52.021469 15858 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:52.020119 15859 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 21.
I20260709 13:07:52.025501 15825 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:51.947861 23021 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:07:52.045715 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 47914.8us
I20260709 13:07:52.048170 23025 raft_consensus.cc:2804] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:52.048925 23025 raft_consensus.cc:697] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:52.058921 15858 raft_consensus.cc:2410] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:07:52.059983 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:52.060604 16533 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:52.062873 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:51.933395 23012 consensus_queue.cc:260] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.15, Last appended by leader: 15, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:52.068887 23012 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:52.073093 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 215008us
W20260709 13:07:52.076284 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 50909.7us
W20260709 13:07:52.076848 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 208124us
I20260709 13:07:51.992636 22659 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:07:52.082887 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 155938us
I20260709 13:07:52.084889 16565 raft_consensus.cc:1275] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:07:52.095428 23028 raft_consensus.cc:2749] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:07:52.096323 23020 log.cc:927] Time spent T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:07:52.106107 15902 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:52.107281 15862 raft_consensus.cc:2468] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 18.
I20260709 13:07:52.050171 23025 consensus_queue.cc:237] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:52.118950 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 71285.5us
I20260709 13:07:52.119880 16533 leader_election.cc:400] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 election: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 19. Current term is 20.
I20260709 13:07:52.121035 16533 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:52.056147 22988 consensus_queue.cc:260] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:52.124466 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:52.124585 22972 consensus_queue.cc:1175] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" responder_term: 16 status { last_received { term: 16 index: 15 } last_committed_idx: 15 last_received_current_leader { term: 16 index: 15 } }
I20260709 13:07:52.092852 22659 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:52.124361 22988 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:52.126951 15974 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:52.129945 15974 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:52.134663 16568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 1498) took 1343 ms. Trace:
I20260709 13:07:52.116217 22921 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:52.128350 22971 raft_consensus.cc:2749] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:52.108170 23013 consensus_queue.cc:1059] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 13:07:52.154103 16568 rpcz_store.cc:276] 0709 13:07:50.791461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:50.791567 (+   106us) service_pool.cc:224] Handling call
0709 13:07:50.793337 (+  1770us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:51.185689 (+392352us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:51.185728 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:51.185793 (+    65us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:51.185847 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:52.134418 (+948571us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b54004790d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:52.134470 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:52.134609 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2453,"raft.queue_time_us":39087,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":84,"spinlock_wait_cycles":488845568,"thread_start_us":176778,"threads_started":1}
I20260709 13:07:52.133471 22945 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:52.153609 22689 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:52.137073 23027 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:07:52.161094 23029 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.183012 23036 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:52.184289 22659 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:52.188447 15934 raft_consensus.cc:1240] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:07:52.131474 22509 consensus_queue.cc:1048] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:07:52.177000 22945 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:52.123656 23021 raft_consensus.cc:697] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:07:52.246127 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 89610.5us
I20260709 13:07:52.266189 15974 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:52.162417 22564 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.173942 22955 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.191982 22689 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:52.250973 22957 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:52.201550 22594 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:52.268653 15974 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:52.264305 22945 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:52.277042 15974 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:52.272593 22803 raft_consensus.cc:2749] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:52.274034 22689 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:52.277678 22594 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:52.290095 23041 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:52.290830 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 180467us
I20260709 13:07:52.280308 22603 raft_consensus.cc:2749] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:07:52.292848 23026 log.cc:927] Time spent T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 13:07:52.293947 16009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 1372) took 1788 ms. Trace:
I20260709 13:07:52.298856 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:52.312493 16011 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:52.312268 16009 rpcz_store.cc:276] 0709 13:07:50.505850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:50.506263 (+   413us) service_pool.cc:224] Handling call
0709 13:07:50.512125 (+  5862us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:51.335159 (+823034us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:52.293705 (+958546us) spinlock_profiling.cc:243] Waited 816 ms on lock 0x7b54001dc754. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:52.293889 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1877279488}
W20260709 13:07:52.300420 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:52.305320 15827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:52.272346 22955 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:52.316663 15929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 2979) took 1111 ms. Trace:
I20260709 13:07:52.317025 15929 rpcz_store.cc:276] 0709 13:07:51.205418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:51.227790 (+ 22372us) service_pool.cc:224] Handling call
0709 13:07:52.300334 (+1072544us) spinlock_profiling.cc:243] Waited 581 ms on lock 0x7b5400328054. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:52.316609 (+ 16275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1335780608}
I20260709 13:07:52.319306 15974 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:52.324290 15929 raft_consensus.cc:1275] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 13 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:07:52.326159 15858 pending_rounds.cc:85] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8: Aborting all ops after (but not including) 7
I20260709 13:07:52.326469 15858 pending_rounds.cc:107] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8: Aborting uncommitted NO_OP operation due to leader change: 18.8
I20260709 13:07:52.326278 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:52.326843 15858 raft_consensus.cc:2889] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:52.328787 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:52.328910 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:52.329445 16004 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 18.
I20260709 13:07:52.332327 15931 raft_consensus.cc:1275] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:07:52.275041 22564 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:52.335665 16011 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 22.
W20260709 13:07:52.338390 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 217411us
I20260709 13:07:52.339085 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:52.339555 16533 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:52.308341 22803 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:52.316283 22659 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.341542 15900 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:52.320287 23017 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.322930 23031 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.328368 23004 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.259229 23029 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:52.346889 16533 leader_election.cc:304] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:52.348706 23030 consensus_queue.cc:1059] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:52.344344 23048 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:07:52.350042 22955 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:52.353547 15900 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:52.357857 23017 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:52.358037 23004 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:52.358680 22955 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:52.360519 15854 raft_consensus.cc:1275] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:07:52.364595 23032 log.cc:927] Time spent T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:07:52.341501 22509 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:52.381916 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:52.311988 23041 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:52.382733 16006 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:52.383049 23048 raft_consensus.cc:2749] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:52.382745 16533 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:52.386689 22873 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:52.387547 23048 raft_consensus.cc:2749] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:07:52.389374 15928 raft_consensus.cc:1240] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:07:52.393934 16533 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:52.398011 16533 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:07:52.399147 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 79923.5us
I20260709 13:07:52.354570 22971 raft_consensus.cc:2749] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:52.400128 15974 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:52.401731 16533 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:52.399206 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:52.402488 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:52.405184 16006 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 19.
I20260709 13:07:52.404980 15855 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 21.
I20260709 13:07:52.381368 23004 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:52.407800 23037 log.cc:927] Time spent T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:07:52.410651 22995 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:52.413676 22679 raft_consensus.cc:3055] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:52.414070 22679 raft_consensus.cc:740] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:52.415221 15853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 2682) took 1363 ms. Trace:
I20260709 13:07:52.416623 23040 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:52.415747 15853 rpcz_store.cc:276] 0709 13:07:51.051507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:51.051626 (+   119us) service_pool.cc:224] Handling call
0709 13:07:51.051968 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:51.120042 (+ 68074us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:07:51.120350 (+   308us) log.cc:844] Serialized 14 byte log entry
0709 13:07:51.235215 (+114865us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:51.235271 (+    56us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:51.235312 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:52.414889 (+1179577us) spinlock_profiling.cc:243] Waited 67.9 ms on lock 0x7b540008dc54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:52.414934 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:52.415064 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":114607,"prepare.queue_time_us":347,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":115025,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":128,"spinlock_wait_cycles":156055936}
I20260709 13:07:52.416986 16567 raft_consensus.cc:1275] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 15 index: 15. Preceding OpId from leader: term: 16 index: 15. (term mismatch)
I20260709 13:07:52.417498 16567 pending_rounds.cc:85] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting all ops after (but not including) 14
I20260709 13:07:52.417776 16567 pending_rounds.cc:107] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting uncommitted NO_OP operation due to leader change: 15.15
I20260709 13:07:52.418013 16567 raft_consensus.cc:2889] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:52.418648 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 2749) took 1090 ms. Trace:
I20260709 13:07:52.419025 16002 rpcz_store.cc:276] 0709 13:07:51.327809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:51.333312 (+  5503us) service_pool.cc:224] Handling call
0709 13:07:51.333590 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:51.457621 (+124031us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:52.418397 (+960776us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b54000eddd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:52.418590 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":274277632}
I20260709 13:07:52.419895 22791 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:52.420495 22791 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:52.424786 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:52.425110 22564 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:52.425513 15859 raft_consensus.cc:1813] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Handling vote request from an unknown peer 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:52.403283 23030 raft_consensus.cc:2804] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:52.425880 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:52.407586 23038 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.354753 22970 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:52.423077 15900 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:52.427922 23041 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:52.428500 23041 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:52.428831 23041 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:52.429147 23041 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:52.430061 22945 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:52.357443 23031 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:52.426530 16002 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 19.
I20260709 13:07:52.433687 23031 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:52.433447 22594 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:52.434549 23009 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:52.389374 23029 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:52.426012 15859 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 15: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:52.351294 22659 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:52.436403 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:52.436993 15861 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 15.
I20260709 13:07:52.437706 15974 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:52.439381 15974 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:52.441075 16533 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:52.332173 23021 consensus_queue.cc:237] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:52.441993 23046 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:52.442490 23046 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:52.442869 23046 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:52.444414 23012 consensus_queue.cc:1059] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:07:52.445077 22659 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:52.445772 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:52.446863 22594 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:52.448020 22803 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:52.448824 23017 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:52.448858 22803 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:52.448239 22667 consensus_meta.cc:287] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.538s	user 0.011s	sys 0.000s
I20260709 13:07:52.450841 23005 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:52.448936 22689 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:52.440217 23038 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:52.453212 23005 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:52.453472 22689 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:52.453959 23038 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:52.454603 22873 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:52.426282 23030 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:52.433686 22945 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:52.459048 23030 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:52.461838 23005 raft_consensus.cc:2804] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:52.423074 22509 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:52.467957 22902 raft_consensus.cc:3055] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:07:52.468947 23005 raft_consensus.cc:697] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:52.488277 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:52.451581 22791 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:52.489056 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 49788.7us
I20260709 13:07:52.494473 15974 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:52.494974 15930 raft_consensus.cc:1240] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:07:52.486124 22902 raft_consensus.cc:740] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:52.497135 22902 consensus_queue.cc:260] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:52.498323 22902 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:52.499711 23036 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:52.500236 23036 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:52.501101 23033 log.cc:927] Time spent T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.213s	user 0.000s	sys 0.000s
I20260709 13:07:52.502550 23036 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:52.503248 22803 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:52.504060 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:52.505064 22791 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:52.508386 23005 consensus_queue.cc:237] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:52.485576 23025 consensus_queue.cc:1059] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:07:52.458025 23009 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:52.517087 23009 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:52.518077 15827 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:52.520211 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:52.522892 15827 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:52.523407 22564 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.523950 22564 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:52.533550 23017 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:52.537654 15827 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:52.538075 15974 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:52.538652 15827 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:52.539130 23052 raft_consensus.cc:2749] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:07:52.535633 22943 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.823s	user 0.018s	sys 0.009s
W20260709 13:07:52.539845 22943 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.836s	user 0.018s	sys 0.010s
I20260709 13:07:52.540308 22899 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:52.540584 15974 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:52.540725 22899 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:52.541126 22899 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:52.541224 22943 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:52.541800 22943 raft_consensus.cc:697] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:52.541746 23052 consensus_queue.cc:1048] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:52.542794 22943 consensus_queue.cc:237] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:52.543920 22659 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:52.543859 23052 consensus_queue.cc:1059] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:52.544453 22659 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:52.544860 22659 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:52.544960 22945 raft_consensus.cc:3055] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:52.545370 22945 raft_consensus.cc:740] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:52.546176 22945 consensus_queue.cc:260] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:52.547106 22667 consensus_queue.cc:1059] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:07:52.547546 22945 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:52.547847 15974 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:52.548594 23038 raft_consensus.cc:3055] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:52.549021 23038 raft_consensus.cc:740] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:52.549079 23052 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:07:52.550325 23038 consensus_queue.cc:260] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:52.568269 23035 log.cc:927] Time spent T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.209s	user 0.000s	sys 0.000s
I20260709 13:07:52.534546 23004 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:52.534379 23031 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.584087 15937 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53636 (request call id 1391) took 1122 ms. Trace:
I20260709 13:07:52.536890 23017 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:52.596014 15937 rpcz_store.cc:276] 0709 13:07:51.462007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:51.637001 (+174994us) service_pool.cc:224] Handling call
0709 13:07:52.583778 (+946777us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b540044bbd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:52.584004 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":270815488}
I20260709 13:07:52.552927 22527 raft_consensus.cc:2749] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:52.605331 15855 raft_consensus.cc:1240] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:52.609758 22824 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:52.612174 23051 log.cc:927] Time spent T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:07:52.612528 22824 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:52.552716 22594 consensus_queue.cc:1059] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 13:07:52.575874 23004 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:52.615548 23004 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:52.616161 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:52.573819 23038 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:52.607409 23017 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:52.630152 15859 raft_consensus.cc:1275] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
W20260709 13:07:52.631057 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:52.599799 23041 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:52.637710 23041 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:52.639734 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:52.640645 16005 raft_consensus.cc:2410] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 24 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 13:07:52.565802 23021 consensus_queue.cc:1059] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 13:07:52.643715 15900 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:52.606711 23031 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:52.644304 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:52.629585 23040 raft_consensus.cc:2749] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:52.646651 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:52.651350 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:52.654431 16011 raft_consensus.cc:1275] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:07:52.654809 15859 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 23.
I20260709 13:07:52.655499 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:52.656343 16006 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 19.
I20260709 13:07:52.656996 16010 raft_consensus.cc:1275] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:07:52.657811 16008 raft_consensus.cc:1240] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:52.640030 15759 catalog_manager.cc:5697] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 10 to 18, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 18 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:52.653581 23031 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:52.660620 22873 consensus_queue.cc:1048] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:07:52.661258 22825 raft_consensus.cc:3055] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:52.661675 22825 raft_consensus.cc:740] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:52.662137 22971 raft_consensus.cc:2804] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:52.661734 15759 catalog_manager.cc:5697] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 15 to 17, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 17 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:07:52.662565 22971 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:52.663069 22971 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:52.662631 22825 consensus_queue.cc:260] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:07:52.665380 23004 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:52.666294 22564 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:52.667587 23004 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:52.626650 23046 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:52.663281 15759 catalog_manager.cc:5697] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 13 to 16, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 16 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:07:52.679400 15974 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:52.680302 23052 raft_consensus.cc:2749] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:07:52.681646 15974 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:52.682442 23052 raft_consensus.cc:2749] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:07:52.683171 22945 consensus_queue.cc:1059] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:52.683660 22899 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:52.684499 22689 raft_consensus.cc:3055] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:52.684878 22689 raft_consensus.cc:740] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:52.671049 22527 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.685832 22899 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:52.691442 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:52.692863 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:52.693970 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:52.691442 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:52.685701 23017 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:52.706099 15759 catalog_manager.cc:5697] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 12 to 15, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 15 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:52.708489 15759 catalog_manager.cc:5697] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 14 to 17, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 17 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:52.685956 23046 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:52.711143 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:52.713759 16006 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:52.698210 22953 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.697922 23031 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.697388 22913 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.697038 23004 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:52.697214 22527 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:52.713567 15759 catalog_manager.cc:5697] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:07:52.685611 23030 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:52.724296 22527 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:52.688716 22873 consensus_queue.cc:1059] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:07:52.726039 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:52.724359 23030 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:52.725313 15759 catalog_manager.cc:5697] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 12 to 15, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 15 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:52.733186 23017 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:52.733639 16004 raft_consensus.cc:1275] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:07:52.707068 22765 consensus_queue.cc:1048] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:07:52.736465 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:52.736274 23004 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:52.735453 15759 catalog_manager.cc:5697] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:52.685670 22689 consensus_queue.cc:260] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:52.741055 15759 catalog_manager.cc:5697] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 7 to 18. New cstate: current_term: 18 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:07:52.744202 15753 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58966 (request call id 246) took 37337 ms (client timeout 15000 ms). Trace:
I20260709 13:07:52.744376 22955 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.750379 22955 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:52.753247 22955 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:52.744776 22913 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:52.754573 16004 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 18.
W20260709 13:07:52.755636 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 72673.2us
W20260709 13:07:52.745613 15753 rpcz_store.cc:269] 0709 13:07:15.406985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.410837 (+  3852us) service_pool.cc:224] Handling call
0709 13:07:52.743945 (+37333108us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:44.558575 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:44.566499 (+  7924us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:44.566554 (+    55us) write_op.cc:435] Acquired schema lock
0709 13:07:44.566570 (+    16us) tablet.cc:662] Decoding operations
0709 13:07:44.572160 (+  5590us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:44.572278 (+   118us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:44.572313 (+    35us) tablet.cc:685] Acquiring locks for 15 operations
0709 13:07:44.573636 (+  1323us) tablet.cc:701] Row locks acquired
0709 13:07:44.573681 (+    45us) write_op.cc:260] PREPARE: finished
0709 13:07:44.573878 (+   197us) write_op.cc:270] Start()
0709 13:07:44.574035 (+   157us) write_op.cc:276] Timestamp: P: 1783602464573813 usec, L: 0
0709 13:07:44.574071 (+    36us) op_driver.cc:348] REPLICATION: starting
0709 13:07:44.731866 (+157795us) log.cc:844] Serialized 5916 byte log entry
0709 13:07:45.526815 (+794949us) op_driver.cc:464] REPLICATION: finished
0709 13:07:46.263126 (+736311us) write_op.cc:301] APPLY: starting
0709 13:07:46.263219 (+    93us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:46.974299 (+711080us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:47.312396 (+338097us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:47.463492 (+151096us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:47.482685 (+ 19193us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=30,key_file_lookups=30,delta_file_lookups=30,mrs_lookups=0
0709 13:07:47.482789 (+   104us) write_op.cc:312] APPLY: finished
0709 13:07:47.483733 (+   944us) log.cc:844] Serialized 139 byte log entry
0709 13:07:47.622988 (+139255us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:47.706309 (+ 83321us) write_op.cc:454] Released schema lock
0709 13:07:47.706388 (+    79us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:47.729037 (+ 22649us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:47.729087 (+    50us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4340325,"reported_tablets":42,"child_traces":[["op",{"apply.queue_time_us":736045,"apply.run_cpu_time_us":64651,"apply.run_wall_time_us":1459269,"cfile_cache_hit":62,"cfile_cache_hit_bytes":66977,"num_ops":15,"prepare.queue_time_us":88009,"prepare.run_cpu_time_us":100106,"prepare.run_wall_time_us":488191,"raft.queue_time_us":428764,"raft.run_cpu_time_us":66606,"raft.run_wall_time_us":528109,"replication_time_us":952568,"thread_start_us":1067409,"threads_started":5,"wal-append.queue_time_us":631178,"wal-append.run_cpu_time_us":250960,"wal-append.run_wall_time_us":2426588}]]}
I20260709 13:07:52.760924 15974 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:52.760994 15759 catalog_manager.cc:5697] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:07:52.765458 16006 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:52.757351 22913 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:52.743151 22971 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:52.769249 22803 consensus_queue.cc:1048] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.006s
I20260709 13:07:52.760097 22955 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:52.776911 15759 catalog_manager.cc:5697] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 16 to 19, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 19 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:07:52.776667 15827 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:52.769846 22943 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:07:52.722540 22953 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:52.755395 23031 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:52.782114 15827 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:52.783524 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:52.786556 15933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 2981) took 1521 ms. Trace:
I20260709 13:07:52.786530 15936 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 18.
I20260709 13:07:52.787045 15933 rpcz_store.cc:276] 0709 13:07:51.265375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:51.265706 (+   331us) service_pool.cc:224] Handling call
0709 13:07:51.275189 (+  9483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:51.663413 (+388224us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:52.786342 (+1122929us) spinlock_profiling.cc:243] Waited 382 ms on lock 0x7b540034a154. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:52.786509 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":879122688}
I20260709 13:07:52.781243 22943 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:52.790642 22943 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:52.791517 15759 catalog_manager.cc:5697] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:52.793037 15759 catalog_manager.cc:5697] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 12 to 14, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 14 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:52.796471 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:52.802349 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:52.803661 15858 raft_consensus.cc:1240] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:52.804112 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:52.803074 15857 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 19.
I20260709 13:07:52.788189 22765 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:52.811048 15759 catalog_manager.cc:5697] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:52.815955 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:52.816787 15859 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:52.779029 22900 consensus_queue.cc:1048] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.024s
I20260709 13:07:52.776455 22955 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:07:52.819394 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:52.822316 16533 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:52.804593 22550 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:52.825718 22816 raft_consensus.cc:2749] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:52.826854 22659 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:52.827797 23059 log.cc:927] Time spent T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:07:52.829061 22659 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:52.832170 15855 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 19.
I20260709 13:07:52.832966 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:52.833755 16011 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:52.816275 23041 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.821439 22765 raft_consensus.cc:697] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:52.839486 22550 raft_consensus.cc:697] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:52.839998 22971 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:52.840651 22971 consensus_queue.cc:1059] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 19.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:07:52.840595 22550 consensus_queue.cc:237] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:52.841845 22676 raft_consensus.cc:3055] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:52.842214 22676 raft_consensus.cc:740] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:52.842358 22953 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:52.842993 22676 consensus_queue.cc:260] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:52.844169 22955 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
I20260709 13:07:52.846891 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:52.848362 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:52.848920 15860 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:52.850121 22679 consensus_queue.cc:260] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:52.851430 22679 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:52.840445 22765 consensus_queue.cc:237] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:52.851779 22955 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.866688 22955 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:52.867223 16535 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:52.868202 16535 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:52.868706 22955 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:52.858112 23029 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:52.858569 22985 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.858913 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:52.869328 23048 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:07:52.871289 15853 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 20.
I20260709 13:07:52.874868 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:52.870136 23030 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:52.846437 15859 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 22.
I20260709 13:07:52.876811 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:07:52.877964 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:52.905859 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:52.906104 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:52.847213 15900 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:52.880249 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:52.907980 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:52.908605 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 61481.6us
I20260709 13:07:52.907462 16002 raft_consensus.cc:1240] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:52.909956 15900 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:52.886827 15860 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 16.
I20260709 13:07:52.873270 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:52.893388 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:52.896580 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:52.899827 22955 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:52.908259 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:52.902920 16011 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 20.
I20260709 13:07:52.888324 16533 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:52.911224 15900 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:07:52.904016 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 118557us
I20260709 13:07:52.908569 16007 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 18.
I20260709 13:07:52.969341 15853 raft_consensus.cc:1275] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:07:52.920207 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:52.974324 15853 raft_consensus.cc:1240] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:52.975010 23005 raft_consensus.cc:2804] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:52.972226 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:52.975841 23005 raft_consensus.cc:697] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:52.976316 15853 raft_consensus.cc:1240] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:52.977065 23005 consensus_queue.cc:237] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:52.933530 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:52.978946 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:52.979997 16007 raft_consensus.cc:2393] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 19: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
W20260709 13:07:52.972354 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 61250.9us
I20260709 13:07:52.981263 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:52.920754 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:52.984265 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:52.983798 15861 raft_consensus.cc:1275] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:07:52.984831 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:52.985978 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 97773.8us
I20260709 13:07:52.986907 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:52.987354 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:52.987716 16011 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 20.
I20260709 13:07:52.922984 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:52.989640 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:52.985746 15933 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 19.
I20260709 13:07:52.993786 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:52.995110 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:52.996016 15859 raft_consensus.cc:3060] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:52.997977 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:53.000752 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:52.893780 22527 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.992867 15759 catalog_manager.cc:5697] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 11 to 18. New cstate: current_term: 18 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:53.004243 15933 raft_consensus.cc:1240] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:52.896888 22980 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.903602 23006 raft_consensus.cc:2804] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:07:52.904634 23030 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:52.904270 23012 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:52.987989 22492 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:52.969341 22953 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:53.015605 15759 catalog_manager.cc:5697] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 14 to 17, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 17 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:07:52.920993 23052 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:53.027364 15859 raft_consensus.cc:2468] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 22.
I20260709 13:07:53.039247 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:52.888998 23041 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:53.039165 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:53.001737 22957 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:07:53.012442 23031 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:52.904178 23029 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:52.972564 22943 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:52.974777 22955 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:52.920181 22985 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:53.028228 23030 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:53.033447 22492 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:53.047426 15759 catalog_manager.cc:5697] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:07:53.043314 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 133192us
I20260709 13:07:53.035089 22953 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:53.044809 15932 pending_rounds.cc:85] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 10
I20260709 13:07:53.084944 15932 pending_rounds.cc:107] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 13:07:53.085286 15932 raft_consensus.cc:2889] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:53.060178 23031 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:53.073385 22594 raft_consensus.cc:3055] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:53.030308 22550 consensus_queue.cc:1048] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.053s
I20260709 13:07:53.028615 22980 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:53.096316 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:53.018051 22527 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:53.104125 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:53.031049 23006 raft_consensus.cc:697] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:53.106137 22594 raft_consensus.cc:740] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:53.081099 15935 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 19.
I20260709 13:07:53.034399 23012 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:53.153184 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:53.156466 23068 log.cc:927] Time spent T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:07:53.161489 23041 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:53.161902 15900 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:53.171898 22689 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:53.178629 22955 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:53.185714 22943 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 24 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:53.186811 23035 log.cc:927] Time spent T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:07:53.187841 23006 consensus_queue.cc:237] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:53.062445 22957 raft_consensus.cc:697] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 22 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:53.123112 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:53.192349 15934 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 22.
I20260709 13:07:53.086557 22953 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:53.108814 15759 catalog_manager.cc:5697] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 14 to 17, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 17 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:53.045332 23052 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:53.205554 15759 catalog_manager.cc:5697] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 11 to 13, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 13 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:07:53.164745 23029 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:53.193310 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:53.084100 22492 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:07:53.228152 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 67109.8us
I20260709 13:07:53.133751 22594 consensus_queue.cc:260] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:53.208904 15929 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:53.209702 15759 catalog_manager.cc:5697] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: config changed from index 3 to 9, term changed from 11 to 13, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4), VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New cstate: current_term: 13 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:53.210862 22527 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:53.208734 23067 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:53.205267 23061 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:53.115777 23031 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:53.287405 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 313336us
W20260709 13:07:53.289851 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 205261us
W20260709 13:07:53.277413 23064 log.cc:927] Time spent T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 13:07:53.308939 15929 raft_consensus.cc:2410] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 20 because replica has last-logged OpId of term: 18 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 15.
I20260709 13:07:53.276453 23076 raft_consensus.cc:2804] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:53.276855 22679 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:53.279935 22509 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:53.287842 22594 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:53.278580 22955 consensus_queue.cc:1048] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 13:07:53.286726 23066 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:07:53.346091 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 76824.7us
I20260709 13:07:53.405261 23091 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:53.406044 23091 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:53.407851 22972 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:53.408583 22972 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:53.410897 22972 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:53.413717 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 122624us
I20260709 13:07:53.413825 23091 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:53.417378 15976 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:53.419838 15976 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:53.419540 23069 consensus_queue.cc:1048] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:07:53.422760 15976 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:53.415729 22953 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:53.426401 22953 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:53.396313 15974 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:53.283567 23067 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:53.448071 23075 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:53.448185 23066 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:53.448868 23075 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:53.450541 23069 raft_consensus.cc:2749] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:07:53.451243 23069 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:07:53.452307 23069 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:53.454012 23069 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:53.454551 23069 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:53.461118 23038 consensus_queue.cc:1059] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:07:53.464169 22508 raft_consensus.cc:3055] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:53.464567 22508 raft_consensus.cc:740] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:53.465514 22508 consensus_queue.cc:260] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:53.468282 23069 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:53.470301 15827 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:53.471804 23069 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:53.473021 23069 raft_consensus.cc:2749] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:53.475054 22984 raft_consensus.cc:3055] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:07:53.475585 22984 raft_consensus.cc:740] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:53.476637 22984 consensus_queue.cc:260] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:53.477991 22984 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:53.481153 22676 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:53.481247 22980 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:53.338554 23005 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:53.339174 23076 raft_consensus.cc:697] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 16 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:53.285437 23061 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:53.512643 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:53.520352 16005 raft_consensus.cc:1240] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:53.339946 22679 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:53.533324 23081 log.cc:927] Time spent T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.215s	user 0.000s	sys 0.000s
I20260709 13:07:53.524622 23096 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:53.524807 22873 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:53.539628 15902 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:53.346036 22509 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:53.493125 22985 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:53.524814 23091 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:53.573540 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 258388us
I20260709 13:07:53.579453 16005 raft_consensus.cc:1240] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:07:53.318974 22957 consensus_queue.cc:237] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:53.590401 15854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 2687) took 1265 ms. Trace:
W20260709 13:07:53.602262 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:53.613633 15854 rpcz_store.cc:276] 0709 13:07:52.325174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:52.330925 (+  5751us) service_pool.cc:224] Handling call
0709 13:07:52.346129 (+ 15204us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:52.360859 (+ 14730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:53.590014 (+1229155us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b54000b5754. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:53.590310 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":32393088}
I20260709 13:07:53.451587 23075 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:53.617087 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:53.564576 22921 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:53.491472 23090 consensus_queue.cc:1048] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.186s
W20260709 13:07:53.618515 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:53.582720 23044 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:53.633016 15825 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:53.582439 22980 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:53.522606 23005 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:53.588332 23009 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:53.616868 22985 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:07:53.629315 23083 log.cc:927] Time spent T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.069s	user 0.002s	sys 0.000s
I20260709 13:07:53.655970 15825 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:53.607597 23096 raft_consensus.cc:697] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 20 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
W20260709 13:07:53.641798 23085 log.cc:927] Time spent T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.112s	user 0.000s	sys 0.002s
I20260709 13:07:53.623605 23012 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:53.608374 22873 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:53.678679 15825 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:53.624191 23091 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:53.681234 15825 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:07:53.671258 23084 log.cc:927] Time spent T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260709 13:07:53.643175 22921 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:53.674227 23087 raft_consensus.cc:2749] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:53.694141 15825 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:07:53.696735 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 225615us
I20260709 13:07:53.683838 23094 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:53.671429 23052 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:53.656409 23044 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:53.717629 23044 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:53.684139 22551 raft_consensus.cc:2749] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:53.719414 23091 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:53.685387 23067 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:53.671582 23076 consensus_queue.cc:237] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:53.723033 22689 consensus_meta.cc:287] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.551s	user 0.009s	sys 0.000s
I20260709 13:07:53.724821 23004 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:53.675467 23009 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:53.726689 15930 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 20.
I20260709 13:07:53.727779 23009 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:53.728684 22873 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:53.729275 23061 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:53.671710 22985 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:53.699951 23040 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:53.730279 23040 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:53.731782 22985 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:53.732224 23040 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:07:53.734107 23030 consensus_meta.cc:287] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.648s	user 0.009s	sys 0.000s
I20260709 13:07:53.734511 23030 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:53.737651 23067 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:53.738065 22824 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:53.738171 23067 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:53.709298 23070 log.cc:927] Time spent T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.188s	user 0.000s	sys 0.001s
I20260709 13:07:53.739471 23005 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:53.740319 23067 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:53.740628 15937 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 2690) took 1047 ms. Trace:
I20260709 13:07:53.740993 15937 rpcz_store.cc:276] 0709 13:07:52.692818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:52.693032 (+   214us) service_pool.cc:224] Handling call
0709 13:07:52.706488 (+ 13456us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:52.714789 (+  8301us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:52.714817 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:52.719108 (+  4291us) log.cc:844] Serialized 26 byte log entry
0709 13:07:52.852777 (+133669us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b18001b6c40. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f480d1 00007f73ea363a0f
0709 13:07:53.063987 (+211210us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:53.064024 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:53.064084 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:53.740345 (+676261us) raft_consensus.cc:1685] finished
0709 13:07:53.740364 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:53.740574 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7890,"raft.queue_time_us":8002,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":5430,"spinlock_wait_cycles":27969792,"thread_start_us":211020,"threads_started":1,"wal-append.queue_time_us":573776}
W20260709 13:07:53.710189 23079 log.cc:927] Time spent T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
W20260709 13:07:53.711447 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:53.748317 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:53.748679 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:53.748956 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:53.749243 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:53.749402 15935 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 16.
I20260709 13:07:53.748996 15933 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 13.
I20260709 13:07:53.750512 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:53.751299 15931 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 19.
W20260709 13:07:53.712301 23082 log.cc:927] Time spent T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:07:53.713716 23017 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:53.752369 23017 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:53.752748 23017 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:53.753340 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:53.753957 15937 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 19.
I20260709 13:07:53.752208 23096 consensus_queue.cc:237] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:53.694638 22755 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:53.758239 22755 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:53.758623 22755 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:53.760401 22508 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:53.763149 23030 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:53.764107 23031 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:53.767401 16569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1567) took 2010 ms. Trace:
I20260709 13:07:53.768322 23066 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:53.767805 16569 rpcz_store.cc:276] 0709 13:07:51.756516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:51.757805 (+  1289us) service_pool.cc:224] Handling call
0709 13:07:51.758073 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:52.772366 (+1014293us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:52.772397 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:52.772445 (+    48us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:52.772474 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:53.767177 (+994703us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400724554. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:53.767222 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:53.767346 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2332160896}
I20260709 13:07:53.772727 22509 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:53.771340 23031 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:53.773160 22921 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:53.773765 23031 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:53.780575 23031 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:53.795928 23030 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:53.796559 23030 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:53.797943 16533 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:53.798805 23030 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:53.799113 23048 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:53.811358 16533 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:53.816833 16533 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:07:53.819273 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:53.821173 15931 raft_consensus.cc:1275] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:07:53.824090 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:53.824873 15933 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 22.
I20260709 13:07:53.799403 23030 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:53.834117 23031 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:53.801290 23043 raft_consensus.cc:2804] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:53.800066 22921 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:53.837348 22985 raft_consensus.cc:2804] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:07:53.839092 23103 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:53.862212 16533 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:53.799845 23048 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:53.839826 23030 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:53.848953 23043 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:53.888224 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:07:53.900421 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 477775us
I20260709 13:07:53.887332 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:53.897706 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:53.898268 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:53.899014 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:53.860194 22985 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:53.859767 23103 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:53.908331 16566 raft_consensus.cc:1240] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:53.908529 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:53.908809 16002 raft_consensus.cc:1240] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:53.910041 16011 raft_consensus.cc:1240] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:53.912606 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:53.907686 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:53.914583 16007 raft_consensus.cc:2393] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 19: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:53.917253 16003 raft_consensus.cc:1240] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 24 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 13:07:53.871546 22679 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:53.919692 16006 raft_consensus.cc:1240] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:53.920183 15902 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:53.921972 15902 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:53.897166 23006 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.008s	sys 0.015s
W20260709 13:07:53.923502 23006 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.008s	sys 0.015s
I20260709 13:07:53.923313 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:53.924126 15902 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:53.925657 16002 raft_consensus.cc:1275] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:07:53.898166 23005 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:53.929100 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:53.933547 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:53.935245 22988 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:53.933080 16561 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 19.
I20260709 13:07:53.937104 16004 raft_consensus.cc:1240] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:53.859611 23031 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:53.939224 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:53.944205 16008 raft_consensus.cc:1240] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:53.944815 22676 raft_consensus.cc:2749] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:53.935758 22988 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:53.945365 22988 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:53.946381 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:53.947359 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:53.948271 16570 raft_consensus.cc:2410] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:07:53.949059 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:53.949604 16570 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 16.
I20260709 13:07:53.950419 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:53.944279 15902 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:53.952441 22676 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:53.952898 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:53.953846 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:53.954598 16565 raft_consensus.cc:2393] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 18: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:53.955327 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:53.956383 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:53.952440 16561 raft_consensus.cc:1275] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:07:53.958297 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:53.959303 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:53.959889 16561 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 19.
I20260709 13:07:53.951403 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:53.960868 16570 raft_consensus.cc:2410] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 13:07:53.960566 22988 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:53.961241 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:53.961977 16561 raft_consensus.cc:2368] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 18. Current term is 19.
I20260709 13:07:53.961850 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:53.962585 16570 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 19.
I20260709 13:07:53.962760 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:53.963358 16561 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 17.
I20260709 13:07:53.963632 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:53.964859 16570 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:53.937166 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:53.975441 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:53.859990 22921 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:53.963099 22988 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:53.964012 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:53.934603 16005 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 19.
W20260709 13:07:53.970225 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:53.897907 23040 consensus_queue.cc:1048] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:07:53.972114 23041 consensus_queue.cc:1059] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:07:53.942330 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:53.916416 22985 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:53.927688 23067 raft_consensus.cc:2749] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:53.902204 23043 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:53.980034 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:53.982024 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:53.995360 16563 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 21.
I20260709 13:07:53.996250 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:53.996834 16563 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 21.
I20260709 13:07:53.998088 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:53.998926 16569 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:54.004387 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:54.005193 16563 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:54.006317 16563 raft_consensus.cc:1240] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:53.964201 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:07:53.982456 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:53.918982 23103 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:53.983700 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:07:53.987213 22984 consensus_meta.cc:287] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.509s	user 0.011s	sys 0.000s
I20260709 13:07:53.941310 23005 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:53.983565 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 289452us
I20260709 13:07:53.936087 22971 consensus_queue.cc:1059] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 13:07:54.023474 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:54.018857 15976 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:54.024158 16010 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 16.
I20260709 13:07:54.019202 16535 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:54.012310 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:54.025465 16005 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 19.
I20260709 13:07:53.973869 15902 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:54.014678 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:54.014982 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:54.026584 16002 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 17.
I20260709 13:07:54.018177 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:07:54.018546 15974 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.021s	sys 0.041s
I20260709 13:07:53.984793 22676 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:07:54.017004 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 319345us
I20260709 13:07:54.017450 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:54.029773 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:54.036505 16004 raft_consensus.cc:2410] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:07:54.039271 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:54.022111 15825 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:54.040877 15825 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:54.022804 16567 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 15.
I20260709 13:07:54.015784 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:07:54.028440 15974 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.022s	sys 0.041s
I20260709 13:07:54.028930 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:54.042886 16003 raft_consensus.cc:2393] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 19: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:54.041471 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:54.044660 16564 raft_consensus.cc:1275] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:07:54.044785 16006 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:07:54.043119 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 596485us
I20260709 13:07:54.051425 15933 raft_consensus.cc:1275] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 16. (term mismatch)
I20260709 13:07:54.051810 15933 pending_rounds.cc:85] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 15
I20260709 13:07:54.052029 15933 pending_rounds.cc:107] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 20.16
I20260709 13:07:54.052263 15933 raft_consensus.cc:2889] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:54.053054 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:54.053793 15930 raft_consensus.cc:2393] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 20: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:54.053102 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:54.061863 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:54.062788 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 89005.3us
I20260709 13:07:54.064455 15902 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:54.062868 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:54.066823 16565 raft_consensus.cc:2410] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:07:54.023517 23005 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:54.075029 15937 raft_consensus.cc:1240] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:54.075915 15862 raft_consensus.cc:1240] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 14. Current term is 16. Ops: []
I20260709 13:07:53.994581 23009 consensus_queue.cc:1059] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.041s
I20260709 13:07:54.079092 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:54.079610 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:54.081427 15930 raft_consensus.cc:1275] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:07:54.081754 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:54.082634 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:54.085088 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:54.085234 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:54.094942 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:54.032294 22527 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:54.015842 23006 consensus_queue.cc:1059] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.052s
I20260709 13:07:54.100205 15859 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:54.072386 23067 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:54.063179 22550 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:54.115976 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:54.116099 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:54.067408 23041 raft_consensus.cc:3055] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:54.063570 23000 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:54.119657 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:54.120568 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:54.068387 22980 raft_consensus.cc:2749] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:54.072496 22765 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.113518 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:54.095536 22492 consensus_meta.cc:287] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.811s	user 0.013s	sys 0.000s
I20260709 13:07:54.123929 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:54.072764 22659 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:54.126498 15902 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:54.032066 22984 consensus_queue.cc:1059] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.084s
I20260709 13:07:54.117162 22527 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:54.128541 15974 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:54.127848 15861 raft_consensus.cc:1275] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:07:54.139212 15933 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:54.136921 16006 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 19.
I20260709 13:07:54.137377 23061 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.145080 23006 raft_consensus.cc:3055] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:07:54.114864 23028 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:07:54.148100 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 82391.2us
I20260709 13:07:54.178320 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:54.130075 22550 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:54.125149 23040 raft_consensus.cc:2749] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:07:54.214716 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 66759us
I20260709 13:07:54.075373 22971 consensus_queue.cc:1048] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.065s
I20260709 13:07:54.127795 23074 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:54.121287 23018 consensus_queue.cc:1059] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 13:07:54.226575 15859 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 24.
I20260709 13:07:54.149787 15853 raft_consensus.cc:1240] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 21. Current term is 22. Ops: []
W20260709 13:07:54.237682 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 260158us
I20260709 13:07:54.138751 23041 raft_consensus.cc:740] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:54.125771 23067 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:54.117691 23076 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:07:54.258392 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 239306us
I20260709 13:07:54.267715 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:54.271147 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:54.307603 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:54.224020 22980 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.315102 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:54.217008 23000 raft_consensus.cc:697] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 20 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
W20260709 13:07:54.292482 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:54.279531 22550 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:54.293584 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:54.299153 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:54.264297 22527 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:54.211215 22492 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:54.309885 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:54.313145 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:54.313598 16006 raft_consensus.cc:1275] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:07:54.261621 23006 raft_consensus.cc:740] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:07:54.293452 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 165004us
W20260709 13:07:54.318635 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:54.323504 23105 log.cc:927] Time spent T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
W20260709 13:07:54.324370 23107 log.cc:927] Time spent T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
W20260709 13:07:54.325090 23106 log.cc:927] Time spent T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.199s	user 0.000s	sys 0.000s
I20260709 13:07:54.325794 22679 raft_consensus.cc:3055] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 LEADER]: Stepping down as leader of term 23
W20260709 13:07:54.325958 23109 log.cc:927] Time spent T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:07:54.326647 22564 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.327198 23099 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:54.331182 23076 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:54.332760 23041 consensus_queue.cc:260] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:54.307631 23066 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:54.347990 22980 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:54.359321 23021 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:07:54.362198 22816 consensus_queue.cc:1048] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:54.367736 22921 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:54.308439 23067 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:54.371501 23108 log.cc:927] Time spent T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:07:54.374608 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:54.376880 23031 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:54.258561 22659 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:54.316259 23075 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:07:54.516882 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:54.380399 23040 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:54.380591 22970 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:54.384722 22825 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:54.385068 16563 raft_consensus.cc:1275] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:07:54.553267 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:07:54.387216 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:54.213864 23017 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:54.395991 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:54.396960 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:54.291446 22957 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:54.405296 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:54.223778 22765 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:54.410535 16570 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 20.
I20260709 13:07:54.425469 23038 raft_consensus.cc:3055] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:54.430347 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:54.626272 22957 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:54.434937 15974 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:54.706343 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 271477us
I20260709 13:07:54.707275 15974 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:54.699962 15853 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:54.286727 23028 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:54.295239 23074 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:54.415290 22945 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:54.438923 22679 raft_consensus.cc:740] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:07:54.505508 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 183084us
I20260709 13:07:54.515470 15827 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:54.462121 23041 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:54.550622 16569 consensus_meta.cc:287] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.551s	user 0.009s	sys 0.000s
I20260709 13:07:54.448675 22564 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:54.507941 23066 raft_consensus.cc:697] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:54.553851 16563 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 19.
I20260709 13:07:54.656653 23090 consensus_queue.cc:1059] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:54.548856 23075 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:54.677734 22594 consensus_meta.cc:287] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 1.329s	user 0.012s	sys 0.000s
I20260709 13:07:54.678179 23017 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 23 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:54.679911 22980 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:54.681519 23009 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.746387 23009 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:54.747820 23100 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.748324 23100 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:07:54.748672 23009 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:54.682013 22921 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.682662 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:54.749665 15855 raft_consensus.cc:2393] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 20: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:54.749333 22921 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:54.685856 22527 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:54.448877 23099 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:54.753643 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:54.688860 23076 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:54.688953 22659 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:54.690384 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:54.686434 16570 raft_consensus.cc:1240] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:54.474269 23000 consensus_queue.cc:237] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 19.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:54.717547 22943 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:07:54.692956 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:54.693573 22988 raft_consensus.cc:3055] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:54.698341 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:07:54.694025 22755 consensus_meta.cc:287] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.935s	user 0.012s	sys 0.000s
I20260709 13:07:54.516243 22667 consensus_queue.cc:1048] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.175s
I20260709 13:07:54.547171 23031 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:54.554447 15860 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 20.
I20260709 13:07:54.278970 23061 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:54.702600 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:54.711179 15854 raft_consensus.cc:2393] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 23: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:54.851590 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:54.852788 15854 raft_consensus.cc:2393] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 23: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:07:54.854343 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:54.523689 23018 consensus_queue.cc:1048] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.176s
I20260709 13:07:54.725013 16569 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 20.
I20260709 13:07:54.725343 16535 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:54.720903 22765 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:54.710698 23091 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:54.853964 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:54.727553 22945 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:54.732230 23090 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.682022 22550 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:54.732982 23075 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:54.734143 22564 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:54.735162 23066 consensus_queue.cc:237] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:54.741427 23074 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:54.741811 23116 consensus_queue.cc:1059] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.013s
I20260709 13:07:54.742949 22679 consensus_queue.cc:260] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:54.712393 22957 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:54.701848 23038 raft_consensus.cc:740] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:54.744565 15853 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 20.
I20260709 13:07:54.693723 23040 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:54.754825 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:54.742419 23043 consensus_meta.cc:287] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.724s	user 0.004s	sys 0.005s
I20260709 13:07:54.548360 23006 consensus_queue.cc:260] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:54.790952 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:54.659770 23096 consensus_queue.cc:1048] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:07:54.802522 16533 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.028s	sys 0.036s
W20260709 13:07:54.811425 15902 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.003s	sys 0.042s
W20260709 13:07:54.902802 16533 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.028s	sys 0.036s
W20260709 13:07:54.902937 15902 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.003s	sys 0.042s
W20260709 13:07:54.903218 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 612253us
I20260709 13:07:54.812805 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:54.813476 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:54.750375 23100 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:54.871980 23018 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.796929 22874 raft_consensus.cc:2749] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:07:54.874253 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 149019us
I20260709 13:07:54.915545 16535 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:54.811928 22659 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:54.877573 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 279198us
I20260709 13:07:54.883023 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:54.811080 22988 raft_consensus.cc:740] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:07:54.873095 22985 consensus_meta.cc:287] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.857s	user 0.003s	sys 0.007s
I20260709 13:07:54.850018 22943 raft_consensus.cc:697] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 24 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:54.891403 15862 raft_consensus.cc:2393] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 20: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:07:54.903637 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 777133us
I20260709 13:07:54.953495 15902 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:54.911569 16533 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:54.908934 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 393533us
I20260709 13:07:54.956169 15827 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:54.913645 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:54.869885 23031 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:54.888824 23123 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:54.959599 16009 raft_consensus.cc:1240] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:54.929176 15859 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 16.
I20260709 13:07:54.930989 16002 raft_consensus.cc:1275] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:07:54.883221 23090 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:54.868988 22755 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:54.950284 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:54.965147 16008 raft_consensus.cc:1240] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:54.965179 15931 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:54.967518 16010 raft_consensus.cc:1275] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:07:54.919492 23091 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.982247 15902 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:54.981024 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:54.915593 23100 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:54.939771 23116 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:54.984076 15935 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
W20260709 13:07:54.984771 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 69353.7us
I20260709 13:07:54.990783 15902 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:54.997380 15902 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:55.009948 15854 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 20: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:07:54.969595 23103 consensus_meta.cc:287] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.946s	user 0.006s	sys 0.006s
I20260709 13:07:54.925596 22816 consensus_queue.cc:1059] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.058s
I20260709 13:07:54.965806 23030 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:54.937391 23101 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.907008 23043 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:55.017266 16569 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:55.022984 16570 raft_consensus.cc:3055] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:07:54.985168 22874 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.025867 16570 raft_consensus.cc:740] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:54.924248 23018 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:54.937763 22659 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:54.986485 23116 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:55.026968 16570 consensus_queue.cc:260] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:54.987044 23122 raft_consensus.cc:2749] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:55.044526 16570 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:55.009052 22921 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:55.008654 23088 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.010327 23134 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.013350 23076 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:55.013216 22564 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:55.011241 23075 raft_consensus.cc:2749] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:54.958503 22985 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:55.009747 23100 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:55.021296 23103 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:55.080211 23126 log.cc:927] Time spent T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
I20260709 13:07:55.081331 23031 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:55.081646 23103 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:55.083027 16568 raft_consensus.cc:2393] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 20: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:55.083648 23090 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:55.045594 23116 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:55.084744 23018 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:55.084952 23099 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:55.077781 22527 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:55.039326 16569 raft_consensus.cc:1275] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:07:55.062966 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 52141.3us
W20260709 13:07:55.031234 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 75186.2us
W20260709 13:07:55.058974 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 120520us
I20260709 13:07:54.987440 23091 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:54.959113 23038 consensus_queue.cc:260] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:55.050143 16566 raft_consensus.cc:2393] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 17: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:55.066287 15853 raft_consensus.cc:1275] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
W20260709 13:07:55.045816 23005 consensus_meta.cc:287] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.927s	user 0.012s	sys 0.001s
I20260709 13:07:55.039924 22874 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:55.033643 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 122157us
I20260709 13:07:55.088811 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:55.119910 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:55.120338 16533 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:55.118306 16562 raft_consensus.cc:2393] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 14: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
W20260709 13:07:55.031491 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 76739.1us
I20260709 13:07:55.069478 15902 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:55.112706 16570 raft_consensus.cc:1275] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 17. (term mismatch)
W20260709 13:07:55.122776 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 53383.4us
I20260709 13:07:55.089046 16535 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:55.129742 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:55.130776 15936 raft_consensus.cc:2393] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 20: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:55.094604 15827 leader_election.cc:400] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 election: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 18. Current term is 19.
I20260709 13:07:55.050122 23087 raft_consensus.cc:2749] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:07:55.041843 23030 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:07:55.083758 16564 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1578) took 1092 ms. Trace:
W20260709 13:07:55.132977 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 44420.9us
W20260709 13:07:55.139420 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 50486us
I20260709 13:07:55.140463 16535 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:55.140714 16006 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 19.
I20260709 13:07:55.028678 23101 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:55.134230 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:55.135294 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:55.065809 23122 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:55.141913 15827 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:55.040994 23132 consensus_queue.cc:1048] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.075s
I20260709 13:07:55.092635 22816 raft_consensus.cc:3055] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:55.147508 15936 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 18.
I20260709 13:07:55.029296 22988 consensus_queue.cc:260] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:55.148777 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 177735us
I20260709 13:07:55.138535 16564 rpcz_store.cc:276] 0709 13:07:53.991526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:54.065507 (+ 73981us) service_pool.cc:224] Handling call
0709 13:07:54.065782 (+   275us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:54.261979 (+196197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:54.262041 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:54.262769 (+   728us) log.cc:844] Serialized 26 byte log entry
0709 13:07:54.697115 (+434346us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:07:54.697153 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:54.697233 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:55.083512 (+386279us) raft_consensus.cc:1685] finished
0709 13:07:55.083536 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:55.083702 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":60406,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":14589,"thread_start_us":580109,"threads_started":2,"wal-append.queue_time_us":593626}
I20260709 13:07:55.157148 15827 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:55.092645 23061 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:55.158480 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:55.158528 16533 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:55.122815 16570 pending_rounds.cc:85] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5: Aborting all ops after (but not including) 16
I20260709 13:07:55.170825 16570 pending_rounds.cc:107] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5: Aborting uncommitted NO_OP operation due to leader change: 21.17
I20260709 13:07:55.171260 16570 raft_consensus.cc:2889] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:07:55.162458 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 66928us
I20260709 13:07:55.095826 22527 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:55.064841 23088 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:55.201337 15929 raft_consensus.cc:1275] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 13:07:55.139927 23076 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:55.139778 23103 raft_consensus.cc:2749] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:55.064363 23134 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:55.048012 22943 consensus_queue.cc:237] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:55.217191 15825 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:55.173449 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:55.249972 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:55.253754 15935 raft_consensus.cc:2393] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 21: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:55.254549 15929 pending_rounds.cc:85] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 8
I20260709 13:07:55.254851 15929 pending_rounds.cc:107] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 15.9
W20260709 13:07:55.174353 15902 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.103.4:43643 timed out after 1.749s (SENT) [suppressed 23 similar messages]
I20260709 13:07:55.186383 16533 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:07:55.263784 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 77496.3us
I20260709 13:07:55.264895 16533 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:55.253670 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:55.266235 15936 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:55.205009 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:55.255127 15929 raft_consensus.cc:2889] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:55.278852 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:55.283499 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 66435us
I20260709 13:07:55.284711 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:55.285630 16007 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:55.149475 23150 raft_consensus.cc:2749] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:55.290143 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:55.290374 16003 raft_consensus.cc:2368] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 21. Current term is 22.
I20260709 13:07:55.290993 15932 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:55.159191 22816 raft_consensus.cc:740] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:07:55.296703 16562 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45244 (request call id 1650) took 1350 ms (client timeout 1793 ms). Trace:
W20260709 13:07:55.297151 16562 rpcz_store.cc:269] 0709 13:07:53.946632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:53.952543 (+  5911us) service_pool.cc:224] Handling call
0709 13:07:55.296493 (+1343950us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54000f6ed4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:55.296646 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2678299392}
I20260709 13:07:55.298427 16563 raft_consensus.cc:1240] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:55.206480 23136 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:55.133246 23005 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:55.202019 23131 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.173137 23123 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:07:55.282822 23130 log.cc:927] Time spent T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.344s	user 0.000s	sys 0.001s
I20260709 13:07:55.335638 15936 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 20.
W20260709 13:07:55.340461 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 166113us
I20260709 13:07:55.171613 23087 raft_consensus.cc:2804] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:55.269421 23030 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:07:55.268791 22667 consensus_queue.cc:1048] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.084s
I20260709 13:07:55.169895 22689 consensus_queue.cc:1059] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.162s
I20260709 13:07:55.349253 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:55.343175 23138 log.cc:927] Time spent T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.265s	user 0.000s	sys 0.001s
I20260709 13:07:55.146919 23075 consensus_queue.cc:1048] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.143s
I20260709 13:07:55.353650 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:55.368914 23120 log.cc:927] Time spent T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:07:55.348999 22550 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:55.362907 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:55.202884 23139 consensus_queue.cc:1048] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.140s
I20260709 13:07:55.156435 23122 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:55.361691 15932 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 20.
I20260709 13:07:55.293871 23140 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:55.366703 16561 raft_consensus.cc:2393] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 23: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:55.299831 23029 raft_consensus.cc:2804] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:55.307294 23040 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:55.372514 23009 raft_consensus.cc:3055] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:07:55.373615 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:55.268033 22527 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:55.375384 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:55.375236 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:55.343988 23150 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:55.337584 23136 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:55.398415 16008 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 20.
I20260709 13:07:55.292520 23076 raft_consensus.cc:697] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 20 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:55.407919 23048 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:55.349833 23061 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:55.400796 16007 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:07:55.401330 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 149898us
I20260709 13:07:55.368404 23132 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.353636 23123 raft_consensus.cc:697] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:07:55.457594 23129 log.cc:927] Time spent T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.301s	user 0.000s	sys 0.001s
I20260709 13:07:55.458156 23123 consensus_queue.cc:237] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:55.367250 23087 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:55.470983 16563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 1520) took 1006 ms. Trace:
I20260709 13:07:55.471534 16563 rpcz_store.cc:276] 0709 13:07:54.464426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:54.737404 (+272978us) service_pool.cc:224] Handling call
0709 13:07:55.298242 (+560838us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:55.307726 (+  9484us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:55.470759 (+163033us) spinlock_profiling.cc:243] Waited 560 ms on lock 0x7b54000f6ed0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:55.470908 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1289007360}
I20260709 13:07:55.356456 23145 raft_consensus.cc:2749] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:07:55.412317 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 146061us
I20260709 13:07:55.479372 16533 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:55.426935 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:55.395495 23043 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:07:55.399313 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 225451us
I20260709 13:07:55.438060 15931 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 23.
I20260709 13:07:55.349380 23131 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:55.437747 22985 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:55.416971 23124 consensus_queue.cc:1048] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.057s
I20260709 13:07:55.412206 23029 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:55.446899 23144 log.cc:927] Time spent T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 13:07:55.399036 22816 consensus_queue.cc:260] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:55.544806 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:55.545487 16008 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:55.450464 23028 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:55.424736 22755 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:55.455569 16566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45244 (request call id 1644) took 1530 ms. Trace:
I20260709 13:07:55.553076 16566 rpcz_store.cc:276] 0709 13:07:53.925386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:53.941991 (+ 16605us) service_pool.cc:224] Handling call
0709 13:07:55.455347 (+1513356us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54006a09d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:55.455495 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2445950208}
W20260709 13:07:55.456717 23152 log.cc:927] Time spent T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:07:55.416867 23009 raft_consensus.cc:740] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:55.462354 15930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 1419) took 1145 ms. Trace:
I20260709 13:07:55.435657 23136 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:55.477272 23087 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:55.413976 23041 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:55.478049 23100 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:55.541199 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:55.514972 16007 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 20.
I20260709 13:07:55.541882 15974 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:07:55.492137 23067 consensus_meta.cc:287] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.977s	user 0.011s	sys 0.000s
I20260709 13:07:55.433501 23099 consensus_queue.cc:1059] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.102s
I20260709 13:07:55.413618 23140 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:55.567847 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:55.568392 22816 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:55.569403 23005 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:55.570226 23134 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:55.572906 22527 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:55.575559 23009 consensus_queue.cc:260] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:55.461752 23061 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:55.532856 22689 raft_consensus.cc:3055] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:55.545611 22874 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:55.441586 23091 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:55.576112 15930 rpcz_store.cc:276] 0709 13:07:54.314407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:54.319281 (+  4874us) service_pool.cc:224] Handling call
0709 13:07:54.319700 (+   419us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:54.403392 (+ 83692us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:54.403438 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:54.419466 (+ 16028us) log.cc:844] Serialized 52 byte log entry
0709 13:07:54.921276 (+501810us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:55.012296 (+ 91020us) log.cc:844] Serialized 14 byte log entry
0709 13:07:55.012832 (+   536us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:55.012922 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:55.459438 (+446516us) raft_consensus.cc:1685] finished
0709 13:07:55.459463 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:55.459631 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":83177,"prepare.queue_time_us":158561,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":83752,"raft.run_cpu_time_us":317,"raft.run_wall_time_us":22784,"thread_start_us":520302,"threads_started":2,"wal-append.queue_time_us":609811}
I20260709 13:07:55.411525 22659 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:55.536118 23150 raft_consensus.cc:2749] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:55.492511 23101 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:55.538025 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:55.538324 23048 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:55.460079 23076 consensus_queue.cc:237] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:55.551395 23029 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:55.589241 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 54255.2us
W20260709 13:07:55.600754 22825 consensus_meta.cc:287] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 1.045s	user 0.008s	sys 0.004s
I20260709 13:07:55.584463 23067 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:55.589485 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 49641.1us
I20260709 13:07:55.470458 23132 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:55.585634 23100 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.583611 23041 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:55.592685 22689 raft_consensus.cc:740] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:55.598752 23167 raft_consensus.cc:3055] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:55.622398 15857 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 19.
I20260709 13:07:55.625526 22492 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:55.632525 15862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58214 (request call id 1328) took 1554 ms. Trace:
I20260709 13:07:55.637532 16561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45244 (request call id 1655) took 1689 ms. Trace:
I20260709 13:07:55.675900 16561 rpcz_store.cc:276] 0709 13:07:53.948135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:53.963976 (+ 15841us) service_pool.cc:224] Handling call
0709 13:07:55.637344 (+1673368us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b5400550ad4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:55.637469 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3114658688}
I20260709 13:07:55.611315 23048 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:55.638185 23171 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:55.638895 23134 consensus_queue.cc:1048] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:07:55.640105 15933 raft_consensus.cc:2368] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 17. Current term is 18.
I20260709 13:07:55.640435 23131 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:55.641373 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:55.642988 23163 log.cc:927] Time spent T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 13:07:55.644901 22679 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:55.647647 23100 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:55.649631 23067 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:55.650139 23132 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:55.627455 15929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 3017) took 1150 ms. Trace:
I20260709 13:07:55.710697 15929 rpcz_store.cc:276] 0709 13:07:54.477138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:54.477305 (+   167us) service_pool.cc:224] Handling call
0709 13:07:54.477591 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:55.282241 (+804650us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:55.627168 (+344927us) spinlock_profiling.cc:243] Waited 723 ms on lock 0x7b540044b6d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:55.627384 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1663552384}
I20260709 13:07:55.575843 23116 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:55.608035 23009 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:55.655318 15853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 2720) took 1242 ms. Trace:
I20260709 13:07:55.652808 23140 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:55.653882 23088 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:55.660809 22689 consensus_queue.cc:260] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:55.609989 23091 raft_consensus.cc:2749] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:55.637594 23061 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:55.611536 23122 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:55.664541 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:55.666309 23087 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:55.666865 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 51654us
I20260709 13:07:55.654791 23041 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:55.668692 22564 consensus_meta.cc:287] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.594s	user 0.009s	sys 0.001s
I20260709 13:07:55.670428 23029 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:55.672736 16008 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 20.
I20260709 13:07:55.663487 15974 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:55.675101 15862 rpcz_store.cc:276] 0709 13:07:54.078424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:54.078684 (+   260us) service_pool.cc:224] Handling call
0709 13:07:55.632333 (+1553649us) spinlock_profiling.cc:243] Waited 811 ms on lock 0x7b54003b5e54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:55.632465 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1865497856}
I20260709 13:07:55.693228 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:55.730422 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:55.698540 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:55.731992 16567 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 19.
I20260709 13:07:55.733071 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:55.733803 16567 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 22.
I20260709 13:07:55.734767 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:55.735374 16567 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:07:55.676445 22957 consensus_meta.cc:287] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.777s	user 0.008s	sys 0.004s
I20260709 13:07:55.692080 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:55.704528 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:55.706257 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:55.673939 23167 raft_consensus.cc:740] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:55.714835 15853 rpcz_store.cc:276] 0709 13:07:54.413089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:54.901184 (+488095us) service_pool.cc:224] Handling call
0709 13:07:54.901529 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:55.097097 (+195568us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:55.655055 (+557958us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b54003b6854. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:55.655259 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":378073984}
I20260709 13:07:55.740566 16561 raft_consensus.cc:2393] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 23: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:55.740723 16562 raft_consensus.cc:2393] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 17: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:55.673183 23136 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:55.724421 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:55.725072 16011 raft_consensus.cc:1240] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:55.742094 16565 raft_consensus.cc:2393] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 21: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:55.694911 16569 raft_consensus.cc:1240] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 16. Current term is 17. Ops: [16.15-16.15]
W20260709 13:07:55.742722 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 79384.7us
I20260709 13:07:55.737986 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:55.739054 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:55.740851 16564 raft_consensus.cc:2393] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 20: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:55.741497 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:55.725557 16007 raft_consensus.cc:1240] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:55.742815 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:55.674775 22508 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.746646 16011 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:55.747311 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:55.751077 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:55.752827 16007 raft_consensus.cc:1240] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:07:55.757649 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:55.759184 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:55.759897 16008 raft_consensus.cc:2393] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 17: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:55.758229 16004 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 15.
I20260709 13:07:55.758637 15974 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:55.688205 23171 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:55.768211 16567 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 20.
I20260709 13:07:55.731307 22564 raft_consensus.cc:2749] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:55.774271 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:55.775475 15974 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:55.749323 22689 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:55.743824 23088 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.754990 23132 raft_consensus.cc:2749] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:07:55.769789 23123 consensus_queue.cc:1059] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
I20260709 13:07:55.794963 16011 raft_consensus.cc:2468] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 20.
I20260709 13:07:55.798084 23136 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:55.796531 15974 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:55.799924 15974 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:55.800902 22943 raft_consensus.cc:2749] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:55.801584 23168 raft_consensus.cc:2804] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:55.802208 23168 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:55.802682 23168 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:55.809142 23123 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.809921 23123 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:55.813980 23123 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:55.814584 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:55.816722 16010 raft_consensus.cc:2393] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 22: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:55.774597 22816 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:55.757400 22508 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:07:55.828454 15854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 2716) took 1416 ms. Trace:
I20260709 13:07:55.829224 15854 rpcz_store.cc:276] 0709 13:07:54.411924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:54.853591 (+441667us) service_pool.cc:224] Handling call
0709 13:07:55.828160 (+974569us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b5400427cd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:55.828364 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":313785728}
I20260709 13:07:55.833619 23038 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:55.843623 22945 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:55.843839 23177 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.844403 23177 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:55.845329 15857 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 2704) took 2998 ms (client timeout 2219 ms). Trace:
W20260709 13:07:55.845749 15857 rpcz_store.cc:269] 0709 13:07:52.847167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:52.847386 (+   219us) service_pool.cc:224] Handling call
0709 13:07:55.845119 (+2997733us) spinlock_profiling.cc:243] Waited 2.73 s on lock 0x7b54001009d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:55.845224 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6284167936}
I20260709 13:07:55.846624 23177 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:55.848232 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:55.848619 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:55.848888 16567 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:55.748884 22957 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:55.851889 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:55.853245 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:55.853796 22957 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:55.854465 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:55.853936 16562 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 20: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:55.855123 16564 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 19.
I20260709 13:07:55.792308 23131 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:55.857275 15976 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:55.857720 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:55.858357 16569 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 20.
I20260709 13:07:55.859408 23087 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:55.817227 23088 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:55.864490 22988 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:55.866313 23029 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:55.867964 23048 leader_election.cc:290] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:55.874639 23176 log.cc:927] Time spent T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:07:55.876789 23100 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:55.881808 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:55.882160 22816 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:55.882788 16570 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 17.
I20260709 13:07:55.851960 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:55.780503 23134 raft_consensus.cc:2749] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:07:55.911444 15860 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 2871) took 1834 ms (client timeout 1695 ms). Trace:
W20260709 13:07:55.912045 15933 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53636 (request call id 1418) took 1602 ms (client timeout 1826 ms). Trace:
W20260709 13:07:55.926964 15933 rpcz_store.cc:269] 0709 13:07:54.309489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:54.309640 (+   151us) service_pool.cc:224] Handling call
0709 13:07:55.911855 (+1602215us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b5400198054. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:55.911973 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2848526720}
I20260709 13:07:55.912801 16006 raft_consensus.cc:1240] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:55.924597 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:55.822351 23167 consensus_queue.cc:260] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:55.925858 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:55.892705 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:55.926067 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 124803us
W20260709 13:07:55.926874 15860 rpcz_store.cc:269] 0709 13:07:54.076589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:54.076744 (+   155us) service_pool.cc:224] Handling call
0709 13:07:55.906585 (+1829841us) spinlock_profiling.cc:243] Waited 407 ms on lock 0x7b54003b6d54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:55.911360 (+  4775us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":935425792}
W20260709 13:07:55.929265 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:55.930277 16569 raft_consensus.cc:1275] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:07:55.935626 16533 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:55.936028 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:55.928928 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:55.924985 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:55.925711 15902 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:55.928462 16010 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 17.
I20260709 13:07:55.965045 16570 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:55.965546 15974 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:55.965699 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:55.966708 16004 raft_consensus.cc:2393] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 22: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:55.966318 16010 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:55.966082 16007 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:55.968299 16567 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 22.
I20260709 13:07:55.969503 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:55.969866 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:55.970971 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:55.971232 16569 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Advancing to term 18
W20260709 13:07:55.965075 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:55.976696 16533 leader_election.cc:400] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) with higher term. Message: Invalid argument: T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 17. Current term is 18.
I20260709 13:07:55.977505 16533 leader_election.cc:403] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:55.976789 15974 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:55.979326 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:55.964488 15976 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:55.985709 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:55.986413 16008 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:55.988656 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:55.989964 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:55.990447 16004 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:55.990650 15827 leader_election.cc:400] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 21. Current term is 22.
I20260709 13:07:55.991096 15827 leader_election.cc:403] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:55.990963 16570 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 22.
W20260709 13:07:55.991971 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 66326.3us
I20260709 13:07:55.992790 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:55.912042 23102 raft_consensus.cc:3055] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:55.987710 15976 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:07:55.995085 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:56.003472 16005 raft_consensus.cc:2410] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:07:56.005924 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:56.006626 16570 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:56.006832 16007 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 18.
I20260709 13:07:56.007350 16569 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 18.
I20260709 13:07:56.008914 16007 raft_consensus.cc:1240] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:07:56.009142 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:55.996517 15827 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:56.004935 15902 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:55.920210 23131 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:55.814107 23168 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:56.043334 23131 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:56.043831 23131 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:56.044948 23168 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:55.986047 23115 raft_consensus.cc:2749] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:56.013351 15827 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:55.986382 23076 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:56.039235 15974 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:55.942366 23136 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:55.974227 22957 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:56.050066 16004 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 17.
I20260709 13:07:55.974525 23006 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:07:56.052102 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 47245.2us
I20260709 13:07:55.986196 22564 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.991494 22603 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:56.011168 23147 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Advancing to term 22
W20260709 13:07:56.055720 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:56.064910 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 51682.9us
I20260709 13:07:56.004835 23100 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:56.060667 15902 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:07:56.001659 15900 connection.cc:256] RPC call timeout handler was delayed by -1.0861996229999704s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2219s for a response
I20260709 13:07:56.036978 23066 raft_consensus.cc:2804] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:56.061961 16004 raft_consensus.cc:1240] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 18. Current term is 19. Ops: [18.9-18.9]
I20260709 13:07:56.063124 16008 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 21.
I20260709 13:07:56.012441 23048 raft_consensus.cc:2749] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) with higher term. Message: Invalid argument: T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 17. Current term is 18.
I20260709 13:07:56.011533 23122 raft_consensus.cc:2749] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:56.005280 23102 raft_consensus.cc:740] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:56.065724 15827 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:56.039713 23116 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:55.973727 22921 consensus_queue.cc:1059] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.076s
I20260709 13:07:56.036357 23171 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:56.011725 23041 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:56.001220 23087 consensus_queue.cc:1048] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 13:07:56.001365 23104 consensus_queue.cc:1048] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.088s
W20260709 13:07:56.114701 15900 connection.cc:256] RPC call timeout handler was delayed by -1.04546565499993s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17930000000000001s for a response
I20260709 13:07:56.196094 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:56.248371 16010 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:56.212332 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:56.250082 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:56.196094 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:56.208829 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:56.182343 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:56.066859 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:07:56.154841 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 89147.6us
I20260709 13:07:56.252817 16005 raft_consensus.cc:2393] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 20: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:56.038784 23142 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:56.115365 15825 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:56.253041 15827 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:07:56.257035 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 141714us
I20260709 13:07:56.183475 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:56.258320 15854 raft_consensus.cc:2393] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 16: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:56.183878 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:56.260106 15933 raft_consensus.cc:1240] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:56.260483 15929 raft_consensus.cc:1240] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 22 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 21. Current term is 22. Ops: []
W20260709 13:07:56.179229 23182 log.cc:927] Time spent T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
W20260709 13:07:56.088500 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:56.064141 23168 raft_consensus.cc:2749] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:56.262250 15853 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:07:56.266126 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:56.273675 15935 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 21.
I20260709 13:07:56.268452 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:56.059219 22970 raft_consensus.cc:2804] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:56.279840 22970 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.261564 23142 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.280287 22970 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:56.280435 23142 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:56.254856 22985 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:56.282886 22985 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:56.171747 22816 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:56.285205 22985 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:56.066421 23136 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:56.068688 23038 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:56.286473 23038 raft_consensus.cc:697] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 20 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:56.287413 23038 consensus_queue.cc:237] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:56.288741 16563 raft_consensus.cc:1240] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:07:56.296221 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:56.173895 23040 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.066545 22957 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:56.285418 15856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43520 (request call id 2711) took 1890 ms. Trace:
I20260709 13:07:56.167063 23179 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:07:56.285885 16568 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45244 (request call id 1648) took 2339 ms (client timeout 1749 ms). Trace:
I20260709 13:07:56.288661 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:56.251641 23100 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.265038 23041 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:56.292044 15974 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:56.309087 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:56.309366 15974 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:56.293578 23177 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:56.294440 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:56.066249 22564 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:56.184074 23126 log.cc:927] Time spent T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:07:56.315768 15858 raft_consensus.cc:1275] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:07:56.159586 22594 raft_consensus.cc:2804] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:56.265961 23090 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:07:56.276549 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:56.270145 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:56.295871 15937 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 17.
I20260709 13:07:56.287331 23136 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:56.176199 23088 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:56.308600 23100 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:56.149386 23031 raft_consensus.cc:2804] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:56.307054 23153 consensus_queue.cc:1059] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 13:07:56.248149 23066 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:56.301256 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:56.333528 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:56.334571 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:56.335457 15855 raft_consensus.cc:2410] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:07:56.119813 22508 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:07:56.270805 23009 consensus_meta.cc:287] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.553s	user 0.003s	sys 0.007s
W20260709 13:07:56.304836 16568 rpcz_store.cc:269] 0709 13:07:53.946007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:53.946150 (+   143us) service_pool.cc:224] Handling call
0709 13:07:56.285645 (+2339495us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54006a1654. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:56.285791 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2612183424}
I20260709 13:07:56.305547 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:56.335453 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:56.336617 15928 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 20.
I20260709 13:07:56.337126 23031 raft_consensus.cc:697] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 20 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:56.337450 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:56.337646 15857 raft_consensus.cc:1275] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:07:56.338161 15928 raft_consensus.cc:2393] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 23: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:56.338531 22564 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:56.339100 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:56.300181 23156 consensus_queue.cc:1048] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:07:56.344017 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:56.300925 15856 rpcz_store.cc:276] 0709 13:07:54.394999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:54.395152 (+   153us) service_pool.cc:224] Handling call
0709 13:07:54.395623 (+   471us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:54.396128 (+   505us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:54.396168 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:54.396901 (+   733us) log.cc:844] Serialized 52 byte log entry
0709 13:07:55.688755 (+1291854us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b1800427e40. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f480d1 00007f73ea363a0f
0709 13:07:55.792350 (+103595us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:55.820994 (+ 28644us) log.cc:844] Serialized 14 byte log entry
0709 13:07:55.821274 (+   280us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:55.821379 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:56.285112 (+463733us) raft_consensus.cc:1685] finished
0709 13:07:56.285210 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:56.285361 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":28439,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":7491,"raft.run_cpu_time_us":294,"raft.run_wall_time_us":17478,"spinlock_wait_cycles":24856448,"thread_start_us":94119,"threads_started":1,"wal-append.queue_time_us":1565471}
I20260709 13:07:56.345213 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:56.304107 23040 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:56.329303 23158 consensus_queue.cc:1048] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.017s
I20260709 13:07:56.348906 23040 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:56.351644 22508 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:56.352483 16569 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 21.
I20260709 13:07:56.303966 23104 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.353132 23104 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:56.354079 22679 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.354298 23041 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:56.354593 22679 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:56.355149 23104 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:56.311056 15935 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 18.
I20260709 13:07:56.319697 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:56.334600 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:56.358672 15858 raft_consensus.cc:2376] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 22. Re-sending same reply.
I20260709 13:07:56.065819 23076 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:56.298848 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:56.359997 15976 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:56.361204 22957 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:56.336658 23066 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:56.363498 16567 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 22.
I20260709 13:07:56.363739 23076 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:56.322959 22492 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.365962 22492 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:56.368489 23153 raft_consensus.cc:3055] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:07:56.368973 23153 raft_consensus.cc:740] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:56.370007 23153 consensus_queue.cc:260] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:56.319293 23140 raft_consensus.cc:2749] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:56.372256 23140 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:56.372656 23140 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.372984 23140 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:56.374135 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:56.322649 23168 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:56.255160 23067 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.375201 23168 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.375412 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:56.375640 23168 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:56.375443 23067 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:56.378399 23067 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:56.379562 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:56.379945 23009 consensus_queue.cc:1048] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:07:56.380728 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:56.381325 15857 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 24.
I20260709 13:07:56.379776 23147 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 21. Current term is 22.
I20260709 13:07:56.323364 22603 raft_consensus.cc:2749] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:56.332641 23090 raft_consensus.cc:697] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 23 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:56.383244 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:56.371484 23153 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:56.385257 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:56.386567 15858 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 22.
I20260709 13:07:56.327217 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:56.338058 23031 consensus_queue.cc:237] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:56.352231 22508 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:56.322369 23177 raft_consensus.cc:697] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 20 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:56.390244 15933 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 18.
I20260709 13:07:56.383460 23090 consensus_queue.cc:237] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:56.390893 23177 consensus_queue.cc:237] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:56.392176 22508 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:56.390110 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:56.395102 15928 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:07:56.311089 23179 raft_consensus.cc:697] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:56.398613 16535 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:56.405747 23043 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:07:56.408844 16002 raft_consensus.cc:1275] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:07:56.409734 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:56.410393 16568 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 22.
I20260709 13:07:56.410748 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:56.411648 16561 raft_consensus.cc:1275] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 17. (term mismatch)
I20260709 13:07:56.412106 16561 pending_rounds.cc:85] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5: Aborting all ops after (but not including) 16
I20260709 13:07:56.411950 15853 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 18.
I20260709 13:07:56.412525 16561 pending_rounds.cc:107] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5: Aborting uncommitted NO_OP operation due to leader change: 23.17
I20260709 13:07:56.412865 16561 raft_consensus.cc:2889] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:56.420277 15928 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 14.
I20260709 13:07:56.262024 23171 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:56.428473 15935 raft_consensus.cc:1240] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:07:56.428907 15933 raft_consensus.cc:1275] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:07:56.434682 15825 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:56.301985 15827 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:56.417824 15976 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:07:56.437129 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 135237us
I20260709 13:07:56.438472 15827 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:56.430563 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:56.321568 22594 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:56.438759 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:56.436944 15825 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:56.449738 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:56.452567 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:56.454696 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 133993us
I20260709 13:07:56.440116 23104 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:56.440515 23040 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.450913 22551 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:56.453744 22594 raft_consensus.cc:3060] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:56.500057 23136 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:56.500864 23147 raft_consensus.cc:2804] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:56.470358 23143 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:56.440289 23048 consensus_queue.cc:1048] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.053s
I20260709 13:07:56.503332 22551 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.509068 23143 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.512177 22551 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:56.512406 23143 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:56.514092 23142 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:56.521783 22492 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:56.443447 23171 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:56.494526 23104 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.493760 23162 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.405323 23179 consensus_queue.cc:237] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:56.542357 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:56.506963 23136 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:56.556810 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:56.565008 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:56.502290 23066 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:07:56.535089 23104 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:56.509572 23147 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.505192 23040 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:56.525305 23131 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:56.577781 23136 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:07:56.588466 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 125455us
W20260709 13:07:56.598158 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 537506us
I20260709 13:07:56.598917 15902 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:07:56.600162 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 161789us
I20260709 13:07:56.587507 23147 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:56.539011 23162 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:56.614493 23031 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:56.611811 15902 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:07:56.672711 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 60970.8us
I20260709 13:07:56.673527 15902 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:56.594276 23018 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:56.653805 15827 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:56.663168 15862 raft_consensus.cc:1275] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:07:56.676314 15902 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:56.677947 22492 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:56.678130 23067 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:56.678542 22492 raft_consensus.cc:697] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 18 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:56.678663 23067 raft_consensus.cc:697] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 17 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:56.679124 23102 consensus_queue.cc:260] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:56.679421 22492 consensus_queue.cc:237] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:56.680541 23102 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:56.679453 23067 consensus_queue.cc:237] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:56.686196 22551 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:56.688687 23040 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:56.621441 23038 consensus_queue.cc:1059] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.059s
I20260709 13:07:56.615363 23052 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.062s
W20260709 13:07:56.698290 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 44565.6us
I20260709 13:07:56.702579 15902 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:56.705535 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:56.713477 16564 raft_consensus.cc:1275] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:07:56.642095 22970 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:56.673142 23031 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:56.660243 22689 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.723423 15902 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:56.724829 15827 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:56.725330 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:56.700009 23090 raft_consensus.cc:2749] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:07:56.716603 23203 log.cc:927] Time spent T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:07:56.710176 22957 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:56.734488 16005 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 18.
I20260709 13:07:56.738967 15827 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:56.728430 23031 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:56.740065 15902 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:07:56.743749 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 306953us
I20260709 13:07:56.724766 23142 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:56.722855 23076 raft_consensus.cc:2749] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:56.778631 15827 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:56.741364 15932 raft_consensus.cc:2393] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 16: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:56.732098 22508 raft_consensus.cc:3055] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:56.705889 23018 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:56.786341 16533 leader_election.cc:304] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:56.715329 23140 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:56.790143 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:56.802479 23140 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:56.802196 23067 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:56.805430 23067 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:56.808468 23067 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:56.737054 22564 raft_consensus.cc:2749] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:56.793042 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:56.715216 23100 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:56.715519 23168 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:56.795374 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 55387.4us
I20260709 13:07:56.798167 16533 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:56.780555 23046 consensus_queue.cc:1048] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.019s
I20260709 13:07:56.799510 22985 raft_consensus.cc:2749] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:07:56.800175 23204 log.cc:927] Time spent T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:07:56.726084 23041 consensus_queue.cc:1048] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.054s
I20260709 13:07:56.800680 23171 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:56.740703 23206 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:07:56.791045 23196 log.cc:927] Time spent T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.117s	user 0.001s	sys 0.001s
I20260709 13:07:56.844203 23162 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:56.784682 23029 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:07:56.849721 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:56.850507 16569 raft_consensus.cc:2393] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 14: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:56.797142 23090 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:56.737671 23153 consensus_queue.cc:1048] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.063s
I20260709 13:07:56.824345 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:56.735826 22689 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:56.874739 23067 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:56.875622 22689 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:56.825455 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:56.876442 16570 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 18.
I20260709 13:07:56.876899 23067 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:56.884202 22594 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:56.885128 23031 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:56.886339 22594 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:56.826953 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:56.887051 23031 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:56.887362 15860 raft_consensus.cc:2393] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 20: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:07:56.889518 15937 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 3024) took 1686 ms (client timeout 2200 ms). Trace:
W20260709 13:07:56.890019 15937 rpcz_store.cc:269] 0709 13:07:55.203491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:55.204690 (+  1199us) service_pool.cc:224] Handling call
0709 13:07:56.889238 (+1684548us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b540008a2d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:56.889380 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2357640960}
I20260709 13:07:56.827078 16533 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:56.831401 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:56.891594 16567 raft_consensus.cc:2393] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 17: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:56.831743 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:56.832007 23123 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 13:07:56.834849 23193 log.cc:927] Time spent T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.235s	user 0.001s	sys 0.000s
I20260709 13:07:56.836596 22970 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:56.837586 22551 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:56.839881 23131 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:56.797349 22508 raft_consensus.cc:740] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
W20260709 13:07:56.791515 23183 log.cc:927] Time spent T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.059s	user 0.004s	sys 0.006s
I20260709 13:07:56.825803 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:56.879344 23066 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:56.824597 23110 raft_consensus.cc:2804] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:56.894906 16568 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 17.
I20260709 13:07:56.876592 16562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1600) took 1025 ms. Trace:
W20260709 13:07:56.911693 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 182147us
W20260709 13:07:56.942638 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:56.995884 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:56.998545 16535 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:56.887084 16561 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 21.
W20260709 13:07:56.952224 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 125253us
I20260709 13:07:57.001664 16533 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:57.002763 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:57.000388 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:57.006933 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:57.007663 16002 raft_consensus.cc:2393] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 18: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:57.007483 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:57.008229 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:57.008256 16008 raft_consensus.cc:2368] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 21. Current term is 22.
I20260709 13:07:57.009920 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:57.010484 16010 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 20.
I20260709 13:07:56.991886 16562 rpcz_store.cc:276] 0709 13:07:55.850733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:55.855839 (+  5106us) service_pool.cc:224] Handling call
0709 13:07:55.856174 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:55.883363 (+ 27189us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:07:56.061705 (+178342us) log.cc:844] Serialized 14 byte log entry
0709 13:07:56.061890 (+   185us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:56.061933 (+    43us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:07:56.061972 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:56.876371 (+814399us) spinlock_profiling.cc:243] Waited 27 ms on lock 0x7b540059c9d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:56.876418 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:56.876533 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":292000,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":78,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":50859,"spinlock_wait_cycles":62037760,"thread_start_us":177965,"threads_started":1}
W20260709 13:07:57.012864 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:57.016417 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:57.018026 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:56.948182 23123 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.828858 23143 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:57.020004 16533 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:57.021713 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:57.026036 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:57.026813 16568 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:56.949489 23029 raft_consensus.cc:697] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 22 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:57.034356 16533 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
W20260709 13:07:57.036293 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:57.036742 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:57.032698 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:56.988667 22985 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:57.038280 16008 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 20.
I20260709 13:07:57.032927 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:57.039808 16007 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 16.
I20260709 13:07:57.037544 15931 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:07:57.002745 23110 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:57.047782 22508 consensus_queue.cc:260] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:57.049160 22508 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:57.052482 23202 log.cc:927] Time spent T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:07:57.052209 23029 consensus_queue.cc:237] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:57.052801 23048 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.956641 23087 consensus_queue.cc:1048] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.079s
I20260709 13:07:57.030122 23168 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:57.053545 23048 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:57.057468 23048 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:56.948477 23090 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:56.950500 23136 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:57.024207 23103 raft_consensus.cc:2804] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:57.076623 23147 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:07:57.077514 23103 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:57.077950 23103 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:57.030112 23040 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:57.023972 23123 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:57.042237 23102 raft_consensus.cc:2764] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 13:07:57.070963 23090 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:57.044902 23043 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:57.073870 23018 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:57.045655 23110 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:57.077049 23136 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 23 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:57.079329 23147 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:57.085430 22957 raft_consensus.cc:697] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 21 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:07:57.115190 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 295114us
I20260709 13:07:57.116379 16568 raft_consensus.cc:2468] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 22.
I20260709 13:07:57.085713 23043 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:57.117800 15976 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:57.120157 22957 consensus_queue.cc:237] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:57.123858 23215 raft_consensus.cc:2804] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:07:57.124470 23215 raft_consensus.cc:697] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:57.125443 23215 consensus_queue.cc:237] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:57.142856 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 293667us
W20260709 13:07:57.152882 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:57.155805 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:57.156827 16002 raft_consensus.cc:2393] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 20: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:57.159750 16010 raft_consensus.cc:1275] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:07:57.161286 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:57.167366 16010 raft_consensus.cc:1240] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 13:07:57.169552 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:57.170394 15935 raft_consensus.cc:2393] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 20: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:57.141228 23043 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:57.176637 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:57.148465 23104 consensus_meta.cc:287] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.557s	user 0.010s	sys 0.000s
I20260709 13:07:57.181100 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:57.182082 15936 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:07:57.183859 15825 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:57.187037 16002 raft_consensus.cc:1240] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:57.191179 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:57.195513 15937 raft_consensus.cc:1275] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
W20260709 13:07:57.179733 23177 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.017s	sys 0.018s
I20260709 13:07:57.199545 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:57.200059 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:57.199940 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:57.201741 15937 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 19.
I20260709 13:07:57.189998 23147 raft_consensus.cc:2804] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:57.204257 23147 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:57.204685 23147 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:57.200632 23177 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.018s	sys 0.018s
I20260709 13:07:57.218551 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:57.221148 15932 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:07:57.187191 23104 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:57.200096 23155 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:07:57.237214 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:57.240413 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:57.241310 15857 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 20.
I20260709 13:07:57.241422 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:57.244261 15932 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 20.
I20260709 13:07:57.244551 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:57.246948 16002 raft_consensus.cc:1240] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:57.248612 15825 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:57.254808 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:57.254680 23018 consensus_queue.cc:1048] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:07:57.264904 16563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 1526) took 1568 ms. Trace:
I20260709 13:07:57.332094 16563 rpcz_store.cc:276] 0709 13:07:55.696581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:55.696717 (+   136us) service_pool.cc:224] Handling call
0709 13:07:56.288514 (+591797us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:56.296413 (+  7899us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:57.264683 (+968270us) spinlock_profiling.cc:243] Waited 574 ms on lock 0x7b54006a1650. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a78ad 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:57.264837 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1319720192}
I20260709 13:07:57.265841 15860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 2884) took 1084 ms. Trace:
I20260709 13:07:57.277613 23104 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:57.244099 23136 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:57.293594 23110 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:57.295362 23103 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:57.295826 23088 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:07:57.293939 23043 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:57.298377 15854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43524 (request call id 2892) took 1022 ms. Trace:
I20260709 13:07:57.350755 15854 rpcz_store.cc:276] 0709 13:07:56.275985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:56.276237 (+   252us) service_pool.cc:224] Handling call
0709 13:07:57.298138 (+1021901us) spinlock_profiling.cc:243] Waited 913 ms on lock 0x7b5400427cd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:57.298302 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2100288256}
I20260709 13:07:57.352641 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:57.353477 15854 raft_consensus.cc:2393] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 17: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:57.246218 23155 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:57.302798 23100 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:57.305802 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:57.364614 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:57.365676 15853 raft_consensus.cc:2393] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 21: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:57.321348 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:57.366290 15928 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 22.
I20260709 13:07:57.321394 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:57.322407 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:57.323863 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:57.326018 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:57.368682 15861 raft_consensus.cc:2393] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 20: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:57.328256 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:57.370772 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:57.332578 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 84016.5us
I20260709 13:07:57.333173 15860 rpcz_store.cc:276] 0709 13:07:56.181944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:56.182087 (+   143us) service_pool.cc:224] Handling call
0709 13:07:57.258557 (+1076470us) spinlock_profiling.cc:243] Waited 636 ms on lock 0x7b540032da54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:07:57.265770 (+  7213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1462351104}
I20260709 13:07:57.336843 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:57.372393 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:57.338537 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:57.372587 15825 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:57.349404 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:57.373054 16004 raft_consensus.cc:2393] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 20: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:57.374384 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:57.374678 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:57.342716 16011 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:07:57.376060 16006 raft_consensus.cc:2393] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 21: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:07:57.343497 23018 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:57.377384 15857 raft_consensus.cc:1240] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 22 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:57.378871 15859 raft_consensus.cc:2393] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 19: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:57.346160 23078 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.380128 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:57.381719 15858 raft_consensus.cc:1240] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:07:57.383754 15855 raft_consensus.cc:2393] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 16: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:57.385823 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:57.386252 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:57.350657 23221 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.381287 15827 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:57.387357 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:57.390470 16011 raft_consensus.cc:2393] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 20: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:57.393195 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:57.393626 16004 raft_consensus.cc:1275] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:07:57.393953 15861 raft_consensus.cc:2393] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 20: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:57.391405 15827 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:57.394304 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:57.394517 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:57.395011 15933 raft_consensus.cc:2393] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 20: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:57.395686 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:57.396202 15934 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:57.395030 15930 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:57.393998 15937 raft_consensus.cc:1275] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:07:57.344271 23123 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:57.400756 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 207413us
I20260709 13:07:57.402309 15974 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:57.410645 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:57.410683 16533 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:57.414489 15974 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:57.348675 23136 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:57.426447 23136 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:57.426710 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:57.427403 16009 raft_consensus.cc:2368] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 22. Current term is 23.
I20260709 13:07:57.428274 23143 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 23 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:57.430486 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:57.431321 15933 raft_consensus.cc:2393] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 20: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:57.389182 22594 raft_consensus.cc:2804] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:57.432132 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:57.432739 15933 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 20.
I20260709 13:07:57.433605 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:57.434176 15933 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 18.
I20260709 13:07:57.395534 23221 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:57.435500 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:57.436625 23221 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:57.393900 23078 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:57.434537 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:57.438026 16009 raft_consensus.cc:2368] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 17. Current term is 18.
I20260709 13:07:57.419610 15902 leader_election.cc:400] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 21. Current term is 22.
I20260709 13:07:57.432816 15931 raft_consensus.cc:1240] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:57.438573 23078 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:57.383173 23018 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:57.439146 23018 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:57.421729 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:57.440243 15932 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:57.443841 15854 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:57.448892 23110 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:57.449440 15902 leader_election.cc:403] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:57.450203 22957 raft_consensus.cc:3060] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Advancing to term 22
W20260709 13:07:57.450610 23223 log.cc:927] Time spent T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:07:57.436484 15900 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:57.356248 23147 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:57.451610 23212 consensus_queue.cc:1059] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:07:57.452176 23102 raft_consensus.cc:2804] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:57.452704 23102 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:57.453099 23102 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:57.453400 23147 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:57.454255 23218 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:57.454963 15900 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:57.394626 23206 raft_consensus.cc:2804] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:57.455760 23218 raft_consensus.cc:2749] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:57.457052 15900 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:57.457806 23218 raft_consensus.cc:2749] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:07:57.458590 23123 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 25 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:57.458981 15900 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:57.459846 15932 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 23.
I20260709 13:07:57.451596 15902 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:57.461557 22957 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 21. Current term is 22.
W20260709 13:07:57.461755 23227 log.cc:927] Time spent T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:07:57.464421 23043 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:57.471928 23218 raft_consensus.cc:2749] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:57.472636 23218 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:57.472512 15902 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:57.473039 23218 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:57.473405 23218 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:57.473443 23217 raft_consensus.cc:2749] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:07:57.474678 16567 raft_consensus.cc:1240] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 25 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 24. Current term is 25. Ops: [24.17-24.17]
I20260709 13:07:57.476302 15900 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:57.477871 15902 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:57.478899 15902 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:57.479024 22970 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:57.405035 23090 consensus_queue.cc:1048] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.026s
W20260709 13:07:57.481287 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 66847.8us
I20260709 13:07:57.481810 15860 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 19.
I20260709 13:07:57.491029 15974 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:57.482892 15902 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:57.486246 15854 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 17.
I20260709 13:07:57.486368 15930 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 19.
I20260709 13:07:57.488793 15934 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 18.
I20260709 13:07:57.493808 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:07:57.496001 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 62126.4us
W20260709 13:07:57.497308 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:57.498569 16006 raft_consensus.cc:2393] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 16: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:57.499177 15900 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:57.386128 23219 consensus_queue.cc:1059] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.110s
I20260709 13:07:57.510244 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:57.511117 16567 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:57.463052 23018 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:57.492166 23215 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:57.507445 16568 raft_consensus.cc:1240] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:57.508103 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:57.489964 23061 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:57.479701 22970 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:57.482751 23155 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:57.490226 23029 raft_consensus.cc:2749] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:57.479645 22988 consensus_queue.cc:1048] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.029s
I20260709 13:07:57.518970 16563 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:57.515010 23157 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:07:57.494624 22594 raft_consensus.cc:697] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 20 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:57.515288 23221 consensus_queue.cc:1059] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.006s
W20260709 13:07:57.522694 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:57.512979 22957 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:57.527714 22970 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:57.524653 16562 raft_consensus.cc:1275] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:07:57.504397 23123 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:57.499051 23043 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:57.526072 23147 raft_consensus.cc:3055] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:07:57.527576 23206 raft_consensus.cc:697] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:57.509390 23038 consensus_queue.cc:1059] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 13:07:57.541695 22594 consensus_queue.cc:237] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:07:57.538506 23076 raft_consensus.cc:2764] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:07:57.527865 23091 consensus_queue.cc:1059] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.022s
I20260709 13:07:57.528182 23088 consensus_queue.cc:1059] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.022s
I20260709 13:07:57.560690 22970 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:57.536835 23179 consensus_queue.cc:1059] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.033s
I20260709 13:07:57.564635 23147 raft_consensus.cc:740] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:57.587229 23206 consensus_queue.cc:237] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:57.588799 23230 log.cc:927] Time spent T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.203s	user 0.002s	sys 0.002s
I20260709 13:07:57.592307 23218 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:57.594626 23218 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:57.594851 23221 raft_consensus.cc:3055] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:57.595238 23221 raft_consensus.cc:740] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:57.596206 23143 raft_consensus.cc:3055] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:57.595876 23221 consensus_queue.cc:260] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:57.596592 23143 raft_consensus.cc:740] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:57.597080 23221 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:57.597383 23143 consensus_queue.cc:260] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:57.598582 23143 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:57.604076 23147 consensus_queue.cc:260] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:57.605787 23147 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:57.606273 23103 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:57.568889 23043 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:57.609695 23043 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:57.610574 23179 raft_consensus.cc:3055] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:07:57.610967 23179 raft_consensus.cc:740] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:57.611284 23102 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:57.611774 23179 consensus_queue.cc:260] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:57.612952 23179 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:57.613452 23102 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:57.565109 23123 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:57.616194 23123 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:57.579903 23091 consensus_queue.cc:1243] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer 31882606e87447fb9a36bbf625129fe8 log is divergent from this leader: its last log entry 22.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
W20260709 13:07:57.662087 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 164517us
I20260709 13:07:57.681519 15827 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:57.689857 15827 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:57.505710 15902 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:07:57.697376 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 191618us
I20260709 13:07:57.697934 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:57.699334 15929 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 19.
I20260709 13:07:57.700984 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:57.705271 15902 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:57.708443 15928 raft_consensus.cc:1240] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 20. Current term is 21. Ops: [20.10-20.10]
I20260709 13:07:57.712651 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:57.713836 15930 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 19.
I20260709 13:07:57.716351 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:57.719874 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:57.720659 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:57.525563 15974 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:57.724052 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:57.725216 15861 raft_consensus.cc:2368] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 18. Current term is 19.
W20260709 13:07:57.725821 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 200294us
I20260709 13:07:57.727519 15974 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:07:57.729360 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:57.557826 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:57.736073 16003 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60156 (request call id 1423) took 1024 ms. Trace:
I20260709 13:07:57.736833 16003 rpcz_store.cc:276] 0709 13:07:56.710890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:56.711011 (+   121us) service_pool.cc:224] Handling call
0709 13:07:56.711324 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:56.888074 (+176750us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:56.888105 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:56.888537 (+   432us) log.cc:844] Serialized 26 byte log entry
0709 13:07:57.243284 (+354747us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:57.243341 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:57.734973 (+491632us) spinlock_profiling.cc:243] Waited 176 ms on lock 0x7b54003ece54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:57.735074 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:57.735182 (+   108us) raft_consensus.cc:1685] finished
0709 13:07:57.735210 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:57.735721 (+   511us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8798,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":5471,"spinlock_wait_cycles":405653888,"thread_start_us":354340,"threads_started":1,"wal-append.queue_time_us":360936}
I20260709 13:07:57.745888 16008 raft_consensus.cc:1275] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:07:57.749203 16005 raft_consensus.cc:1275] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:07:57.751478 15974 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:57.554178 16563 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 23.
I20260709 13:07:57.769174 23029 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.770156 23018 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:57.770426 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:57.771294 23066 raft_consensus.cc:2749] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:57.770218 23029 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:57.780468 23021 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.781363 23043 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:57.786840 16567 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 21.
I20260709 13:07:57.824179 23088 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:57.825346 23088 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:57.827999 22603 raft_consensus.cc:2749] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:57.830492 23009 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:57.831358 23009 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:57.834373 23102 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:57.834616 23090 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:07:57.835415 23102 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:57.838634 23102 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:57.839985 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:57.841009 16563 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:57.845090 23123 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.845904 23123 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:57.848728 23021 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:57.771320 23018 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.090s
W20260709 13:07:57.890296 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:57.890540 23061 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:57.891145 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:57.891445 23110 raft_consensus.cc:2749] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:07:57.891989 16562 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:07:57.892938 23202 log.cc:927] Time spent T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
W20260709 13:07:57.928025 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 170696us
I20260709 13:07:57.927865 15929 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 17.
I20260709 13:07:57.927448 15902 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:57.940479 23177 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.941085 23078 raft_consensus.cc:3055] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:57.941609 23078 raft_consensus.cc:740] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:57.941543 23177 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:57.944075 23091 consensus_queue.cc:1048] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.061s
I20260709 13:07:57.946287 23046 raft_consensus.cc:3055] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:07:57.946885 23046 raft_consensus.cc:740] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:57.886495 23043 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:57.965667 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 99398.1us
I20260709 13:07:57.946816 23133 consensus_queue.cc:1048] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.157s
I20260709 13:07:57.969880 22988 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:57.970561 22988 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:57.971027 22988 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:57.971349 23076 raft_consensus.cc:2804] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:07:57.971387 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 227629us
W20260709 13:07:57.975750 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:57.979831 23218 raft_consensus.cc:2749] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:57.980625 23029 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:57.950953 23009 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:57.969738 22679 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:57.943087 23078 consensus_queue.cc:260] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:57.984710 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 289367us
I20260709 13:07:57.877321 23168 consensus_queue.cc:1059] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.156s
I20260709 13:07:57.995813 23133 consensus_queue.cc:1048] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:57.996655 15825 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:58.007547 16562 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 23.
I20260709 13:07:58.007781 23090 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:58.008466 23090 raft_consensus.cc:697] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 19 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:58.009059 23217 raft_consensus.cc:2749] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:58.009245 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:58.009804 23090 consensus_queue.cc:237] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:58.010711 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:07:58.009753 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.015357 16563 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 19.
I20260709 13:07:58.017916 15902 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:58.007547 15825 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:07:58.055536 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 58506.9us
W20260709 13:07:58.068313 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 338470us
I20260709 13:07:58.054227 23177 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:58.066609 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 59254.5us
I20260709 13:07:58.073499 23009 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:58.073792 23245 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:07:58.074553 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 575298us
I20260709 13:07:58.075188 23250 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.076193 23250 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:58.079949 23248 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:58.080772 23248 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:58.083112 23248 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:58.090523 16566 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 1525) took 2394 ms. Trace:
W20260709 13:07:58.073557 23240 log.cc:927] Time spent T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:07:58.097817 23247 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:58.113821 15974 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:58.115499 15902 leader_election.cc:400] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 election: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 22. Current term is 23.
I20260709 13:07:58.119218 15902 leader_election.cc:403] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 13:07:58.120388 23247 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:58.120981 23247 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:58.061370 23021 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:58.124966 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 57788.6us
I20260709 13:07:58.074553 23245 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:58.128911 23245 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:58.132249 16561 raft_consensus.cc:1275] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:07:58.160980 16563 raft_consensus.cc:1275] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:07:58.110777 23046 consensus_queue.cc:260] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:58.168934 16562 raft_consensus.cc:1240] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 20. Current term is 21. Ops: [18.13-20.14]
I20260709 13:07:58.079576 23250 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:58.169802 23088 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:58.163754 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:07:58.180824 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 63404.2us
I20260709 13:07:58.181171 23174 raft_consensus.cc:2749] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:58.181942 16010 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:07:58.181964 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 68389.2us
I20260709 13:07:58.187953 23018 consensus_queue.cc:1059] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 19.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:58.188427 23046 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:58.189252 23038 raft_consensus.cc:3055] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:07:58.189641 23038 raft_consensus.cc:740] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:58.190528 23038 consensus_queue.cc:260] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:58.192049 23038 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:58.195808 23009 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:58.196483 16563 raft_consensus.cc:1240] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:58.197834 15974 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:58.196177 15825 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:58.198616 23009 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:58.199354 23043 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:07:58.199975 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.201217 23205 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:58.201723 23205 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:58.202122 23205 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:58.204455 15825 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:07:58.206601 22970 consensus_meta.cc:287] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.629s	user 0.006s	sys 0.005s
I20260709 13:07:58.214238 23247 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:58.207118 22970 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:58.170871 23123 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:58.118047 16566 rpcz_store.cc:276] 0709 13:07:55.695588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:55.695760 (+   172us) service_pool.cc:224] Handling call
0709 13:07:55.696131 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:55.712211 (+ 16080us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:55.712257 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:55.712858 (+   601us) log.cc:844] Serialized 26 byte log entry
0709 13:07:55.923281 (+210423us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:07:56.206182 (+282901us) log.cc:844] Serialized 14 byte log entry
0709 13:07:56.248785 (+ 42603us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:58.089728 (+1840943us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b54004790d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:58.089828 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:58.089939 (+   111us) raft_consensus.cc:1685] finished
0709 13:07:58.090061 (+   122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:58.090310 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":107924,"prepare.queue_time_us":252257,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":26,"raft.queue_time_us":43104,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":3561,"spinlock_wait_cycles":36214656,"thread_start_us":317051,"threads_started":2,"wal-append.queue_time_us":262780,"wal-append.run_cpu_time_us":2454,"wal-append.run_wall_time_us":1267844}
W20260709 13:07:58.302825 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:58.335561 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.355410 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:58.349669 15902 leader_election.cc:400] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 election: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 17. Current term is 18.
I20260709 13:07:58.358424 15902 leader_election.cc:403] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:07:58.354363 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:07:58.352406 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 154693us
I20260709 13:07:58.292992 23122 raft_consensus.cc:2749] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 22. Current term is 23.
I20260709 13:07:58.350616 16561 raft_consensus.cc:1275] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:07:58.360821 15933 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
W20260709 13:07:58.361270 22957 consensus_meta.cc:287] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.814s	user 0.014s	sys 0.013s
I20260709 13:07:58.361744 22957 raft_consensus.cc:2749] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:58.364416 16010 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 21.
I20260709 13:07:58.360868 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:58.368350 16569 raft_consensus.cc:1240] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 22. Current term is 23. Ops: [22.17-22.17]
W20260709 13:07:58.360846 23251 log.cc:927] Time spent T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.192s	user 0.005s	sys 0.004s
I20260709 13:07:58.362975 15976 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
W20260709 13:07:58.369452 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 213056us
W20260709 13:07:58.371053 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.372413 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:58.380686 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:07:58.381451 15861 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:58.384835 23018 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.385560 23018 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:58.386122 22970 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:58.379467 23218 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 17. Current term is 18.
I20260709 13:07:58.373802 23174 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:58.378041 23177 raft_consensus.cc:2804] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:58.392663 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:58.366647 23245 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:58.406298 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:58.393553 23174 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:58.413841 23174 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:58.420054 23248 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:58.420818 23248 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:58.423223 23248 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:58.426163 23242 log.cc:927] Time spent T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.302s	user 0.002s	sys 0.001s
I20260709 13:07:58.427129 23052 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:58.411078 23088 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:58.440047 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:58.428197 23218 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:07:58.429140 23122 consensus_queue.cc:1048] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:07:58.458804 23052 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:58.461370 23052 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:58.465286 23122 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:07:58.474413 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:58.474754 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:58.475185 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:58.475397 16003 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 19.
I20260709 13:07:58.476485 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:58.477301 16005 raft_consensus.cc:2393] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 17: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
W20260709 13:07:58.477871 23239 log.cc:927] Time spent T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.363s	user 0.001s	sys 0.002s
W20260709 13:07:58.478593 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:58.480806 16534 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.485330 23018 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:58.486399 23167 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:58.489799 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:58.490762 15857 raft_consensus.cc:2393] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 17: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:58.491472 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:58.492079 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:58.492401 23123 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.492689 15936 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:58.493062 23123 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:58.494298 23253 log.cc:927] Time spent T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 13:07:58.497166 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:58.498003 15931 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 17.
I20260709 13:07:58.499240 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:58.501099 23123 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:58.506378 15936 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 20.
I20260709 13:07:58.440551 15974 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:58.442050 22957 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:58.442377 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:58.444123 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 264427us
I20260709 13:07:58.445501 23088 raft_consensus.cc:697] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:58.447217 23076 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:58.448662 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:58.450116 23245 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:58.453572 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:07:58.454882 23236 log.cc:927] Time spent T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.407s	user 0.000s	sys 0.003s
I20260709 13:07:58.490583 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:58.508632 22988 consensus_meta.cc:287] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.537s	user 0.013s	sys 0.005s
I20260709 13:07:58.508986 23123 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.511667 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:58.513768 16005 raft_consensus.cc:1240] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 20. Current term is 21. Ops: [20.10-20.10]
I20260709 13:07:58.514870 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:58.407948 23177 raft_consensus.cc:697] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 17 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:58.435230 15861 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 21.
I20260709 13:07:58.488910 16566 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 18.
I20260709 13:07:58.520834 15825 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:58.434145 23247 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:58.523514 15933 raft_consensus.cc:1240] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:58.525063 15853 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 20.
W20260709 13:07:58.535513 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.524093 23076 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:58.545184 15825 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:58.545291 16564 raft_consensus.cc:2393] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 17: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:58.548409 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:58.551644 15825 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:58.553360 16562 raft_consensus.cc:2410] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 21 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:07:58.554384 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:58.549835 15976 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:58.559815 16566 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 14.
I20260709 13:07:58.561213 23206 raft_consensus.cc:2804] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:07:58.561640 23206 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:58.561975 23206 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:58.562453 23167 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:58.562996 23167 raft_consensus.cc:697] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 20 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:58.558374 23247 raft_consensus.cc:2804] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:58.563908 23247 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:58.564484 23247 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:58.563948 23167 consensus_queue.cc:237] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:58.550410 23046 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:07:58.564891 15931 raft_consensus.cc:1275] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:07:58.565234 23046 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:58.566269 16568 raft_consensus.cc:1275] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:07:58.567883 23177 consensus_queue.cc:237] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:58.569348 23087 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.569928 23087 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:58.571429 15856 raft_consensus.cc:1275] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 15. (term mismatch)
I20260709 13:07:58.571825 15856 pending_rounds.cc:85] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8: Aborting all ops after (but not including) 14
I20260709 13:07:58.563963 23052 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:07:58.572134 15856 pending_rounds.cc:107] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8: Aborting uncommitted NO_OP operation due to leader change: 18.15
I20260709 13:07:58.572475 15856 raft_consensus.cc:2889] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:58.572378 23052 raft_consensus.cc:697] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 23 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:58.574935 23052 consensus_queue.cc:237] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:58.576923 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:58.546414 23123 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:58.585436 23174 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:58.567972 16569 raft_consensus.cc:1275] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:07:58.587409 15859 raft_consensus.cc:2393] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 18: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:07:58.536168 23250 raft_consensus.cc:2749] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:58.593849 22508 consensus_queue.cc:1048] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:58.594125 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:58.594755 15931 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:07:58.595611 15862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 1356) took 1746 ms. Trace:
I20260709 13:07:58.596078 15862 rpcz_store.cc:276] 0709 13:07:56.848796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:56.848959 (+   163us) service_pool.cc:224] Handling call
0709 13:07:56.872295 (+ 23336us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:56.948015 (+ 75720us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:07:57.328452 (+380437us) log.cc:844] Serialized 14 byte log entry
0709 13:07:57.668209 (+339757us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:57.669845 (+  1636us) log.cc:844] Serialized 52 byte log entry
0709 13:07:57.670041 (+   196us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:07:57.670130 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:57.670273 (+   143us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:58.454235 (+783962us) spinlock_profiling.cc:243] Waited 2.8 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:07:58.595134 (+140899us) raft_consensus.cc:1685] finished
0709 13:07:58.595239 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:58.595385 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":78613,"prepare.queue_time_us":461689,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":182105,"raft.run_cpu_time_us":442,"raft.run_wall_time_us":47142,"spinlock_wait_cycles":6430080,"thread_start_us":678993,"threads_started":2,"wal-append.queue_time_us":591562}
I20260709 13:07:58.598567 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:07:58.602070 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.603016 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:58.602593 23088 consensus_queue.cc:237] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:58.605854 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 209469us
I20260709 13:07:58.602109 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:58.608628 15974 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:58.608810 16008 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:58.533285 22988 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:58.616163 16005 raft_consensus.cc:1275] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:07:58.617040 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.611342 23046 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:58.586418 23205 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:58.615665 23248 raft_consensus.cc:2749] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:07:58.632454 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 168828us
I20260709 13:07:58.636369 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:58.592286 22957 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:58.620293 23123 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:58.668517 23074 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:58.690821 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 82313.5us
I20260709 13:07:58.698494 23123 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:07:58.747779 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.747905 15937 raft_consensus.cc:1240] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 16. Current term is 17. Ops: []
W20260709 13:07:58.792112 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 47499.2us
I20260709 13:07:58.748044 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:58.742877 23167 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.794364 23167 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:58.796061 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.797209 23167 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:58.734429 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 254835us
I20260709 13:07:58.797712 23017 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.798543 23017 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:58.799929 16533 leader_election.cc:400] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 election: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 18. Current term is 19.
I20260709 13:07:58.800086 22988 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:58.800277 16533 leader_election.cc:403] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:07:58.800879 23043 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 18. Current term is 19.
W20260709 13:07:58.804412 23255 log.cc:927] Time spent T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:07:58.805024 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:58.805928 15933 raft_consensus.cc:2393] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 18: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:58.805076 23218 consensus_queue.cc:1059] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.016s
I20260709 13:07:58.807884 23218 raft_consensus.cc:3055] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:07:58.808393 23218 raft_consensus.cc:740] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:58.803194 16533 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:58.810071 23043 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:58.810525 23043 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:58.810922 23043 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:58.699013 23206 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:58.809615 23218 consensus_queue.cc:260] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:58.748103 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:58.817646 15902 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:58.719590 23078 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:07:58.740247 15900 connection.cc:256] RPC call timeout handler was delayed by -2.1077968120000605s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1837s for a response
I20260709 13:07:58.820688 16008 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 20.
I20260709 13:07:58.748876 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:58.794238 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:58.829816 16002 raft_consensus.cc:2410] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 25 because replica has last-logged OpId of term: 24 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 13:07:58.827447 16004 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 16.
I20260709 13:07:58.825959 16533 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:58.819564 23110 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.854440 15902 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:58.860522 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:58.822240 23174 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:58.866425 15929 raft_consensus.cc:1275] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:07:58.724272 23247 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:58.869175 16008 raft_consensus.cc:2393] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 20: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:58.869410 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:58.870208 15936 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:58.831293 23122 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:58.748250 23212 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.171s
I20260709 13:07:58.748005 23136 consensus_queue.cc:1059] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.226s
I20260709 13:07:58.725263 23074 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:58.837486 23123 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:58.875566 23245 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:58.887030 23218 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:58.914674 16011 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:07:58.924005 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 74431.8us
W20260709 13:07:58.918634 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.889534 23205 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:07:58.924923 15900 connection.cc:256] RPC call timeout handler was delayed by -1.9799968119999676s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19790000000000002s for a response
I20260709 13:07:58.884384 23122 raft_consensus.cc:697] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 18 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:07:58.929821 15900 connection.cc:256] RPC call timeout handler was delayed by -1.8062968120000278s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21720000000000003s for a response
W20260709 13:07:58.930968 15900 connection.cc:256] RPC call timeout handler was delayed by -1.7675968119999652s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2215s for a response
W20260709 13:07:58.931687 15900 connection.cc:256] RPC call timeout handler was delayed by -1.0961968120000165s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2961s for a response
W20260709 13:07:58.932077 15900 connection.cc:256] RPC call timeout handler was delayed by -1.087196812000002s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.29710000000000003s for a response
I20260709 13:07:58.935786 23122 consensus_queue.cc:237] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:58.936930 23074 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:58.937162 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:58.939594 23167 raft_consensus.cc:3055] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:07:58.940167 23167 raft_consensus.cc:740] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:58.941111 23167 consensus_queue.cc:260] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:58.942278 23167 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:58.943107 16011 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 19.
W20260709 13:07:58.944463 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.910223 23029 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:58.884989 23110 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:58.929831 22957 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:58.954773 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 129030us
W20260709 13:07:58.958747 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 146722us
I20260709 13:07:58.964413 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:58.951571 23006 raft_consensus.cc:2804] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:58.969559 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:58.917647 23245 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:58.971124 23258 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:07:58.973644 16535 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:58.979588 23017 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:58.980674 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:58.990186 15827 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
W20260709 13:07:58.991063 23264 log.cc:927] Time spent T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:07:58.992333 15827 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:58.969147 23029 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:59.000280 15932 raft_consensus.cc:2410] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 22 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
W20260709 13:07:59.001881 23262 log.cc:927] Time spent T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:07:58.970104 23090 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:59.005046 23090 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:58.969013 23123 leader_election.cc:290] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:59.008502 15856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58214 (request call id 1352) took 2632 ms. Trace:
I20260709 13:07:59.008930 15856 rpcz_store.cc:276] 0709 13:07:56.376269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:56.376399 (+   130us) service_pool.cc:224] Handling call
0709 13:07:56.376769 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:58.573177 (+2196408us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:59.008171 (+434994us) spinlock_profiling.cc:243] Waited 2.19 s on lock 0x7b54003b6354. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:07:59.008418 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5045872000}
I20260709 13:07:59.010267 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:59.010795 23267 log.cc:927] Time spent T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.051s	user 0.002s	sys 0.000s
I20260709 13:07:59.011039 16004 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 18.
I20260709 13:07:59.011592 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:59.012629 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:07:59.013715 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:59.014870 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:59.015829 23245 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:59.018029 15933 raft_consensus.cc:2368] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 18. Current term is 19.
I20260709 13:07:59.012035 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:59.005533 23110 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:58.951129 23076 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:58.971779 23258 raft_consensus.cc:697] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 21 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:59.028589 15936 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 20.
I20260709 13:07:59.012265 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:59.010875 23247 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:59.031167 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:59.034046 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:59.033115 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:59.037715 16007 raft_consensus.cc:2368] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 18. Current term is 19.
I20260709 13:07:58.987792 23006 raft_consensus.cc:697] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:59.023245 23090 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:59.039758 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:59.040688 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:59.040941 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:59.042666 15936 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 20.
I20260709 13:07:59.048703 15934 raft_consensus.cc:3060] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:59.050520 15929 raft_consensus.cc:3060] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:59.043663 23110 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:07:59.052485 16006 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:59.052489 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:59.056450 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 64235.5us
I20260709 13:07:59.057472 15827 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:59.058207 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:59.059755 23018 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:59.059562 23156 consensus_queue.cc:1059] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.032s
I20260709 13:07:59.060822 23156 raft_consensus.cc:3055] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:59.061264 23156 raft_consensus.cc:740] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:59.062445 23156 consensus_queue.cc:260] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:59.063889 23269 log.cc:927] Time spent T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:07:59.064850 23029 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:59.071661 16006 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 17.
I20260709 13:07:59.076152 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:59.076253 23006 consensus_queue.cc:237] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:59.093320 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:59.094868 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:59.095882 23110 raft_consensus.cc:697] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 23 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:59.096025 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:59.096643 16566 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:59.096951 23110 consensus_queue.cc:237] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:59.101528 23123 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:59.077445 23272 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:59.102080 23123 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:59.102528 23272 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:59.103884 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:59.080027 23273 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:59.104786 23272 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:59.104671 23273 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:59.105383 23087 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:59.105950 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:59.106829 23273 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:59.108716 16535 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:59.081836 23076 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:59.084270 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 46707.2us
I20260709 13:07:59.086251 15825 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:59.086891 23029 consensus_queue.cc:1048] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:59.044152 23136 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:59.115216 15929 raft_consensus.cc:2468] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 21.
I20260709 13:07:59.120030 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:59.125468 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:59.125764 16561 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:59.123888 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:59.124586 15902 leader_election.cc:400] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 election: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 18. Current term is 19.
I20260709 13:07:59.103474 23206 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:59.127168 15934 raft_consensus.cc:2468] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 19.
I20260709 13:07:59.126298 15936 raft_consensus.cc:2410] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 17 because replica has last-logged OpId of term: 16 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 14.
I20260709 13:07:59.128764 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:59.126971 15902 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:59.130633 16566 raft_consensus.cc:2468] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 18.
I20260709 13:07:59.132958 15902 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:59.131646 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:59.134164 15902 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:59.065454 23258 consensus_queue.cc:237] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:59.139276 22957 raft_consensus.cc:2749] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:59.141738 23156 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:59.143931 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:07:59.144716 15759 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58966 (request call id 247) took 20540 ms (client timeout 15000 ms). Trace:
I20260709 13:07:59.129820 23052 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.146106 23052 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:59.147930 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:59.148465 23052 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:59.148365 15923 tablet_service.cc:1558] Processing DeleteTablet for tablet 679229ffe7114facacb8e3a2c4c6d226 with delete_type TABLET_DATA_TOMBSTONED (TS 8430db80a39d4868bcf7641c16de35ea not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:58226
I20260709 13:07:59.148697 15933 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:07:59.129676 23205 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:59.147848 23136 consensus_queue.cc:1059] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:59.149950 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:07:59.154340 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:59.117447 23272 raft_consensus.cc:2749] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:07:59.155197 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:59.156059 16561 raft_consensus.cc:2468] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 19.
I20260709 13:07:59.149039 22508 consensus_queue.cc:1048] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:07:59.156774 23076 raft_consensus.cc:2804] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:59.165131 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:59.167373 15937 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:59.169039 16570 raft_consensus.cc:2393] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 19: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:07:59.172106 15974 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:59.173307 15976 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:59.173416 15974 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:59.175102 23248 raft_consensus.cc:2749] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:07:59.175868 23088 raft_consensus.cc:2804] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:59.176349 23088 raft_consensus.cc:697] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 19 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:59.177253 23088 consensus_queue.cc:237] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:59.180583 23174 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:59.180714 23076 raft_consensus.cc:697] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 18 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:07:59.185854 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:59.190658 15933 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 15.
I20260709 13:07:59.156137 23245 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:07:59.199357 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:59.165526 23205 raft_consensus.cc:697] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 20 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
W20260709 13:07:59.145063 15759 rpcz_store.cc:269] 0709 13:07:38.604265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:38.640638 (+ 36373us) service_pool.cc:224] Handling call
0709 13:07:59.144499 (+20503861us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:54.473326 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:54.662618 (+189292us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:54.662690 (+    72us) write_op.cc:435] Acquired schema lock
0709 13:07:54.662717 (+    27us) tablet.cc:662] Decoding operations
0709 13:07:54.713666 (+ 50949us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:54.751349 (+ 37683us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:54.751879 (+   530us) tablet.cc:685] Acquiring locks for 20 operations
0709 13:07:55.374425 (+622546us) tablet.cc:701] Row locks acquired
0709 13:07:55.403484 (+ 29059us) write_op.cc:260] PREPARE: finished
0709 13:07:55.433869 (+ 30385us) write_op.cc:270] Start()
0709 13:07:55.493505 (+ 59636us) write_op.cc:276] Timestamp: P: 1783602475415692 usec, L: 0
0709 13:07:55.493557 (+    52us) op_driver.cc:348] REPLICATION: starting
0709 13:07:55.601757 (+108200us) log.cc:844] Serialized 7816 byte log entry
0709 13:07:56.292088 (+690331us) op_driver.cc:464] REPLICATION: finished
0709 13:07:56.616023 (+323935us) write_op.cc:301] APPLY: starting
0709 13:07:56.676812 (+ 60789us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:57.136423 (+459611us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:57.136546 (+   123us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:57.609254 (+472708us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:57.609561 (+   307us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=40,key_file_lookups=40,delta_file_lookups=40,mrs_lookups=0
0709 13:07:57.609605 (+    44us) write_op.cc:312] APPLY: finished
0709 13:07:57.610035 (+   430us) log.cc:844] Serialized 180 byte log entry
0709 13:07:58.058611 (+448576us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:58.095514 (+ 36903us) write_op.cc:454] Released schema lock
0709 13:07:58.095659 (+   145us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:58.096306 (+   647us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:58.096337 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1575230,"reported_tablets":42,"child_traces":[["op",{"apply.queue_time_us":301228,"apply.run_cpu_time_us":70879,"apply.run_wall_time_us":1485265,"cfile_cache_hit":82,"cfile_cache_hit_bytes":87927,"num_ops":20,"prepare.queue_time_us":574957,"prepare.run_cpu_time_us":56775,"prepare.run_wall_time_us":1557454,"raft.queue_time_us":308694,"raft.run_cpu_time_us":19481,"raft.run_wall_time_us":218731,"replication_time_us":757354,"thread_start_us":1372511,"threads_started":5,"wal-append.queue_time_us":771490,"wal-append.run_cpu_time_us":29602,"wal-append.run_wall_time_us":2330475}]]}
I20260709 13:07:59.206508 23090 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:59.228595 15936 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 21.
I20260709 13:07:59.164969 23046 consensus_queue.cc:1048] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.044s
I20260709 13:07:59.206221 22508 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.227738 23090 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:59.243639 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:59.253585 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 142860us
I20260709 13:07:59.230511 23123 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:59.252189 15825 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:59.250669 15827 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:59.214325 23247 consensus_queue.cc:1048] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 13:07:59.237000 23245 raft_consensus.cc:697] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 17 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:07:59.265029 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 188367us
W20260709 13:07:59.269567 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 96322.4us
I20260709 13:07:59.264566 16535 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:59.270430 15902 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:59.249866 15746 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet dfeaf1f6660d48848ae69710739427a0 with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER failed with leader eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica eb1e0493af1e46dbb6dec7e25525bab5 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:07:59.296392 15854 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 20.
I20260709 13:07:59.263989 22970 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:59.301702 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:59.273778 23123 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:59.239022 22508 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:59.298835 16535 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:59.297909 16533 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:59.309438 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:59.309994 16008 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 20.
I20260709 13:07:59.310645 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:59.311218 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:59.273191 23017 raft_consensus.cc:2804] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:07:59.302371 16568 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 19.
I20260709 13:07:59.291582 23029 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:07:59.307471 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 56889us
I20260709 13:07:59.308380 23123 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:07:59.315258 23287 tablet_replica.cc:333] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:07:59.316452 22508 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:59.321376 15760 catalog_manager.cc:5697] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 5 to 13, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 13 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
W20260709 13:07:59.324090 23265 log.cc:927] Time spent T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.335s	user 0.000s	sys 0.002s
I20260709 13:07:59.324747 23090 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:59.325397 23017 raft_consensus.cc:697] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:59.326035 23283 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.326818 23206 raft_consensus.cc:3055] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:59.333756 15937 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 18.
I20260709 13:07:59.350965 15827 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:59.335021 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 161705us
I20260709 13:07:59.338537 23287 raft_consensus.cc:2243] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Raft consensus shutting down.
W20260709 13:07:59.342381 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 43614.9us
I20260709 13:07:59.305639 23258 raft_consensus.cc:2749] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:59.345690 23017 consensus_queue.cc:237] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:59.330521 23205 consensus_queue.cc:237] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:59.315596 22970 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:59.349239 23217 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:59.349712 23206 raft_consensus.cc:740] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
W20260709 13:07:59.353869 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:59.341869 23174 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:59.361984 23279 log.cc:927] Time spent T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:07:59.363330 23268 raft_consensus.cc:2749] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:07:59.363792 23285 raft_consensus.cc:2804] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:59.366977 15976 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:59.336596 23123 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:59.340433 23074 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.266165 23076 consensus_queue.cc:237] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:59.381127 15825 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:07:59.389365 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:59.386667 16535 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:59.392680 16006 raft_consensus.cc:1275] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:07:59.390210 15935 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:07:59.389253 15760 catalog_manager.cc:5697] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
I20260709 13:07:59.395092 15827 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:07:59.395834 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 123799us
I20260709 13:07:59.400708 16008 raft_consensus.cc:1275] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:07:59.401764 23294 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:59.402307 23294 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:59.403614 23287 raft_consensus.cc:2272] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:59.404544 23294 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:59.404969 22970 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:59.387008 23248 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:59.368496 23283 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:59.383831 23122 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:59.393242 23167 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:07:59.394533 23078 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.397756 23258 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:07:59.425858 15974 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:59.398094 23285 raft_consensus.cc:697] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:59.409788 23110 raft_consensus.cc:2804] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:59.411724 23247 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:59.393306 23074 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:59.385397 23174 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:59.410295 23006 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:59.412617 23248 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:59.421271 16535 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:59.442128 15760 catalog_manager.cc:5697] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 7 to 12, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 12 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
W20260709 13:07:59.438021 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 60063.5us
I20260709 13:07:59.387598 23217 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:59.458072 23217 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:59.459254 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:07:59.460850 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 46559.2us
I20260709 13:07:59.464107 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:59.440510 23078 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:59.467170 23078 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:59.415915 23245 consensus_queue.cc:237] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:59.467949 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:59.444486 23052 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:59.468807 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:59.445693 16533 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:07:59.434651 23122 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:59.473395 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:59.447345 23074 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:07:59.447924 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 55807.3us
W20260709 13:07:59.448108 23043 consensus_meta.cc:287] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.637s	user 0.011s	sys 0.001s
I20260709 13:07:59.448385 23283 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:59.474706 16568 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 19.
I20260709 13:07:59.475824 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:59.434944 23258 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:59.480032 15827 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:59.448642 23285 consensus_queue.cc:237] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:59.479169 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:59.484534 16569 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 21.
I20260709 13:07:59.485332 16004 raft_consensus.cc:1240] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 19 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:59.477792 16567 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 18.
I20260709 13:07:59.478786 23052 raft_consensus.cc:697] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:07:59.482137 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:59.483798 16564 raft_consensus.cc:1240] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:59.455278 23248 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:59.465389 23006 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:59.485091 23298 consensus_queue.cc:1048] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.010s
W20260709 13:07:59.495173 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 49635.2us
I20260709 13:07:59.495795 23087 consensus_queue.cc:1048] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:07:59.497718 23291 log.cc:927] Time spent T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.152s	user 0.000s	sys 0.002s
I20260709 13:07:59.498418 16533 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:59.489076 23287 ts_tablet_manager.cc:1916] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:07:59.500308 23247 raft_consensus.cc:2749] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:59.501500 15859 raft_consensus.cc:1240] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:59.483589 23217 consensus_queue.cc:1048] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 13:07:59.435791 23110 raft_consensus.cc:697] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:59.504310 23248 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:59.505589 23110 consensus_queue.cc:237] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:07:59.506307 23295 log.cc:927] Time spent T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:07:59.494418 23006 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:59.507128 23052 consensus_queue.cc:237] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:59.507838 23006 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:59.508327 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:59.510810 15861 raft_consensus.cc:1240] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 22 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:59.511516 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:59.512028 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:59.512173 15855 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 20.
I20260709 13:07:59.512265 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:59.512629 15853 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 19.
I20260709 13:07:59.512949 15860 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 20.
I20260709 13:07:59.485531 23283 raft_consensus.cc:2804] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:07:59.488646 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:59.523097 15825 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:07:59.537149 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 48821.8us
I20260709 13:07:59.540513 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:59.544389 15936 raft_consensus.cc:1240] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 19. Current term is 20. Ops: []
W20260709 13:07:59.547994 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 49673.4us
I20260709 13:07:59.521091 23122 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:59.481088 23043 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:59.539476 23247 raft_consensus.cc:2804] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:59.559597 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:59.548610 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:07:59.557942 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:59.559491 15932 raft_consensus.cc:1240] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 18 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:59.526875 23283 raft_consensus.cc:697] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:07:59.566134 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:59.568918 15859 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 21.
I20260709 13:07:59.570811 15933 raft_consensus.cc:1240] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 17 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:59.572247 16562 raft_consensus.cc:1240] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:59.573179 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:59.573688 16533 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:59.574512 23268 raft_consensus.cc:2749] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:59.574965 16533 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:59.576061 23297 raft_consensus.cc:2749] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:07:59.577590 23174 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:59.577267 15860 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "8430db80a39d4868bcf7641c16de35ea"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:07:59.560617 23247 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:59.578665 23247 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:59.579219 15937 raft_consensus.cc:1275] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:07:59.580386 23248 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:59.520952 23206 consensus_queue.cc:260] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:59.592549 23248 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.593068 23248 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:07:59.595145 23248 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:59.598762 15760 catalog_manager.cc:5697] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 9 to 12, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 12 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
I20260709 13:07:59.611194 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:59.615828 15936 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:07:59.616261 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:59.618512 15932 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:07:59.619498 15827 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:59.567164 22508 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:59.648523 15976 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:59.652117 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:07:59.652917 15936 raft_consensus.cc:2393] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 19: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
W20260709 13:07:59.656723 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 81819.8us
I20260709 13:07:59.664453 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:07:59.669504 15856 raft_consensus.cc:1240] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 20 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:59.670251 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:59.673640 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:59.674856 16533 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:59.630326 23218 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.621918 23156 consensus_queue.cc:1059] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:07:59.654513 23284 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:59.689698 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:59.653895 23087 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:59.672873 23074 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:59.622660 23017 consensus_queue.cc:1059] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.058s
I20260709 13:07:59.654124 22508 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:07:59.708734 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 89339.8us
I20260709 13:07:59.702255 23268 raft_consensus.cc:2749] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:59.702668 23043 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:07:59.703562 23074 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:59.720763 15827 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:59.719492 15902 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:59.722332 15902 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:59.700419 23218 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:59.724687 15860 raft_consensus.cc:606] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Received request to transfer leadership to TS 8430db80a39d4868bcf7641c16de35ea
W20260709 13:07:59.741349 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:59.709697 23156 consensus_queue.cc:1243] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Peer 8430db80a39d4868bcf7641c16de35ea log is divergent from this leader: its last log entry 17.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:07:59.721638 15760 catalog_manager.cc:5697] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 12 to 16, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 16 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:59.744494 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 82262.3us
W20260709 13:07:59.723479 23293 log.cc:927] Time spent T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.256s	user 0.000s	sys 0.005s
I20260709 13:07:59.723106 23074 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:59.718519 23017 consensus_queue.cc:1243] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer 8430db80a39d4868bcf7641c16de35ea log is divergent from this leader: its last log entry 18.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:07:59.718772 23123 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:59.725629 23258 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:59.745141 23076 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:59.743898 23087 raft_consensus.cc:2749] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:07:59.701099 23283 consensus_queue.cc:237] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:59.777151 22508 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:59.779433 16570 raft_consensus.cc:1240] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 18. Current term is 19. Ops: []
W20260709 13:07:59.779724 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:59.771518 16533 leader_election.cc:400] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 election: Vote denied by peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) with higher term. Message: Invalid argument: T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 18. Current term is 19.
I20260709 13:07:59.775951 23076 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:07:59.771976 15760 catalog_manager.cc:5697] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 12 to 14, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 14 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:59.745033 23122 consensus_queue.cc:1059] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.028s
W20260709 13:07:59.790573 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 48033.4us
W20260709 13:07:59.782140 23302 log.cc:927] Time spent T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:07:59.795992 23218 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:07:59.802208 23038 raft_consensus.cc:3055] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:07:59.787045 23305 log.cc:927] Time spent T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
W20260709 13:07:59.815729 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 47073.8us
I20260709 13:07:59.820911 16533 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:59.821043 23205 consensus_queue.cc:1048] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:59.785789 23076 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:59.819373 23206 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:59.830022 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:59.749221 23217 consensus_queue.cc:1059] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 13:07:59.832690 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:59.834877 16562 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 22.
I20260709 13:07:59.840523 16570 raft_consensus.cc:1275] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:07:59.840541 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:59.842464 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:59.843837 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:59.844420 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:59.847035 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:59.847369 16007 raft_consensus.cc:1275] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:07:59.847517 16566 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 17.
W20260709 13:07:59.846616 15827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:59.845193 16568 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 21.
I20260709 13:07:59.824090 23122 raft_consensus.cc:3055] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:59.853272 16010 raft_consensus.cc:1275] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:07:59.853690 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:59.851023 15974 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:59.852066 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:59.852005 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:59.855737 16569 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:07:59.824386 23038 raft_consensus.cc:740] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:07:59.858858 15974 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:07:59.856518 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:59.860630 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:59.861665 16533 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:07:59.864112 16568 raft_consensus.cc:1275] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:07:59.864655 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:59.865453 16569 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:59.831343 22970 consensus_queue.cc:1059] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.033s
I20260709 13:07:59.870239 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:59.840766 23284 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:59.861435 15974 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:07:59.880967 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:07:59.875712 15854 raft_consensus.cc:1240] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:59.877082 16005 raft_consensus.cc:1275] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:07:59.877491 16561 raft_consensus.cc:1275] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:07:59.878275 16010 raft_consensus.cc:1275] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 13:07:59.852787 23122 raft_consensus.cc:740] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:07:59.922948 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 61570.9us
I20260709 13:07:59.882982 16003 raft_consensus.cc:2393] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 17: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:07:59.841480 23288 consensus_queue.cc:1059] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 13:07:59.862910 23248 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:59.946607 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:07:59.870606 23245 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:59.966078 15856 raft_consensus.cc:1275] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:07:59.969051 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:07:59.946162 23311 consensus_queue.cc:1059] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 13:07:59.971205 23288 raft_consensus.cc:3055] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:59.950596 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:07:59.883414 22970 consensus_queue.cc:1243] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer 85e997ba5003439781ec4f1f8db0c3ee log is divergent from this leader: its last log entry 19.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:07:59.858999 23006 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:59.995862 16569 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 18.
I20260709 13:07:59.944355 23123 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:59.994146 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:00.001354 15976 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:07:59.966638 23248 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:08:00.010561 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:59.948997 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:00.005761 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:08:00.030678 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 169054us
I20260709 13:08:00.032226 23122 consensus_queue.cc:260] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:07:59.981879 23288 raft_consensus.cc:740] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:07:59.983207 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:07:59.868975 23174 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:59.867064 23294 consensus_queue.cc:1059] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.110s
I20260709 13:08:00.088160 23294 raft_consensus.cc:3055] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:00.088596 23294 raft_consensus.cc:740] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:08:00.089639 23294 consensus_queue.cc:260] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:00.091076 23294 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:59.871764 23247 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:00.093822 23247 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:07:59.991947 23038 consensus_queue.cc:260] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:00.095719 23038 raft_consensus.cc:3060] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:59.982007 16570 raft_consensus.cc:2393] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 25: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:08:00.066776 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:00.108311 15860 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 21.
I20260709 13:08:00.004500 23043 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.109498 23043 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:00.073171 16564 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:00.116606 23043 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:00.120165 16566 raft_consensus.cc:1275] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:08:00.075786 16533 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:07:59.983953 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:00.130988 16007 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 17.
I20260709 13:08:00.010087 23311 consensus_queue.cc:1243] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer 31882606e87447fb9a36bbf625129fe8 log is divergent from this leader: its last log entry 20.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
W20260709 13:08:00.084012 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 135189us
I20260709 13:08:00.133239 23311 consensus_queue.cc:1048] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:08:00.133613 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:00.134742 23043 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:08:00.134842 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 59211.9us
I20260709 13:08:00.135635 16533 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:00.136385 23123 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:00.145927 16564 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 21.
I20260709 13:08:00.150363 23123 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:00.010512 23248 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:00.038985 23289 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:00.167014 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:00.167989 16009 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 25.
I20260709 13:08:00.173983 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:00.174533 15853 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:59.945619 23268 raft_consensus.cc:2804] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:00.180435 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:08:00.153491 23314 log.cc:927] Time spent T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.194s	user 0.001s	sys 0.000s
I20260709 13:08:00.179840 23122 raft_consensus.cc:3060] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:08:00.185626 23317 log.cc:927] Time spent T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:08:00.184530 23268 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:00.213850 23268 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:00.216235 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:00.217453 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:00.218178 15855 raft_consensus.cc:2393] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 17: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:08:00.219096 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:08:00.221370 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:00.222472 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:00.223717 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:00.224375 15858 raft_consensus.cc:2393] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 21: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:08:00.225046 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:00.225710 15858 raft_consensus.cc:2393] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 20: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:08:00.226397 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:00.185703 15857 raft_consensus.cc:1275] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 14. (term mismatch)
I20260709 13:08:00.194765 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:00.080013 23017 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.200446 15933 raft_consensus.cc:1275] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 15. (term mismatch)
I20260709 13:08:00.235255 15933 pending_rounds.cc:85] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 14
I20260709 13:08:00.227389 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:00.235563 15933 pending_rounds.cc:107] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 18.15
I20260709 13:08:00.236025 15933 raft_consensus.cc:2889] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:00.236833 15861 raft_consensus.cc:1275] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:08:00.237880 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:00.239070 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:00.239941 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:00.241405 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:08:00.271909 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 269295us
I20260709 13:08:00.203491 15854 raft_consensus.cc:2410] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:08:00.205348 15862 raft_consensus.cc:1275] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:08:00.242581 23091 raft_consensus.cc:3055] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:00.239598 15861 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 21.
I20260709 13:08:00.233843 15857 pending_rounds.cc:85] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8: Aborting all ops after (but not including) 13
I20260709 13:08:00.277745 15857 pending_rounds.cc:107] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8: Aborting uncommitted NO_OP operation due to leader change: 20.14
I20260709 13:08:00.278023 15857 raft_consensus.cc:2889] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:08:00.184952 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:00.249342 23303 log.cc:927] Time spent T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.558s	user 0.004s	sys 0.001s
I20260709 13:08:00.252441 15853 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 21.
I20260709 13:08:00.252609 23245 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:00.253573 23301 raft_consensus.cc:3055] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:08:00.072324 22957 consensus_queue.cc:1048] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.163s
I20260709 13:08:00.081909 23313 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:00.194965 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:00.258173 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:08:00.297569 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 135176us
I20260709 13:08:00.171458 23289 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:00.208770 23006 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:00.262184 23315 log.cc:927] Time spent T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.394s	user 0.002s	sys 0.000s
W20260709 13:08:00.152784 23258 consensus_queue.cc:1175] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "85e997ba5003439781ec4f1f8db0c3ee" responder_term: 18 status { last_received { term: 18 index: 16 } last_committed_idx: 16 last_received_current_leader { term: 18 index: 16 } }
I20260709 13:08:00.264168 23288 consensus_queue.cc:260] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:00.264348 23076 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:00.265164 23017 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:08:00.234602 15900 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.103.1:38033 timed out after 1.932s (SENT) [suppressed 18 similar messages]
W20260709 13:08:00.266906 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:00.267158 23052 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:00.267351 23324 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.288054 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:00.289683 15857 raft_consensus.cc:1240] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 23 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 22. Current term is 23. Ops: [22.17-22.17]
I20260709 13:08:00.277303 23091 raft_consensus.cc:740] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:00.302145 15856 raft_consensus.cc:2410] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 18 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 13:08:00.258456 23043 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:00.307492 15860 raft_consensus.cc:2393] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 17: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:08:00.296218 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:00.312146 16533 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:00.314301 16564 raft_consensus.cc:1275] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:08:00.316135 15855 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 21.
I20260709 13:08:00.316645 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:00.344549 15862 raft_consensus.cc:2410] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 21 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
W20260709 13:08:00.306124 23283 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.000s	sys 0.013s
I20260709 13:08:00.313222 23289 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:00.302196 23245 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:08:00.317626 23074 consensus_meta.cc:287] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.546s	user 0.010s	sys 0.000s
I20260709 13:08:00.383980 23074 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:00.386530 23074 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:00.334743 23288 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:00.301788 23301 raft_consensus.cc:740] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:08:00.337170 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 102585us
I20260709 13:08:00.337832 23017 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:00.337699 23052 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:00.302443 23174 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:00.339514 15854 raft_consensus.cc:1275] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:08:00.393935 16570 raft_consensus.cc:1240] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:00.342061 15858 raft_consensus.cc:1240] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:00.342962 15859 raft_consensus.cc:2410] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 21 because replica has last-logged OpId of term: 20 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 9.
I20260709 13:08:00.345451 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:00.345588 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:00.347321 23324 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:00.351106 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:00.351544 23076 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:00.352126 15929 raft_consensus.cc:1275] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:08:00.397425 15930 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:08:00.353703 15931 raft_consensus.cc:1275] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:08:00.318810 22508 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.354593 15936 raft_consensus.cc:1275] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:08:00.366634 15746 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet dfeaf1f6660d48848ae69710739427a0 with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER failed with leader eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica eb1e0493af1e46dbb6dec7e25525bab5 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:00.367121 16566 raft_consensus.cc:1240] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:00.368513 15902 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:00.369565 22957 consensus_queue.cc:1048] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:08:00.370239 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:00.369940 23248 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:00.374545 23091 consensus_queue.cc:260] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:00.379437 23313 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:00.376677 23123 raft_consensus.cc:2749] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:08:00.380218 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 68190.1us
I20260709 13:08:00.325528 23088 consensus_queue.cc:1048] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.073s
I20260709 13:08:00.388823 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:08:00.397522 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:00.366343 15937 raft_consensus.cc:1275] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
W20260709 13:08:00.425499 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 112466us
W20260709 13:08:00.377364 23283 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.000s	sys 0.013s
I20260709 13:08:00.410336 15974 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:08:00.413980 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 45554.3us
I20260709 13:08:00.427512 15976 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:00.415381 15857 raft_consensus.cc:2393] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 20: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:08:00.421674 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:00.422638 16533 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:00.434195 15862 raft_consensus.cc:2368] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 17. Current term is 18.
I20260709 13:08:00.434664 16533 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:00.388089 16004 raft_consensus.cc:1240] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 19. Current term is 21. Ops: []
I20260709 13:08:00.400345 23074 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:00.416898 23247 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.425141 23313 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:00.427594 23076 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.436025 23087 consensus_queue.cc:1048] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 13:08:00.457032 23076 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:00.459177 23076 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:00.442178 23273 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:00.459255 23301 consensus_queue.cc:260] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:00.459957 23273 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:00.460430 23273 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:00.460566 23301 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:00.442579 23043 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:00.444891 23324 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:00.435137 23218 consensus_queue.cc:1059] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.013s
I20260709 13:08:00.481518 23218 raft_consensus.cc:3055] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:00.481897 23218 raft_consensus.cc:740] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:08:00.482770 23218 consensus_queue.cc:260] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:00.450244 23248 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:00.450183 23052 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:08:00.486835 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 50418.2us
I20260709 13:08:00.488409 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:00.490294 15855 raft_consensus.cc:1275] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:08:00.455786 23321 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:00.440604 23074 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:00.476712 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:00.491997 16569 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 23.
I20260709 13:08:00.485162 15854 raft_consensus.cc:1275] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 21 index: 10. (term mismatch)
I20260709 13:08:00.492898 15854 pending_rounds.cc:85] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8: Aborting all ops after (but not including) 9
I20260709 13:08:00.493202 15854 pending_rounds.cc:107] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8: Aborting uncommitted NO_OP operation due to leader change: 20.10
I20260709 13:08:00.493470 15854 raft_consensus.cc:2889] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:00.485747 15930 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 18.
I20260709 13:08:00.419550 22508 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:00.484994 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:00.414729 23174 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:00.499327 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:00.501523 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:00.505295 15902 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:00.506565 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:00.507643 15862 raft_consensus.cc:2410] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 18 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 13:08:00.496671 23088 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:00.511880 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:00.518316 15853 raft_consensus.cc:1275] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 15. (term mismatch)
I20260709 13:08:00.518556 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:00.518828 15853 pending_rounds.cc:85] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8: Aborting all ops after (but not including) 14
I20260709 13:08:00.519173 15853 pending_rounds.cc:107] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8: Aborting uncommitted NO_OP operation due to leader change: 17.15
I20260709 13:08:00.519201 15930 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 23.
I20260709 13:08:00.520475 15933 raft_consensus.cc:1275] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:08:00.516995 15937 raft_consensus.cc:2410] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 19 because replica has last-logged OpId of term: 18 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 15.
I20260709 13:08:00.517853 16569 raft_consensus.cc:1240] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 13:08:00.522138 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:00.519556 15853 raft_consensus.cc:2889] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:00.516332 15825 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:08:00.516852 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:00.518270 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:00.549139 15857 raft_consensus.cc:2410] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:08:00.528573 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:00.552985 16533 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:00.472072 23247 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:00.559583 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:00.557502 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:00.525885 23088 raft_consensus.cc:697] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:00.559583 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:00.560719 15862 raft_consensus.cc:2376] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 21. Re-sending same reply.
I20260709 13:08:00.564075 23327 consensus_queue.cc:1059] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.140s
I20260709 13:08:00.564319 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:00.564508 22508 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:00.564986 15862 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 21.
I20260709 13:08:00.497442 23156 consensus_queue.cc:1048] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.057s
I20260709 13:08:00.565994 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 24 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:00.566144 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:00.567652 15860 raft_consensus.cc:1275] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:08:00.525619 23321 consensus_queue.cc:1059] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 13:08:00.568434 23321 raft_consensus.cc:3055] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:00.568764 23321 raft_consensus.cc:740] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:08:00.569242 23074 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:00.569569 23321 consensus_queue.cc:260] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:00.570714 23321 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:00.515451 23331 raft_consensus.cc:2749] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:00.520627 23167 consensus_queue.cc:1048] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.051s
I20260709 13:08:00.573495 15929 raft_consensus.cc:1275] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:08:00.574654 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:00.575304 15857 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:00.580837 23174 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:00.574677 15936 raft_consensus.cc:2410] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 21 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
I20260709 13:08:00.581712 23313 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:00.582083 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:00.582928 15929 raft_consensus.cc:2410] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 22 because replica has last-logged OpId of term: 18 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 15.
I20260709 13:08:00.556612 23043 consensus_queue.cc:1048] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 13:08:00.587896 23245 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:00.589208 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:00.589900 15862 raft_consensus.cc:2368] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 20. Current term is 21.
I20260709 13:08:00.556232 23090 raft_consensus.cc:2804] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:00.593562 23091 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:00.593628 23123 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:00.594177 23123 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:00.590637 15857 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 21.
I20260709 13:08:00.594614 23123 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:00.589977 23245 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:00.525453 23258 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:08:00.600131 23258 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:00.600489 23258 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:00.601989 15928 raft_consensus.cc:1240] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:00.603601 23090 raft_consensus.cc:697] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:08:00.605098 23247 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:00.604887 23090 consensus_queue.cc:237] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:00.550539 23287 ts_tablet_manager.cc:1929] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 13.8
I20260709 13:08:00.606137 23287 log.cc:1199] T 679229ffe7114facacb8e3a2c4c6d226 P 8430db80a39d4868bcf7641c16de35ea: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-1-root/wals/679229ffe7114facacb8e3a2c4c6d226
I20260709 13:08:00.607528 23287 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:58226 (request call id 16) took 1459 ms. Trace:
I20260709 13:08:00.607753 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:00.607924 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 24 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:00.608774 15937 raft_consensus.cc:2410] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 24 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 16.
I20260709 13:08:00.607966 23287 rpcz_store.cc:276] 0709 13:07:59.147865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:59.148117 (+   252us) service_pool.cc:224] Handling call
0709 13:07:59.314669 (+166552us) ts_tablet_manager.cc:1151] Deleting tablet 679229ffe7114facacb8e3a2c4c6d226
0709 13:07:59.314769 (+   100us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:08:00.269156 (+954387us) tablet_metadata.cc:665] Metadata flushed
0709 13:08:00.520769 (+251613us) tablet_metadata.cc:665] Metadata flushed
0709 13:08:00.607430 (+ 86661us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8945,"tablet-delete.queue_time_us":165409,"thread_start_us":87440,"threads_started":1}
I20260709 13:08:00.609822 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:00.554303 23218 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:08:00.620934 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:00.621155 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:00.621773 16010 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:00.561733 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:00.622543 15861 raft_consensus.cc:2393] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 17: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:08:00.624010 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 71108.5us
W20260709 13:08:00.629156 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:00.640146 15856 raft_consensus.cc:2368] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 20. Current term is 21.
I20260709 13:08:00.514720 23283 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:00.641811 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 231561us
I20260709 13:08:00.643011 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:00.656036 16533 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:00.650980 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:00.665733 16006 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 21.
I20260709 13:08:00.518098 23110 consensus_queue.cc:1048] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.126s
I20260709 13:08:00.669234 15974 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:00.670699 15974 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:00.672510 15974 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:00.673576 23318 consensus_queue.cc:1048] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.006s
I20260709 13:08:00.661563 23283 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:00.674387 15744 catalog_manager.cc:5028] TS 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): tablet 679229ffe7114facacb8e3a2c4c6d226 (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:08:00.676425 15900 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:08:00.677331 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 243685us
I20260709 13:08:00.646554 23313 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:00.681882 23301 raft_consensus.cc:2749] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:08:00.682535 23289 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:00.685241 16566 raft_consensus.cc:1275] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:08:00.685289 22957 consensus_queue.cc:1048] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:08:00.689208 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:00.690339 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:00.690951 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:00.666723 23043 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:00.675716 23306 raft_consensus.cc:2804] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:00.709125 15827 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:00.681442 23174 raft_consensus.cc:2804] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:00.692194 23318 raft_consensus.cc:2749] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:00.699975 15900 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:00.722250 16010 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 18.
W20260709 13:08:00.728612 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 103510us
I20260709 13:08:00.729689 16533 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:00.731948 16533 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:08:00.751041 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 45442.2us
I20260709 13:08:00.755513 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:00.768621 16569 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 20.
I20260709 13:08:00.707095 23306 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:08:00.714159 23283 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:00.713737 23174 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:08:00.727267 23301 raft_consensus.cc:2749] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:00.781172 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:00.781821 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:00.786106 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:00.786830 15928 raft_consensus.cc:2393] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 21: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:08:00.782624 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:00.748267 23017 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:00.736984 23297 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:00.726137 23324 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:00.701079 23268 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:00.710870 23273 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:00.784555 23332 consensus_queue.cc:1048] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 13:08:00.810895 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:00.812179 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:00.812309 23273 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:00.752336 23206 consensus_queue.cc:1048] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:08:00.749245 23038 consensus_queue.cc:1048] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.050s
I20260709 13:08:00.816628 16570 raft_consensus.cc:1240] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 21. Current term is 22. Ops: []
I20260709 13:08:00.803809 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:00.804128 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:00.804973 16006 raft_consensus.cc:1275] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:08:00.805604 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 115726us
I20260709 13:08:00.774623 23088 consensus_queue.cc:237] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:00.774075 23306 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:00.823215 15900 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:00.785451 23174 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:00.803253 23123 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:00.832860 16006 raft_consensus.cc:1240] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 21. Current term is 22. Ops: []
W20260709 13:08:00.861037 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:00.789105 23283 consensus_queue.cc:1048] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.044s
I20260709 13:08:00.824518 23297 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.881719 15974 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:00.887183 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:00.820344 23324 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:00.892129 15928 raft_consensus.cc:1275] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:08:00.894673 15929 raft_consensus.cc:1240] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 18. Current term is 19. Ops: []
W20260709 13:08:00.903824 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:00.880722 23283 consensus_queue.cc:1243] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer 8430db80a39d4868bcf7641c16de35ea log is divergent from this leader: its last log entry 18.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
W20260709 13:08:00.906807 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 158447us
W20260709 13:08:00.907637 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 88015.7us
I20260709 13:08:00.910303 15900 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:08:00.913584 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 176993us
I20260709 13:08:00.921221 16533 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:00.922286 16533 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:00.878626 23074 consensus_queue.cc:1048] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 13:08:00.933643 16006 raft_consensus.cc:1240] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 19. Current term is 20. Ops: []
W20260709 13:08:00.934582 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:00.906206 23301 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:00.937824 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:00.938689 15827 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:08:00.939612 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:00.941998 15827 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:00.904987 23335 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:00.949827 15827 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:00.883951 23297 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:00.957038 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:00.922891 22970 consensus_queue.cc:1048] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:08:00.957465 15858 raft_consensus.cc:1240] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 20 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:00.957693 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:00.923045 23122 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:00.963402 15856 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:00.957096 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:01.001138 15857 raft_consensus.cc:2393] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 21: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:08:00.941398 23218 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.006680 23218 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:00.950433 23038 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:08:01.008404 16010 raft_consensus.cc:1275] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:08:00.953698 23248 consensus_queue.cc:1048] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 13:08:01.007656 23324 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:01.010504 15902 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:01.011155 23297 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:01.011739 23324 raft_consensus.cc:2749] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:08:01.014485 15902 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:01.014964 23006 consensus_queue.cc:1059] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:08:01.015875 23006 raft_consensus.cc:3055] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:08:01.015981 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 64780.6us
I20260709 13:08:01.016270 23006 raft_consensus.cc:740] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:08:01.005141 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:01.017160 15900 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:00.955719 23289 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 23 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:00.959707 23206 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:01.019323 22970 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:08:01.019639 15936 raft_consensus.cc:1275] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 17. (term mismatch)
I20260709 13:08:01.019990 15936 pending_rounds.cc:85] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 16
I20260709 13:08:01.020249 15936 pending_rounds.cc:107] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 22.17
I20260709 13:08:01.020498 15936 raft_consensus.cc:2889] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:01.020427 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:01.021937 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:08:01.022243 23350 log.cc:927] Time spent T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:08:01.022571 16569 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 23.
I20260709 13:08:01.008708 23218 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:01.017861 23123 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:00.933861 23273 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:00.953889 23258 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:00.977188 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:01.019964 22970 consensus_queue.cc:1048] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:08:01.039925 23297 raft_consensus.cc:2749] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:08:00.977995 23336 log.cc:927] Time spent T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.228s	user 0.000s	sys 0.001s
I20260709 13:08:01.029524 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:01.050308 16561 raft_consensus.cc:1240] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 20. Current term is 21. Ops: []
I20260709 13:08:00.946075 23284 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:08:01.008594 23038 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:08:01.036338 23351 log.cc:927] Time spent T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:08:01.057142 15935 raft_consensus.cc:1240] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 24 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 23. Current term is 24. Ops: []
W20260709 13:08:01.033234 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 65465.2us
I20260709 13:08:01.041335 15862 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:08:01.028306 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 106094us
I20260709 13:08:00.961799 23110 raft_consensus.cc:2749] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:08:00.984138 23343 log.cc:927] Time spent T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
W20260709 13:08:00.985330 23346 log.cc:927] Time spent T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 13:08:00.984777 23345 log.cc:927] Time spent T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 13:08:01.049309 23349 log.cc:927] Time spent T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:08:01.043579 23289 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.965059 23122 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:01.109424 23122 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:01.061877 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:01.017192 23006 consensus_queue.cc:260] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:01.063879 16533 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:01.060988 15827 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
W20260709 13:08:01.113399 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 52876.2us
I20260709 13:08:01.113459 15862 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 22.
I20260709 13:08:01.039885 23324 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:01.115170 23324 raft_consensus.cc:697] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 21 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:08:01.115247 23354 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:01.115789 23354 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:01.116148 23354 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:01.116201 23324 consensus_queue.cc:237] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:01.066524 23123 consensus_queue.cc:1048] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.052s
I20260709 13:08:01.017390 15928 raft_consensus.cc:1275] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:08:00.976012 23342 consensus_queue.cc:1059] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.091s
I20260709 13:08:00.957104 23359 consensus_queue.cc:1048] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.111s
W20260709 13:08:01.121690 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 57879.1us
W20260709 13:08:01.062407 23341 log.cc:927] Time spent T 00000000000000000000000000000000 P d33cebb189b14a989836214fdbafb898: Append to log took a long time: real 0.354s	user 0.000s	sys 0.002s
I20260709 13:08:01.122731 16533 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:01.063259 16565 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 20.
I20260709 13:08:01.089043 16008 raft_consensus.cc:1275] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:08:01.089504 23306 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:01.091753 15974 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:08:01.092293 23321 consensus_meta.cc:287] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.521s	user 0.008s	sys 0.004s
I20260709 13:08:01.056924 23074 consensus_queue.cc:1048] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.039s
I20260709 13:08:01.128619 23342 raft_consensus.cc:3055] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:01.066591 23046 consensus_queue.cc:1059] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.030s
I20260709 13:08:01.129022 23342 raft_consensus.cc:740] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:01.129642 23046 raft_consensus.cc:3055] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:08:01.129868 23006 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:01.101794 23258 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 24 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:01.130762 23017 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:01.131655 15858 raft_consensus.cc:1275] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:08:01.130045 23046 raft_consensus.cc:740] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:01.134485 23046 consensus_queue.cc:260] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:01.139364 16533 leader_election.cc:400] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 election: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 20. Current term is 21.
I20260709 13:08:01.106055 23284 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:01.055714 23038 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:01.143942 16533 leader_election.cc:403] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
W20260709 13:08:01.140725 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 49079.3us
I20260709 13:08:01.145679 15974 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:01.147183 15974 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:01.105679 23174 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:01.066931 23268 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:01.143030 23283 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:01.145256 23284 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:01.082033 23297 consensus_queue.cc:1059] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.069s
I20260709 13:08:01.141180 23043 raft_consensus.cc:2749] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:08:01.121652 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 104545us
I20260709 13:08:01.102525 15856 raft_consensus.cc:2468] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 21.
I20260709 13:08:01.173868 16533 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:01.119774 23289 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:01.176717 15827 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:01.143374 23074 raft_consensus.cc:2749] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:01.179041 16006 raft_consensus.cc:1240] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:01.149322 23313 raft_consensus.cc:2749] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:01.178910 15974 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:01.197196 15974 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
W20260709 13:08:01.148113 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 132011us
I20260709 13:08:01.199381 15900 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:01.191329 15827 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:01.202148 15974 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:08:01.192811 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 135082us
I20260709 13:08:01.154824 22508 consensus_queue.cc:1048] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.041s
I20260709 13:08:01.206543 15900 leader_election.cc:400] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 17. Current term is 18.
I20260709 13:08:01.206841 15900 leader_election.cc:403] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:01.179621 23123 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 20. Current term is 21.
I20260709 13:08:01.191854 23046 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:01.207109 16533 leader_election.cc:304] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:01.229442 16533 leader_election.cc:400] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 20. Current term is 21.
I20260709 13:08:01.129889 23342 consensus_queue.cc:260] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:01.217631 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:01.224092 15902 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:01.195226 23017 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.195079 23206 consensus_queue.cc:1048] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 13:08:01.224696 23320 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.208158 23327 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:01.179586 23078 consensus_queue.cc:1048] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:08:01.237298 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:01.210333 23052 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:08:01.241590 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:01.241901 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:01.238204 15976 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:01.250103 15862 raft_consensus.cc:2368] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 20. Current term is 21.
I20260709 13:08:01.213680 23313 raft_consensus.cc:2804] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:01.252084 15827 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:01.228219 23218 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:01.254302 15827 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:01.205796 23301 consensus_queue.cc:1048] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:08:01.227862 23091 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:01.230626 23283 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:01.238245 23076 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:08:01.234880 23043 raft_consensus.cc:2749] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:01.236464 23367 raft_consensus.cc:2749] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 17. Current term is 18.
I20260709 13:08:01.256361 15827 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:01.252933 23324 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:01.268726 23359 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:01.252569 23088 consensus_queue.cc:1048] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:08:01.269038 23340 raft_consensus.cc:2749] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:08:01.246596 23247 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:08:01.322183 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 65880.5us
I20260709 13:08:01.239328 23321 consensus_queue.cc:1048] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 13:08:01.245966 23074 consensus_queue.cc:1059] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.075s
I20260709 13:08:01.268849 23110 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:01.337085 23273 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:01.239773 23017 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:08:01.338985 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 93859us
I20260709 13:08:01.269886 23076 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:01.244445 23320 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:01.249858 23335 consensus_queue.cc:1059] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.094s
I20260709 13:08:01.321154 23283 raft_consensus.cc:697] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:08:01.340771 23362 log.cc:927] Time spent T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:08:01.258477 23090 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.082s
I20260709 13:08:01.328033 22957 consensus_queue.cc:1059] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:08:01.369804 23289 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:01.357299 23076 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:01.429755 23372 raft_consensus.cc:2749] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:08:01.332371 23324 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:01.448545 23324 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:01.449041 23324 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:01.448851 23321 consensus_queue.cc:1048] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:08:01.449698 23324 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:01.450742 23321 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:01.432152 23258 consensus_queue.cc:1048] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.020s
I20260709 13:08:01.453284 23258 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.453765 23258 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:01.378659 23018 raft_consensus.cc:993] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8: : Instructing follower 8430db80a39d4868bcf7641c16de35ea to start an election
I20260709 13:08:01.456169 23018 raft_consensus.cc:1081] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Signalling peer 8430db80a39d4868bcf7641c16de35ea to start an election
I20260709 13:08:01.459635 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:08:01.367358 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 116787us
I20260709 13:08:01.304059 23091 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:01.320533 23361 consensus_queue.cc:1048] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.211s
I20260709 13:08:01.371240 23366 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.357332 23359 raft_consensus.cc:2749] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:01.462168 16565 raft_consensus.cc:2393] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 22: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:08:01.434206 23306 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:01.478201 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:01.443252 23038 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:01.464015 23006 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.360356 23273 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:01.483897 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:01.451371 23321 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:01.494927 15856 raft_consensus.cc:2410] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 26 because replica has last-logged OpId of term: 24 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 13:08:01.497303 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:01.501497 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:01.480618 23258 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:01.511489 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:01.512984 16565 raft_consensus.cc:2393] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 20: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:08:01.513235 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:01.513517 16533 leader_election.cc:400] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 election: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 20. Current term is 21.
I20260709 13:08:01.513890 16533 leader_election.cc:403] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:08:01.514040 16011 raft_consensus.cc:2410] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 26 because replica has last-logged OpId of term: 24 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 13:08:01.514762 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:01.517779 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:01.518774 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:01.473775 23366 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:01.493853 23324 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:01.508610 23052 raft_consensus.cc:697] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 22 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:01.530155 16005 raft_consensus.cc:1240] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:01.490650 23006 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:01.525272 23167 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:01.521903 23313 raft_consensus.cc:697] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:01.543798 23205 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.564927 23297 consensus_queue.cc:1059] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 13:08:01.523394 23283 consensus_queue.cc:237] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:01.547946 23361 consensus_queue.cc:1048] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.033s
I20260709 13:08:01.573706 23091 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:01.615566 23273 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:01.616544 23273 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:01.617147 23273 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:01.619745 23375 raft_consensus.cc:3060] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:08:01.624372 23388 log.cc:927] Time spent T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:08:01.592324 23122 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:01.628963 15929 raft_consensus.cc:1240] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 21. Current term is 22. Ops: []
I20260709 13:08:01.629858 23122 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:01.630478 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:01.630715 23038 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 25 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:01.631553 23273 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:01.632310 23076 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:01.633620 15933 raft_consensus.cc:1240] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 24 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 13:08:01.635835 23375 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 20. Current term is 21.
I20260709 13:08:01.587857 23313 consensus_queue.cc:237] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:01.636540 23375 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:01.637012 23375 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:01.576979 23205 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:01.668586 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:01.610157 23320 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:01.672513 23319 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.674549 23319 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:01.677098 23319 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:01.592190 23354 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:01.679893 23354 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:01.680439 23366 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:01.693681 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:01.648813 23052 consensus_queue.cc:237] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:01.695475 23375 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:01.696154 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:01.697106 16010 raft_consensus.cc:2393] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 23: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:08:01.697263 23321 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:08:01.699100 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 370347us
I20260709 13:08:01.710711 15928 raft_consensus.cc:1275] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:08:01.654088 23371 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:08:01.721321 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 379695us
I20260709 13:08:01.730420 15827 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:01.733250 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:01.736510 15827 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:01.748289 16009 raft_consensus.cc:1240] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 13:08:01.751030 23205 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:01.753643 23284 consensus_meta.cc:287] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.564s	user 0.010s	sys 0.000s
I20260709 13:08:01.754163 23284 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:08:01.754899 23369 log.cc:927] Time spent T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.492s	user 0.003s	sys 0.000s
I20260709 13:08:01.755810 23399 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.756325 23399 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:01.738106 15928 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
 from {username='slave'} at 127.0.0.1:58390
I20260709 13:08:01.758221 23076 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 24 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:01.758392 23377 consensus_queue.cc:1048] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.030s
I20260709 13:08:01.759451 16568 raft_consensus.cc:3055] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:08:01.723932 23248 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.760304 23248 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:01.762498 23248 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:01.760542 23017 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:01.762961 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:01.764312 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:01.765352 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:01.765544 23391 raft_consensus.cc:3055] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:08:01.766012 23391 raft_consensus.cc:740] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:08:01.758713 23399 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:01.759881 16568 raft_consensus.cc:740] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:08:01.778990 16568 consensus_queue.cc:260] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:01.780769 16568 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:01.769892 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:01.770087 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:01.770941 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:01.758589 15928 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:01.783013 15928 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:01.766927 23391 consensus_queue.cc:260] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:08:01.731837 23046 consensus_meta.cc:287] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.502s	user 0.004s	sys 0.008s
I20260709 13:08:01.782269 16009 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 17.
I20260709 13:08:01.787796 16569 raft_consensus.cc:2393] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 25: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:08:01.789011 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:01.793798 15900 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:01.797652 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:01.798655 15932 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:01.803051 16565 raft_consensus.cc:1275] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 13 index: 14. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:08:01.738950 23361 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.812177 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:01.817838 15930 raft_consensus.cc:3060] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:01.809939 16002 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 22.
I20260709 13:08:01.819231 16009 raft_consensus.cc:1275] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:08:01.814122 15937 pending_rounds.cc:85] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 15
I20260709 13:08:01.814579 23076 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:01.819963 15937 pending_rounds.cc:107] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 18.16
I20260709 13:08:01.843806 15937 raft_consensus.cc:2889] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:08:01.807097 23327 consensus_meta.cc:287] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.543s	user 0.012s	sys 0.000s
I20260709 13:08:01.892747 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:08:01.900807 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 99356.3us
I20260709 13:08:01.940769 16533 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:01.720062 23371 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:01.890794 23366 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.967871 16568 raft_consensus.cc:1275] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 13:08:01.954480 23205 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:08:01.940682 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 142978us
I20260709 13:08:01.803979 23090 consensus_queue.cc:1048] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.142s
I20260709 13:08:01.889971 23380 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:01.945364 23327 raft_consensus.cc:2749] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:01.940645 23018 raft_consensus.cc:2749] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:01.986065 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:01.986651 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:01.986951 15862 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:08:01.976270 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:01.968547 16568 pending_rounds.cc:85] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting all ops after (but not including) 11
I20260709 13:08:01.989451 16568 pending_rounds.cc:107] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:08:01.989781 16568 raft_consensus.cc:2889] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:01.988977 15861 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:08:01.984879 15930 raft_consensus.cc:2468] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 22.
I20260709 13:08:01.996872 16533 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:01.998095 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:01.998786 15854 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 20.
I20260709 13:08:02.008436 23384 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:02.009081 23384 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:01.890501 23361 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:02.011214 16009 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 25.
I20260709 13:08:01.979483 23380 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:02.045761 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 49029.3us
I20260709 13:08:02.066421 23380 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:02.089242 23404 raft_consensus.cc:2804] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:08:02.099967 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:02.101146 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:01.998782 23247 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.011274 23384 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:01.971382 23123 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:02.099783 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:02.104228 15862 raft_consensus.cc:1240] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 21. Current term is 22. Ops: [21.10-21.10]
I20260709 13:08:02.104452 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:01.980448 23017 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:08:02.013187 23400 log.cc:927] Time spent T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 13:08:02.100159 23218 consensus_meta.cc:287] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.841s	user 0.012s	sys 0.001s
I20260709 13:08:02.064435 15974 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:08:02.045902 23301 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.239032 23301 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:02.241410 23301 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:02.119365 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:08:02.242744 23395 log.cc:927] Time spent T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.617s	user 0.002s	sys 0.000s
I20260709 13:08:02.187296 15928 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:02.120636 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:02.110827 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 24 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:02.224753 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:02.227711 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:02.011849 23392 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:08:02.246395 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 125884us
W20260709 13:08:02.166589 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:01.973493 23366 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:01.979125 23205 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:02.171828 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:02.121896 23017 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:08:02.167840 23408 log.cc:927] Time spent T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.102s	user 0.005s	sys 0.000s
I20260709 13:08:02.179967 23375 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.282390 15827 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
W20260709 13:08:02.160897 23402 log.cc:927] Time spent T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:08:02.207513 23297 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:02.277490 23327 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:02.210325 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 24 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:02.312824 16567 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:02.060343 23284 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 24 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:02.238406 23043 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:02.167819 23218 raft_consensus.cc:2749] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:08:02.265837 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 201468us
I20260709 13:08:02.269297 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:02.269503 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:02.326749 15933 raft_consensus.cc:2393] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 22: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:08:02.272257 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:02.273203 23174 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:08:02.196602 23370 log.cc:927] Time spent T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.863s	user 0.004s	sys 0.006s
I20260709 13:08:02.267359 16568 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 18.
I20260709 13:08:02.275974 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:02.279125 15928 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:02.221366 15932 raft_consensus.cc:2468] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 21.
I20260709 13:08:02.343420 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:02.312424 23327 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:02.346808 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:02.347579 15933 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 18.
I20260709 13:08:02.286561 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:02.119359 23404 raft_consensus.cc:697] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:08:02.189498 23342 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:02.290421 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:02.200750 23361 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:02.282900 15854 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:08:02.207989 15934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58402 (request call id 2758) took 1317 ms. Trace:
W20260709 13:08:02.357805 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 382500us
I20260709 13:08:02.307217 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:02.360369 15854 raft_consensus.cc:606] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Received request to transfer leadership to TS 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:02.360953 15854 raft_consensus.cc:614] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:02.118255 23247 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:02.290124 23375 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:02.267405 23354 consensus_queue.cc:1048] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 13:08:02.328478 23405 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.271940 23397 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:02.324350 15974 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:08:02.331506 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 49080.1us
W20260709 13:08:02.372026 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 47754.1us
I20260709 13:08:02.274067 23392 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:02.344990 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:02.346133 15936 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 20.
I20260709 13:08:02.314898 23297 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:02.266314 16563 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 19.
I20260709 13:08:02.386698 16010 raft_consensus.cc:1240] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:02.358250 15934 rpcz_store.cc:276] 0709 13:08:00.890946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:00.892371 (+  1425us) service_pool.cc:224] Handling call
0709 13:08:00.892770 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:00.893197 (+   427us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:00.893236 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:00.893798 (+   562us) log.cc:844] Serialized 26 byte log entry
0709 13:08:01.136681 (+242883us) spinlock_profiling.cc:243] Waited 995 us on lock 0x7b18004220e0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72892a6 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9
0709 13:08:01.149647 (+ 12966us) spinlock_profiling.cc:243] Waited 4.25 ms on lock 0x7b18004220e0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f480d1 00007f73ea363a0f
0709 13:08:01.575460 (+425813us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:08:01.575546 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:01.575638 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:02.207737 (+632099us) raft_consensus.cc:1685] finished
0709 13:08:02.207757 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:02.207879 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":107,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":10255,"spinlock_wait_cycles":12071680,"thread_start_us":425608,"threads_started":1,"wal-append.queue_time_us":906033}
I20260709 13:08:02.373265 15854 raft_consensus.cc:1240] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 21. Current term is 22. Ops: [19.11-21.12]
I20260709 13:08:02.344264 16005 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 18.
I20260709 13:08:02.354687 23327 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:02.288167 23018 consensus_queue.cc:1059] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.038s
I20260709 13:08:02.329041 23043 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:02.386339 15935 raft_consensus.cc:2393] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 24: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:08:02.389031 15827 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:08:02.386739 23331 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:02.396039 23371 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:02.392714 23090 consensus_queue.cc:1059] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.019s
I20260709 13:08:02.403069 23218 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:02.397929 23393 consensus_queue.cc:1048] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 13:08:02.409113 23331 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:08:02.428543 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 164471us
I20260709 13:08:02.429998 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:02.430150 23423 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 13:08:02.431344 23419 log.cc:927] Time spent T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:08:02.419986 23404 consensus_queue.cc:237] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:02.373143 23405 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:02.376250 23397 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:02.437251 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:02.441007 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:08:02.443240 15746 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 83a1b79cdcf046c78453695c798a14a5 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica eb1e0493af1e46dbb6dec7e25525bab5 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:02.447633 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:02.450278 23356 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:02.450920 15935 raft_consensus.cc:1240] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 20. Current term is 21. Ops: []
I20260709 13:08:02.453450 15934 raft_consensus.cc:1240] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 21. Current term is 22. Ops: []
I20260709 13:08:02.455051 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:02.426880 23331 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:02.455188 23405 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:02.460034 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:02.437963 23423 raft_consensus.cc:697] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 25 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:08:02.464568 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:02.427050 23218 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:02.472918 15928 raft_consensus.cc:1240] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 24 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 23. Current term is 24. Ops: [23.16-23.16]
I20260709 13:08:02.493732 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:02.494980 16567 raft_consensus.cc:2468] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 24.
I20260709 13:08:02.497659 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:02.465917 23017 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:02.504499 15862 raft_consensus.cc:1240] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 14. Current term is 15. Ops: []
I20260709 13:08:02.512435 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:02.513592 15937 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 19.
W20260709 13:08:02.520412 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:02.561007 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:02.562860 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:02.566079 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:02.567188 15933 raft_consensus.cc:2410] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:08:02.573581 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:02.574240 16567 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 18.
W20260709 13:08:02.574765 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 185828us
I20260709 13:08:02.570926 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: true dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:02.576073 16561 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:02.576720 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:02.577114 15827 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:02.577317 16567 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 17.
I20260709 13:08:02.512431 23247 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:02.568650 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:02.612318 15862 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 18.
I20260709 13:08:02.570567 23361 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:02.624389 16008 raft_consensus.cc:1275] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:08:02.575759 23043 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:02.630890 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:02.648540 16533 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:02.649586 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:02.650277 23043 raft_consensus.cc:2804] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:02.650897 23043 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:02.651353 23043 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:02.610785 23078 raft_consensus.cc:2749] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:08:02.612150 23297 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:02.668987 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41662 (request call id 2786) took 1161 ms. Trace:
I20260709 13:08:02.670275 23433 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.670387 16562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1676) took 1187 ms. Trace:
I20260709 13:08:02.675899 16002 rpcz_store.cc:276] 0709 13:08:01.499397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:01.505073 (+  5676us) service_pool.cc:224] Handling call
0709 13:08:02.661021 (+1155948us) spinlock_profiling.cc:243] Waited 292 ms on lock 0x7b5400305a54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:08:02.661170 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":670453120}
I20260709 13:08:02.701709 16562 rpcz_store.cc:276] 0709 13:08:01.483148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:01.483923 (+   775us) service_pool.cc:224] Handling call
0709 13:08:01.484215 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:01.673691 (+189476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:01.673735 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:01.729641 (+ 55906us) log.cc:844] Serialized 26 byte log entry
0709 13:08:01.966159 (+236518us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:08:01.966211 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:02.669807 (+703596us) spinlock_profiling.cc:243] Waited 80.4 ms on lock 0x7b540059bd54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:08:02.669882 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:02.669935 (+    53us) raft_consensus.cc:1685] finished
0709 13:08:02.669952 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:02.670276 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":51255,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":97,"spinlock_wait_cycles":184988544,"thread_start_us":249477,"threads_started":2,"wal-append.queue_time_us":489281}
W20260709 13:08:02.671219 23411 log.cc:927] Time spent T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:08:02.716338 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:02.726189 16561 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 19.
I20260709 13:08:02.646216 23361 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:02.733198 15931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 3152) took 1427 ms. Trace:
I20260709 13:08:02.717715 23379 raft_consensus.cc:3055] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:08:02.619607 23423 consensus_queue.cc:237] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:02.733731 15931 rpcz_store.cc:276] 0709 13:08:01.306281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:01.306468 (+   187us) service_pool.cc:224] Handling call
0709 13:08:01.334540 (+ 28072us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:01.334742 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:08:01.389136 (+ 54394us) log.cc:844] Serialized 14 byte log entry
0709 13:08:02.671212 (+1282076us) spinlock_profiling.cc:243] Waited 41.4 ms on lock 0x7b180040c3c0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72b8455 00007f73ea3a0060 00007f73ea3b2716 00007f73ea3c3287 00007f73ea3c3208
0709 13:08:02.732674 (+ 61462us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:02.732744 (+    70us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:02.732814 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:02.732985 (+   171us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:02.733129 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14590,"prepare.queue_time_us":52392,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":120818,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":137,"spinlock_wait_cycles":95111040,"thread_start_us":61296,"threads_started":1}
I20260709 13:08:02.723045 23392 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:08:02.754714 23432 log.cc:927] Time spent T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:08:02.705659 23433 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:02.739173 23379 raft_consensus.cc:740] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:02.797374 23379 consensus_queue.cc:260] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:02.798811 23379 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:02.766536 23218 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:08:02.755662 23431 log.cc:927] Time spent T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 13:08:02.780529 23397 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:02.782116 23361 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:02.790107 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:02.792320 23359 raft_consensus.cc:3055] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:08:02.793258 23433 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:08:02.794603 23380 consensus_meta.cc:287] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.676s	user 0.007s	sys 0.004s
I20260709 13:08:02.795110 23327 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:02.806941 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:02.808141 15861 raft_consensus.cc:1275] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
W20260709 13:08:02.812270 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 142356us
W20260709 13:08:02.820454 23374 log.cc:927] Time spent T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 1.476s	user 0.004s	sys 0.000s
W20260709 13:08:02.819255 23428 log.cc:927] Time spent T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.264s	user 0.002s	sys 0.000s
I20260709 13:08:02.814674 15930 raft_consensus.cc:1240] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 18. Current term is 19. Ops: []
W20260709 13:08:02.820266 23444 log.cc:927] Time spent T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.051s	user 0.000s	sys 0.002s
I20260709 13:08:02.819825 23442 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.821692 15857 raft_consensus.cc:1275] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:08:02.835836 16561 raft_consensus.cc:1240] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 13:08:02.838465 16009 raft_consensus.cc:1240] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:08:02.830088 15900 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:02.839820 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:02.843030 16003 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 20.
I20260709 13:08:02.823479 23429 raft_consensus.cc:3055] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:08:02.843828 22957 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:02.844081 15900 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:02.824658 23434 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:02.844910 22957 raft_consensus.cc:2749] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:02.845413 23434 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:02.844036 23429 raft_consensus.cc:740] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:08:02.849609 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:02.849718 23429 consensus_queue.cc:260] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:02.850457 16563 raft_consensus.cc:2410] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
W20260709 13:08:02.850572 23394 log.cc:927] Time spent T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 1.238s	user 0.001s	sys 0.001s
I20260709 13:08:02.850843 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:02.850854 23429 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:02.821374 23359 raft_consensus.cc:740] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:08:02.847467 23434 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:02.812906 23043 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:02.843988 23327 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:02.855877 23442 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:02.831594 23380 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:02.861924 15855 raft_consensus.cc:2368] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 20. Current term is 21.
I20260709 13:08:02.853017 23359 consensus_queue.cc:260] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:02.862491 15827 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:08:02.864580 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 216137us
I20260709 13:08:02.839699 23454 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:02.837407 23433 consensus_queue.cc:1059] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 19.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 13:08:02.884840 23454 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:02.898928 23076 raft_consensus.cc:3055] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:08:02.901312 23076 raft_consensus.cc:740] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:08:02.902280 23076 consensus_queue.cc:260] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:02.868172 23354 raft_consensus.cc:2749] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:02.909229 16533 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:02.903148 23380 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:02.865618 23404 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.888199 23359 raft_consensus.cc:3060] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:08:02.907183 23268 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 13:08:02.902993 23454 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:02.921608 23076 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:02.919451 23404 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:02.918529 16533 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:02.910560 23284 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:02.934013 23268 raft_consensus.cc:2804] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:02.910877 23455 consensus_queue.cc:1048] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 13:08:02.932453 23043 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:02.920493 23354 consensus_queue.cc:1048] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:08:02.943198 23366 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:02.997061 15936 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58390 (request call id 3164) took 1244 ms. Trace:
I20260709 13:08:02.997637 15936 rpcz_store.cc:276] 0709 13:08:01.752366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:01.765120 (+ 12754us) service_pool.cc:224] Handling call
0709 13:08:02.996790 (+1231670us) spinlock_profiling.cc:243] Waited 580 ms on lock 0x7b54003b8b54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:08:02.996940 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1334243200}
I20260709 13:08:02.999449 16006 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60156 (request call id 1460) took 1502 ms. Trace:
I20260709 13:08:02.999993 16006 rpcz_store.cc:276] 0709 13:08:01.496960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:01.497058 (+    98us) service_pool.cc:224] Handling call
0709 13:08:02.999201 (+1502143us) spinlock_profiling.cc:243] Waited 483 ms on lock 0x7b54001f8bd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:08:02.999377 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1110607104}
I20260709 13:08:03.004299 23006 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:02.938292 23284 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:03.007762 23442 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:03.008092 23452 log.cc:927] Time spent T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:08:03.015174 23391 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:03.017611 16006 raft_consensus.cc:1275] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:08:03.009040 23327 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:03.018640 23327 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:03.020637 23327 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:03.006953 23006 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:03.009126 23436 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:08:03.030915 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 148552us
I20260709 13:08:03.033213 15827 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:03.069342 23466 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:03.070822 15827 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:03.076059 23466 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:08:03.076930 23456 log.cc:927] Time spent T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:08:03.078706 23284 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:03.042251 23436 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:03.085978 23436 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:03.027746 23043 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:03.096505 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: true dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:03.095083 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:03.098619 15853 raft_consensus.cc:3055] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:03.098956 15853 raft_consensus.cc:740] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:08:03.098742 15858 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 18.
I20260709 13:08:03.110088 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:03.099916 15853 consensus_queue.cc:260] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:03.110867 15856 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 17.
I20260709 13:08:03.110997 15853 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:03.112135 16567 raft_consensus.cc:1240] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 23. Current term is 24. Ops: []
I20260709 13:08:03.114015 15861 raft_consensus.cc:1275] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:08:03.115981 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:03.116410 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:03.117136 15857 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 21.
I20260709 13:08:03.122568 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:03.052590 23397 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:08:03.124792 23397 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:03.127220 23397 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:03.103257 23361 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:08:03.131088 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:03.110266 23043 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:03.135066 23043 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:03.137537 15853 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 19.
I20260709 13:08:03.137840 23356 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:03.029683 23006 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:08:03.143179 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 69669.1us
I20260709 13:08:03.144623 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:03.138420 23356 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:03.147162 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:03.150933 16567 raft_consensus.cc:2393] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 19: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:08:03.151075 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:03.151924 16006 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 20.
I20260709 13:08:03.147768 15827 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:08:03.120693 23404 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:03.148779 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:03.149298 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:03.155306 16565 raft_consensus.cc:1275] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:08:03.156246 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:03.156167 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:03.157512 16007 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 16.
I20260709 13:08:03.155349 16535 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:03.158941 16006 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 19.
I20260709 13:08:03.159386 16567 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:08:03.138633 23205 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:03.220602 15855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43520 (request call id 2772) took 1116 ms. Trace:
I20260709 13:08:03.221037 15855 rpcz_store.cc:276] 0709 13:08:02.103847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:02.104169 (+   322us) service_pool.cc:224] Handling call
0709 13:08:03.211564 (+1107395us) spinlock_profiling.cc:243] Waited 752 ms on lock 0x7b54002ebb54. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:08:03.220501 (+  8937us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1728470912}
I20260709 13:08:03.113767 23268 raft_consensus.cc:697] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:08:03.163471 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:03.165493 23375 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:03.230049 23268 consensus_queue.cc:237] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:03.173508 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:03.175058 15929 raft_consensus.cc:1275] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:08:03.241180 15902 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:03.242091 15902 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:03.242137 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:03.242767 15937 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:08:03.242949 15902 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:08:03.243777 23273 raft_consensus.cc:3055] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:03.244216 23273 raft_consensus.cc:740] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:08:03.245100 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:03.244961 23273 consensus_queue.cc:260] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:03.246189 23273 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:08:03.246659 23468 log.cc:927] Time spent T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.058s	user 0.001s	sys 0.001s
I20260709 13:08:03.247385 15929 raft_consensus.cc:1240] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 24 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 13:08:03.199913 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:03.179847 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:03.258561 15932 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:03.181478 23289 raft_consensus.cc:3055] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:08:03.183842 23006 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:03.160604 23469 log.cc:927] Time spent T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 13:08:03.184443 16561 raft_consensus.cc:1275] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:08:03.187835 23470 raft_consensus.cc:2749] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:08:03.189715 23331 consensus_meta.cc:287] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.726s	user 0.010s	sys 0.002s
I20260709 13:08:03.218421 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:03.218755 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:03.188203 23043 consensus_queue.cc:1059] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 13:08:03.236120 23391 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:03.236241 23167 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:03.247848 23288 consensus_queue.cc:1048] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:08:03.300873 23441 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:03.310106 23288 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:03.310205 23441 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:03.310724 23288 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:08:03.249024 23406 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:03.311863 23406 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:03.312284 23406 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:03.260439 23289 raft_consensus.cc:740] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:03.313987 23289 consensus_queue.cc:260] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:03.314664 16009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41662 (request call id 2791) took 1014 ms. Trace:
I20260709 13:08:03.315079 23289 raft_consensus.cc:3060] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:03.249477 23366 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:03.315114 16009 rpcz_store.cc:276] 0709 13:08:02.300713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:02.341272 (+ 40559us) service_pool.cc:224] Handling call
0709 13:08:02.341698 (+   426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:02.842755 (+501057us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:03.314434 (+471679us) spinlock_profiling.cc:243] Waited 496 ms on lock 0x7b54000ed8d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:08:03.314608 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1141784448}
I20260709 13:08:03.315819 23436 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:03.316067 23366 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:03.316532 23366 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:03.316401 23436 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:03.318570 23436 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:03.177810 15930 raft_consensus.cc:1275] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 16. (term mismatch)
I20260709 13:08:03.336818 15930 pending_rounds.cc:85] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 15
I20260709 13:08:03.191118 16535 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:03.253659 15902 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:03.337069 15930 pending_rounds.cc:107] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 20.16
I20260709 13:08:03.337778 15930 raft_consensus.cc:2889] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:08:03.337896 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 84313.6us
I20260709 13:08:03.254864 23327 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:08:03.262460 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 46140.1us
I20260709 13:08:03.338704 15902 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:03.200297 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:03.200604 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:03.345127 15933 raft_consensus.cc:2393] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 19: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:08:03.345796 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:08:03.345297 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 155046us
I20260709 13:08:03.341540 15902 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:03.347860 16535 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:03.350153 16535 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:03.265712 15937 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 21.
I20260709 13:08:03.263482 15931 raft_consensus.cc:2393] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 19: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:08:03.273510 15932 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 19.
I20260709 13:08:03.277303 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:03.279268 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:03.161516 23422 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:03.281313 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:03.356920 15858 raft_consensus.cc:3060] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:03.281260 16565 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
W20260709 13:08:03.283143 23439 log.cc:927] Time spent T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.095s	user 0.005s	sys 0.002s
I20260709 13:08:03.278056 23018 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:08:03.283434 23427 log.cc:927] Time spent T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.098s	user 0.000s	sys 0.008s
I20260709 13:08:03.284296 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:03.365497 15853 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 19.
I20260709 13:08:03.285331 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:03.286458 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:03.287598 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:03.367333 15861 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 21.
I20260709 13:08:03.256582 23392 consensus_queue.cc:1059] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 13:08:03.203750 23076 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:03.368662 23076 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:08:03.251876 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 88294.6us
I20260709 13:08:03.376605 23397 raft_consensus.cc:3055] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:08:03.377035 23397 raft_consensus.cc:740] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:03.377842 23397 consensus_queue.cc:260] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:03.379101 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:03.379861 16007 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 21.
I20260709 13:08:03.385046 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:03.386922 23090 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:03.320257 23167 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:08:03.279438 23419 log.cc:927] Time spent T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.055s	user 0.004s	sys 0.000s
I20260709 13:08:03.328810 23288 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:03.302829 23331 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:08:03.330313 15826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:03.299012 23381 raft_consensus.cc:2804] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:08:03.399222 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:03.402252 15858 raft_consensus.cc:2468] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 19.
I20260709 13:08:03.327131 23391 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:03.333179 23088 raft_consensus.cc:493] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:08:03.336226 23404 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:03.361791 15856 raft_consensus.cc:2393] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 19: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:08:03.346859 23434 raft_consensus.cc:2804] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:03.346859 23470 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:03.394389 16535 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:03.387521 23090 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:03.417862 23090 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:03.398634 16570 raft_consensus.cc:1240] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 18. Current term is 19. Ops: [16.15-18.16]
I20260709 13:08:03.399513 23375 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:03.330825 23356 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:03.357514 23043 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:03.419757 16563 raft_consensus.cc:1240] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 20. Current term is 21. Ops: []
W20260709 13:08:03.336002 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 44995.9us
I20260709 13:08:03.402967 16568 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:03.355353 23433 raft_consensus.cc:2764] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:08:03.361232 23422 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:03.404462 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:03.400903 23454 raft_consensus.cc:2804] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:03.412036 23088 raft_consensus.cc:515] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:03.412915 23441 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:03.414561 23391 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:03.441978 16568 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 21.
I20260709 13:08:03.415084 23361 raft_consensus.cc:3055] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:03.414902 23404 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:03.347127 23327 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:03.392096 23018 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:03.419312 23375 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:03.352583 23414 raft_consensus.cc:2749] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:03.423856 16535 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:03.453760 16535 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:03.426666 15976 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:03.371592 23366 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:03.410074 23434 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:03.429800 23450 raft_consensus.cc:2749] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:08:03.431552 23422 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:03.455453 16535 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:08:03.458112 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 205354us
I20260709 13:08:03.441182 23331 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:03.481061 15855 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 22.
I20260709 13:08:03.430015 23288 consensus_queue.cc:1048] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 13:08:03.488098 15825 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:08:03.444665 23361 raft_consensus.cc:740] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:08:03.500205 15856 raft_consensus.cc:1275] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:08:03.500509 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:03.481451 23434 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:03.481016 23342 raft_consensus.cc:2804] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:03.478607 23359 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:03.519266 23359 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:03.521775 23359 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:03.485833 15974 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:08:03.515868 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 174393us
I20260709 13:08:03.516666 23342 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:03.523895 23342 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:03.525923 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:03.507934 23123 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:03.484321 23297 consensus_queue.cc:1048] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.036s
I20260709 13:08:03.535457 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:03.536538 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:03.537986 16006 raft_consensus.cc:1240] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:03.540401 23405 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:03.540964 23405 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:03.543063 23405 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:03.544838 15858 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ac847593d6724522bda12a2207f91c31"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:08:03.545009 23366 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:03.545421 15858 raft_consensus.cc:606] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Received request to transfer leadership to TS 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:03.545822 15858 raft_consensus.cc:614] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:03.508425 23436 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:03.549240 23436 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:03.510288 22508 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:03.549664 23436 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:03.550040 22508 raft_consensus.cc:697] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 21 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:08:03.550928 22508 consensus_queue.cc:237] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:03.552395 23379 raft_consensus.cc:2749] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:08:03.534049 23470 raft_consensus.cc:2749] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:03.557008 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:03.549641 15902 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:03.556919 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:08:03.561748 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 181314us
I20260709 13:08:03.563545 15902 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:08:03.569244 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 160508us
I20260709 13:08:03.569388 15827 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:03.571602 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:03.574918 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:03.576648 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:03.532025 23123 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:03.577049 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:03.586575 16533 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:03.584777 15827 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:03.569201 23283 raft_consensus.cc:2749] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:03.569669 23381 raft_consensus.cc:697] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 24 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:08:03.543548 23167 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:03.608611 23088 leader_election.cc:290] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:03.609925 23375 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:03.611752 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:03.611457 23366 consensus_queue.cc:1048] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:08:03.654299 16008 raft_consensus.cc:2393] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 18: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:08:03.586851 23123 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:03.586494 23361 consensus_queue.cc:260] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:03.756474 23361 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:03.756769 23433 raft_consensus.cc:2749] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:03.605687 23407 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:08:03.618803 23475 log.cc:927] Time spent T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:08:03.787781 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:03.789475 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:03.790786 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:03.797750 23414 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:03.798545 23414 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:08:03.807788 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 352378us
I20260709 13:08:03.587978 23342 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:08:03.625981 23476 log.cc:927] Time spent T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 13:08:03.626603 15858 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:08:03.825362 15858 raft_consensus.cc:606] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Received request to transfer leadership to TS 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:03.627562 23297 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:03.628892 23018 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:03.630196 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:03.630872 23482 raft_consensus.cc:2749] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:08:03.605546 23356 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:03.633090 23404 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:03.634459 23327 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:03.582686 23090 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:03.837285 23090 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:03.837749 23090 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:03.607333 16533 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:03.647820 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:08:03.840065 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 232813us
I20260709 13:08:03.840801 16533 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:03.632820 23406 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:03.613029 16011 raft_consensus.cc:1275] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 25 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:08:03.588212 23442 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:03.672669 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:03.726377 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:03.845058 23442 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:03.847254 23442 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:03.740368 23381 consensus_queue.cc:237] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:03.742238 23422 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:03.849347 23422 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:03.745908 23436 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:03.850950 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:03.851996 23436 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:03.648605 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:03.853052 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:03.853261 23436 raft_consensus.cc:2804] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:03.853653 15929 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:03.853750 23436 raft_consensus.cc:697] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:08:03.854717 23436 consensus_queue.cc:237] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:03.855803 15935 raft_consensus.cc:2393] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 22: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:08:03.871124 16009 raft_consensus.cc:1240] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:03.872601 15929 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 21.
I20260709 13:08:03.811751 23156 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:03.869055 23470 consensus_queue.cc:1048] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:08:03.885820 23167 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:03.887632 23076 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:03.887993 23043 raft_consensus.cc:2749] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:03.888967 23090 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:03.891057 23090 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:03.893091 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:03.895761 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:08:03.897385 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:03.899181 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:08:03.900206 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 377090us
I20260709 13:08:03.901021 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:03.903081 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:03.903365 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:03.898473 23156 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:03.911597 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:03.912477 16008 raft_consensus.cc:1240] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:03.916961 23342 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:03.929956 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:03.918830 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:03.892927 23454 raft_consensus.cc:697] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 19 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:08:03.924153 23123 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:03.927132 23429 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:08:03.932047 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:03.928435 23156 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:03.938210 15974 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:03.944063 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:03.944746 16569 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:08:03.957633 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 394104us
I20260709 13:08:03.907851 23288 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:03.968226 15974 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:03.965339 15902 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:03.969549 15974 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:03.971237 15825 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:08:03.963371 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 389770us
I20260709 13:08:03.977654 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:03.978853 15900 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:08:03.950636 23342 raft_consensus.cc:2749] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:08:03.986065 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:03.986184 15902 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:03.986749 16005 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:03.964015 23393 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:03.992390 15858 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:08:03.993106 15858 raft_consensus.cc:606] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 25 LEADER]: Received request to transfer leadership to TS 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:03.975081 23289 raft_consensus.cc:2749] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:03.991546 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:03.996723 16008 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:03.997834 15900 leader_election.cc:400] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 20. Current term is 21.
I20260709 13:08:03.988013 23488 raft_consensus.cc:2749] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:08:03.978531 23372 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:03.998350 15900 leader_election.cc:403] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:08:04.001933 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:04.003122 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:03.980382 23442 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:08:04.008836 16007 raft_consensus.cc:2393] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 18: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:08:03.964190 23434 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:03.970566 23479 consensus_queue.cc:1059] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.025s
I20260709 13:08:03.974397 23288 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:03.995751 23076 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:03.992684 23482 raft_consensus.cc:2764] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 13:08:04.021407 15825 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:08:04.024711 16569 raft_consensus.cc:2468] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:08:04.006379 23372 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:03.991835 23366 consensus_queue.cc:1048] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
W20260709 13:08:04.043104 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 48758.4us
W20260709 13:08:04.049043 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 206734us
I20260709 13:08:03.999943 23393 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:04.024214 23076 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:04.022840 23289 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:04.023567 23414 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:04.056907 16008 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 21.
I20260709 13:08:04.031064 23273 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:04.034163 23372 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:04.006986 15902 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:04.052268 16533 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:08:04.065016 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 58084.2us
I20260709 13:08:04.066445 15902 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:04.048933 23156 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:04.042201 23297 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.068440 16533 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:04.059899 23076 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:04.071581 16533 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:04.053572 23493 raft_consensus.cc:2749] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033) with higher term. Message: Invalid argument: T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 20. Current term is 21.
I20260709 13:08:04.075700 23123 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:04.073351 23297 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:04.073571 23156 raft_consensus.cc:697] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 21 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:08:04.085644 23156 consensus_queue.cc:237] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:04.065099 23289 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:04.089841 23289 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:04.023517 23454 consensus_queue.cc:237] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:04.030673 23492 consensus_queue.cc:1059] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:08:04.071657 23006 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:04.067832 23482 raft_consensus.cc:2749] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:04.095629 16004 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 19.
I20260709 13:08:04.106031 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:04.134095 16008 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:04.134949 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:04.114820 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:04.104274 16533 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:04.137669 16569 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:04.139084 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:04.139933 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:04.140151 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:04.110050 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:04.141040 15929 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 18.
W20260709 13:08:04.096354 23335 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.021s	sys 0.057s
I20260709 13:08:04.145805 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:04.135788 23038 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.106935 23393 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:04.165912 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:04.167214 16008 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 21.
I20260709 13:08:04.101884 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:04.149257 15825 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:08:04.147315 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:04.116281 16567 raft_consensus.cc:1240] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:04.149183 16005 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 22.
I20260709 13:08:04.095768 23414 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:04.152172 16569 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 20.
I20260709 13:08:04.106673 23043 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:04.187250 15902 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:04.189527 16569 raft_consensus.cc:1275] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 25 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:08:04.096045 23397 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:08:04.192806 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 218644us
I20260709 13:08:04.115581 23372 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:04.122097 23366 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.106518 23288 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:08:04.155112 23335 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.022s	sys 0.057s
I20260709 13:08:04.100656 23483 consensus_queue.cc:1059] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.080s
I20260709 13:08:04.174551 23038 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:04.155103 23283 raft_consensus.cc:2749] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:04.216048 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:04.198946 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:04.199393 16006 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:04.200446 23407 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:04.200837 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:04.122996 23434 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:04.106978 23006 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:04.203220 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:04.203914 23488 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:04.111868 23123 consensus_queue.cc:1048] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.075s
I20260709 13:08:04.117385 23482 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:04.207327 15902 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:04.099550 23405 raft_consensus.cc:2749] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:04.208160 16535 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:08:04.210583 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 189193us
I20260709 13:08:04.210206 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:04.221462 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:04.099037 23436 consensus_queue.cc:1048] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.111s
I20260709 13:08:04.241649 15853 raft_consensus.cc:1275] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:08:04.242702 15974 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:04.229274 15856 raft_consensus.cc:2393] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 18: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:08:04.236430 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:04.238413 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:04.240738 15855 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:04.240487 15827 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:04.247062 16006 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 20.
I20260709 13:08:04.248235 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:04.248418 16535 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:04.231101 23006 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:04.252897 15827 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:04.219892 23407 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:04.254070 15861 raft_consensus.cc:2393] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 22: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:08:04.259652 16002 raft_consensus.cc:1275] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:08:04.259632 15902 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:04.263835 15827 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:04.212860 23366 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:04.268805 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:04.273562 16005 raft_consensus.cc:1240] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 13:08:04.273658 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:04.275635 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:04.239224 23482 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:04.278014 15827 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:04.279044 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:04.249727 23335 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.277979 23247 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:08:04.282301 15929 raft_consensus.cc:1275] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:08:04.252900 23501 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.255350 23043 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:04.289213 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:04.254734 23500 raft_consensus.cc:2764] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:08:04.252658 23504 raft_consensus.cc:2749] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:04.260689 23407 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:04.281843 23414 consensus_queue.cc:1048] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.018s
I20260709 13:08:04.279871 23405 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:04.297255 15974 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:04.302506 15855 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 18.
I20260709 13:08:04.299705 15928 raft_consensus.cc:2393] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 19: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:08:04.306555 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:08:04.312419 23412 raft_consensus.cc:2749] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:04.313449 23412 consensus_queue.cc:1048] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:08:04.314905 23412 consensus_queue.cc:1048] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:08:04.285046 23167 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:08:04.311348 23498 log.cc:927] Time spent T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:08:04.318615 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 25 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:04.289896 23205 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:04.293787 23335 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:04.337193 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:04.338379 23372 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:04.339551 23335 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:04.341060 23038 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:04.335371 23043 raft_consensus.cc:697] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:08:04.342434 23043 consensus_queue.cc:237] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:04.346702 23482 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:04.277112 23434 consensus_queue.cc:1059] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 13:08:04.334939 23393 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:04.348650 23406 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:04.332315 23366 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:04.353279 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:04.354475 16563 raft_consensus.cc:2410] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 24 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 13:08:04.335204 15936 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58390 (request call id 3178) took 1096 ms. Trace:
I20260709 13:08:04.359956 15936 rpcz_store.cc:276] 0709 13:08:03.238623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:03.248481 (+  9858us) service_pool.cc:224] Handling call
0709 13:08:03.248753 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:03.413521 (+164768us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:03.413551 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:03.413591 (+    40us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:08:03.413621 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:04.334914 (+921293us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b54002da5d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:08:04.334980 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:04.335137 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":378495360}
I20260709 13:08:04.329805 23006 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:04.307024 23247 raft_consensus.cc:697] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 22 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:08:04.331858 16011 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 22.
I20260709 13:08:04.276109 23489 raft_consensus.cc:2804] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:04.371515 23489 raft_consensus.cc:697] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:08:04.371810 23247 consensus_queue.cc:237] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:04.372495 23489 consensus_queue.cc:237] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:08:04.376937 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 66989.5us
I20260709 13:08:04.355468 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:04.378909 16563 raft_consensus.cc:2368] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 19. Current term is 20.
I20260709 13:08:04.382977 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:08:04.377048 23391 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.843s	user 0.007s	sys 0.000s
I20260709 13:08:04.388149 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceaf02d91e334d5eba077b59ccce3a38" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:04.378032 23405 raft_consensus.cc:697] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 20 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:08:04.387439 23391 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.855s	user 0.007s	sys 0.000s
I20260709 13:08:04.392105 23006 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:04.392431 23405 consensus_queue.cc:237] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:04.393141 15974 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:04.394106 23088 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:04.396109 15976 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:08:04.401224 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 112727us
I20260709 13:08:04.403553 15827 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:04.301417 23501 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:04.407202 15976 leader_election.cc:400] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 election: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 19. Current term is 20.
I20260709 13:08:04.407696 15976 leader_election.cc:403] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:08:04.409881 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:04.403725 23288 raft_consensus.cc:2804] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:04.436072 23372 raft_consensus.cc:2749] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for earlier term 19. Current term is 20.
I20260709 13:08:04.449126 16533 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:04.454134 23331 raft_consensus.cc:3055] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:08:04.454613 23331 raft_consensus.cc:740] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:04.451257 16533 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:08:04.459329 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 55846.6us
I20260709 13:08:04.460333 15827 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:04.462852 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:04.466282 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:08:04.458253 23481 log.cc:927] Time spent T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 13:08:04.470557 15827 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:08:04.525506 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 55029.8us
I20260709 13:08:04.528999 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:04.529817 16009 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 23.
I20260709 13:08:04.460310 23284 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:04.533773 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:04.550465 15936 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 22.
I20260709 13:08:04.464834 23442 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:04.461453 23006 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:04.524668 23429 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.461234 23288 raft_consensus.cc:697] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 17 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:04.610780 23391 raft_consensus.cc:3055] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:04.468901 23289 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:04.511098 23377 raft_consensus.cc:2749] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:04.624989 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:04.633968 23391 raft_consensus.cc:740] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:08:04.455494 23331 consensus_queue.cc:260] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:04.655932 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:04.630643 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:04.677726 16009 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 19.
I20260709 13:08:04.680807 16008 raft_consensus.cc:1240] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:08:04.552657 23205 raft_consensus.cc:2749] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:08:04.692292 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:04.693110 15856 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 22.
I20260709 13:08:04.703732 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:04.704135 15858 raft_consensus.cc:1240] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:04.705011 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:04.713239 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:04.713855 15853 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:08:04.689376 23076 consensus_meta.cc:287] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.588s	user 0.010s	sys 0.000s
I20260709 13:08:04.703330 23167 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:04.715358 15861 raft_consensus.cc:1240] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 17. Current term is 18. Ops: []
I20260709 13:08:04.655284 23429 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:04.683175 23289 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:04.738547 23482 raft_consensus.cc:493] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.743160 23482 raft_consensus.cc:515] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:04.745247 23482 leader_election.cc:290] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:04.736284 23504 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:08:04.746191 23504 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:04.748452 23504 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:04.725150 23167 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:04.753275 15853 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 18.
I20260709 13:08:04.753733 23167 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:04.732003 23391 consensus_queue.cc:260] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:08:04.756696 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 61790.4us
I20260709 13:08:04.762725 23391 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:04.732700 23331 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:04.648720 23288 consensus_queue.cc:237] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:04.720444 23076 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:04.794823 15827 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:08:04.779690 15902 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:08:04.834996 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 55415us
I20260709 13:08:04.839147 15929 raft_consensus.cc:1240] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 21. Current term is 22. Ops: [21.16-21.16]
W20260709 13:08:04.841482 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 46783.1us
I20260709 13:08:04.825456 23377 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:08:04.844913 23319 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:08:04.845397 23319 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:04.844991 23377 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:04.845748 23319 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:04.847287 23377 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:04.855628 23319 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:04.864003 23319 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 24 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:04.890542 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:04.891403 16009 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:04.896289 23504 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:04.896914 23504 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:04.897320 23504 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:04.830152 23508 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:08:04.782846 23492 consensus_queue.cc:1059] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.086s
I20260709 13:08:04.782127 23482 consensus_queue.cc:1059] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.089s
I20260709 13:08:04.919149 16009 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 24.
I20260709 13:08:04.929505 23076 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:04.930606 23429 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:04.931413 23289 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:04.937115 23484 raft_consensus.cc:3055] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:08:04.937572 23484 raft_consensus.cc:740] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:08:04.938344 23484 consensus_queue.cc:260] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:04.939090 23484 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:04.922606 23366 raft_consensus.cc:3055] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:08:04.940068 23366 raft_consensus.cc:740] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:08:04.940897 23273 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:04.912704 23508 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:04.941413 23273 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:04.942916 23508 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:04.951035 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:08:04.954401 23496 log.cc:927] Time spent T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.763s	user 0.004s	sys 0.030s
I20260709 13:08:04.955596 16562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1703) took 1369 ms. Trace:
I20260709 13:08:04.956462 16562 rpcz_store.cc:276] 0709 13:08:03.586427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:03.586587 (+   160us) service_pool.cc:224] Handling call
0709 13:08:03.586889 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:03.587321 (+   432us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:08:03.587365 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:03.604168 (+ 16803us) log.cc:844] Serialized 26 byte log entry
0709 13:08:03.838576 (+234408us) spinlock_profiling.cc:243] Waited 30.1 ms on lock 0x7b180038a040. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e729de07 00007f73f00f8ab9 00007f73e72ba69b 00007f73e72b893f 00007f73e72bac02 00007f73e9f4220b 00007f73e9f47d23 00007f73e9f480d1 00007f73ea363a0f
0709 13:08:04.025930 (+187354us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:04.025990 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:04.026089 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:04.953857 (+927768us) spinlock_profiling.cc:243] Waited 16.5 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:08:04.955366 (+  1509us) raft_consensus.cc:1685] finished
0709 13:08:04.955388 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:04.955538 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":91,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":13399,"spinlock_wait_cycles":107256448,"thread_start_us":187138,"threads_started":1,"wal-append.queue_time_us":535334}
I20260709 13:08:04.953990 23273 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:04.940922 23366 consensus_queue.cc:260] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:08:04.959604 23512 log.cc:927] Time spent T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.587s	user 0.013s	sys 0.008s
I20260709 13:08:04.960053 23366 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:08:04.962292 15901 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:04.963718 23273 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:04.976166 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:04.979393 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:04.983754 16009 raft_consensus.cc:1240] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 17. Current term is 18. Ops: []
W20260709 13:08:04.984597 23407 consensus_meta.cc:287] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.649s	user 0.007s	sys 0.003s
I20260709 13:08:04.985018 23407 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:04.986121 15859 raft_consensus.cc:2393] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 19: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:08:04.988060 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:04.989670 23407 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:04.990082 16002 raft_consensus.cc:2393] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 16: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:08:04.993147 16566 raft_consensus.cc:1240] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 22. Current term is 23. Ops: []
I20260709 13:08:04.998862 16562 raft_consensus.cc:1240] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 21. Current term is 22. Ops: []
I20260709 13:08:05.011826 23379 consensus_queue.cc:1059] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:08:05.014366 23331 raft_consensus.cc:3055] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:08:05.014355 23484 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:05.015434 23484 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:05.018757 23484 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:05.020346 23331 raft_consensus.cc:740] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:05.022318 23504 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:05.023425 23331 consensus_queue.cc:260] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:05.025699 23331 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:05.037231 23289 consensus_queue.cc:1059] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:08:05.041857 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:05.047644 15900 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:05.049331 15900 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:05.049966 23522 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:05.047961 15935 raft_consensus.cc:1275] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:08:05.050349 23484 raft_consensus.cc:2749] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:05.055591 16008 raft_consensus.cc:1275] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:08:05.051491 15900 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:08:05.057475 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:05.060765 23407 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:05.061371 23273 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:05.061276 23407 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:05.061883 23273 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:05.063817 23361 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.063339 23407 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:05.064419 23361 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:05.066902 23361 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:05.067360 23361 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.067688 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:05.068235 15935 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
I20260709 13:08:05.073793 15935 raft_consensus.cc:1275] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:08:05.074065 23297 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:05.059187 23484 raft_consensus.cc:2804] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:05.079404 23484 raft_consensus.cc:697] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 18 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:08:05.080410 23484 consensus_queue.cc:237] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:05.084453 23327 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:05.085188 23327 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:05.086970 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:05.087826 16005 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 16.
I20260709 13:08:05.088034 23273 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:05.088665 23361 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:05.091127 16009 raft_consensus.cc:1275] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:08:05.092053 16011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41674 (request call id 2909) took 1114 ms. Trace:
I20260709 13:08:05.092504 16011 rpcz_store.cc:276] 0709 13:08:03.977253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:03.977400 (+   147us) service_pool.cc:224] Handling call
0709 13:08:05.091868 (+1114468us) spinlock_profiling.cc:243] Waited 353 ms on lock 0x7b54000eecd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:08:05.091991 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":811395968}
I20260709 13:08:05.095472 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:05.096150 16009 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 17.
I20260709 13:08:05.097544 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:05.099349 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:05.100157 15930 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 16.
I20260709 13:08:05.055953 23522 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:05.095079 23361 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:05.125955 23247 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.742s	user 0.010s	sys 0.003s
W20260709 13:08:05.126545 23247 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.744s	user 0.010s	sys 0.003s
I20260709 13:08:05.127401 15855 raft_consensus.cc:1240] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 22 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 13:08:05.104764 23522 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:05.141680 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:05.176095 15853 raft_consensus.cc:2393] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 24: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:08:05.112313 23273 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.151619 15859 raft_consensus.cc:1240] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:05.164445 23450 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:05.167795 15759 catalog_manager.cc:5697] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 18 to 19, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 19 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:05.190626 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:05.171135 23206 raft_consensus.cc:993] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8: : Instructing follower 85e997ba5003439781ec4f1f8db0c3ee to start an election
I20260709 13:08:05.148176 23429 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:05.190860 16568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:05.189138 16533 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:05.191253 16570 raft_consensus.cc:3060] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:05.193863 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b3676041725488fb6ffad37889b386b" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:05.194371 16533 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:05.194782 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:05.195447 16567 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 16.
I20260709 13:08:05.191941 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:05.197774 16003 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:05.192678 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:05.192152 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:05.194624 16568 raft_consensus.cc:1240] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:05.201591 16565 raft_consensus.cc:1275] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:08:05.205878 16566 raft_consensus.cc:1275] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 13 index: 14. Preceding OpId from leader: term: 19 index: 17. (index mismatch)
I20260709 13:08:05.223623 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:05.194245 23247 raft_consensus.cc:1081] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Signalling peer 85e997ba5003439781ec4f1f8db0c3ee to start an election
I20260709 13:08:05.209685 23006 raft_consensus.cc:2804] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:05.193599 23361 consensus_queue.cc:1048] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.028s
I20260709 13:08:05.192073 23273 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:05.201325 23429 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:05.193703 23450 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:05.213372 23436 raft_consensus.cc:2804] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:05.233520 16562 raft_consensus.cc:1275] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:08:05.234972 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:05.209640 23404 consensus_queue.cc:1048] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.020s
I20260709 13:08:05.228588 23327 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:05.235872 16565 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 23.
I20260709 13:08:05.227838 23504 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:05.229789 23006 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:05.243054 23513 log.cc:927] Time spent T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.867s	user 0.006s	sys 0.000s
I20260709 13:08:05.244541 23006 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:05.236956 23436 raft_consensus.cc:493] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:05.248765 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:05.253079 23273 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:05.254827 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:05.255455 16004 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 22.
I20260709 13:08:05.257234 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:05.258755 15931 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 22.
I20260709 13:08:05.247545 23436 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:05.261499 16533 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:05.264616 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:05.263998 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:05.272356 16566 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:05.293785 23361 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:05.294317 23361 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:05.294699 23361 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:05.298198 23429 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:05.321489 16566 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 24.
I20260709 13:08:05.322082 16570 raft_consensus.cc:2468] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 18.
I20260709 13:08:05.322458 23504 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:05.328271 23504 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:05.331254 23504 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:05.337184 16003 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 20.
I20260709 13:08:05.343760 23484 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:05.351712 23504 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:05.363955 23327 raft_consensus.cc:3055] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:08:05.364423 23327 raft_consensus.cc:740] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:05.365352 23327 consensus_queue.cc:260] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:05.366588 23327 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:05.366380 23504 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:05.368506 23406 raft_consensus.cc:3055] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:05.368913 23406 raft_consensus.cc:740] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:08:05.369673 23406 consensus_queue.cc:260] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:05.370263 23504 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:05.370872 23406 raft_consensus.cc:3060] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:08:05.377902 23510 log.cc:927] Time spent T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 1.004s	user 0.008s	sys 0.013s
I20260709 13:08:05.383864 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:05.384534 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 25 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:08:05.392380 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:05.392904 15902 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:05.399536 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:05.405476 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:05.406780 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:05.411196 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:05.412480 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:05.413604 15858 raft_consensus.cc:1240] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 21. Current term is 22. Ops: []
I20260709 13:08:05.419112 23501 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:05.429602 15928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53636 (request call id 1494) took 1282 ms. Trace:
I20260709 13:08:05.430029 15928 rpcz_store.cc:276] 0709 13:08:04.146704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:04.147118 (+   414us) service_pool.cc:224] Handling call
0709 13:08:05.429394 (+1282276us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b540019c3d4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:08:05.429542 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":298709888}
I20260709 13:08:05.431535 15929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53636 (request call id 1496) took 1114 ms. Trace:
I20260709 13:08:05.431945 15929 rpcz_store.cc:276] 0709 13:08:04.317580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:04.318172 (+   592us) service_pool.cc:224] Handling call
0709 13:08:04.321581 (+  3409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:04.839736 (+518155us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:05.431294 (+591558us) spinlock_profiling.cc:243] Waited 517 ms on lock 0x7b540044cad4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3a8e01 00007f73ea3a783e 00007f73ef36c419 00007f73e88e3bd4 00007f73e88e3b62 00007f73e88e3a91 00007f73e88e3a0c 00007f73e88e2cc2 00007f73e86e348c 00007f73e86e28c6
0709 13:08:05.431478 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1189478656}
I20260709 13:08:05.433131 15934 raft_consensus.cc:1240] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:05.438743 23283 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:05.439313 23283 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:05.441484 23283 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:05.441320 23273 consensus_queue.cc:1059] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:08:05.442193 23273 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:05.442744 23283 consensus_queue.cc:1059] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:08:05.442703 23273 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:05.443503 23123 raft_consensus.cc:3055] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:08:05.443934 23123 raft_consensus.cc:740] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:08:05.444749 23123 consensus_queue.cc:260] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:05.445271 23273 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:05.457576 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:05.458329 16561 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 24.
I20260709 13:08:05.462450 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:05.467293 23283 consensus_queue.cc:1048] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:08:05.475018 15746 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet dfeaf1f6660d48848ae69710739427a0 with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER failed with leader eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica eb1e0493af1e46dbb6dec7e25525bab5 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:05.478374 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:05.445966 23123 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:05.446657 23297 raft_consensus.cc:3055] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:05.480732 23297 raft_consensus.cc:740] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:08:05.481647 23297 consensus_queue.cc:260] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:05.482774 23297 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:05.484045 23405 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:05.484625 23405 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:05.488576 15902 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:05.487399 23405 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:05.518146 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:05.528940 15862 raft_consensus.cc:3060] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:08:05.518198 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:05.529814 15859 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
W20260709 13:08:05.526129 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:05.532392 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 68172.9us
I20260709 13:08:05.535631 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:05.537501 15900 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:05.538882 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:05.541425 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:08:05.541592 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 45641.9us
W20260709 13:08:05.542107 16533 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:05.542665 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 54166.3us
I20260709 13:08:05.543650 15902 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:05.545221 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:05.488292 23504 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:05.547725 23504 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:08:05.550083 23504 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
I20260709 13:08:05.488380 23433 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:05.550912 23433 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:05.559139 15856 raft_consensus.cc:1275] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:08:05.562991 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:05.565519 15862 raft_consensus.cc:2468] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 21.
I20260709 13:08:05.567883 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:05.570663 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:05.572006 15974 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:05.575729 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:05.576745 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:05.577644 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:05.578523 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:08:05.515276 23489 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.140s	user 0.002s	sys 0.029s
I20260709 13:08:05.583957 15856 raft_consensus.cc:1275] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 19 index: 17. (index mismatch)
I20260709 13:08:05.585309 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:05.586771 23156 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.587409 23156 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:05.590047 23156 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:05.579932 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:05.581037 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:05.374579 23436 raft_consensus.cc:515] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:05.540760 23381 raft_consensus.cc:2749] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:05.583501 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:05.597910 16533 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:05.566179 23366 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:08:05.584062 23489 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.209s	user 0.002s	sys 0.030s
I20260709 13:08:05.566514 23405 raft_consensus.cc:2804] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:05.562659 23404 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:05.608075 16533 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:05.374351 23484 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:05.615074 16533 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:08:05.616784 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 60102.4us
I20260709 13:08:05.533006 23006 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:05.618911 16533 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:05.624207 15902 leader_election.cc:304] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:05.632053 15827 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:05.633795 16004 raft_consensus.cc:1275] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:08:05.635142 16004 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051"
dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:41674
I20260709 13:08:05.635634 23206 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:08:05.638049 23206 raft_consensus.cc:697] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 24 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:08:05.635237 15827 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:05.639161 23441 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:05.639822 23441 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:05.640339 23441 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:05.640537 15759 catalog_manager.cc:5697] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 17 to 21, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 21 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:05.639253 23206 consensus_queue.cc:237] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:05.650897 15759 catalog_manager.cc:5697] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 17 to 19, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 19 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:05.653394 15759 catalog_manager.cc:5697] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 14 to 16, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 16 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:05.655109 16533 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:05.656922 15760 catalog_manager.cc:5244] ChangeConfig:REMOVE_PEER RPC for tablet 679229ffe7114facacb8e3a2c4c6d226 with cas_config_opid_index 3: aborting the task: latest config opid_index 9; task opid_index 3
I20260709 13:08:05.636384 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:05.635736 16004 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:05.661661 16004 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:05.655489 15759 catalog_manager.cc:5697] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 13 to 17, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 17 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:05.611011 23043 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:08:05.670931 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:05.595711 23289 consensus_queue.cc:1048] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.028s
I20260709 13:08:05.610266 23405 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:08:05.622105 23497 raft_consensus.cc:2804] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:08:05.659509 15760 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58448 (request call id 311) took 53362 ms (client timeout 15000 ms). Trace:
I20260709 13:08:05.610598 23450 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:05.603487 23319 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.023s
I20260709 13:08:05.676252 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:08:05.670893 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 46806.2us
I20260709 13:08:05.622619 23381 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:05.624439 23273 raft_consensus.cc:2749] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:05.623024 23375 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:08:05.666500 15759 catalog_manager.cc:5697] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 15 to 18, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 18 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:05.684314 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:05.624951 23493 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:05.688431 15902 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:05.693261 16562 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 19.
I20260709 13:08:05.695554 15759 catalog_manager.cc:5697] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 19 to 22. New cstate: current_term: 22 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:08:05.696648 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 163511us
I20260709 13:08:05.616228 23404 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:08:05.701038 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 158823us
I20260709 13:08:05.673768 22970 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:08:05.691854 23529 log.cc:927] Time spent T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.347s	user 0.001s	sys 0.000s
I20260709 13:08:05.697974 15976 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:05.674160 23342 raft_consensus.cc:2749] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:05.622017 23205 consensus_queue.cc:1048] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.072s
I20260709 13:08:05.701944 15827 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:05.682677 23405 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:05.688300 23497 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:05.690387 23078 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.672923 23156 consensus_queue.cc:1048] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 13:08:05.690557 23018 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.691223 23283 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:08:05.745131 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 47224us
W20260709 13:08:05.751073 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 49209.3us
W20260709 13:08:05.682528 15760 rpcz_store.cc:269] 0709 13:07:12.297696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.297974 (+   278us) service_pool.cc:224] Handling call
0709 13:08:05.659374 (+53361400us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:00.441934 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:00.442624 (+   690us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:00.442679 (+    55us) write_op.cc:435] Acquired schema lock
0709 13:08:00.442699 (+    20us) tablet.cc:662] Decoding operations
0709 13:08:00.443599 (+   900us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:00.443708 (+   109us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:00.443743 (+    35us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:08:00.444362 (+   619us) tablet.cc:701] Row locks acquired
0709 13:08:00.444411 (+    49us) write_op.cc:260] PREPARE: finished
0709 13:08:00.444594 (+   183us) write_op.cc:270] Start()
0709 13:08:00.444755 (+   161us) write_op.cc:276] Timestamp: P: 1783602480444529 usec, L: 0
0709 13:08:00.444795 (+    40us) op_driver.cc:348] REPLICATION: starting
0709 13:08:00.445473 (+   678us) log.cc:844] Serialized 2459 byte log entry
0709 13:08:01.512552 (+1067079us) op_driver.cc:464] REPLICATION: finished
0709 13:08:01.787583 (+275031us) write_op.cc:301] APPLY: starting
0709 13:08:01.812843 (+ 25260us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:02.426651 (+613808us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:02.448879 (+ 22228us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:02.846585 (+397706us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:02.846909 (+   324us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=12,key_file_lookups=12,delta_file_lookups=12,mrs_lookups=0
0709 13:08:02.846954 (+    45us) write_op.cc:312] APPLY: finished
0709 13:08:02.847254 (+   300us) log.cc:844] Serialized 65 byte log entry
0709 13:08:02.945975 (+ 98721us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:02.967088 (+ 21113us) write_op.cc:454] Released schema lock
0709 13:08:02.968348 (+  1260us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:03.009780 (+ 41432us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:03.011342 (+  1562us) write_op.cc:454] Released schema lock
0709 13:08:03.321146 (+309804us) spinlock_profiling.cc:243] Waited 1.76 ms on lock 0x7b18003287e0. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e72bb3f3 00007f73e72bd7c1 00007f73e72bd779 00007f73e72bd709 00007f73e72bd6d1 00007f73e72bc9cd 000000000038f354 000000000038f199 00007f73e729e83a
Metrics: {"mutex_wait_us":2350757,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":200127,"apply.run_cpu_time_us":40218,"apply.run_wall_time_us":1240517,"cfile_cache_hit":26,"cfile_cache_hit_bytes":29267,"num_ops":6,"prepare.queue_time_us":453300,"prepare.run_cpu_time_us":25770,"prepare.run_wall_time_us":156593,"raft.queue_time_us":170207,"raft.run_cpu_time_us":14818,"raft.run_wall_time_us":167611,"replication_time_us":1044699,"spinlock_wait_cycles":4056960,"thread_start_us":539050,"threads_started":5,"wal-append.queue_time_us":391354,"wal-append.run_cpu_time_us":19660,"wal-append.run_wall_time_us":2810113}]]}
W20260709 13:08:05.737437 23511 log.cc:927] Time spent T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 1.363s	user 0.005s	sys 0.005s
I20260709 13:08:05.702490 23043 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:05.769966 16002 raft_consensus.cc:1240] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Rejecting Update request from peer eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:08:05.757279 23018 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:05.780560 23018 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:05.751477 23497 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:08:05.755659 23525 log.cc:927] Time spent T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:08:05.702085 23493 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:05.702620 23381 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:05.784282 23043 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:05.750764 23541 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:08:05.754662 23542 raft_consensus.cc:2749] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:05.755416 23356 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:05.699755 15759 catalog_manager.cc:5697] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 13 to 22, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 22 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:05.839098 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 23 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:05.838904 15759 catalog_manager.cc:5697] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 17 to 18, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 18 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:05.840297 16003 raft_consensus.cc:2410] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 23 because replica has last-logged OpId of term: 21 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 9.
I20260709 13:08:05.840989 15759 catalog_manager.cc:5697] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 13 to 20, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 20 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:08:05.780634 23522 consensus_meta.cc:287] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.590s	user 0.010s	sys 0.000s
I20260709 13:08:05.809943 16004 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:05.844420 16008 raft_consensus.cc:1240] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 16. Current term is 17. Ops: []
I20260709 13:08:05.802578 16562 raft_consensus.cc:1275] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:08:05.742586 22970 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:05.846887 16004 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:08:05.798607 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 143576us
I20260709 13:08:05.842438 15759 catalog_manager.cc:5697] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 14 to 16, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 16 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:05.848078 15759 catalog_manager.cc:5697] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 16 to 19. New cstate: current_term: 19 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:05.851162 15759 catalog_manager.cc:5697] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 13 to 18, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 18 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:05.852568 15759 catalog_manager.cc:5697] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 13 to 18, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 18 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:05.854092 15759 catalog_manager.cc:5697] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 6 to 19. New cstate: current_term: 19 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:08:05.856351 23540 log.cc:927] Time spent T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
W20260709 13:08:05.857128 23534 log.cc:927] Time spent T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 13:08:05.859508 23542 consensus_queue.cc:1048] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:08:05.863595 23377 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.988816 23377 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:05.990964 23377 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:05.867954 23544 raft_consensus.cc:2749] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:08:05.991874 23544 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:05.992365 23544 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:05.994628 23544 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:05.804145 23283 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:05.995581 23283 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:05.998652 23283 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:05.873937 23484 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:05.877171 23545 consensus_queue.cc:1048] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:08:05.897699 23247 raft_consensus.cc:993] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8: : Instructing follower 85e997ba5003439781ec4f1f8db0c3ee to start an election
I20260709 13:08:06.000129 23247 raft_consensus.cc:1081] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 25 LEADER]: Signalling peer 85e997ba5003439781ec4f1f8db0c3ee to start an election
I20260709 13:08:06.001075 22970 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:05.900560 23404 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:05.757031 23078 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:05.879022 23405 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:05.827312 23541 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:06.031951 23361 consensus_meta.cc:287] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.733s	user 0.010s	sys 0.000s
W20260709 13:08:06.041945 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 354622us
I20260709 13:08:05.849851 23522 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:08:06.044674 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 356309us
I20260709 13:08:06.037243 23541 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:08:06.057523 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 485529us
I20260709 13:08:06.058764 15974 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:06.031701 23297 consensus_queue.cc:1048] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.022s
I20260709 13:08:05.910281 15827 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:05.929631 23043 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:05.871507 23441 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:05.975934 23493 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:05.979820 23436 leader_election.cc:290] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:05.980247 16536 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:05.981384 23381 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:05.843873 23433 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:05.985374 23006 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:06.007052 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 120511us
W20260709 13:08:06.010511 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 103407us
I20260709 13:08:06.075260 16533 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:08:06.060070 15901 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.000s	sys 0.006s
W20260709 13:08:06.075984 15901 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.000s	sys 0.007s
I20260709 13:08:06.044135 23361 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:06.082778 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:08:06.061664 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 332413us
I20260709 13:08:06.064837 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:06.040588 23497 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:06.099953 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:06.061096 23288 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:08:05.927431 23406 consensus_meta.cc:287] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.554s	user 0.005s	sys 0.000s
I20260709 13:08:05.906347 15759 catalog_manager.cc:5697] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 10 to 16, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 16 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:06.081939 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:08:06.063457 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 153299us
I20260709 13:08:06.063694 15974 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:08:06.130075 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 66935.6us
I20260709 13:08:06.130811 15974 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:08:06.132529 23547 log.cc:927] Time spent T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:08:06.088295 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:06.060225 23553 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.045s
I20260709 13:08:06.089438 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:08:06.139894 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 52933.7us
I20260709 13:08:06.090186 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:06.090775 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:06.091567 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:06.078715 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:06.082202 16533 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:06.067026 23533 raft_consensus.cc:2804] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:06.098476 15900 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:06.098872 16002 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:06.100522 16010 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:08:06.105425 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:06.097301 23549 raft_consensus.cc:2749] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:06.106736 16566 raft_consensus.cc:1275] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:08:06.110328 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:06.112334 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:06.108237 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:08:06.086436 23327 consensus_meta.cc:287] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.719s	user 0.011s	sys 0.002s
I20260709 13:08:06.137274 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:06.163828 16009 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 18.
I20260709 13:08:06.164759 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:06.165820 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:06.166219 16010 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 18.
I20260709 13:08:06.166471 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:06.166754 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:06.167128 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:06.138124 15974 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:06.081542 15936 raft_consensus.cc:1275] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:08:06.143733 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:08:06.143055 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 44665.6us
I20260709 13:08:06.144392 16561 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 16.
I20260709 13:08:06.145248 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:06.169809 15900 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:06.142922 23331 raft_consensus.cc:2749] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:06.146453 23088 consensus_queue.cc:1059] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 13:08:06.154119 23361 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:06.172356 23078 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:08:06.159593 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 77445.5us
I20260709 13:08:06.173622 16533 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:06.174887 16005 raft_consensus.cc:1275] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 24 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:08:06.175246 23441 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:06.175940 23006 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:06.146152 23533 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:06.168449 23501 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.144016 23557 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:06.177991 23501 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:06.180075 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 23 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:06.180863 15933 raft_consensus.cc:2410] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 23 because replica has last-logged OpId of term: 21 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 9.
I20260709 13:08:06.181111 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:06.181699 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:06.182274 15934 raft_consensus.cc:1275] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 14. (term mismatch)
I20260709 13:08:06.182632 15934 pending_rounds.cc:85] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 13
I20260709 13:08:06.182829 15934 pending_rounds.cc:107] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 21.14
I20260709 13:08:06.183038 15934 raft_consensus.cc:2889] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:06.185158 15931 raft_consensus.cc:1275] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:08:06.181702 15932 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 18.
I20260709 13:08:06.137531 15759 catalog_manager.cc:5697] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 18 to 21, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 21 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:08:06.186142 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 48121.6us
I20260709 13:08:06.187207 15759 catalog_manager.cc:5697] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 16 to 20, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 20 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:06.167743 16002 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 17.
I20260709 13:08:06.189793 15759 catalog_manager.cc:5697] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 13 to 16, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 16 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:06.177454 23533 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:06.186311 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:06.192127 15759 catalog_manager.cc:5697] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 12 to 17, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 17 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:06.194113 15759 catalog_manager.cc:5697] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 10 to 16, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 16 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:06.196128 15759 catalog_manager.cc:5697] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 13 to 15, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 15 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:06.157212 23543 raft_consensus.cc:2749] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:08:06.156731 23405 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:06.210523 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:06.212117 16010 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e"
dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
 from {username='slave'} at 127.0.0.1:41674
I20260709 13:08:06.212517 16002 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:06.205669 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:06.208055 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:06.218890 16007 raft_consensus.cc:2368] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 16. Current term is 17.
I20260709 13:08:06.208528 16535 leader_election.cc:304] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:06.221916 16535 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:06.222913 23549 raft_consensus.cc:2804] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:06.224393 23550 raft_consensus.cc:2804] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:06.224977 23550 raft_consensus.cc:697] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:08:06.223419 23549 raft_consensus.cc:697] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:08:06.225910 23550 consensus_queue.cc:237] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:06.226100 23549 consensus_queue.cc:237] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:06.229857 15825 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:06.230993 15825 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:06.231244 23205 raft_consensus.cc:2749] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:06.231832 23206 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:06.232594 23501 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:06.234182 23088 raft_consensus.cc:3055] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:06.234692 23088 raft_consensus.cc:740] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:06.198311 23379 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.235870 23288 consensus_queue.cc:1059] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:08:06.220708 16002 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 23.
I20260709 13:08:06.236166 23379 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:06.238364 23288 raft_consensus.cc:3055] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:08:06.238956 23379 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:06.235505 15974 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:06.235532 23088 consensus_queue.cc:260] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:06.242558 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:06.233819 15825 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:06.243242 16567 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 20.
W20260709 13:08:06.244406 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 74655.5us
I20260709 13:08:06.244962 15825 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:06.245641 15900 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:06.246133 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:06.247007 15928 raft_consensus.cc:2393] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 22: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:08:06.163537 23297 raft_consensus.cc:2749] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:06.255841 15825 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:06.244609 16535 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:06.264012 16010 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 25 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:06.264499 16010 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:08:06.197530 15759 catalog_manager.cc:5697] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 13 to 21, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 21 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:06.278184 15976 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:08:06.280939 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 121382us
I20260709 13:08:06.282171 23541 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:06.284590 15827 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:08:06.286170 23541 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 25 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:08:06.289551 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 54150.3us
I20260709 13:08:06.291255 15974 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:06.243038 23088 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:06.161911 23090 consensus_queue.cc:1059] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.096s
I20260709 13:08:06.292974 23206 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:08:06.297842 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 53307.8us
I20260709 13:08:06.246799 23545 raft_consensus.cc:2749] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:08:06.307545 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 61955.9us
I20260709 13:08:06.306139 16010 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 26 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:06.238969 23288 raft_consensus.cc:740] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:08:06.343781 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 52605.3us
I20260709 13:08:06.344769 15974 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:06.315286 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:06.346277 16567 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:06.297624 23553 consensus_queue.cc:1048] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.055s
I20260709 13:08:06.293233 23551 raft_consensus.cc:2749] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:06.315632 23522 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 23 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:06.310446 23043 raft_consensus.cc:2804] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:08:06.308781 23429 consensus_queue.cc:1048] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.049s
I20260709 13:08:06.279734 23078 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:06.298727 23558 raft_consensus.cc:2749] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:06.289112 23557 raft_consensus.cc:697] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 18 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:06.366906 15974 leader_election.cc:304] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:06.370857 15974 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:06.314374 23018 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:08:06.341933 23366 consensus_meta.cc:287] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea: Time spent flushing consensus metadata: real 0.729s	user 0.009s	sys 0.002s
I20260709 13:08:06.315727 23331 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:08:06.340054 23406 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:06.339514 23543 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.367991 23372 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:06.364799 23327 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:06.372169 23392 raft_consensus.cc:2749] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:06.342504 23501 consensus_queue.cc:1048] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.063s
I20260709 13:08:06.393831 23379 raft_consensus.cc:2749] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:08:06.432377 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 126986us
I20260709 13:08:06.375749 23381 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:06.395810 23018 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:06.436348 16567 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 25.
I20260709 13:08:06.440567 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:06.444090 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a538795cdae403fae5941538575596b" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:06.394634 23545 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:06.426187 23372 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:06.404748 23366 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:08:06.448540 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 77782.2us
I20260709 13:08:06.472167 15974 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:08:06.450084 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:06.474025 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:06.450809 15854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:06.452381 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:06.456786 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:06.458129 15936 raft_consensus.cc:3060] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:08:06.420084 23078 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:08:06.443544 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 173018us
I20260709 13:08:06.459304 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:06.478065 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:06.478475 15928 raft_consensus.cc:3060] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:06.462105 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:06.470597 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:06.477253 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:06.477588 15861 raft_consensus.cc:1275] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:08:06.443295 23406 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.395408 23247 consensus_queue.cc:1048] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.066s
I20260709 13:08:06.483502 15825 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:06.487030 16011 raft_consensus.cc:1275] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:08:06.459735 23018 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:06.497179 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:06.511188 15937 raft_consensus.cc:3060] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:06.507063 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:06.512054 15856 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 22.
I20260709 13:08:06.425714 23543 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:06.513283 15976 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:06.470674 23372 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:06.550194 15930 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 19.
I20260709 13:08:06.531023 15928 raft_consensus.cc:2468] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 17.
I20260709 13:08:06.559315 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:06.559973 15861 raft_consensus.cc:3055] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:08:06.560045 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:06.560362 15861 raft_consensus.cc:740] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:08:06.567939 15936 raft_consensus.cc:2468] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 18.
I20260709 13:08:06.570827 15974 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:08:06.484563 23497 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:06.505376 23327 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:06.584569 23327 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:06.585004 23327 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:06.498198 23545 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.487798 23078 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:06.585825 23545 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:06.587059 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:06.587363 23078 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:06.460106 23381 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:06.581912 15937 raft_consensus.cc:2468] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 23.
I20260709 13:08:06.575201 16011 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 22.
I20260709 13:08:06.583698 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:06.561064 15861 consensus_queue.cc:260] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:08:06.580709 23555 log.cc:927] Time spent T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:08:06.589844 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:06.434149 23288 consensus_queue.cc:260] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:06.610522 15861 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:06.610414 16563 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 23.
I20260709 13:08:06.613873 15976 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:06.577010 23379 raft_consensus.cc:2804] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:08:06.617285 23319 raft_consensus.cc:2749] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:06.510327 23559 consensus_queue.cc:1048] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.102s
I20260709 13:08:06.510438 23406 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:06.588029 23545 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:08:06.626600 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 143232us
I20260709 13:08:06.633034 15827 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:08:06.677731 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:06.680997 16563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:06.684840 15861 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:08:06.679880 15827 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:06.693344 15974 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:08:06.635341 23541 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.542766 23557 consensus_queue.cc:237] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:06.665395 23558 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.628170 23508 consensus_queue.cc:1048] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.063s
I20260709 13:08:06.665242 23545 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:08:06.668771 23580 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:06.744472 23578 raft_consensus.cc:2804] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:08:06.755909 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 76102.7us
I20260709 13:08:06.715101 23545 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:06.738701 23580 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:06.745745 23379 raft_consensus.cc:697] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 25 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:06.709239 23541 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:06.715957 23558 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:06.778412 23247 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:06.806859 23541 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:06.757561 23422 consensus_queue.cc:1048] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.023s
W20260709 13:08:06.776358 23565 log.cc:927] Time spent T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.260s	user 0.000s	sys 0.001s
I20260709 13:08:06.810516 23379 consensus_queue.cc:237] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:06.767570 23545 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:06.806730 15858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:06.761628 23578 raft_consensus.cc:697] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:06.766372 23587 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:08:06.769589 23580 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:06.822934 15858 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 22.
W20260709 13:08:06.827401 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 70744.1us
W20260709 13:08:06.826524 23547 log.cc:927] Time spent T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:08:06.834544 23578 consensus_queue.cc:237] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:06.839248 23584 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.839826 23584 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:06.841897 23584 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:06.843794 23587 raft_consensus.cc:697] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 22 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:08:06.844686 23587 consensus_queue.cc:237] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:06.846575 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:06.846909 16563 raft_consensus.cc:1275] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:08:06.848026 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:06.873123 23543 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:06.873783 23381 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:08:06.874662 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 199916us
I20260709 13:08:06.885901 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:08:06.891372 23548 log.cc:927] Time spent T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.841s	user 0.007s	sys 0.000s
I20260709 13:08:06.896677 23406 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:06.897349 23018 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:06.900352 16010 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 26 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:06.901197 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: true dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:06.901839 15861 raft_consensus.cc:3055] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:08:06.884639 23457 consensus_queue.cc:1048] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 13:08:06.902175 15861 raft_consensus.cc:740] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:08:06.902954 15861 consensus_queue.cc:260] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:06.903889 15861 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:08:06.904657 23564 log.cc:927] Time spent T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.529s	user 0.002s	sys 0.000s
I20260709 13:08:06.911643 15861 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 26.
I20260709 13:08:06.913467 15974 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:08:06.914268 23594 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:08:06.914749 23594 raft_consensus.cc:697] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 26 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:06.915560 23594 consensus_queue.cc:237] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:06.918426 23558 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:06.895191 23372 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:08:06.937367 23088 consensus_meta.cc:287] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.632s	user 0.010s	sys 0.000s
I20260709 13:08:06.937791 23372 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:06.942975 23580 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:08:06.946969 23566 log.cc:927] Time spent T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.517s	user 0.001s	sys 0.000s
W20260709 13:08:06.948340 23570 log.cc:927] Time spent T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.479s	user 0.002s	sys 0.000s
W20260709 13:08:06.950860 23569 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
W20260709 13:08:06.951206 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 63039.2us
I20260709 13:08:06.960321 15825 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:06.907518 23327 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:06.963819 23605 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:06.966161 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:06.958787 23594 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.969350 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:06.972993 23327 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:06.977702 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:06.980758 15934 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:06.981766 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
W20260709 13:08:06.983444 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 678524us
I20260709 13:08:06.991972 15853 raft_consensus.cc:2393] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 22: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:08:06.992655 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 649472us
I20260709 13:08:06.996150 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:06.984486 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:08:07.001936 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:06.981024 23533 consensus_meta.cc:287] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.734s	user 0.010s	sys 0.001s
I20260709 13:08:07.012604 15859 raft_consensus.cc:2393] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 22: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:08:07.014894 23550 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.980192 23605 raft_consensus.cc:697] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 19 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:08:06.999657 23043 raft_consensus.cc:697] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:08:07.021222 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: true dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:06.978207 23544 raft_consensus.cc:493] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:08:07.032299 16004 raft_consensus.cc:1275] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:08:07.038750 15934 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 19.
I20260709 13:08:07.041846 16570 raft_consensus.cc:1240] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:07.044714 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:07.045305 16564 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:08:07.046216 15976 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:07.047231 16533 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:08:07.045254 23603 log.cc:927] Time spent T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:08:07.015445 23550 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:07.046200 23205 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.984071 23594 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:07.067809 16564 raft_consensus.cc:2468] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 21.
I20260709 13:08:07.068892 15976 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:07.069720 23372 raft_consensus.cc:2804] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:07.070258 23372 raft_consensus.cc:697] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 21 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:07.074304 15902 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:07.030458 23533 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:07.076887 23018 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:07.077462 23594 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:07.078344 23558 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:08:07.078552 23545 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:07.079896 15856 raft_consensus.cc:1240] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 19. Current term is 20. Ops: [19.17-19.17]
I20260709 13:08:07.080166 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:07.080787 15861 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 21.
I20260709 13:08:07.081460 23205 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:08:07.084192 23599 log.cc:927] Time spent T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:08:07.084044 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:07.084715 16564 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 22.
I20260709 13:08:07.085740 15976 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:07.052084 23297 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:07.017072 23605 consensus_queue.cc:237] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:07.075493 15902 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:07.043301 23544 raft_consensus.cc:515] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:07.086835 23381 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:08:07.087203 23580 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 23 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:07.083595 23205 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:07.093215 23406 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:07.102664 23406 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:07.099244 23356 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:08:07.105134 23406 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:07.093708 23543 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:07.105449 23356 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:07.105912 23543 raft_consensus.cc:697] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 20 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:08:07.106789 23543 consensus_queue.cc:237] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:07.094154 23331 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:07.078950 23558 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:07.097894 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:07.063644 23587 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:07.109290 15902 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:08:07.109508 15974 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.103.2:43009 timed out after 1.708s (SENT) [suppressed 3 similar messages]
I20260709 13:08:07.110637 15827 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:07.065598 23569 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:08:07.125965 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:07.111604 23331 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:07.126884 23576 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:07.124457 23542 raft_consensus.cc:2749] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:08:07.127292 23608 log.cc:927] Time spent T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:08:07.128583 15902 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:07.134090 15902 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:07.132592 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:07.129354 15855 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Advancing to term 24
I20260709 13:08:07.107489 23356 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:07.129716 23331 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:07.071208 23372 consensus_queue.cc:237] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:08:07.144091 23600 log.cc:927] Time spent T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
I20260709 13:08:07.145778 23533 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:07.146970 15902 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:07.148159 15902 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:07.149752 23331 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:07.152768 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:07.153337 16566 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:07.154986 16569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45246 (request call id 1625) took 1012 ms. Trace:
I20260709 13:08:07.156141 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:07.157557 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:07.158017 16568 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45244 (request call id 1755) took 1047 ms. Trace:
I20260709 13:08:07.159657 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:08:07.161058 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 113907us
I20260709 13:08:07.108657 23297 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:07.162276 16533 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:07.162402 23123 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:07.163170 16533 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:07.163647 23297 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:07.138407 23288 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:07.164609 15861 pending_rounds.cc:85] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8: Aborting all ops after (but not including) 16
I20260709 13:08:07.164919 15861 pending_rounds.cc:107] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8: Aborting uncommitted NO_OP operation due to leader change: 23.17
I20260709 13:08:07.165140 15861 raft_consensus.cc:2889] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:07.169405 23381 raft_consensus.cc:2749] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:07.169667 23553 raft_consensus.cc:2749] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:07.170130 23381 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:07.132467 23406 raft_consensus.cc:2749] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:07.131157 23018 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:07.132887 23576 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:07.174768 16566 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 23.
I20260709 13:08:07.174932 23558 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:07.131194 23569 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:07.130831 23587 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:07.143754 23434 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:07.178473 15855 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 24.
W20260709 13:08:07.177598 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 47264.6us
W20260709 13:08:07.162004 23567 log.cc:927] Time spent T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.657s	user 0.002s	sys 0.002s
I20260709 13:08:07.163842 23497 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:07.067036 23577 consensus_queue.cc:1059] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
W20260709 13:08:07.140883 23607 log.cc:927] Time spent T 00000000000000000000000000000000 P d33cebb189b14a989836214fdbafb898: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
W20260709 13:08:07.177527 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 125533us
I20260709 13:08:07.174993 15902 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:07.176036 15976 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:07.157671 23549 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:07.202548 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:07.196501 23450 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:07.158422 16568 rpcz_store.cc:276] 0709 13:08:06.110730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.110917 (+   187us) service_pool.cc:224] Handling call
0709 13:08:06.111412 (+   495us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:08:06.116526 (+  5114us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:08:06.429146 (+312620us) log.cc:844] Serialized 14 byte log entry
0709 13:08:07.011636 (+582490us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:08:07.012278 (+   642us) log.cc:844] Serialized 78 byte log entry
0709 13:08:07.012494 (+   216us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:08:07.021211 (+  8717us) log.cc:844] Serialized 14 byte log entry
0709 13:08:07.044935 (+ 23724us) log.cc:844] Serialized 14 byte log entry
0709 13:08:07.157457 (+112522us) spinlock_profiling.cc:243] Waited 4.01 ms on lock 0x7b18004c3900. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73f0183de1 00007f73f01905b1 00007f73e7235c0a 00007f73f0975219 00007f73e72bda40 00007f73e72b84e4 00007f73e72b8455 00007f73ea3a0060 00007f73ea3b2716 00007f73ea3c3287 00007f73ea3c3208
0709 13:08:07.157624 (+   167us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:07.157786 (+   162us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:07.157824 (+    38us) raft_consensus.cc:1685] finished
0709 13:08:07.157836 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:07.157970 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":424615,"prepare.queue_time_us":275586,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":5895,"raft.queue_time_us":434877,"raft.run_cpu_time_us":301,"raft.run_wall_time_us":5956,"spinlock_wait_cycles":15638528,"thread_start_us":421876,"threads_started":2,"wal-append.queue_time_us":242033}
I20260709 13:08:07.190899 15825 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:08:07.193580 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:07.155344 16569 rpcz_store.cc:276] 0709 13:08:06.142303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.142684 (+   381us) service_pool.cc:224] Handling call
0709 13:08:06.143011 (+   327us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:06.147507 (+  4496us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:08:06.430583 (+283076us) log.cc:844] Serialized 14 byte log entry
0709 13:08:07.011119 (+580536us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:07.030843 (+ 19724us) log.cc:844] Serialized 52 byte log entry
0709 13:08:07.031013 (+   170us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:08:07.031044 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:07.031106 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:07.154773 (+123667us) raft_consensus.cc:1685] finished
0709 13:08:07.154791 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:07.154932 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":484680,"prepare.queue_time_us":328790,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":407875,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":6704,"thread_start_us":266092,"threads_started":2,"wal-append.queue_time_us":361225}
I20260709 13:08:07.194813 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:07.176625 23576 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:07.197924 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:07.200327 16562 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45232 (request call id 1736) took 1130 ms. Trace:
I20260709 13:08:07.213016 16562 rpcz_store.cc:276] 0709 13:08:06.069929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.070145 (+   216us) service_pool.cc:224] Handling call
0709 13:08:06.070466 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:06.075484 (+  5018us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:06.075530 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:06.088875 (+ 13345us) log.cc:844] Serialized 26 byte log entry
0709 13:08:06.296085 (+207210us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:08:06.296122 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:06.296183 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:07.161463 (+865280us) spinlock_profiling.cc:243] Waited 61.6 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:08:07.200034 (+ 38571us) raft_consensus.cc:1685] finished
0709 13:08:07.200056 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:07.200247 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":590,"raft.queue_time_us":8660,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":5849,"spinlock_wait_cycles":141575552,"thread_start_us":153881,"threads_started":1,"wal-append.queue_time_us":309148}
I20260709 13:08:07.158265 15862 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 22.
I20260709 13:08:07.206154 16011 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:07.213017 15859 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 18.
I20260709 13:08:07.197527 16533 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:07.207638 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:07.216637 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:07.217181 16562 raft_consensus.cc:3060] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:07.204418 15902 leader_election.cc:400] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 election: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 16. Current term is 17.
I20260709 13:08:07.209275 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:07.218765 15902 leader_election.cc:403] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:08:07.202692 23569 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:07.221499 16565 raft_consensus.cc:2410] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 19 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:08:07.177196 23331 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 23 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:07.225133 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:08:07.226389 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 116915us
I20260709 13:08:07.226569 15934 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 24.
I20260709 13:08:07.190397 23558 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:07.233950 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:07.099900 23043 consensus_queue.cc:237] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:07.239442 16011 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 19.
I20260709 13:08:07.254788 15935 raft_consensus.cc:1240] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 13:08:07.252089 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:07.236107 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:07.236886 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c138dd864152446f83386a4cd43c6efc" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:07.282961 15936 raft_consensus.cc:3060] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:07.283069 15932 raft_consensus.cc:3060] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:07.236765 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:07.236444 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:07.285616 15930 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 22.
I20260709 13:08:07.235270 15974 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:08:07.235832 15929 raft_consensus.cc:1240] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:07.203747 23434 raft_consensus.cc:2749] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:07.288507 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:07.289326 15928 raft_consensus.cc:2393] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 22: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:08:07.300527 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:07.304796 16562 raft_consensus.cc:2468] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 24.
I20260709 13:08:07.203859 23006 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:07.297557 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:07.335879 15859 raft_consensus.cc:3060] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:08:07.210139 23545 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 24 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:07.336979 16570 raft_consensus.cc:1240] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 20. Current term is 21. Ops: [20.17-20.17]
I20260709 13:08:07.338112 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:07.227644 23406 raft_consensus.cc:2749] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 16. Current term is 17.
I20260709 13:08:07.306994 15827 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:08:07.351001 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 44087.7us
I20260709 13:08:07.227870 23580 raft_consensus.cc:2804] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:08:07.308041 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:07.310415 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:07.317135 15936 raft_consensus.cc:2468] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:08:07.226970 23436 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:08:07.291060 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 58076.3us
I20260709 13:08:07.359388 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:07.339491 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:07.360002 15900 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:08:07.366155 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:07.366855 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:07.367102 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:07.290095 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d68c78f0d94e0eb90543b642d1f1af" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:07.367605 16565 raft_consensus.cc:2368] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 22. Current term is 23.
I20260709 13:08:07.367733 16561 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 22.
I20260709 13:08:07.341723 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:07.332302 23434 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:07.343585 16009 raft_consensus.cc:1240] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 21 LEADER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 20. Current term is 21. Ops: [20.17-20.17]
I20260709 13:08:07.343444 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:07.343991 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:07.372453 16006 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 22.
I20260709 13:08:07.373350 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:07.345695 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:07.203155 23549 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:07.220467 23497 consensus_queue.cc:1048] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.087s
I20260709 13:08:07.345258 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:07.224032 23573 consensus_queue.cc:1048] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.110s
I20260709 13:08:07.295578 23577 raft_consensus.cc:2804] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:08:07.354100 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:08:07.342170 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 114931us
I20260709 13:08:07.368700 15933 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:07.295564 23558 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:07.247399 23533 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:07.364814 15900 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:08:07.380985 23559 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:07.381413 23559 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:07.381775 23559 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:07.382511 15928 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 22.
I20260709 13:08:07.383035 23543 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:07.344626 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecbdcd9cd6c944e7a9277f7c8f191e96" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:08:07.384871 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 163693us
I20260709 13:08:07.341647 15932 raft_consensus.cc:2468] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 25.
I20260709 13:08:07.347131 23006 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:07.371467 23580 raft_consensus.cc:697] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:07.397049 15933 raft_consensus.cc:2468] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 23.
I20260709 13:08:07.399148 15928 raft_consensus.cc:1275] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:08:07.401243 15974 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:07.398532 15902 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:07.392942 15932 raft_consensus.cc:1275] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:08:07.394392 15934 raft_consensus.cc:1275] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 13:08:07.384867 23610 log.cc:927] Time spent T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.289s	user 0.000s	sys 0.002s
I20260709 13:08:07.370617 23406 consensus_queue.cc:1059] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 18.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.020s
I20260709 13:08:07.408509 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:07.409179 15862 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 21.
I20260709 13:08:07.411962 23543 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:07.412420 23543 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:07.409319 23557 raft_consensus.cc:2749] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:07.403618 23366 raft_consensus.cc:2749] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:08:07.425699 23615 log.cc:927] Time spent T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.221s	user 0.002s	sys 0.000s
I20260709 13:08:07.427645 16002 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 22.
I20260709 13:08:07.428465 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:07.429757 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:07.430538 23006 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:07.378355 23545 consensus_queue.cc:1059] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 13:08:07.403496 15902 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:07.391511 23533 raft_consensus.cc:697] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 19 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:08:07.377686 23434 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:07.391973 23577 raft_consensus.cc:697] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 24 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:08:07.457454 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 54033.5us
I20260709 13:08:07.452979 15859 raft_consensus.cc:2468] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 23.
I20260709 13:08:07.464695 16533 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:07.466657 15902 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:07.402645 23544 leader_election.cc:290] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:07.465621 23405 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:07.471402 23623 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:08:07.473596 15827 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:07.479821 15932 raft_consensus.cc:1275] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 25 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:08:07.401356 23542 consensus_queue.cc:1048] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.076s
I20260709 13:08:07.488730 15900 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:07.491928 15827 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:07.498477 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:07.499022 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 25 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:07.402721 23594 consensus_queue.cc:1059] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.077s
I20260709 13:08:07.509605 15827 leader_election.cc:400] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 22. Current term is 23.
I20260709 13:08:07.510087 15827 leader_election.cc:403] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:07.481057 23406 raft_consensus.cc:2749] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:07.486847 23078 raft_consensus.cc:2749] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:08:07.500106 23618 log.cc:927] Time spent T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 13:08:07.511624 23006 consensus_queue.cc:1048] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.016s
I20260709 13:08:07.523479 23043 consensus_queue.cc:1048] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.005s
I20260709 13:08:07.535835 23577 consensus_queue.cc:237] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:07.518034 23429 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:07.567837 23429 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting leader election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:08:07.568322 23429 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:08:07.572274 23624 log.cc:927] Time spent T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:08:07.487013 23381 consensus_queue.cc:1059] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.063s
W20260709 13:08:07.543427 23629 log.cc:927] Time spent T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:08:07.565303 23543 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:08:07.573829 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 77270.4us
I20260709 13:08:07.559183 16009 raft_consensus.cc:1240] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 22. Current term is 23. Ops: []
W20260709 13:08:07.574833 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 61941.5us
I20260709 13:08:07.512499 23569 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 22. Current term is 23.
I20260709 13:08:07.590402 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 25 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:07.590530 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:07.600513 23434 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:07.612156 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e422bf1c2eb46fcb2fe33e4eb00f888" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:07.610003 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:07.613322 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:07.615052 16009 raft_consensus.cc:1275] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:08:07.498399 23580 consensus_queue.cc:237] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:07.587486 23406 raft_consensus.cc:3055] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:08:07.617733 16569 raft_consensus.cc:1275] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 25 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:08:07.623111 23406 raft_consensus.cc:740] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:08:07.625919 16561 raft_consensus.cc:1240] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:07.632607 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:07.548386 23533 consensus_queue.cc:237] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:07.647964 15902 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:07.626459 23549 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:07.661053 23406 consensus_queue.cc:260] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.17, Last appended by leader: 17, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:07.645838 23558 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:07.661990 23434 raft_consensus.cc:2749] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:08:07.658615 23331 consensus_queue.cc:1059] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:08:07.657953 23594 consensus_queue.cc:1048] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.033s
W20260709 13:08:07.759117 23450 consensus_meta.cc:287] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent flushing consensus metadata: real 0.554s	user 0.006s	sys 0.004s
I20260709 13:08:07.750017 16533 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
W20260709 13:08:07.781452 15900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:07.816336 23627 log.cc:927] Time spent T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:08:07.816926 23601 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:07.817121 23587 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:07.820119 23550 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:07.821738 23559 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:07.822767 15932 raft_consensus.cc:1240] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 18. Current term is 19. Ops: []
I20260709 13:08:07.823459 23429 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:07.825796 23429 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:07.826341 23543 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 23 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:07.827634 23406 raft_consensus.cc:3060] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:08:07.805049 23567 log.cc:927] Time spent T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.150s	user 0.000s	sys 0.003s
W20260709 13:08:07.805348 23630 log.cc:927] Time spent T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 13:08:07.833782 16568 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:08:07.809743 23450 raft_consensus.cc:2749] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:07.842638 15860 raft_consensus.cc:1275] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:08:07.842903 15862 raft_consensus.cc:1275] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:08:07.843235 23288 raft_consensus.cc:3055] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:07.844417 15859 raft_consensus.cc:1275] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 26 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 13:08:07.846448 16568 raft_consensus.cc:1275] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 26 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 13:08:07.865062 15974 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
W20260709 13:08:07.868765 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 220920us
I20260709 13:08:07.871487 15902 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:07.883807 15902 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:07.843608 23288 raft_consensus.cc:740] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:07.845340 23559 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:07.890193 15902 leader_election.cc:304] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:07.848493 23489 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:07.893903 15902 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:07.880652 23405 raft_consensus.cc:2749] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:07.880873 23392 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:07.901574 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:07.895587 23543 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:07.899787 23631 raft_consensus.cc:2749] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:07.904549 23580 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:08:07.916056 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 166118us
I20260709 13:08:07.917943 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:07.918521 15931 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 22.
I20260709 13:08:07.920670 16561 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 22.
I20260709 13:08:07.920969 16568 raft_consensus.cc:1275] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 15. (term mismatch)
I20260709 13:08:07.921483 16568 pending_rounds.cc:85] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5: Aborting all ops after (but not including) 14
I20260709 13:08:07.921762 16568 pending_rounds.cc:107] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 13:08:07.922049 16568 raft_consensus.cc:2889] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:07.902308 23489 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:07.908892 23522 raft_consensus.cc:2749] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:07.923995 23549 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:07.905648 23392 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:07.933712 16533 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:07.937811 16533 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:07.925212 23580 raft_consensus.cc:697] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:07.931609 23549 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:07.944183 23392 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:07.940236 23634 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:07.914354 23288 consensus_queue.cc:260] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:07.942294 23497 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:08:07.951264 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 86297.6us
I20260709 13:08:07.950879 23549 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:07.948793 23580 consensus_queue.cc:237] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:08:08.049178 15977 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:07.959501 15974 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:08:08.052644 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:08.053893 15862 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:07.975219 23288 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:08.066725 16570 raft_consensus.cc:1275] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:08:08.067564 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:08.052414 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:08.029102 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:08.040884 15928 raft_consensus.cc:1275] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:08:08.075336 15936 raft_consensus.cc:2393] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 19: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:08:08.076166 15855 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:08.027405 15827 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:08.122817 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:08.123937 15928 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 19.
I20260709 13:08:08.126238 23553 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:08.078723 23289 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.117959 23379 raft_consensus.cc:2804] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:08:08.132839 23379 raft_consensus.cc:697] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:08.134389 23379 consensus_queue.cc:237] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:07.965873 16002 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41674 (request call id 2966) took 1119 ms. Trace:
W20260709 13:08:08.055353 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 95899.2us
I20260709 13:08:08.136337 16002 rpcz_store.cc:276] 0709 13:08:06.846083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.846213 (+   130us) service_pool.cc:224] Handling call
0709 13:08:07.965707 (+1119494us) spinlock_profiling.cc:243] Waited 580 ms on lock 0x7b5400259ed4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea311571 00007f73ea3ac2fa 00007f73ef36db17 00007f73e88e86d4 00007f73e88e8662 00007f73e88e8591 00007f73e88e850c 00007f73e88e77c2 00007f73e86e348c 00007f73e86e28c6 00007f73e86e226f
0709 13:08:07.965818 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1334505728}
I20260709 13:08:07.964685 23342 raft_consensus.cc:3055] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:08:08.140131 23342 raft_consensus.cc:740] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:08.140072 23327 consensus_queue.cc:1048] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:08:08.141906 23342 consensus_queue.cc:260] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:08.142421 23327 consensus_queue.cc:1048] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:08:08.143249 23342 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:08.148093 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:08.148598 23650 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:08:08.148880 23392 raft_consensus.cc:493] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.149410 23650 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:07.960409 23633 consensus_queue.cc:1048] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.188s
W20260709 13:08:08.151968 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 124998us
I20260709 13:08:08.149516 23392 raft_consensus.cc:515] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:08.167920 23585 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:08:08.168853 23631 consensus_queue.cc:1048] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:08:07.981523 23549 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:07.970036 23366 raft_consensus.cc:3055] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:08.174031 16010 raft_consensus.cc:1275] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:08:07.964485 23557 raft_consensus.cc:3055] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:08:08.129750 23289 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:08.180265 15929 raft_consensus.cc:1240] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 22. Current term is 23. Ops: []
I20260709 13:08:08.180562 23580 consensus_queue.cc:1048] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:08:08.184365 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:08.185297 16004 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 20.
I20260709 13:08:08.184406 23377 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:08.186868 23377 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:08.187893 23557 raft_consensus.cc:740] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:08.189560 23377 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:08.189283 23557 consensus_queue.cc:260] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:08.190478 23557 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:08.191808 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:08.192564 16561 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 20.
I20260709 13:08:08.194365 23580 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:08.196946 23289 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:08.197575 15902 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:08.199786 23406 raft_consensus.cc:2749] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:08.175534 15862 raft_consensus.cc:2468] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 23.
I20260709 13:08:08.196959 23392 leader_election.cc:290] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:07.982822 23489 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:08.195214 23580 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:08.175552 23585 raft_consensus.cc:697] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 22 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:08:08.208739 23650 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:08.224246 23549 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:08.231741 23078 consensus_queue.cc:1059] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:08:08.233096 23018 raft_consensus.cc:3055] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:08:08.233603 23018 raft_consensus.cc:740] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:08:08.235314 23585 consensus_queue.cc:237] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:08.238652 23366 raft_consensus.cc:740] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:08:08.234967 23018 consensus_queue.cc:260] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:08.232088 15825 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:08.245602 23356 raft_consensus.cc:2749] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:08:08.246644 23018 raft_consensus.cc:3060] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:08.252209 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:08.254340 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:08.255295 15855 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 19.
I20260709 13:08:08.257462 23392 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 13:08:08.248124 15827 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:08.277326 15862 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 26.
I20260709 13:08:08.280103 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:08.282722 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:08.296813 23543 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:08.297549 23543 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:08:08.304642 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 56729.9us
I20260709 13:08:08.221527 23327 consensus_queue.cc:1048] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.071s
I20260709 13:08:08.332618 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:08.335835 23392 consensus_queue.cc:1059] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:08.340004 23489 consensus_queue.cc:1048] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:08:08.347450 23289 raft_consensus.cc:3055] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:08:08.348172 23289 raft_consensus.cc:740] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:08.348866 23601 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:08.349503 23601 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:08.349313 23289 consensus_queue.cc:260] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:08.349974 23601 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:08.358991 15928 raft_consensus.cc:1275] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:08:08.364217 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:08.365234 23366 consensus_queue.cc:260] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 19.17, Last appended by leader: 17, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:08.367066 23366 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:08.372465 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:08.377584 15932 raft_consensus.cc:2393] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 22: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:08:08.379832 23553 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.380705 23553 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:08.394676 23580 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:08.395897 15974 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:08:08.396442 23289 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:08:08.397455 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 69898.8us
I20260709 13:08:08.383576 23553 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:08.405673 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:08.406498 16006 raft_consensus.cc:1275] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:08:08.416024 16002 raft_consensus.cc:2410] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 13:08:08.416121 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:08.417062 15932 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:08.420918 23327 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:08.421526 23327 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:08.421967 23327 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:08:08.451568 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 121139us
I20260709 13:08:08.467712 15862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:08.476405 15936 raft_consensus.cc:1240] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 13:08:08.482801 15859 raft_consensus.cc:1275] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:08:08.464632 23543 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:08.488440 23580 raft_consensus.cc:493] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.489380 23580 raft_consensus.cc:515] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:08.490198 15976 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:08.491294 23288 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:08:08.492609 23577 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:08:08.493028 15827 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:08.493458 23557 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:08.493883 23557 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:08.494266 23557 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:08.496495 23573 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:08.498148 15930 raft_consensus.cc:1275] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:08:08.498978 16561 raft_consensus.cc:1240] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 15. Current term is 16. Ops: []
I20260709 13:08:08.500241 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:08.501138 15935 raft_consensus.cc:2393] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 23: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:08:08.503011 16533 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:08.505133 15900 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:08:08.506392 15900 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:08:08.506953 23123 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:08:08.513727 16006 raft_consensus.cc:1275] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:08:08.515237 23559 raft_consensus.cc:2804] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:08:08.515928 23366 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:08:08.516610 23366 raft_consensus.cc:697] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 19 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:08:08.517694 23366 consensus_queue.cc:237] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:08.518821 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:08.522625 23392 consensus_queue.cc:1048] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:08:08.523905 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:08.524586 16010 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 23.
I20260709 13:08:08.527009 15937 raft_consensus.cc:1275] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:08:08.528815 23206 consensus_queue.cc:1048] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:08:08.531343 23559 raft_consensus.cc:697] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 23 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:08:08.533223 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:08.533519 16009 raft_consensus.cc:1275] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:08:08.533874 15933 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 19.
I20260709 13:08:08.534780 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:08.537761 23559 consensus_queue.cc:237] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:08.538990 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:08.540558 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:08.542052 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:08.507460 23123 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:08.543256 23123 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:08:08.546303 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:08.559132 15932 raft_consensus.cc:2468] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 23.
I20260709 13:08:08.560914 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50c1e8d433a407aa9fae9cb9866f184" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:08.564174 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:08.566138 23297 consensus_queue.cc:1048] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:08:08.567536 15936 raft_consensus.cc:1275] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 21 index: 17. (term mismatch)
I20260709 13:08:08.568009 15936 pending_rounds.cc:85] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea: Aborting all ops after (but not including) 16
I20260709 13:08:08.568251 15936 pending_rounds.cc:107] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea: Aborting uncommitted NO_OP operation due to leader change: 20.17
I20260709 13:08:08.568522 15936 raft_consensus.cc:2889] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:08:08.519480 23657 log.cc:927] Time spent T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:08:08.566324 16535 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:08.572731 16533 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:08:08.575428 15859 raft_consensus.cc:1240] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 22. Current term is 23. Ops: []
I20260709 13:08:08.571457 23580 leader_election.cc:290] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:08.595821 15827 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:08.594516 23576 consensus_queue.cc:1048] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:08:08.608958 15974 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:08:08.610890 23288 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:08.611627 15974 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:08.611442 23288 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:08.615150 15900 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:08:08.628226 15975 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:08.628604 23489 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:08.629031 23489 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:08.629451 23489 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:08:08.632064 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 66004.6us
I20260709 13:08:08.617359 23288 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:08.634370 23650 raft_consensus.cc:2749] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:08.605854 23577 consensus_queue.cc:1048] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 13:08:08.642421 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:08:08.644531 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 57094us
I20260709 13:08:08.642428 23550 raft_consensus.cc:2804] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:08:08.652473 23550 raft_consensus.cc:697] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:08:08.653599 23550 consensus_queue.cc:237] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:08:08.671468 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 52299.4us
I20260709 13:08:08.672590 15976 leader_election.cc:304] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:08.618377 23356 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:08.683449 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b7d73618ae489990e21e8cc312177d" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:08.670218 23553 consensus_queue.cc:1048] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.028s
I20260709 13:08:08.600208 23573 consensus_queue.cc:1048] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.070s
I20260709 13:08:08.689594 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:08.690291 16004 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 19.
I20260709 13:08:08.612049 23331 consensus_queue.cc:1048] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.066s
I20260709 13:08:08.691823 23649 raft_consensus.cc:2749] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:08.625937 23631 raft_consensus.cc:2749] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:08.702906 23551 consensus_queue.cc:1048] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 13:08:08.681389 23356 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:08.704203 23356 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:08.705689 16568 raft_consensus.cc:1275] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:08:08.735353 23557 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:08.737740 23557 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:08.743615 15859 raft_consensus.cc:1275] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:08:08.754446 23356 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:08.758119 23356 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 23 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:08.743037 23489 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:08.759617 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:08.791392 15928 raft_consensus.cc:2393] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 23: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:08:08.763316 23601 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:08.768105 23327 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:08.723343 23288 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:08.795957 16008 raft_consensus.cc:1240] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:08.775393 15751 catalog_manager.cc:5697] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 8 to 15, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 15 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:08.795583 23288 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:08.793586 23327 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:08.800550 23123 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:08.849514 15902 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:08.869094 16562 raft_consensus.cc:1240] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 27 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 26. Current term is 27. Ops: [26.19-26.19]
I20260709 13:08:08.852759 23543 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:08.875576 15932 raft_consensus.cc:1275] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:08:08.871784 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:08.880770 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:08.881861 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:08.882855 16568 raft_consensus.cc:1275] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:08:08.877197 23543 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:08.899704 16565 raft_consensus.cc:1240] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 20. Current term is 21. Ops: []
I20260709 13:08:08.896267 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:08.878050 23553 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:08.904814 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:08.905349 15936 raft_consensus.cc:1240] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 23 LEADER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 22. Current term is 23. Ops: []
I20260709 13:08:08.871948 23288 consensus_queue.cc:1059] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.019s
W20260709 13:08:08.916952 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 283377us
I20260709 13:08:08.901592 23543 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:08.974516 16569 raft_consensus.cc:1240] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 LEADER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 22. Current term is 23. Ops: []
I20260709 13:08:08.889196 23557 consensus_queue.cc:1048] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:08:08.921452 15976 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:08.988502 16008 raft_consensus.cc:1240] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
W20260709 13:08:08.996191 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 78559.3us
W20260709 13:08:09.024663 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 103337us
I20260709 13:08:08.986421 23489 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:09.021801 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:09.036266 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:09.045578 15976 leader_election.cc:304] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:09.029865 23288 raft_consensus.cc:2749] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:09.009672 23553 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:09.059821 23682 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:09.044337 23678 consensus_queue.cc:1048] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.181s
I20260709 13:08:09.065521 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:08:09.075929 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 94762.2us
I20260709 13:08:09.056965 23123 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 27 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:09.094838 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:09.111447 23649 consensus_queue.cc:1048] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:08:09.113890 23649 consensus_queue.cc:1059] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:08:09.115149 23649 raft_consensus.cc:3055] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:08:09.109786 15976 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:09.117827 23679 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:08:09.115599 23649 raft_consensus.cc:740] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:09.127592 23649 consensus_queue.cc:260] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:09.135960 16007 raft_consensus.cc:1240] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 20. Current term is 21. Ops: [20.15-20.15]
W20260709 13:08:09.153127 23666 log.cc:927] Time spent T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.156s	user 0.001s	sys 0.001s
W20260709 13:08:09.153644 23681 log.cc:927] Time spent T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
W20260709 13:08:09.156559 23675 log.cc:927] Time spent T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:08:09.157280 23553 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:08:09.158257 23670 log.cc:927] Time spent T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
I20260709 13:08:09.157377 23543 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:09.094348 23682 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:08:09.164336 23671 log.cc:927] Time spent T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.194s	user 0.001s	sys 0.001s
W20260709 13:08:09.173964 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 88269.8us
I20260709 13:08:09.179553 16005 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 23.
I20260709 13:08:09.105290 23392 consensus_queue.cc:1048] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.074s
I20260709 13:08:09.188694 23156 consensus_queue.cc:1059] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:08:09.173774 23377 consensus_queue.cc:1059] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.012s
I20260709 13:08:09.194639 23695 raft_consensus.cc:3055] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:09.195091 23695 raft_consensus.cc:740] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:08:09.197124 23682 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:09.198341 23577 raft_consensus.cc:3055] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:08:09.198747 23577 raft_consensus.cc:740] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:09.183379 23631 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:09.230887 23631 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:09.229238 15751 catalog_manager.cc:5697] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 7 to 19, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 19 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:09.235154 23685 log.cc:927] Time spent T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.162s	user 0.000s	sys 0.002s
I20260709 13:08:09.154953 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:08:09.240516 23676 log.cc:927] Time spent T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.177s	user 0.001s	sys 0.000s
I20260709 13:08:09.210178 23577 consensus_queue.cc:260] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:09.241694 23577 raft_consensus.cc:3060] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:09.248195 23695 consensus_queue.cc:260] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:09.249716 23695 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:09.228320 15759 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58966 (request call id 248) took 8679 ms. Trace:
I20260709 13:08:09.253520 23601 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 23 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:09.234685 15751 catalog_manager.cc:5697] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 7 to 21, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 21 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
W20260709 13:08:09.256822 23672 log.cc:927] Time spent T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.260s	user 0.001s	sys 0.001s
W20260709 13:08:09.262863 23680 log.cc:927] Time spent T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.229s	user 0.001s	sys 0.000s
I20260709 13:08:09.264199 15859 raft_consensus.cc:3055] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:08:09.264633 15859 raft_consensus.cc:740] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:08:09.265488 15859 consensus_queue.cc:260] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:09.266718 15859 raft_consensus.cc:3060] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:09.266957 23543 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:09.268361 23006 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:09.269034 23006 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:09.271365 23006 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:09.277910 15933 raft_consensus.cc:1240] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 22. Current term is 23. Ops: []
I20260709 13:08:09.288118 23289 raft_consensus.cc:2749] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:09.281716 16566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:09.292956 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:08:09.298542 15974 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:09.299324 15974 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:09.306967 15859 raft_consensus.cc:1275] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 17. (term mismatch)
I20260709 13:08:09.308135 15859 pending_rounds.cc:85] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8: Aborting all ops after (but not including) 16
I20260709 13:08:09.317930 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:09.318747 16005 raft_consensus.cc:2393] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 21: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:08:09.321940 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:09.317947 23289 raft_consensus.cc:2749] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:08:09.326402 15859 pending_rounds.cc:107] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8: Aborting uncommitted NO_OP operation due to leader change: 22.17
I20260709 13:08:09.326771 15859 raft_consensus.cc:2889] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:09.316294 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:09.355126 15974 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:09.364857 15855 raft_consensus.cc:1240] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Rejecting Update request from peer 85e997ba5003439781ec4f1f8db0c3ee for earlier term 22. Current term is 23. Ops: [21.10-22.11]
I20260709 13:08:09.367120 15862 raft_consensus.cc:1275] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 13. (term mismatch)
I20260709 13:08:09.367599 15862 pending_rounds.cc:85] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8: Aborting all ops after (but not including) 12
I20260709 13:08:09.367882 15862 pending_rounds.cc:107] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8: Aborting uncommitted NO_OP operation due to leader change: 22.13
I20260709 13:08:09.368147 15862 raft_consensus.cc:2889] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:08:09.379241 15828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:09.383792 23327 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:08:09.384696 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:09.374941 15751 catalog_manager.cc:5697] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 11 to 18. New cstate: current_term: 18 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:09.403334 23689 consensus_queue.cc:1059] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.086s
I20260709 13:08:09.404291 23689 raft_consensus.cc:3055] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:08:09.404657 23689 raft_consensus.cc:740] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:08:09.404855 23682 consensus_queue.cc:1059] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:08:09.405216 15751 catalog_manager.cc:5697] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 18 to 22. New cstate: current_term: 22 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:09.408454 15751 catalog_manager.cc:5697] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 12 to 20, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 20 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:09.410189 23649 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:08:09.402616 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:08:09.415238 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 226098us
I20260709 13:08:09.251773 15759 rpcz_store.cc:276] 0709 13:08:00.549286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:00.556264 (+  6978us) service_pool.cc:224] Handling call
0709 13:08:09.228061 (+8671797us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:06.697899 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:06.829166 (+131267us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:06.829995 (+   829us) write_op.cc:435] Acquired schema lock
0709 13:08:06.832330 (+  2335us) tablet.cc:662] Decoding operations
0709 13:08:06.864252 (+ 31922us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:06.864314 (+    62us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:06.864340 (+    26us) tablet.cc:685] Acquiring locks for 23 operations
0709 13:08:06.865908 (+  1568us) tablet.cc:701] Row locks acquired
0709 13:08:06.865927 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:08:06.866060 (+   133us) write_op.cc:270] Start()
0709 13:08:06.866171 (+   111us) write_op.cc:276] Timestamp: P: 1783602486866015 usec, L: 0
0709 13:08:06.866196 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:08:06.866871 (+   675us) log.cc:844] Serialized 8956 byte log entry
0709 13:08:07.520293 (+653422us) op_driver.cc:464] REPLICATION: finished
0709 13:08:07.623517 (+103224us) write_op.cc:301] APPLY: starting
0709 13:08:07.635738 (+ 12221us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:07.962031 (+326293us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:07.962150 (+   119us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:08.132899 (+170749us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:08.133428 (+   529us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=46,key_file_lookups=46,delta_file_lookups=46,mrs_lookups=0
0709 13:08:08.133517 (+    89us) write_op.cc:312] APPLY: finished
0709 13:08:08.134277 (+   760us) log.cc:844] Serialized 204 byte log entry
0709 13:08:08.176956 (+ 42679us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:08.223543 (+ 46587us) write_op.cc:454] Released schema lock
0709 13:08:08.223618 (+    75us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:08.224238 (+   620us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:08.224263 (+    25us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":647648,"reported_tablets":42,"child_traces":[["op",{"apply.queue_time_us":50356,"apply.run_cpu_time_us":42686,"apply.run_wall_time_us":603362,"cfile_cache_hit":94,"cfile_cache_hit_bytes":100497,"num_ops":23,"prepare.queue_time_us":141502,"prepare.run_cpu_time_us":14222,"prepare.run_wall_time_us":305642,"raft.queue_time_us":126619,"raft.run_cpu_time_us":8994,"raft.run_wall_time_us":113935,"replication_time_us":624864,"spinlock_wait_cycles":335744,"thread_start_us":214942,"threads_started":4,"wal-append.queue_time_us":170949,"wal-append.run_cpu_time_us":6016,"wal-append.run_wall_time_us":2086922}]]}
I20260709 13:08:09.423933 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:09.425904 16011 raft_consensus.cc:1275] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 10. (term mismatch)
I20260709 13:08:09.426414 16011 pending_rounds.cc:85] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee: Aborting all ops after (but not including) 9
I20260709 13:08:09.426676 16011 pending_rounds.cc:107] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 13:08:09.426957 16011 raft_consensus.cc:2889] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:09.436381 15902 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:09.411216 15751 catalog_manager.cc:5697] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 13 to 21, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 21 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:09.441871 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:09.443382 16011 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:09.440311 15827 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
W20260709 13:08:09.445016 23634 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.002s	sys 0.012s
I20260709 13:08:09.446033 23576 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:08:09.446805 23576 raft_consensus.cc:697] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 23 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
W20260709 13:08:09.446813 23634 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.003s	sys 0.012s
I20260709 13:08:09.447870 23576 consensus_queue.cc:237] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:09.453877 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:09.454577 16008 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:09.438192 23366 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:09.470698 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:09.471181 23557 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:09.475406 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:09.475242 23557 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:09.445695 15751 catalog_manager.cc:5697] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 17 to 20, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 20 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:09.481954 15751 catalog_manager.cc:5697] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 15 to 23, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 23 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:09.443624 23559 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
W20260709 13:08:09.484335 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 108406us
I20260709 13:08:09.490340 16011 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 20.
I20260709 13:08:09.457887 23406 consensus_queue.cc:1048] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:08:09.501998 15751 catalog_manager.cc:5697] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 14 to 20, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 20 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:09.505688 16008 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 23.
I20260709 13:08:09.509536 23650 consensus_queue.cc:1048] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:08:09.510954 16009 raft_consensus.cc:1275] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 23 index: 11. (term mismatch)
I20260709 13:08:09.511402 16009 pending_rounds.cc:85] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting all ops after (but not including) 10
I20260709 13:08:09.511660 16009 pending_rounds.cc:107] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee: Aborting uncommitted NO_OP operation due to leader change: 22.11
I20260709 13:08:09.511932 16009 raft_consensus.cc:2889] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:08:09.512739 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 147978us
I20260709 13:08:09.492367 23342 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.510257 15751 catalog_manager.cc:5697] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 10 to 16. New cstate: current_term: 16 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:09.525699 23707 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.526714 16535 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:09.526546 23707 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:09.528911 23674 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:08:09.518780 23342 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:09.536038 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:09.536842 16564 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:09.537091 23631 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:09.494320 23559 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:09.539274 23707 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:09.536183 23342 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:08:09.562791 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 44084.9us
I20260709 13:08:09.568127 15856 raft_consensus.cc:1275] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 23 index: 11. (index mismatch)
I20260709 13:08:09.570149 15859 raft_consensus.cc:1275] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:08:09.582369 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:08:09.586869 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 58763.9us
I20260709 13:08:09.583252 15860 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 19.
I20260709 13:08:09.603838 15751 catalog_manager.cc:5697] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 13 to 19, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 19 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:09.608649 15934 raft_consensus.cc:1275] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:08:09.610402 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:09.611300 16535 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:09.613565 16570 raft_consensus.cc:1240] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 20. Current term is 21. Ops: []
I20260709 13:08:09.617471 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:09.618112 16011 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 16.
I20260709 13:08:09.619704 15902 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:09.621186 15974 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:09.566412 23707 consensus_queue.cc:1048] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.037s
I20260709 13:08:09.630949 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:09.634743 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:09.628579 23631 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:09.638346 23631 raft_consensus.cc:697] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 20 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:08:09.639446 23631 consensus_queue.cc:237] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:09.614665 23489 raft_consensus.cc:2749] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:08:09.649700 16564 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 23.
I20260709 13:08:09.634797 15751 catalog_manager.cc:5697] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 14 to 20. New cstate: current_term: 20 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:09.659576 15900 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:08:09.659971 23708 log.cc:927] Time spent T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 13:08:09.661500 23366 raft_consensus.cc:2749] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:09.663995 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:09.635918 23123 consensus_queue.cc:1048] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 13:08:09.678534 23366 consensus_queue.cc:1048] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:08:09.703012 23377 raft_consensus.cc:3055] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:09.703536 23377 raft_consensus.cc:740] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:08:09.704533 23377 consensus_queue.cc:260] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:09.705864 23377 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:09.711274 15827 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:08:09.682130 23406 consensus_queue.cc:1048] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:08:09.736886 23689 consensus_queue.cc:260] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:09.738255 23689 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:09.746321 23559 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:09.652704 15751 catalog_manager.cc:5697] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 12 to 16, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 16 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:09.784765 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 151725us
W20260709 13:08:09.841447 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 130226us
I20260709 13:08:09.765684 23577 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
W20260709 13:08:09.849433 23708 log.cc:927] Time spent T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:08:09.844033 23679 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:08:09.867162 16036 heartbeater.cc:646] Failed to heartbeat to 127.14.103.62:42985 (5 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.14.103.62:42985 timed out after 15.000s (SENT)
W20260709 13:08:09.878729 23715 log.cc:927] Time spent T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 13:08:09.857424 23577 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:09.882493 23577 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:09.843941 23006 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:09.883873 23006 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:09.886076 23006 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:09.887799 23693 log.cc:927] Time spent T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:08:09.891916 23631 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.892452 23631 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:09.893271 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:09.894538 23631 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:09.895161 15751 catalog_manager.cc:5697] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 16 to 24. New cstate: current_term: 24 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:09.897853 23573 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:09.897914 15751 catalog_manager.cc:5697] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 19 to 23. New cstate: current_term: 23 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:09.898681 23573 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:09.899642 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:09.900974 23573 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:08:09.903609 23714 log.cc:927] Time spent T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.235s	user 0.000s	sys 0.002s
I20260709 13:08:09.903716 16007 raft_consensus.cc:2410] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 27 because replica has last-logged OpId of term: 26 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 18.
I20260709 13:08:09.855369 23690 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:08:09.909224 23690 raft_consensus.cc:697] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 23 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:08:09.910152 23690 consensus_queue.cc:237] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:09.911710 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:09.912446 16007 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 16.
I20260709 13:08:09.854380 23549 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:09.914054 23549 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:09.916201 23549 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:08:09.918004 23710 log.cc:927] Time spent T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.385s	user 0.000s	sys 0.001s
W20260709 13:08:09.922761 23711 log.cc:927] Time spent T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.386s	user 0.001s	sys 0.000s
I20260709 13:08:09.931464 23679 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:09.931980 23679 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:09.935040 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:09.939646 16569 raft_consensus.cc:2393] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 16: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:08:09.940196 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:09.948338 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:09.947768 16570 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 19.
I20260709 13:08:09.951227 15930 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 20.
I20260709 13:08:09.960582 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
W20260709 13:08:09.970126 15746 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet dfeaf1f6660d48848ae69710739427a0 with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER failed with leader eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica eb1e0493af1e46dbb6dec7e25525bab5 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:09.978435 15902 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:10.007375 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:10.008404 15856 raft_consensus.cc:2393] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 20: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
W20260709 13:08:10.011178 23702 log.cc:927] Time spent T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.479s	user 0.000s	sys 0.012s
I20260709 13:08:10.015949 23559 raft_consensus.cc:2804] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:10.016757 23559 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:10.017216 23559 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:10.016068 16010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41674 (request call id 3007) took 1045 ms. Trace:
I20260709 13:08:10.019611 16010 rpcz_store.cc:276] 0709 13:08:08.971135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:08.985101 (+ 13966us) service_pool.cc:224] Handling call
0709 13:08:09.012691 (+ 27590us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:09.069910 (+ 57219us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:08:09.092539 (+ 22629us) log.cc:844] Serialized 14 byte log entry
0709 13:08:09.417974 (+325435us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:09.470363 (+ 52389us) log.cc:844] Serialized 52 byte log entry
0709 13:08:09.470557 (+   194us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:08:09.470600 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:09.470678 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:10.007076 (+536398us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b4400022b4c. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e71e1c85 00007f73e71e26a2 00007f73e9f541db 00007f73e9f4faaf 00007f73e9f4386e 00007f73e9f42bb6 00007f73e9f427d8 00007f73e9f4c221 00007f73e9f4c1d9 00007f73e9f4c169
0709 13:08:10.015767 (+  8691us) raft_consensus.cc:1685] finished
0709 13:08:10.015867 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:10.016010 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38899,"prepare.queue_time_us":16860,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":49,"raft.queue_time_us":35477,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":13770,"spinlock_wait_cycles":29268352,"thread_start_us":259525,"threads_started":1,"wal-append.queue_time_us":346598}
I20260709 13:08:10.024801 15902 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:08:10.045138 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 56142.9us
I20260709 13:08:10.052450 16533 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:10.064846 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:10.072510 16007 raft_consensus.cc:2368] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 19. Current term is 20.
I20260709 13:08:10.076184 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:10.054816 23577 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:10.125913 15827 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:08:10.167414 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 86393us
I20260709 13:08:10.078526 23404 raft_consensus.cc:2804] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:10.066052 23631 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:10.168715 15976 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:10.116302 23689 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.183706 16007 raft_consensus.cc:1240] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 20. Current term is 21. Ops: [20.15-20.15]
W20260709 13:08:10.185644 15903 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:10.168998 23356 raft_consensus.cc:2804] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:10.172356 23404 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:10.081063 23682 consensus_queue.cc:1048] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.094s
I20260709 13:08:10.171839 23695 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:10.195937 23695 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:10.198037 23695 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:10.193389 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:10.191648 23550 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:10.152966 23577 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:10.202518 23678 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:10.200469 23550 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:10.208262 23557 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:10.192533 23356 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:10.192834 23404 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:10.212612 23550 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:10.212999 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:10.211372 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:10.226859 15974 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:10.231379 23557 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:10.222205 23356 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:10.202873 23689 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:10.251199 16010 raft_consensus.cc:1275] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:08:10.253741 16535 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
W20260709 13:08:10.254710 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 224620us
I20260709 13:08:10.256538 23006 raft_consensus.cc:2804] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:10.256970 23006 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:10.257316 23006 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:10.268236 16535 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:10.273463 16002 raft_consensus.cc:1275] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:08:10.275121 15902 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:10.281591 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:10.282263 15853 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 20.
I20260709 13:08:10.284006 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:10.286579 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:10.289381 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:10.290081 16005 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 27.
I20260709 13:08:10.286393 23692 raft_consensus.cc:2749] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:10.285211 23550 raft_consensus.cc:2749] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 13:08:10.294040 23404 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:10.321882 23404 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:10.311365 23689 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:10.312784 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:10.324410 16005 raft_consensus.cc:2410] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for term 24 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
I20260709 13:08:10.326740 23559 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:10.353981 16567 raft_consensus.cc:2393] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 27: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:08:10.316025 16535 leader_election.cc:400] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 19. Current term is 20.
I20260709 13:08:10.360864 16535 leader_election.cc:403] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:10.362664 23550 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:08:10.370992 23550 raft_consensus.cc:2749] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 19. Current term is 20.
W20260709 13:08:10.372061 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 56297.6us
I20260709 13:08:10.373622 23577 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:10.303298 23682 consensus_queue.cc:1048] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 13:08:10.375463 23550 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:10.428362 23631 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:10.377516 23692 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:10.380316 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:10.429487 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:10.389447 16569 raft_consensus.cc:1275] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:08:10.402117 23689 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:10.429461 23692 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:10.383963 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:10.431986 16570 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:10.429009 23550 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:10.430980 23689 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:10.434413 23691 consensus_queue.cc:1048] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:08:10.434825 23689 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:10.436681 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:10.385283 23006 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:10.411113 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:10.438828 15860 raft_consensus.cc:2393] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 20: Already voted for candidate 31882606e87447fb9a36bbf625129fe8 in this term.
I20260709 13:08:10.439724 23006 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:10.445449 23550 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:10.447935 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:10.450500 16005 raft_consensus.cc:2393] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 20: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:08:10.449488 23692 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:10.358999 23559 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:10.459800 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:10.462045 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:10.463620 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:10.464277 16005 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:10.464694 15930 raft_consensus.cc:2368] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 20. Current term is 21.
I20260709 13:08:10.487282 15974 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:10.490123 16004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:10.503595 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae068fc0a8a84b148d068df2b4463b8c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:10.507083 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:10.495504 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:10.506968 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:10.509438 15853 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 20.
I20260709 13:08:10.511215 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:10.512714 15859 raft_consensus.cc:3060] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:10.513783 16533 leader_election.cc:400] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 election: Vote denied by peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) with higher term. Message: Invalid argument: T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 20. Current term is 21.
I20260709 13:08:10.514215 16533 leader_election.cc:403] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:08:10.429056 23631 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:10.517498 23631 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:10.518400 23557 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:10.518678 23356 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:10.518946 23557 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:08:10.521096 23557 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:10.494827 23734 raft_consensus.cc:2749] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:10.558856 15859 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 21.
W20260709 13:08:10.645352 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 80448.3us
W20260709 13:08:10.563095 23679 consensus_meta.cc:287] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee: Time spent flushing consensus metadata: real 0.631s	user 0.009s	sys 0.002s
W20260709 13:08:10.652624 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 43905.9us
I20260709 13:08:10.653647 15827 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:10.601130 16533 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:10.656908 15929 raft_consensus.cc:1275] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:08:10.657970 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:10.563617 23577 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:08:10.679940 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 78895.3us
I20260709 13:08:10.687016 23156 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:08:10.709527 15900 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:10.678043 23725 consensus_queue.cc:1048] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 13:08:10.661422 23679 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:10.717422 15900 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:10.734081 15900 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:08:10.686476 23577 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:10.710027 23156 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:10.713204 23289 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.742663 16005 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 17.
I20260709 13:08:10.748710 16009 raft_consensus.cc:1240] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:10.741547 23682 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:08:10.755946 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 74112.9us
I20260709 13:08:10.758234 15902 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:10.742002 23749 raft_consensus.cc:2749] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:08:10.785640 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 98742.5us
I20260709 13:08:10.747491 23691 raft_consensus.cc:2804] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:10.748000 23156 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:08:10.800426 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:10.777150 23682 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:08:10.813997 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 67877us
I20260709 13:08:10.815078 16562 raft_consensus.cc:1275] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:08:10.788401 23356 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:10.821717 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:10.795316 23406 raft_consensus.cc:2749] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:10.825356 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:10.840284 16562 raft_consensus.cc:2368] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 16. Current term is 17.
I20260709 13:08:10.840433 16010 raft_consensus.cc:2368] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 16. Current term is 17.
I20260709 13:08:10.797749 23489 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:10.754777 23289 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:10.840971 16570 raft_consensus.cc:2468] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 21.
I20260709 13:08:10.843889 15859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:10.844504 15859 raft_consensus.cc:3060] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:10.848460 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:10.857779 15859 raft_consensus.cc:2468] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 17.
I20260709 13:08:10.818553 23682 raft_consensus.cc:3060] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:10.806977 23691 raft_consensus.cc:697] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 21 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:08:10.917085 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 170443us
I20260709 13:08:10.920765 15974 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:10.909163 23674 raft_consensus.cc:2749] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009) with higher term. Message: Invalid argument: T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for earlier term 20. Current term is 21.
W20260709 13:08:10.946581 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 66324.9us
I20260709 13:08:10.947903 16533 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:10.955160 23392 raft_consensus.cc:2749] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:10.974366 23673 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:10.940533 23206 consensus_queue.cc:1048] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 13:08:10.937378 23748 consensus_queue.cc:1059] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:08:10.992594 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:08:10.986069 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 65940.9us
I20260709 13:08:11.001828 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:10.990327 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:10.968273 23751 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:11.002996 15853 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 20.
I20260709 13:08:11.000653 16533 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:10.992187 23739 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.996374 23679 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:08:11.021530 23744 log.cc:927] Time spent T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.306s	user 0.001s	sys 0.000s
I20260709 13:08:11.033200 15827 leader_election.cc:400] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 16. Current term is 17.
I20260709 13:08:11.041579 15827 leader_election.cc:403] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
W20260709 13:08:11.025952 23745 log.cc:927] Time spent T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.278s	user 0.001s	sys 0.000s
I20260709 13:08:11.050397 23763 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643) with higher term. Message: Invalid argument: T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 16. Current term is 17.
I20260709 13:08:11.052129 23289 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:11.034468 23739 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:11.055359 23739 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:11.029425 23762 raft_consensus.cc:2804] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:08:11.046278 23577 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:10.985313 23673 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:11.067440 16005 raft_consensus.cc:2393] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 20: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:08:11.071493 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:11.071800 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:11.072966 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:11.072796 15937 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 23.
I20260709 13:08:10.971903 23691 consensus_queue.cc:237] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.17, Last appended by leader: 17, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:11.077744 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:11.070426 15928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:11.081408 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:11.082093 15937 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 19.
I20260709 13:08:11.080132 15861 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 20.
I20260709 13:08:11.084199 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:11.024595 23078 consensus_queue.cc:1048] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.047s
I20260709 13:08:11.090571 15974 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:11.093575 15974 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:08:11.079409 23762 raft_consensus.cc:697] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:08:11.111068 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 63757us
I20260709 13:08:11.104400 23577 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:08:11.118731 15900 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:08:11.120332 15974 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:08:11.127612 23543 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:11.128120 23543 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:11.128592 23543 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:11.126104 23678 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:11.124773 23580 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:08:11.132126 23678 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:11.132558 23580 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:11.132776 23678 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:11.133060 23580 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:11.124931 23765 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:08:11.138294 23765 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:08:11.126547 23746 log.cc:927] Time spent T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.350s	user 0.002s	sys 0.003s
W20260709 13:08:11.148134 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 142116us
I20260709 13:08:11.112991 23682 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:11.167220 23366 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.167830 23366 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:08:11.168463 23759 log.cc:927] Time spent T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:08:11.170414 23366 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:11.147538 23734 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.173070 23734 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:11.173502 23765 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:11.175240 23734 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:11.180569 23682 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:11.189020 23678 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:11.190932 23580 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:11.210960 23673 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:11.144922 23762 consensus_queue.cc:237] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:08:11.216006 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 49461.8us
I20260709 13:08:11.216620 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:11.238483 15827 leader_election.cc:400] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 16 election: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for earlier term 16. Current term is 17.
I20260709 13:08:11.237689 16562 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 21.
I20260709 13:08:11.238654 15974 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:11.255491 23734 raft_consensus.cc:2749] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:11.271304 23749 raft_consensus.cc:3055] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:08:11.271811 23749 raft_consensus.cc:740] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:08:11.237495 23543 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:11.272598 23749 consensus_queue.cc:260] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:11.273613 23749 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:11.273876 23543 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:11.275429 23678 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:11.280555 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:11.296401 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:11.303602 15861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:11.304446 15861 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:11.306419 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:11.311476 16005 raft_consensus.cc:1240] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 24 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 23. Current term is 24. Ops: []
I20260709 13:08:11.314854 16570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:11.321863 16564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:11.322764 16564 raft_consensus.cc:2410] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:08:11.328094 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:11.298418 23678 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.343044 15827 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:11.348769 23580 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 24 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:11.350068 23678 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:11.352797 23678 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:11.336230 23694 consensus_queue.cc:1059] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.018s
I20260709 13:08:11.361330 15855 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 20.
I20260709 13:08:11.369670 15853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:11.374467 15853 raft_consensus.cc:2410] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:08:11.379432 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:11.383921 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:11.396171 15935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:11.397099 15935 raft_consensus.cc:3060] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:11.378116 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:11.401322 15936 raft_consensus.cc:2410] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:08:11.403951 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:11.404635 16007 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 21.
I20260709 13:08:11.406563 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:11.407362 16005 raft_consensus.cc:2393] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 20: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:08:11.409508 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:11.410216 16002 raft_consensus.cc:2368] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 20. Current term is 21.
W20260709 13:08:11.412885 23774 log.cc:927] Time spent T 00000000000000000000000000000000 P d33cebb189b14a989836214fdbafb898: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
W20260709 13:08:11.417264 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 74345.4us
I20260709 13:08:11.419512 23694 raft_consensus.cc:2749] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:08:11.420938 23156 consensus_meta.cc:287] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8: Time spent flushing consensus metadata: real 0.592s	user 0.006s	sys 0.005s
I20260709 13:08:11.421438 23156 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:11.428128 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:11.428033 15900 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
W20260709 13:08:11.437657 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 57885.7us
I20260709 13:08:11.449469 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:11.454620 16011 raft_consensus.cc:1240] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 19. Current term is 20. Ops: []
I20260709 13:08:11.457762 23601 raft_consensus.cc:3055] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:08:11.458271 23601 raft_consensus.cc:740] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:08:11.423663 23156 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 27 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:11.475518 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:11.480796 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:11.482667 16569 raft_consensus.cc:2393] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 27: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:08:11.481639 15902 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:08:11.484910 23770 log.cc:927] Time spent T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 13:08:11.486549 23689 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:11.487002 23689 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:11.487330 23689 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:11.459167 23601 consensus_queue.cc:260] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:11.444731 23749 raft_consensus.cc:2804] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:11.487624 15861 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 21.
I20260709 13:08:11.511263 15974 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:08:11.512225 23673 raft_consensus.cc:2804] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:11.503083 15902 leader_election.cc:400] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 election: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 20. Current term is 21.
I20260709 13:08:11.513643 15902 leader_election.cc:403] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:08:11.516057 16002 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 27.
I20260709 13:08:11.512848 23673 raft_consensus.cc:697] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:11.523535 23673 consensus_queue.cc:237] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:11.526292 15827 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:11.496899 23601 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:11.504287 23749 raft_consensus.cc:697] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 20 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:08:11.534382 15974 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:11.498827 23689 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:11.531314 23690 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:08:11.550599 23577 raft_consensus.cc:2749] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:11.553213 23690 raft_consensus.cc:697] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 27 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:08:11.556989 15974 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:11.560163 15974 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:08:11.567088 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:11.568356 23734 raft_consensus.cc:2749] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:08:11.561882 23557 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:11.554266 23690 consensus_queue.cc:237] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:11.666314 15853 raft_consensus.cc:1275] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
I20260709 13:08:11.703298 23559 consensus_queue.cc:1048] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:08:11.716899 15935 raft_consensus.cc:2468] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 21.
I20260709 13:08:11.688678 23749 consensus_queue.cc:237] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:11.718775 23748 raft_consensus.cc:2749] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 20. Current term is 21.
W20260709 13:08:11.753916 23780 log.cc:927] Time spent T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:08:11.778173 23689 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:11.816843 16002 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58"
dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
mode: GRACEFUL
new_leader_uuid: "8430db80a39d4868bcf7641c16de35ea"
 from {username='slave'} at 127.0.0.1:43208
I20260709 13:08:11.817680 16002 raft_consensus.cc:606] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Received request to transfer leadership to TS 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:11.818178 16002 raft_consensus.cc:614] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:08:11.823232 23781 log.cc:927] Time spent T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 13:08:11.853973 16006 raft_consensus.cc:1275] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:08:11.856573 15853 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:08:11.857465 15853 raft_consensus.cc:606] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Received request to transfer leadership to TS eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:11.857883 15853 raft_consensus.cc:614] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:11.862080 15853 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:08:11.862793 15853 raft_consensus.cc:606] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 23 LEADER]: Received request to transfer leadership to TS eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:11.861032 23760 consensus_queue.cc:1048] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 13:08:11.870767 15853 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd"
dest_uuid: "31882606e87447fb9a36bbf625129fe8"
mode: GRACEFUL
new_leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:57012
I20260709 13:08:11.871753 15853 raft_consensus.cc:606] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 23 LEADER]: Received request to transfer leadership to TS eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:11.891077 16006 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def"
dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
mode: GRACEFUL
new_leader_uuid: "8430db80a39d4868bcf7641c16de35ea"
 from {username='slave'} at 127.0.0.1:43208
I20260709 13:08:11.891821 16006 raft_consensus.cc:606] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Received request to transfer leadership to TS 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:11.892259 16006 raft_consensus.cc:614] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:11.928244 23533 raft_consensus.cc:993] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8: : Instructing follower eb1e0493af1e46dbb6dec7e25525bab5 to start an election
I20260709 13:08:11.928902 23533 raft_consensus.cc:1081] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 23 LEADER]: Signalling peer eb1e0493af1e46dbb6dec7e25525bab5 to start an election
I20260709 13:08:11.961534 16569 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01"
dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:45246
I20260709 13:08:11.962189 16569 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:11.962535 16569 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:11.974882 23704 raft_consensus.cc:993] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8: : Instructing follower eb1e0493af1e46dbb6dec7e25525bab5 to start an election
I20260709 13:08:11.975571 23704 raft_consensus.cc:1081] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 23 LEADER]: Signalling peer eb1e0493af1e46dbb6dec7e25525bab5 to start an election
I20260709 13:08:12.003063 16562 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd"
dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
 from {username='slave'} at 127.0.0.1:45246
I20260709 13:08:12.003868 16562 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:12.004266 16562 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:12.015398 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:08:12.017328 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:12.023334 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:12.024017 16567 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:12.024480 16570 raft_consensus.cc:1275] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 19 index: 17. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
W20260709 13:08:12.032080 15825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:12.040041 16567 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 22.
I20260709 13:08:11.982491 16569 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:12.046940 16562 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:12.049770 15902 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:12.043092 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:12.054204 16009 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:12.053669 16562 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 24 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:12.058357 16569 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 24 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:12.057282 23406 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:08:12.062189 15974 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:12.067101 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: true dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:12.067786 15936 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:12.069309 23580 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:12.070922 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: true dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:12.071569 16005 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:12.071477 23406 raft_consensus.cc:697] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 22 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:08:12.042827 23691 consensus_queue.cc:1048] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.023s
I20260709 13:08:12.077217 23406 consensus_queue.cc:237] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:12.090282 16005 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 24.
I20260709 13:08:12.092989 16535 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:12.093993 23404 raft_consensus.cc:2804] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:08:12.094498 23404 raft_consensus.cc:697] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:08:12.095410 23404 consensus_queue.cc:237] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:12.122159 15853 raft_consensus.cc:1275] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:08:12.123751 23734 consensus_queue.cc:1048] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:08:12.125533 16009 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 22.
I20260709 13:08:12.133435 15857 raft_consensus.cc:1275] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:08:12.133986 15936 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 24.
I20260709 13:08:12.137782 16533 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:12.140564 23633 raft_consensus.cc:2804] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 13:08:12.140928 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 76672.4us
I20260709 13:08:12.141142 23633 raft_consensus.cc:697] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:08:12.169591 23633 consensus_queue.cc:237] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:12.177302 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: true dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:12.177956 15860 raft_consensus.cc:3055] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:08:12.178288 15860 raft_consensus.cc:740] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:08:12.178988 15860 consensus_queue.cc:260] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:12.180236 15860 raft_consensus.cc:3060] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:12.180298 23404 consensus_queue.cc:1048] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:08:12.191778 15856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: true dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:12.197386 15936 raft_consensus.cc:1275] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:08:12.198917 15856 raft_consensus.cc:3055] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:08:12.199331 15856 raft_consensus.cc:740] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:08:12.204080 15860 raft_consensus.cc:2468] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 24.
I20260709 13:08:12.206781 23734 consensus_queue.cc:1048] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.012s
I20260709 13:08:12.234640 15856 consensus_queue.cc:260] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:12.265375 15856 raft_consensus.cc:3060] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:12.262380 15759 catalog_manager.cc:5697] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:12.276466 15759 catalog_manager.cc:5697] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 11 to 19, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 19 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:12.278851 15759 catalog_manager.cc:5697] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:12.280814 15759 catalog_manager.cc:5697] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 19 to 21, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 21 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:12.330648 15855 raft_consensus.cc:1275] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:08:12.338864 23682 consensus_queue.cc:1048] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:08:12.341665 23633 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:12.342229 23633 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:12.344476 23633 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:12.348341 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:12.349203 16009 raft_consensus.cc:2410] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 for term 28 because replica has last-logged OpId of term: 26 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 18.
I20260709 13:08:12.359418 15759 catalog_manager.cc:5697] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 13 to 23. New cstate: current_term: 23 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:12.361907 15759 catalog_manager.cc:5697] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 18 to 21, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 21 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:12.365264 15759 catalog_manager.cc:5697] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:12.367585 15759 catalog_manager.cc:5697] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 17 to 19, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 19 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:12.369953 15759 catalog_manager.cc:5697] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:08:12.376776 15751 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58426 (request call id 347) took 11798 ms (client timeout 15000 ms). Trace:
W20260709 13:08:12.377362 15751 rpcz_store.cc:269] 0709 13:08:00.578771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:00.579026 (+   255us) service_pool.cc:224] Handling call
0709 13:08:12.376700 (+11797674us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:10.749060 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:10.974450 (+225390us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:10.995072 (+ 20622us) write_op.cc:435] Acquired schema lock
0709 13:08:10.995098 (+    26us) tablet.cc:662] Decoding operations
0709 13:08:11.211410 (+216312us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:11.211515 (+   105us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:11.211554 (+    39us) tablet.cc:685] Acquiring locks for 16 operations
0709 13:08:11.212985 (+  1431us) tablet.cc:701] Row locks acquired
0709 13:08:11.213032 (+    47us) write_op.cc:260] PREPARE: finished
0709 13:08:11.213240 (+   208us) write_op.cc:270] Start()
0709 13:08:11.213433 (+   193us) write_op.cc:276] Timestamp: P: 1783602491213161 usec, L: 0
0709 13:08:11.213456 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:08:11.214204 (+   748us) log.cc:844] Serialized 6394 byte log entry
0709 13:08:11.713686 (+499482us) op_driver.cc:464] REPLICATION: finished
0709 13:08:11.860570 (+146884us) write_op.cc:301] APPLY: starting
0709 13:08:11.860669 (+    99us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:12.012130 (+151461us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:12.012258 (+   128us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:12.140363 (+128105us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:12.140691 (+   328us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=32,mrs_lookups=0
0709 13:08:12.140739 (+    48us) write_op.cc:312] APPLY: finished
0709 13:08:12.141184 (+   445us) log.cc:844] Serialized 148 byte log entry
0709 13:08:12.164446 (+ 23262us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:12.205856 (+ 41410us) write_op.cc:454] Released schema lock
0709 13:08:12.206472 (+   616us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:12.208412 (+  1940us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:12.208459 (+    47us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":596858,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":146576,"apply.run_cpu_time_us":24689,"apply.run_wall_time_us":347799,"cfile_cache_hit":66,"cfile_cache_hit_bytes":71167,"num_ops":16,"prepare.queue_time_us":397979,"prepare.run_cpu_time_us":27703,"prepare.run_wall_time_us":657584,"raft.queue_time_us":283674,"raft.run_cpu_time_us":70740,"raft.run_wall_time_us":94724,"replication_time_us":491493,"spinlock_wait_cycles":51712,"thread_start_us":519574,"threads_started":4,"wal-append.queue_time_us":143249}]]}
I20260709 13:08:12.385118 15860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:12.384989 15759 catalog_manager.cc:5697] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 12 to 22. New cstate: current_term: 22 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:12.386386 15759 catalog_manager.cc:5697] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 19 to 25, leader changed from eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 25 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:12.397863 15759 catalog_manager.cc:5697] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 16 to 17, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 17 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:12.400923 16009 raft_consensus.cc:1275] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:08:12.401508 15759 catalog_manager.cc:5697] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 9 to 19. New cstate: current_term: 19 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:12.408425 15759 catalog_manager.cc:5697] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 19 to 23. New cstate: current_term: 23 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:12.405709 23682 consensus_queue.cc:1048] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:08:12.404505 15856 raft_consensus.cc:2468] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 24.
I20260709 13:08:12.414207 15759 catalog_manager.cc:5697] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 21 to 22, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 22 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:12.425012 15759 catalog_manager.cc:5697] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:12.428604 15759 catalog_manager.cc:5697] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 20 to 21, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 21 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:12.432468 16533 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:12.432804 15759 catalog_manager.cc:5697] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:12.434185 15759 catalog_manager.cc:5697] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 12 to 15, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 15 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:12.436510 23674 raft_consensus.cc:2749] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:08:12.446082 16569 raft_consensus.cc:1275] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 27 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 13:08:12.435833 15759 catalog_manager.cc:5697] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:12.449276 15759 catalog_manager.cc:5697] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 23 to 24, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 24 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:12.450999 15759 catalog_manager.cc:5697] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:12.453502 15759 catalog_manager.cc:5697] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 21 to 23, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 23 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:12.477495 16002 raft_consensus.cc:1275] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 27 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 13:08:12.482569 23723 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:12.490758 23723 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:12.492889 23723 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:12.494057 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:12.494731 15936 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 20.
I20260709 13:08:12.496579 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:12.497519 16005 raft_consensus.cc:2393] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 20: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:08:12.499071 15825 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:08:12.501626 23018 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:12.506840 23018 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:12.507215 23018 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:12.479404 23690 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 16, Time since last communication: 0.024s
I20260709 13:08:12.528157 23699 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 13:08:12.561725 23018 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:12.582206 23018 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:12.583534 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:12.587781 16002 raft_consensus.cc:1275] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:08:12.597420 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:12.598333 16002 raft_consensus.cc:2393] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 20: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:08:12.598244 15933 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 20.
I20260709 13:08:12.599638 15825 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:08:12.605262 23682 consensus_queue.cc:1048] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.050s
W20260709 13:08:12.650606 23807 log.cc:927] Time spent T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 13:08:12.657939 23725 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:08:12.674883 23809 log.cc:927] Time spent T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:08:12.708936 23725 raft_consensus.cc:697] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 20 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
W20260709 13:08:12.738209 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 108450us
I20260709 13:08:12.740406 16009 raft_consensus.cc:1275] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 24 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:08:12.742519 23633 consensus_queue.cc:1048] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:08:12.762707 23725 consensus_queue.cc:237] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:08:12.784543 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 184969us
I20260709 13:08:12.805342 23689 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:12.805938 23689 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:12.885838 16569 raft_consensus.cc:1275] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:08:12.899510 15857 raft_consensus.cc:1275] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:08:12.912647 23824 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:12.933777 15855 consensus_queue.cc:237] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:08:12.970738 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 95019.6us
W20260709 13:08:12.986469 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 103164us
I20260709 13:08:12.977861 23689 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:13.017788 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:13.024292 16009 raft_consensus.cc:2393] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 24: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
W20260709 13:08:13.038604 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 54609.7us
I20260709 13:08:13.041175 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:13.015062 23828 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:13.073712 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 55710.2us
I20260709 13:08:12.986503 23749 consensus_queue.cc:1048] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.057s
I20260709 13:08:13.074504 23828 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:12.952981 23824 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:08:13.136744 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 108614us
I20260709 13:08:13.156504 15857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:13.157202 15857 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 24.
W20260709 13:08:13.177304 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 299594us
W20260709 13:08:13.180981 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 123585us
I20260709 13:08:13.192598 23824 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:13.229133 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:13.236678 23828 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:13.232502 23825 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.244181 23825 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:13.246619 23825 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:13.255975 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8" is_pre_election: true
I20260709 13:08:13.256618 15855 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 24.
I20260709 13:08:13.259383 15974 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:08:13.277930 23673 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:08:13.278549 23673 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:13.278985 23673 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:13.315505 16569 raft_consensus.cc:1275] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:08:13.318790 15900 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:13.322014 16002 raft_consensus.cc:1275] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:08:13.327891 23406 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:08:13.328332 23406 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:13.328749 23406 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:08:13.336144 23845 log.cc:927] Time spent T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
W20260709 13:08:13.337990 15825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 126773us
I20260709 13:08:13.338833 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:13.345580 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:13.351377 23835 consensus_queue.cc:1048] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:08:13.475827 15937 raft_consensus.cc:1275] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 24 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
W20260709 13:08:13.481209 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 134875us
I20260709 13:08:13.485080 15853 raft_consensus.cc:1275] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
W20260709 13:08:13.477376 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 49700.7us
I20260709 13:08:13.488386 16002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:13.489188 16010 raft_consensus.cc:1275] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:08:13.477295 15934 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 20.
I20260709 13:08:13.501999 23674 consensus_queue.cc:1048] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:08:13.535907 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:13.554667 15974 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:08:13.520936 23406 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:13.592541 23406 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 24 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:13.545238 23404 consensus_queue.cc:1048] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.048s
I20260709 13:08:13.599942 15930 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 24.
I20260709 13:08:13.527889 23832 consensus_queue.cc:1048] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.067s
W20260709 13:08:13.607132 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 65842.7us
I20260709 13:08:13.606237 23849 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:08:13.613173 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 74045.2us
I20260709 13:08:13.613981 23849 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:13.624320 23156 consensus_queue.cc:1048] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 13:08:13.639369 23813 consensus_queue.cc:1048] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:13.656268 15855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "31882606e87447fb9a36bbf625129fe8"
I20260709 13:08:13.645608 23849 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:13.682787 15855 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 24.
I20260709 13:08:13.688059 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:13.691169 15930 raft_consensus.cc:1275] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:08:13.703341 15900 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea; no voters: 
I20260709 13:08:13.733297 23673 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:13.734578 16002 raft_consensus.cc:2393] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 in current term 25: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
W20260709 13:08:13.734879 23850 log.cc:927] Time spent T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.235s	user 0.002s	sys 0.002s
I20260709 13:08:13.742568 23849 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:13.744650 16008 raft_consensus.cc:1240] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:08:13.749541 23849 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:13.710450 23156 consensus_queue.cc:1048] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 13:08:13.726779 23817 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:08:13.762499 23817 raft_consensus.cc:697] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 24 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:08:13.763446 23817 consensus_queue.cc:237] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:13.782133 15827 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:13.784071 23429 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:13.784235 23725 consensus_queue.cc:1059] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:08:13.785356 23725 raft_consensus.cc:3055] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:08:13.785753 23725 raft_consensus.cc:740] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:08:13.787015 23725 consensus_queue.cc:260] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:13.788532 23725 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:13.842092 23673 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 25 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:278: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 5
/apache/dev/git/kudu/src/kudu/util/test_util.cc:402: Failure
Failed
Timed out waiting for assertion to pass.
I20260709 13:08:13.859014 14748 tablet_server.cc:179] TabletServer@127.14.103.1:0 shutting down...
W20260709 13:08:13.863659 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 140859us
W20260709 13:08:13.896201 16533 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:13.900811 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 185143us
I20260709 13:08:13.904603 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:13.940361 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
W20260709 13:08:13.964972 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 68798.8us
W20260709 13:08:13.975075 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 50157.6us
I20260709 13:08:13.977864 23429 raft_consensus.cc:2749] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:08:14.009790 15900 consensus_peers.cc:597] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:14.066346 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 56586.3us
W20260709 13:08:14.067632 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 83864.2us
W20260709 13:08:14.083313 15974 leader_election.cc:336] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:14.081776 23691 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:14.104079 15974 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:14.135505 15759 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58966 (request call id 249) took 3674 ms. Trace:
I20260709 13:08:14.101838 23853 raft_consensus.cc:2955] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Committing config change with OpId 16.9: config changed from index 3 to 9, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:08:14.145802 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 68211us
I20260709 13:08:14.136147 15759 rpcz_store.cc:276] 0709 13:08:10.450075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.450230 (+   155us) service_pool.cc:224] Handling call
0709 13:08:14.124050 (+3673820us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:12.650547 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:12.699440 (+ 48893us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:12.699474 (+    34us) write_op.cc:435] Acquired schema lock
0709 13:08:12.699491 (+    17us) tablet.cc:662] Decoding operations
0709 13:08:12.701149 (+  1658us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:12.701244 (+    95us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:12.701274 (+    30us) tablet.cc:685] Acquiring locks for 23 operations
0709 13:08:12.702816 (+  1542us) tablet.cc:701] Row locks acquired
0709 13:08:12.702834 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:08:12.702975 (+   141us) write_op.cc:270] Start()
0709 13:08:12.703106 (+   131us) write_op.cc:276] Timestamp: P: 1783602492702927 usec, L: 0
0709 13:08:12.703133 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 13:08:12.732846 (+ 29713us) log.cc:844] Serialized 8890 byte log entry
0709 13:08:12.874565 (+141719us) op_driver.cc:464] REPLICATION: finished
0709 13:08:12.979707 (+105142us) write_op.cc:301] APPLY: starting
0709 13:08:12.983572 (+  3865us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:13.332103 (+348531us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:13.332235 (+   132us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:13.806895 (+474660us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:13.807220 (+   325us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=46,key_file_lookups=46,delta_file_lookups=46,mrs_lookups=0
0709 13:08:13.807264 (+    44us) write_op.cc:312] APPLY: finished
0709 13:08:13.807700 (+   436us) log.cc:844] Serialized 204 byte log entry
0709 13:08:13.807864 (+   164us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:13.808052 (+   188us) write_op.cc:454] Released schema lock
0709 13:08:13.808114 (+    62us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:13.836117 (+ 28003us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:13.836149 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":112236,"reported_tablets":33,"child_traces":[["op",{"apply.queue_time_us":93415,"apply.run_cpu_time_us":35398,"apply.run_wall_time_us":860144,"cfile_cache_hit":94,"cfile_cache_hit_bytes":100497,"num_ops":23,"prepare.queue_time_us":122483,"prepare.run_cpu_time_us":6462,"prepare.run_wall_time_us":87973,"replication_time_us":171227,"thread_start_us":89756,"threads_started":2}]]}
I20260709 13:08:14.155637 16561 raft_consensus.cc:2955] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Committing config change with OpId 16.9: config changed from index 3 to 9, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:14.139843 23691 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } attrs { replace: true } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:14.165153 23691 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:14.166787 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:14.168251 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:14.152937 23740 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:08:14.174507 15900 consensus_peers.cc:597] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:14.170742 15974 consensus_peers.cc:597] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:14.205472 15902 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:08:14.207487 15974 leader_election.cc:336] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:14.208214 15974 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:08:14.221768 15827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 43898.7us
W20260709 13:08:14.223202 15974 consensus_peers.cc:597] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:14.225589 16009 raft_consensus.cc:2955] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Committing config change with OpId 16.9: config changed from index 3 to 9, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:08:14.224694 15974 consensus_peers.cc:597] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:14.222591 23870 log.cc:927] Time spent T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea: Append to log took a long time: real 0.225s	user 0.000s	sys 0.001s
I20260709 13:08:14.221724 23682 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:08:14.263962 15827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.239282 23740 raft_consensus.cc:2749] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 13:08:14.276283 16533 consensus_peers.cc:597] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:14.294137 16533 consensus_peers.cc:597] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:14.355629 16533 consensus_peers.cc:597] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:14.364835 16533 consensus_peers.cc:597] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:14.371909 15900 consensus_peers.cc:597] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:14.373464 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 167205us
W20260709 13:08:14.392923 16533 consensus_peers.cc:597] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:14.408445 16533 consensus_peers.cc:597] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:14.478549 23853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:41940 (request call id 0) took 1595 ms. Trace:
I20260709 13:08:14.479151 23853 rpcz_store.cc:276] 0709 13:08:12.882382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:12.882524 (+   142us) service_pool.cc:224] Handling call
0709 13:08:12.966604 (+ 84080us) log.cc:844] Serialized 494 byte log entry
0709 13:08:14.478033 (+1511429us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3893,"raft.run_cpu_time_us":1008,"raft.run_wall_time_us":1006,"thread_start_us":238627,"threads_started":2,"wal-append.queue_time_us":189204}
I20260709 13:08:14.533042 15744 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 15301761b2ec47fe82ae2f431c8b34ee with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:08:14.551007 15974 consensus_peers.cc:597] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:14.551725 15974 consensus_peers.cc:597] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:14.562824 15974 consensus_peers.cc:597] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:14.568537 15974 consensus_peers.cc:597] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:14.573778 15974 consensus_peers.cc:597] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:14.573948 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 127770us
W20260709 13:08:14.581132 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 146277us
W20260709 13:08:14.580777 15900 consensus_peers.cc:597] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:14.584940 16010 raft_consensus.cc:1240] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 25 FOLLOWER]: Rejecting Update request from peer 8430db80a39d4868bcf7641c16de35ea for earlier term 24. Current term is 25. Ops: []
W20260709 13:08:14.588739 15900 leader_election.cc:336] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:14.581543 16533 consensus_peers.cc:597] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:14.614683 15753 catalog_manager.cc:5697] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:14.627072 23691 consensus_queue.cc:1059] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.008s
I20260709 13:08:14.637729 23691 raft_consensus.cc:3055] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:08:14.649139 15753 catalog_manager.cc:5697] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 10 to 19, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 19 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:14.638787 23691 raft_consensus.cc:740] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:08:14.660648 16533 consensus_peers.cc:597] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:14.662724 15900 consensus_peers.cc:597] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:14.680802 23018 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
W20260709 13:08:14.681445 15900 consensus_peers.cc:597] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:14.681459 23018 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:14.680708 15753 catalog_manager.cc:5697] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 14 to 16, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 16 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
W20260709 13:08:14.683238 15900 consensus_peers.cc:597] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:14.686575 15900 consensus_peers.cc:597] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:14.683579 16533 consensus_peers.cc:597] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:14.658394 23691 consensus_queue.cc:260] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:14.693466 15753 catalog_manager.cc:5697] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 11 to 19, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 19 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
I20260709 13:08:14.703404 23018 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:14.708652 15753 catalog_manager.cc:5697] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 13 to 20. New cstate: current_term: 20 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
I20260709 13:08:14.694226 23691 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:08:14.693450 15900 consensus_peers.cc:597] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:14.724634 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:14.723086 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:14.735121 23018 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:14.736299 23018 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:14.738873 23356 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:14.739749 23356 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:14.742643 23356 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:14.745811 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:14.745807 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b6e87097733428bb2167c6f308fe589" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:14.748618 15827 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:14.749987 23356 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:14.774201 23832 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:14.774964 23832 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:14.781810 23832 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:14.783048 15825 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:14.783325 23156 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:14.784225 23156 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:14.788049 23704 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:14.788733 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:14.790429 23156 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:14.791399 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd0dfa7599d64160b42b750a38e5c239" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:14.793686 23018 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:14.795886 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:14.798534 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:14.804134 23156 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:14.805219 15827 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:14.807025 23356 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:14.812125 23860 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:14.812729 23860 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:14.815995 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:14.817421 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:14.824759 23156 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:14.830802 23156 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:14.834113 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:14.836138 23356 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:14.840875 23356 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:14.843163 23356 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:14.850597 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:14.850689 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:14.852785 23860 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:14.856755 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:14.849918 15825 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:14.858896 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:14.859215 15827 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:14.864710 15825 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:14.868774 23018 raft_consensus.cc:493] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:14.868999 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:14.869560 15827 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:14.867812 23860 raft_consensus.cc:2749] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:14.869313 23018 raft_consensus.cc:515] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:14.866784 23690 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:14.872864 23018 leader_election.cc:290] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:14.877929 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:14.878942 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dc06813d5b7464fbdb39cde50f5ab48" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:14.881165 15825 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:14.870469 23704 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:08:14.881944 23018 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:14.873687 23832 raft_consensus.cc:2749] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:14.885998 15827 leader_election.cc:304] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:14.887449 23704 raft_consensus.cc:2749] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:14.981011 23356 raft_consensus.cc:493] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:08:14.990115 23690 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:08:14.987928 23356 raft_consensus.cc:515] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:15.016795 23356 leader_election.cc:290] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:15.018643 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:15.024199 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f401f200ca34a14a539f8a74675eab4" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:14.995739 23690 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:15.027434 15827 leader_election.cc:304] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:15.029568 23018 raft_consensus.cc:2749] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:15.024199 23860 raft_consensus.cc:493] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:08:15.057071 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:15.055933 23690 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:15.056977 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:15.036142 23860 raft_consensus.cc:515] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:15.061898 23690 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.062439 23690 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:15.064497 23860 leader_election.cc:290] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:15.065385 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:15.067145 15827 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:15.067932 23860 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:15.068748 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:15.068943 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679229ffe7114facacb8e3a2c4c6d226" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:15.064963 23690 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:15.070883 15827 leader_election.cc:304] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:15.071055 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:15.071623 23690 raft_consensus.cc:2749] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:15.071707 16009 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 21.
I20260709 13:08:15.075888 15827 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:15.076627 23690 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:15.076999 23690 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:15.077343 23690 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:15.080997 23860 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:15.081586 23860 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:15.083809 23860 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:15.086100 23690 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:15.086551 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:15.089037 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:15.090116 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:15.091851 23690 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:15.093037 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:15.094719 16010 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:15.093091 15827 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:15.098366 23860 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:15.102730 16010 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 22.
I20260709 13:08:15.105134 15827 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:15.106251 23690 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:08:15.106899 23690 raft_consensus.cc:697] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 22 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:08:15.110816 23356 raft_consensus.cc:493] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:15.109025 23690 consensus_queue.cc:237] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:15.112205 23356 raft_consensus.cc:515] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:15.114332 23356 leader_election.cc:290] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:15.116007 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:15.128966 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd61c62d0f0489291627ee496b25051" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:15.132688 15827 leader_election.cc:304] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:15.135426 23356 raft_consensus.cc:2749] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:15.166035 23860 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:15.166810 23860 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:15.171804 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:15.174865 23860 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:15.175060 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2be2d6f06bc47a39f26307ae9372194" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:15.177743 15827 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:15.179045 23690 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:15.198994 23690 raft_consensus.cc:493] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:08:15.199608 23690 raft_consensus.cc:515] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:15.201103 15756 catalog_manager.cc:5697] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 10 to 15. New cstate: current_term: 15 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
I20260709 13:08:15.201901 23690 leader_election.cc:290] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:15.203575 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:15.205003 23690 raft_consensus.cc:493] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:08:15.205159 23860 raft_consensus.cc:493] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:08:15.205699 23690 raft_consensus.cc:515] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:15.205817 23860 raft_consensus.cc:515] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:15.207855 23690 leader_election.cc:290] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:15.208022 23860 leader_election.cc:290] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:15.207273 15756 catalog_manager.cc:5697] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 12 to 13, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 13 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
I20260709 13:08:15.209869 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:15.210573 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:15.213456 23860 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:08:15.213814 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f4b066dd3d464699e6b00ebf1cdc44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:15.214041 23860 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:15.213991 15756 catalog_manager.cc:5697] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
I20260709 13:08:15.209806 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eabbcb31af804f2dac6af5731e18a4dd" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:15.217801 15827 leader_election.cc:304] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:15.204936 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952bd0ea3bba4d38ad3383fdf572cd01" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:15.218540 23860 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:15.228330 23860 raft_consensus.cc:2749] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:15.230940 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:15.231003 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:15.232276 15825 leader_election.cc:304] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:08:15.220175 15753 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58448 (request call id 313) took 20500 ms (client timeout 15000 ms). Trace:
I20260709 13:08:15.239868 23690 raft_consensus.cc:2749] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:08:15.234683 15753 rpcz_store.cc:269] 0709 13:07:54.720313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:54.720538 (+   225us) service_pool.cc:224] Handling call
0709 13:08:15.220073 (+20499535us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:14.838355 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:14.838954 (+   599us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:14.838987 (+    33us) write_op.cc:435] Acquired schema lock
0709 13:08:14.839004 (+    17us) tablet.cc:662] Decoding operations
0709 13:08:14.839655 (+   651us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:14.844543 (+  4888us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:14.844575 (+    32us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:08:14.845333 (+   758us) tablet.cc:701] Row locks acquired
0709 13:08:14.845367 (+    34us) write_op.cc:260] PREPARE: finished
0709 13:08:14.845534 (+   167us) write_op.cc:270] Start()
0709 13:08:14.845660 (+   126us) write_op.cc:276] Timestamp: P: 1783602494845485 usec, L: 0
0709 13:08:14.845697 (+    37us) op_driver.cc:348] REPLICATION: starting
0709 13:08:14.866607 (+ 20910us) log.cc:844] Serialized 2079 byte log entry
0709 13:08:15.024943 (+158336us) op_driver.cc:464] REPLICATION: finished
0709 13:08:15.117826 (+ 92883us) write_op.cc:301] APPLY: starting
0709 13:08:15.117906 (+    80us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:15.120993 (+  3087us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:15.121081 (+    88us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:15.123772 (+  2691us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:15.124007 (+   235us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=10,mrs_lookups=0
0709 13:08:15.124086 (+    79us) write_op.cc:312] APPLY: finished
0709 13:08:15.124350 (+   264us) log.cc:844] Serialized 57 byte log entry
0709 13:08:15.125067 (+   717us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:15.125160 (+    93us) write_op.cc:454] Released schema lock
0709 13:08:15.125194 (+    34us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:15.125597 (+   403us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:15.125628 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":20440,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":91668,"apply.run_cpu_time_us":7204,"apply.run_wall_time_us":7745,"cfile_cache_hit":22,"cfile_cache_hit_bytes":25077,"num_ops":5,"prepare.queue_time_us":123380,"prepare.run_cpu_time_us":12427,"prepare.run_wall_time_us":59844,"raft.queue_time_us":54295,"raft.run_cpu_time_us":31545,"raft.run_wall_time_us":90338,"replication_time_us":159986,"thread_start_us":187029,"threads_started":4,"wal-append.queue_time_us":56046}]]}
I20260709 13:08:15.245890 15827 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:15.246768 23356 raft_consensus.cc:2749] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:15.247211 15827 leader_election.cc:304] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:15.248883 23356 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:15.257901 23018 raft_consensus.cc:2749] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:08:15.259332 23356 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:15.261759 23356 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:15.262717 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:15.263375 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffbfc0e667484157953a66ae2b697013" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:15.270684 23860 raft_consensus.cc:493] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:08:15.271299 23860 raft_consensus.cc:515] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:15.273697 23880 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:15.274334 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:15.274312 23880 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:15.277554 23880 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:15.274523 23860 leader_election.cc:290] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:15.279070 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:15.279881 16010 raft_consensus.cc:2368] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 21. Current term is 22.
I20260709 13:08:15.281096 15902 leader_election.cc:400] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 21. Current term is 22.
I20260709 13:08:15.281427 15902 leader_election.cc:403] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:15.267135 15827 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:15.287541 23814 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:15.296705 23018 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:15.287703 23880 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:08:15.297290 23018 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:15.295997 23356 raft_consensus.cc:2749] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:15.292708 23814 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:15.309926 23814 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:15.301246 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c04fe1420dfe4602909deef4baa3f115" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:15.314678 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:15.316963 15827 leader_election.cc:304] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:15.318539 23356 raft_consensus.cc:2749] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:15.325827 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:08:15.330974 15900 leader_election.cc:336] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:15.334492 15827 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:15.336148 15753 catalog_manager.cc:5697] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 24 to 27, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 27 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:15.337081 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:15.338518 23814 raft_consensus.cc:2749] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:15.339634 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32207d388e0e4185898655b74ae976b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:15.340078 23018 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:15.337692 15753 catalog_manager.cc:5697] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 23 to 24, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 24 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:15.342280 15753 catalog_manager.cc:5697] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 13 to 24, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 24 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:15.343351 15827 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:15.344027 23018 raft_consensus.cc:2749] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:15.343988 15753 catalog_manager.cc:5697] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 18 to 22, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 22 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:15.346176 23018 raft_consensus.cc:493] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:08:15.346010 15753 catalog_manager.cc:5697] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 18 to 20, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 20 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:15.346722 23018 raft_consensus.cc:515] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:15.349750 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:15.352313 15923 tablet_service.cc:1558] Processing DeleteTablet for tablet 679229ffe7114facacb8e3a2c4c6d226 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 9)) from {username='slave'} at 127.0.0.1:58226
I20260709 13:08:15.347461 15753 catalog_manager.cc:5697] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 18 to 21, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 21 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:15.354532 15753 catalog_manager.cc:5697] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 15 to 17, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 17 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:15.357765 23880 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251) with higher term. Message: Invalid argument: T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea for earlier term 21. Current term is 22.
W20260709 13:08:15.352618 15756 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58434 (request call id 346) took 40984 ms (client timeout 15000 ms). Trace:
W20260709 13:08:15.358898 15756 rpcz_store.cc:269] 0709 13:07:34.410894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:34.430257 (+ 19363us) service_pool.cc:224] Handling call
0709 13:08:15.352495 (+40922238us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:15.237029 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:15.237597 (+   568us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:15.237635 (+    38us) write_op.cc:435] Acquired schema lock
0709 13:08:15.237655 (+    20us) tablet.cc:662] Decoding operations
0709 13:08:15.238283 (+   628us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:15.238348 (+    65us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:15.238381 (+    33us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:08:15.238737 (+   356us) tablet.cc:701] Row locks acquired
0709 13:08:15.238758 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:08:15.238912 (+   154us) write_op.cc:270] Start()
0709 13:08:15.239030 (+   118us) write_op.cc:276] Timestamp: P: 1783602495238864 usec, L: 0
0709 13:08:15.239052 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:08:15.239756 (+   704us) log.cc:844] Serialized 1308 byte log entry
0709 13:08:15.248525 (+  8769us) op_driver.cc:464] REPLICATION: finished
0709 13:08:15.251161 (+  2636us) write_op.cc:301] APPLY: starting
0709 13:08:15.251249 (+    88us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:15.253053 (+  1804us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:15.253160 (+   107us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:15.267077 (+ 13917us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:15.267237 (+   160us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=6,mrs_lookups=0
0709 13:08:15.267283 (+    46us) write_op.cc:312] APPLY: finished
0709 13:08:15.267493 (+   210us) log.cc:844] Serialized 41 byte log entry
0709 13:08:15.267645 (+   152us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:15.267763 (+   118us) write_op.cc:454] Released schema lock
0709 13:08:15.267808 (+    45us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:15.268166 (+   358us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:15.268193 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":789887,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":2383,"apply.run_cpu_time_us":5109,"apply.run_wall_time_us":16983,"cfile_cache_hit":12,"cfile_cache_hit_bytes":12570,"num_ops":3,"prepare.queue_time_us":114,"prepare.run_cpu_time_us":3066,"prepare.run_wall_time_us":11241,"replication_time_us":9338}]]}
I20260709 13:08:15.365476 15744 catalog_manager.cc:5028] TS 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): tablet 679229ffe7114facacb8e3a2c4c6d226 (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:08:15.374998 23018 leader_election.cc:290] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:15.378754 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78530df4cdb742eda67781671b0431c6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:15.381131 15827 leader_election.cc:304] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:15.384097 23814 raft_consensus.cc:2749] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:15.399641 23690 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:15.401255 23690 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:15.403332 23690 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:15.403928 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:15.413177 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d9c19e253ce4d53bb9e2f2241d3cafc" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:15.415439 15827 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:15.417434 23018 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:08:15.435602 15751 catalog_manager.cc:5697] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 15 to 20, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 20 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
I20260709 13:08:15.437496 15751 catalog_manager.cc:5697] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
I20260709 13:08:15.441505 15751 catalog_manager.cc:5697] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 19 to 23. New cstate: current_term: 23 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
I20260709 13:08:15.443198 15751 catalog_manager.cc:5697] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 19 to 21, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 21 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:15.445013 15751 catalog_manager.cc:5697] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:15.449172 15751 catalog_manager.cc:5697] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 13 to 20, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 20 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
I20260709 13:08:15.450443 15751 catalog_manager.cc:5697] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 16 to 17, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 17 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
I20260709 13:08:15.451310 15751 catalog_manager.cc:5697] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 19 to 21, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 21 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
I20260709 13:08:15.453176 15751 catalog_manager.cc:5697] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 16 to 19. New cstate: current_term: 19 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:15.455384 15751 catalog_manager.cc:5697] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:15.465910 15751 catalog_manager.cc:5697] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 19 to 20, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 31882606e87447fb9a36bbf625129fe8 (127.14.103.1). New cstate: current_term: 20 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:15.517836 16009 raft_consensus.cc:1275] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:08:15.522341 23018 consensus_queue.cc:1048] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:08:15.644665 15753 catalog_manager.cc:5697] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: config changed from index 3 to 9, VOTER 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) evicted. New cstate: current_term: 16 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:15.677742 15930 raft_consensus.cc:1275] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Refusing update from remote peer 31882606e87447fb9a36bbf625129fe8: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:08:15.681622 23018 consensus_queue.cc:1048] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:08:15.705541 15923 tablet_service.cc:1558] Processing DeleteTablet for tablet 15301761b2ec47fe82ae2f431c8b34ee with delete_type TABLET_DATA_TOMBSTONED (TS 8430db80a39d4868bcf7641c16de35ea not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:58226
I20260709 13:08:15.707645 23888 tablet_replica.cc:333] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:08:15.708920 23888 raft_consensus.cc:2243] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:15.709954 23888 raft_consensus.cc:2272] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:15.713727 23888 ts_tablet_manager.cc:1916] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:08:15.708815 15751 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58426 (request call id 348) took 2909 ms. Trace:
I20260709 13:08:15.719497 15751 rpcz_store.cc:276] 0709 13:08:12.799279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:12.843338 (+ 44059us) service_pool.cc:224] Handling call
0709 13:08:15.708739 (+2865401us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:15.500630 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:15.501169 (+   539us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:15.501204 (+    35us) write_op.cc:435] Acquired schema lock
0709 13:08:15.501217 (+    13us) tablet.cc:662] Decoding operations
0709 13:08:15.502269 (+  1052us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:15.502323 (+    54us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:15.502353 (+    30us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:08:15.503269 (+   916us) tablet.cc:701] Row locks acquired
0709 13:08:15.503287 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:08:15.503445 (+   158us) write_op.cc:270] Start()
0709 13:08:15.528184 (+ 24739us) write_op.cc:276] Timestamp: P: 1783602495503397 usec, L: 0
0709 13:08:15.528212 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 13:08:15.528774 (+   562us) log.cc:844] Serialized 4389 byte log entry
0709 13:08:15.530692 (+  1918us) op_driver.cc:464] REPLICATION: finished
0709 13:08:15.531166 (+   474us) write_op.cc:301] APPLY: starting
0709 13:08:15.531247 (+    81us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:15.536970 (+  5723us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:15.537064 (+    94us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:15.575633 (+ 38569us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:15.575896 (+   263us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=22,mrs_lookups=0
0709 13:08:15.575948 (+    52us) write_op.cc:312] APPLY: finished
0709 13:08:15.576274 (+   326us) log.cc:844] Serialized 105 byte log entry
0709 13:08:15.576468 (+   194us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:15.576613 (+   145us) write_op.cc:454] Released schema lock
0709 13:08:15.576682 (+    69us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:15.591524 (+ 14842us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:15.591562 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1259,"reported_tablets":38,"child_traces":[["op",{"apply.queue_time_us":256,"apply.run_cpu_time_us":14081,"apply.run_wall_time_us":60273,"cfile_cache_hit":44,"cfile_cache_hit_bytes":46090,"num_ops":11,"prepare.queue_time_us":109,"prepare.run_cpu_time_us":3876,"prepare.run_wall_time_us":28452,"replication_time_us":2368,"spinlock_wait_cycles":55040}]]}
I20260709 13:08:15.731230 14748 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:15.736331 14748 tablet_replica.cc:333] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:15.737465 14748 raft_consensus.cc:2243] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:08:15.741323 14748 raft_consensus.cc:2272] T 975ecd3d27dd45ba9a554ac0ee591e0e P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:15.752353 15752 catalog_manager.cc:5697] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:15.753825 14748 tablet_replica.cc:333] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:15.766304 14748 raft_consensus.cc:2243] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:15.768697 14748 raft_consensus.cc:2272] T 7b6e87097733428bb2167c6f308fe589 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:15.773793 23356 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.774283 23356 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:15.777841 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:15.778169 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:15.778731 15934 raft_consensus.cc:2410] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 26 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 14.
I20260709 13:08:15.778748 16009 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 25.
I20260709 13:08:15.777304 23356 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:15.781060 15827 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:08:15.781827 23356 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:08:15.782220 23356 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:15.782588 23356 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:08:15.786334 14748 tablet_replica.cc:333] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:15.806902 14748 raft_consensus.cc:2243] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:15.792553 23356 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:15.811220 14748 raft_consensus.cc:2272] T dd0dfa7599d64160b42b750a38e5c239 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:15.815869 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:15.816517 15934 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:08:15.821240 23356 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 26 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:15.824011 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:15.824632 16010 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:08:15.827383 15934 raft_consensus.cc:2410] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 31882606e87447fb9a36bbf625129fe8 for term 26 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 14.
I20260709 13:08:15.838747 16010 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31882606e87447fb9a36bbf625129fe8 in term 26.
I20260709 13:08:15.847412 15827 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:15.851385 14748 tablet_replica.cc:333] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:15.857379 23156 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:08:15.858016 23156 raft_consensus.cc:697] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 26 LEADER]: Becoming Leader. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:08:15.858862 23156 consensus_queue.cc:237] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:15.870756 14748 raft_consensus.cc:2243] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:08:15.875025 14748 raft_consensus.cc:2272] T 8849099536af497bae91c72da893681c P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:15.878655 23888 ts_tablet_manager.cc:1929] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 16.8
I20260709 13:08:15.879019 23882 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.884943 23882 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:15.887360 23882 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:15.890168 23888 log.cc:1199] T 15301761b2ec47fe82ae2f431c8b34ee P 8430db80a39d4868bcf7641c16de35ea: Deleting WAL directory at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0/minicluster-data/ts-1-root/wals/15301761b2ec47fe82ae2f431c8b34ee
I20260709 13:08:15.896589 15744 catalog_manager.cc:5028] TS 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): tablet 15301761b2ec47fe82ae2f431c8b34ee (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:08:15.899721 14748 tablet_replica.cc:333] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:15.903321 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:15.930924 16010 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 26.
I20260709 13:08:15.931106 14748 raft_consensus.cc:2243] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:15.932046 14748 raft_consensus.cc:2272] T d2be2d6f06bc47a39f26307ae9372194 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:15.932204 15902 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:08:15.933023 23691 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:08:15.933444 23691 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:15.933909 23691 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:08:15.933795 15900 leader_election.cc:336] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:15.942683 14748 tablet_replica.cc:333] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:16.026438 14748 raft_consensus.cc:2243] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:16.027282 14748 raft_consensus.cc:2272] T 6dc06813d5b7464fbdb39cde50f5ab48 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:16.090068 14748 tablet_replica.cc:333] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:16.124687 14748 raft_consensus.cc:2243] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:08:16.091260 23691 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:16.295498 14748 raft_consensus.cc:2272] T 4fcbf678894c4a1eb80c68b5b53acaef P 31882606e87447fb9a36bbf625129fe8 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:16.316238 23691 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 27 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:16.342697 15754 catalog_manager.cc:5697] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:16.359925 14748 tablet_replica.cc:333] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:16.361061 14748 raft_consensus.cc:2243] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:08:16.361924 14748 raft_consensus.cc:2272] T 260d22f804a345ed8f055536994fcb14 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:16.408604 15752 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58426 (request call id 346) took 37457 ms (client timeout 15000 ms). Trace:
I20260709 13:08:16.431516 23690 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:16.440285 23690 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:16.442505 23690 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:16.445497 14748 tablet_replica.cc:333] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:16.446923 14748 raft_consensus.cc:2243] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:16.450243 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab61d516f4754d22b660c6fa5d052e3c" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:16.450888 16010 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:08:16.447583 14748 raft_consensus.cc:2272] T bbd61c62d0f0489291627ee496b25051 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:16.466547 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:16.480098 15937 raft_consensus.cc:1240] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 27 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 26. Current term is 27. Ops: []
I20260709 13:08:16.481177 16010 raft_consensus.cc:2468] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 27.
W20260709 13:08:16.476636 15900 leader_election.cc:336] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:16.481815 16009 raft_consensus.cc:1240] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 27 FOLLOWER]: Rejecting Update request from peer 31882606e87447fb9a36bbf625129fe8 for earlier term 26. Current term is 27. Ops: []
I20260709 13:08:16.483142 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471dbd2e16264fe5837e8b7ea3601f94" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:16.475643 23905 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:16.487421 23905 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:16.489748 23905 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:08:16.419397 15752 rpcz_store.cc:269] 0709 13:07:38.960813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:38.970017 (+  9204us) service_pool.cc:224] Handling call
0709 13:08:16.408473 (+37438456us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:15.879332 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:15.879866 (+   534us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:15.879900 (+    34us) write_op.cc:435] Acquired schema lock
0709 13:08:15.879916 (+    16us) tablet.cc:662] Decoding operations
0709 13:08:15.880256 (+   340us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:15.880309 (+    53us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:15.880338 (+    29us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:08:15.880515 (+   177us) tablet.cc:701] Row locks acquired
0709 13:08:15.880532 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:08:15.880714 (+   182us) write_op.cc:270] Start()
0709 13:08:15.880828 (+   114us) write_op.cc:276] Timestamp: P: 1783602495880620 usec, L: 0
0709 13:08:15.880854 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:08:15.881433 (+   579us) log.cc:844] Serialized 538 byte log entry
0709 13:08:15.883291 (+  1858us) op_driver.cc:464] REPLICATION: finished
0709 13:08:15.884015 (+   724us) write_op.cc:301] APPLY: starting
0709 13:08:15.884097 (+    82us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:15.884944 (+   847us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:15.885036 (+    92us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:15.885751 (+   715us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:15.885918 (+   167us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 13:08:15.885969 (+    51us) write_op.cc:312] APPLY: finished
0709 13:08:15.886154 (+   185us) log.cc:844] Serialized 25 byte log entry
0709 13:08:15.886334 (+   180us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:15.886422 (+    88us) write_op.cc:454] Released schema lock
0709 13:08:15.886465 (+    43us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:15.886804 (+   339us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:15.886835 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":200089,"reported_tablets":46,"spinlock_wait_cycles":3456,"child_traces":[["op",{"apply.queue_time_us":508,"apply.run_cpu_time_us":2760,"apply.run_wall_time_us":2760,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4190,"num_ops":1,"prepare.queue_time_us":2282,"prepare.run_cpu_time_us":2352,"prepare.run_wall_time_us":2352,"replication_time_us":2319}]]}
I20260709 13:08:16.502971 14748 tablet_replica.cc:333] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
W20260709 13:08:16.506201 15900 consensus_peers.cc:597] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:16.519717 14748 raft_consensus.cc:2243] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:16.520398 14748 raft_consensus.cc:2272] T 679229ffe7114facacb8e3a2c4c6d226 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:16.495873 23814 consensus_queue.cc:1059] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.012s
I20260709 13:08:16.521205 23814 consensus_queue.cc:1243] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [LEADER]: Peer 8430db80a39d4868bcf7641c16de35ea log is divergent from this leader: its last log entry 24.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:08:16.521749 23814 raft_consensus.cc:3055] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:08:16.522114 23814 raft_consensus.cc:740] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 31882606e87447fb9a36bbf625129fe8, State: Running, Role: LEADER
I20260709 13:08:16.522981 23814 consensus_queue.cc:260] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:16.524199 23814 raft_consensus.cc:3060] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:08:16.543118 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:16.550539 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3e05ffd87b4bbcb04b315d3bfdf8ed" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:16.551854 14748 tablet_replica.cc:333] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:16.554278 14748 raft_consensus.cc:2243] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:08:16.569398 14748 raft_consensus.cc:2272] T e236a7bf118740c9b3d88227bed5670e P 31882606e87447fb9a36bbf625129fe8 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:16.576231 15902 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:08:16.582948 23904 raft_consensus.cc:2804] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:08:16.583535 23904 raft_consensus.cc:697] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 27 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:08:16.584493 23904 consensus_queue.cc:237] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:16.586643 15827 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:16.588965 23156 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:08:16.591142 15974 consensus_peers.cc:597] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:16.627966 15754 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58426 (request call id 345) took 61671 ms (client timeout 15000 ms). Trace:
I20260709 13:08:16.593955 14748 tablet_replica.cc:333] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:16.602814 15827 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:08:16.609767 16533 consensus_peers.cc:597] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:16.629647 16533 consensus_peers.cc:597] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:16.630821 23814 raft_consensus.cc:2749] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:08:16.636754 23156 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:16.637465 23156 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:16.640709 23156 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:08:16.641346 15900 consensus_peers.cc:597] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:16.642048 14748 raft_consensus.cc:2243] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:16.642865 14748 pending_rounds.cc:70] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8: Trying to abort 1 pending ops.
W20260709 13:08:16.642351 15900 consensus_peers.cc:597] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:16.643450 14748 pending_rounds.cc:77] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8: Aborting op as it isn't in flight: id { term: 24 index: 17 } timestamp: 7305635807936839680 op_type: NO_OP noop_request { }
I20260709 13:08:16.643987 14748 raft_consensus.cc:2889] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:08:16.644402 14748 raft_consensus.cc:2272] T eabbcb31af804f2dac6af5731e18a4dd P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:16.684213 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:08:16.698360 15974 consensus_peers.cc:597] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:16.698359 16533 consensus_peers.cc:597] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:16.700120 15974 consensus_peers.cc:597] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:16.628556 15754 rpcz_store.cc:269] 0709 13:07:14.960620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.969616 (+  8996us) service_pool.cc:224] Handling call
0709 13:08:16.627868 (+61658252us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:16.372353 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:16.372885 (+   532us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:16.372917 (+    32us) write_op.cc:435] Acquired schema lock
0709 13:08:16.372930 (+    13us) tablet.cc:662] Decoding operations
0709 13:08:16.373325 (+   395us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:16.373382 (+    57us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:16.373410 (+    28us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:08:16.373584 (+   174us) tablet.cc:701] Row locks acquired
0709 13:08:16.373597 (+    13us) write_op.cc:260] PREPARE: finished
0709 13:08:16.373732 (+   135us) write_op.cc:270] Start()
0709 13:08:16.373886 (+   154us) write_op.cc:276] Timestamp: P: 1783602496373688 usec, L: 0
0709 13:08:16.373906 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:08:16.374541 (+   635us) log.cc:844] Serialized 538 byte log entry
0709 13:08:16.376869 (+  2328us) op_driver.cc:464] REPLICATION: finished
0709 13:08:16.377391 (+   522us) write_op.cc:301] APPLY: starting
0709 13:08:16.377487 (+    96us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:16.378477 (+   990us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:16.378592 (+   115us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:16.379466 (+   874us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:16.379666 (+   200us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 13:08:16.381988 (+  2322us) write_op.cc:312] APPLY: finished
0709 13:08:16.386845 (+  4857us) log.cc:844] Serialized 25 byte log entry
0709 13:08:16.387019 (+   174us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:16.387110 (+    91us) write_op.cc:454] Released schema lock
0709 13:08:16.387156 (+    46us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:16.387539 (+   383us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:16.387572 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1216915,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":281,"apply.run_cpu_time_us":3314,"apply.run_wall_time_us":10097,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4190,"num_ops":1,"prepare.queue_time_us":124,"prepare.run_cpu_time_us":2464,"prepare.run_wall_time_us":2465,"replication_time_us":2812,"spinlock_wait_cycles":1920}]]}
I20260709 13:08:16.706254 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d0cc5018ee4a9fb3d196c783814c44" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:08:16.709790 16533 consensus_peers.cc:597] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:16.721545 15827 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:16.723052 23018 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:16.727996 14748 tablet_replica.cc:333] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
W20260709 13:08:16.731578 15974 consensus_peers.cc:597] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:16.734426 14748 raft_consensus.cc:2243] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:16.732731 23690 raft_consensus.cc:493] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:08:16.739157 14748 raft_consensus.cc:2272] T 32207d388e0e4185898655b74ae976b1 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:16.739104 23690 raft_consensus.cc:515] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:08:16.740535 15755 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58426 (request call id 344) took 87253 ms (client timeout 15000 ms). Trace:
W20260709 13:08:16.740998 15755 rpcz_store.cc:269] 0709 13:06:49.486925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:49.488460 (+  1535us) service_pool.cc:224] Handling call
0709 13:06:49.883258 (+394798us) spinlock_profiling.cc:243] Waited 1.31 ms on lock 0x7b5400030890. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73e7143db1 00007f73e7283b71 00007f73e72840b2 00007f73f01fda89 00007f73f0195da6 00007f73f0167088 00007f73f026537e 00007f73ee7262a4 00007f73ee726232 00007f73ee726161 00007f73ee7260dc
0709 13:08:16.740323 (+86857065us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":152510,"reported_tablets":46,"spinlock_wait_cycles":3023616}
I20260709 13:08:16.742309 23690 leader_election.cc:290] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:16.742697 14748 tablet_replica.cc:333] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:16.743750 14748 raft_consensus.cc:2243] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:16.744410 14748 raft_consensus.cc:2272] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:16.748800 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:16.751005 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40e7e6076e8d475da059aad457e0e8b1" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:16.753458 15827 leader_election.cc:304] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:16.754742 23018 raft_consensus.cc:493] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:16.755302 23018 raft_consensus.cc:515] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:16.754742 23907 raft_consensus.cc:2749] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:16.757712 23018 leader_election.cc:290] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:16.758426 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:16.760926 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741afa89185645afa8b1c82d31a75748" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:16.776154 15827 leader_election.cc:304] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:16.747581 14748 tablet_replica.cc:333] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:16.778158 23690 raft_consensus.cc:2749] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:08:16.781312 16533 consensus_peers.cc:597] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:16.778277 15758 catalog_manager.cc:5697] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 18 to 19, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 19 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:16.779048 14748 raft_consensus.cc:2243] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:16.795441 14748 raft_consensus.cc:2272] T 2f401f200ca34a14a539f8a74675eab4 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:16.800448 14748 tablet_replica.cc:333] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:16.801298 14748 raft_consensus.cc:2243] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:08:16.802392 14748 raft_consensus.cc:2272] T e71b32319bee4f979532b3ac6a16f2bd P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:16.811405 23907 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:16.812034 23907 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:16.814311 23907 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:16.816457 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:16.818876 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c766f90082d54f299b03f3e9d75d2db0" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:16.821549 15827 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:16.822973 23690 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:16.825312 14748 tablet_replica.cc:333] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:16.826454 14748 raft_consensus.cc:2243] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:08:16.827701 14748 raft_consensus.cc:2272] T c65db3107b664e9c8ce9be2cd2cb36c6 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:16.830813 14748 tablet_replica.cc:333] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:16.831647 14748 raft_consensus.cc:2243] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:16.836710 14748 raft_consensus.cc:2272] T 73f4b066dd3d464699e6b00ebf1cdc44 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:16.840149 15900 consensus_peers.cc:597] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:16.845294 14748 tablet_replica.cc:333] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:16.847033 14748 raft_consensus.cc:2243] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 24 LEADER]: Raft consensus shutting down.
W20260709 13:08:16.855854 15974 consensus_peers.cc:597] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:16.856117 14748 raft_consensus.cc:2272] T 0a10bba95cd1421aa3fc332639e97319 P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:16.859417 14748 tablet_replica.cc:333] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:16.892181 14748 raft_consensus.cc:2243] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:16.892853 14748 pending_rounds.cc:70] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8: Trying to abort 1 pending ops.
I20260709 13:08:16.893138 14748 pending_rounds.cc:77] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8: Aborting op as it isn't in flight: id { term: 26 index: 15 } timestamp: 7305635823058763776 op_type: NO_OP noop_request { }
I20260709 13:08:16.893491 14748 raft_consensus.cc:2889] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:08:16.893834 14748 raft_consensus.cc:2272] T ab61d516f4754d22b660c6fa5d052e3c P 31882606e87447fb9a36bbf625129fe8 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:16.896605 14748 tablet_replica.cc:333] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:16.897491 14748 raft_consensus.cc:2243] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:16.903885 14748 raft_consensus.cc:2272] T 78530df4cdb742eda67781671b0431c6 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:16.909827 14748 tablet_replica.cc:333] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:16.910567 14748 raft_consensus.cc:2243] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:08:16.911612 14748 raft_consensus.cc:2272] T ac847593d6724522bda12a2207f91c31 P 31882606e87447fb9a36bbf625129fe8 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:16.920029 23906 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:16.920717 23906 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:16.922947 14748 tablet_replica.cc:333] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:16.923995 14748 raft_consensus.cc:2243] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:16.924651 14748 raft_consensus.cc:2272] T 741afa89185645afa8b1c82d31a75748 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:16.927847 15974 leader_election.cc:336] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:16.928944 23906 leader_election.cc:290] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:16.927910 14748 tablet_replica.cc:333] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
W20260709 13:08:16.936681 15900 consensus_peers.cc:597] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:16.938308 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:16.953653 14748 raft_consensus.cc:2243] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:16.954315 14748 raft_consensus.cc:2272] T 952bd0ea3bba4d38ad3383fdf572cd01 P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:16.945943 16533 consensus_peers.cc:597] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:16.964679 15974 consensus_peers.cc:597] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:16.964869 15976 leader_election.cc:304] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:16.974922 23895 raft_consensus.cc:2749] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:16.976125 14748 tablet_replica.cc:333] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
W20260709 13:08:16.977479 15900 consensus_peers.cc:597] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:16.976992 14748 raft_consensus.cc:2243] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:08:16.999267 14748 raft_consensus.cc:2272] T 83a1b79cdcf046c78453695c798a14a5 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:17.008039 23907 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:17.008703 23907 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:17.011404 23907 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:16.995564 15900 consensus_peers.cc:597] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:17.013033 15900 consensus_peers.cc:597] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:17.013610 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:17.013610 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b72a30273dd49e883c237c7e5ad237b" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:17.016491 23908 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:17.033708 23908 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:17.035836 23908 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:08:17.024782 16533 consensus_peers.cc:597] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:17.037781 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:08:17.041476 15974 leader_election.cc:336] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:17.041712 23907 raft_consensus.cc:493] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader eb1e0493af1e46dbb6dec7e25525bab5)
I20260709 13:08:17.041954 15974 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:17.042351 23907 raft_consensus.cc:515] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:17.043082 23908 raft_consensus.cc:2749] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:17.044552 23907 leader_election.cc:290] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:17.046401 15827 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:17.047686 23156 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:17.048588 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:17.049081 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888cce851f074f3c969de10b02ff95ad" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:17.050657 23156 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:17.051159 23156 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:17.052459 15827 leader_election.cc:304] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:17.053256 23690 raft_consensus.cc:2749] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:17.025138 14748 tablet_replica.cc:333] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:17.055526 14748 raft_consensus.cc:2243] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:17.056026 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:17.059590 23156 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:17.056198 14748 raft_consensus.cc:2272] T ffbfc0e667484157953a66ae2b697013 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:17.065145 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:17.066924 15827 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:08:17.067540 16533 consensus_peers.cc:597] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:17.068027 23156 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:17.079059 14748 tablet_replica.cc:333] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:17.084463 16006 raft_consensus.cc:1275] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 27 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
W20260709 13:08:17.081621 16533 consensus_peers.cc:597] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:17.086047 23902 consensus_queue.cc:1048] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:08:17.088310 14748 raft_consensus.cc:2243] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Raft consensus shutting down.
W20260709 13:08:17.089766 15900 consensus_peers.cc:597] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:17.090986 14748 raft_consensus.cc:2272] T 91d0cc5018ee4a9fb3d196c783814c44 P 31882606e87447fb9a36bbf625129fe8 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:17.102125 14748 tablet_replica.cc:333] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:17.103008 14748 raft_consensus.cc:2243] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:17.107908 15923 tablet_service.cc:1558] Processing DeleteTablet for tablet 15301761b2ec47fe82ae2f431c8b34ee with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 9)) from {username='slave'} at 127.0.0.1:58226
W20260709 13:08:17.108937 15758 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58434 (request call id 347) took 18626 ms (client timeout 15000 ms). Trace:
I20260709 13:08:17.109886 15919 tablet_service.cc:1558] Processing DeleteTablet for tablet 679229ffe7114facacb8e3a2c4c6d226 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 9)) from {username='slave'} at 127.0.0.1:58226
W20260709 13:08:17.109453 15758 rpcz_store.cc:269] 0709 13:07:58.482371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:58.482839 (+   468us) service_pool.cc:224] Handling call
0709 13:08:16.813417 (+18330578us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:16.813450 (+    33us) write_op.cc:454] Released schema lock
0709 13:08:17.108862 (+295412us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:16.802944 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:16.803487 (+   543us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:16.803521 (+    34us) write_op.cc:435] Acquired schema lock
0709 13:08:16.803538 (+    17us) tablet.cc:662] Decoding operations
0709 13:08:16.803901 (+   363us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:16.803965 (+    64us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:16.803996 (+    31us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:08:16.804193 (+   197us) tablet.cc:701] Row locks acquired
0709 13:08:16.804212 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:08:16.804525 (+   313us) write_op.cc:270] Start()
0709 13:08:16.804835 (+   310us) write_op.cc:276] Timestamp: P: 1783602496804407 usec, L: 0
0709 13:08:16.804881 (+    46us) op_driver.cc:348] REPLICATION: starting
0709 13:08:16.805744 (+   863us) log.cc:844] Serialized 538 byte log entry
0709 13:08:16.808503 (+  2759us) op_driver.cc:464] REPLICATION: finished
0709 13:08:16.809340 (+   837us) write_op.cc:301] APPLY: starting
0709 13:08:16.809424 (+    84us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:16.810261 (+   837us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:16.810358 (+    97us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:16.811197 (+   839us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:16.811379 (+   182us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 13:08:16.811434 (+    55us) write_op.cc:312] APPLY: finished
0709 13:08:16.811618 (+   184us) log.cc:844] Serialized 25 byte log entry
0709 13:08:16.811809 (+   191us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:16.811897 (+    88us) write_op.cc:454] Released schema lock
0709 13:08:16.811944 (+    47us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":258769,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":527,"apply.run_cpu_time_us":2899,"apply.run_wall_time_us":2899,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4190,"num_ops":1,"prepare.queue_time_us":1326,"prepare.run_cpu_time_us":3090,"prepare.run_wall_time_us":5313,"replication_time_us":3505}]]}
I20260709 13:08:17.122448 14748 raft_consensus.cc:2272] T c04fe1420dfe4602909deef4baa3f115 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:17.132174 14748 tablet_replica.cc:333] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:17.133507 14748 raft_consensus.cc:2243] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:17.135943 15759 catalog_manager.cc:5697] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 9 to 20. New cstate: current_term: 20 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
I20260709 13:08:17.152045 23907 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:17.155859 23907 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:17.158298 23907 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:17.161388 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:17.162441 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e1a3c90a16e428fb2b328aff66b4ea6" candidate_uuid: "31882606e87447fb9a36bbf625129fe8" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:17.171180 15827 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31882606e87447fb9a36bbf625129fe8; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:17.172000 23907 raft_consensus.cc:2749] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:17.172900 14748 raft_consensus.cc:2272] T 9b72a30273dd49e883c237c7e5ad237b P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:17.173830 23899 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:17.174361 23899 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:17.176918 15760 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58448 (request call id 314) took 5178 ms. Trace:
I20260709 13:08:17.177047 15744 catalog_manager.cc:5028] TS 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): tablet 15301761b2ec47fe82ae2f431c8b34ee (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:08:17.178872 15744 catalog_manager.cc:5028] TS 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): tablet 679229ffe7114facacb8e3a2c4c6d226 (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:08:17.179452 15760 rpcz_store.cc:276] 0709 13:08:11.955098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.955294 (+   196us) service_pool.cc:224] Handling call
0709 13:08:17.133678 (+5178384us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":94519,"reported_tablets":46}
W20260709 13:08:17.185110 16533 leader_election.cc:336] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:17.185688 23899 leader_election.cc:290] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:17.182130 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:17.195729 16011 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 19.
I20260709 13:08:17.200416 16535 leader_election.cc:304] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:08:17.204809 23899 raft_consensus.cc:2804] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:17.205276 23899 raft_consensus.cc:493] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:17.205650 23899 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:17.155491 15759 catalog_manager.cc:5697] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 19 to 24. New cstate: current_term: 24 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:17.218662 23904 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:17.219364 23904 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:17.222059 23904 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:17.226541 15900 leader_election.cc:336] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:17.228732 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:17.229496 16005 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 20.
I20260709 13:08:17.232451 15902 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:08:17.247252 23904 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:17.247749 23904 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:17.248024 14748 tablet_replica.cc:333] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:17.248160 23904 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:08:17.251833 15974 consensus_peers.cc:597] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:17.253087 14748 raft_consensus.cc:2243] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:08:17.275884 14748 raft_consensus.cc:2272] T 7c0fef632f9a4485838855b553a79163 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:17.282037 23899 raft_consensus.cc:515] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:17.293922 23899 leader_election.cc:290] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:17.297552 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8849099536af497bae91c72da893681c" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:17.298442 16006 raft_consensus.cc:3060] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:08:17.301271 16533 leader_election.cc:336] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:17.287369 14748 tablet_replica.cc:333] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:17.308799 14748 raft_consensus.cc:2243] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:17.309496 14748 raft_consensus.cc:2272] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:17.311807 16006 raft_consensus.cc:2468] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 20.
I20260709 13:08:17.318178 14748 tablet_replica.cc:333] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:17.326704 14748 raft_consensus.cc:2243] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:17.326929 16535 leader_election.cc:304] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:08:17.327420 14748 raft_consensus.cc:2272] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:17.328238 23762 raft_consensus.cc:2804] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:17.330488 14748 tablet_replica.cc:333] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:17.331373 14748 raft_consensus.cc:2243] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:08:17.332387 14748 raft_consensus.cc:2272] T 15301761b2ec47fe82ae2f431c8b34ee P 31882606e87447fb9a36bbf625129fe8 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:17.321635 23904 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:17.337589 23904 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:17.342531 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:17.343397 16006 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:17.345353 14748 tablet_replica.cc:333] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:17.352638 15923 tablet_service.cc:1558] Processing DeleteTablet for tablet 15301761b2ec47fe82ae2f431c8b34ee with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 9)) from {username='slave'} at 127.0.0.1:58226
I20260709 13:08:17.354529 15744 catalog_manager.cc:5028] TS 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): tablet 15301761b2ec47fe82ae2f431c8b34ee (table test-workload [id=88d8dc2b76ea4d919f35c7c1e6af81f4]) successfully deleted
I20260709 13:08:17.351809 14748 raft_consensus.cc:2243] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:08:17.356709 14748 raft_consensus.cc:2272] T dfeaf1f6660d48848ae69710739427a0 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:17.362782 14748 tablet_replica.cc:333] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
W20260709 13:08:17.363827 15900 leader_election.cc:336] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:17.370798 16006 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 21.
I20260709 13:08:17.372042 15759 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58434 (request call id 348) took 2997 ms. Trace:
I20260709 13:08:17.385459 15759 rpcz_store.cc:276] 0709 13:08:14.375061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:14.378751 (+  3690us) service_pool.cc:224] Handling call
0709 13:08:17.352928 (+2974177us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:17.223989 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:17.224500 (+   511us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:17.224530 (+    30us) write_op.cc:435] Acquired schema lock
0709 13:08:17.224545 (+    15us) tablet.cc:662] Decoding operations
0709 13:08:17.224928 (+   383us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:17.224983 (+    55us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:17.225007 (+    24us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:08:17.225320 (+   313us) tablet.cc:701] Row locks acquired
0709 13:08:17.225339 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:08:17.225470 (+   131us) write_op.cc:270] Start()
0709 13:08:17.225570 (+   100us) write_op.cc:276] Timestamp: P: 1783602497225428 usec, L: 0
0709 13:08:17.225602 (+    32us) op_driver.cc:348] REPLICATION: starting
0709 13:08:17.226147 (+   545us) log.cc:844] Serialized 923 byte log entry
0709 13:08:17.228386 (+  2239us) op_driver.cc:464] REPLICATION: finished
0709 13:08:17.228792 (+   406us) write_op.cc:301] APPLY: starting
0709 13:08:17.228903 (+   111us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:17.230425 (+  1522us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:17.230520 (+    95us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:17.231853 (+  1333us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:17.232020 (+   167us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=4,mrs_lookups=0
0709 13:08:17.232065 (+    45us) write_op.cc:312] APPLY: finished
0709 13:08:17.232289 (+   224us) log.cc:844] Serialized 33 byte log entry
0709 13:08:17.232457 (+   168us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:17.232556 (+    99us) write_op.cc:454] Released schema lock
0709 13:08:17.232602 (+    46us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:17.232932 (+   330us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:17.232968 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4478,"reported_tablets":47,"spinlock_wait_cycles":1920,"child_traces":[["op",{"apply.queue_time_us":146,"apply.run_cpu_time_us":4006,"apply.run_wall_time_us":4129,"cfile_cache_hit":8,"cfile_cache_hit_bytes":8380,"num_ops":2,"prepare.queue_time_us":119,"prepare.run_cpu_time_us":2412,"prepare.run_wall_time_us":2412,"replication_time_us":2372}]]}
I20260709 13:08:17.381392 15760 catalog_manager.cc:5697] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 24 to 27. New cstate: current_term: 27 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
I20260709 13:08:17.385912 23915 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:17.452314 14748 raft_consensus.cc:2243] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:08:17.452314 15902 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:08:17.381851 23899 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:08:17.457149 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 71599.6us
I20260709 13:08:17.459923 15760 catalog_manager.cc:5697] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "31882606e87447fb9a36bbf625129fe8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } }
I20260709 13:08:17.466216 14748 raft_consensus.cc:2272] T 850b6b8a2f54468fb87ea6a22c5c870f P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:17.462406 23880 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:17.458285 23762 raft_consensus.cc:697] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:08:17.481863 23762 consensus_queue.cc:237] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:17.463150 23899 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:17.496049 23899 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:17.458252 23915 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:17.517493 23915 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:17.529983 14748 tablet_replica.cc:333] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:17.531160 14748 raft_consensus.cc:2243] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Raft consensus shutting down.
W20260709 13:08:17.537026 15974 consensus_peers.cc:597] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:17.575502 15757 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58448 (request call id 312) took 39638 ms (client timeout 15000 ms). Trace:
W20260709 13:08:17.576018 15757 rpcz_store.cc:269] 0709 13:07:37.937316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:37.955291 (+ 17975us) service_pool.cc:224] Handling call
0709 13:08:17.571090 (+39615799us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":619760,"reported_tablets":46}
I20260709 13:08:17.559147 23899 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:17.582957 23899 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:17.585170 23899 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:17.588642 14748 raft_consensus.cc:2272] T 626a74f434ad4e70bed42f8b404a6e81 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:17.594156 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:08:17.608390 15900 leader_election.cc:336] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:17.564703 23880 raft_consensus.cc:697] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 21 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:08:17.632436 23880 consensus_queue.cc:237] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:17.622800 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:17.639446 16569 raft_consensus.cc:2468] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 14.
I20260709 13:08:17.641767 14748 tablet_replica.cc:333] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:17.642613 14748 raft_consensus.cc:2243] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:08:17.644317 15902 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:08:17.645642 23920 raft_consensus.cc:2804] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:08:17.646139 23920 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:17.646548 23920 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:08:17.649372 14748 raft_consensus.cc:2272] T e9577f76f40348e9b009255d441d012a P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:17.675024 23920 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:17.678810 23920 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:17.680696 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:17.681334 16569 raft_consensus.cc:3060] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:08:17.683589 15900 leader_election.cc:336] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:17.684041 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fcbf678894c4a1eb80c68b5b53acaef" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:17.684854 15934 raft_consensus.cc:2393] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 15: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:08:17.692070 16569 raft_consensus.cc:2468] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 15.
I20260709 13:08:17.694073 15902 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:08:17.694891 23880 raft_consensus.cc:2804] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:08:17.695467 23880 raft_consensus.cc:697] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 15 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:08:17.696631 23880 consensus_queue.cc:237] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:17.699148 14748 tablet_replica.cc:333] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:17.713176 14748 raft_consensus.cc:2243] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:08:17.712203 15757 catalog_manager.cc:5697] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 20 to 21, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 21 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:17.721293 14748 raft_consensus.cc:2272] T 9f65380951b7478bad13f7cbe01eece2 P 31882606e87447fb9a36bbf625129fe8 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:17.722023 15760 catalog_manager.cc:5697] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 19 to 20, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 20 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:17.734890 14748 tablet_replica.cc:333] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:17.736020 14748 raft_consensus.cc:2243] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:17.736953 14748 raft_consensus.cc:2272] T 471dbd2e16264fe5837e8b7ea3601f94 P 31882606e87447fb9a36bbf625129fe8 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:17.740280 14748 tablet_replica.cc:333] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:17.741235 14748 raft_consensus.cc:2243] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:17.747184 14748 raft_consensus.cc:2272] T 40e7e6076e8d475da059aad457e0e8b1 P 31882606e87447fb9a36bbf625129fe8 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:17.746896 15757 catalog_manager.cc:5697] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 14 to 15, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 15 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:17.755972 14748 tablet_replica.cc:333] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:17.756778 14748 raft_consensus.cc:2243] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:17.757355 14748 raft_consensus.cc:2272] T 888cce851f074f3c969de10b02ff95ad P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:17.760222 14748 tablet_replica.cc:333] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
W20260709 13:08:17.778761 16533 leader_election.cc:336] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:17.779346 16533 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:17.779947 14748 raft_consensus.cc:2243] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 19 LEADER]: Raft consensus shutting down.
W20260709 13:08:17.782472 16533 leader_election.cc:336] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:17.782945 16533 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:17.781214 23923 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:08:17.783663 23917 raft_consensus.cc:2749] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:08:17.799901 14748 raft_consensus.cc:2272] T 1b76daed20584023b02d5f1bc7964f58 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:17.802829 14748 tablet_replica.cc:333] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:17.820226 14748 raft_consensus.cc:2243] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:17.820890 14748 raft_consensus.cc:2272] T 05f069356f3b4f72a756c3367d8713b7 P 31882606e87447fb9a36bbf625129fe8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:17.823385 14748 tablet_replica.cc:333] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:17.824043 14748 raft_consensus.cc:2243] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:17.824641 14748 raft_consensus.cc:2272] T c766f90082d54f299b03f3e9d75d2db0 P 31882606e87447fb9a36bbf625129fe8 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:17.827380 14748 tablet_replica.cc:333] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:17.868122 14748 raft_consensus.cc:2243] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:08:17.869349 14748 raft_consensus.cc:2272] T 3a0a5e0b84774549a054f45adf4adf59 P 31882606e87447fb9a36bbf625129fe8 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:17.874306 14748 tablet_replica.cc:333] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8: stopping tablet replica
I20260709 13:08:17.900388 14748 raft_consensus.cc:2243] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:08:17.901597 14748 raft_consensus.cc:2272] T ca01a56f016b4291b8abfddb75b74dd9 P 31882606e87447fb9a36bbf625129fe8 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:17.905894 23924 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:17.906446 23924 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:17.908627 23924 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:17.914098 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:17.914412 23906 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:17.915022 23906 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:17.917162 23906 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:08:17.917207 15900 leader_election.cc:336] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:17.918551 15902 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:17.920666 23922 raft_consensus.cc:2749] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:08:17.920871 15974 leader_election.cc:336] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:17.925523 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:17.928325 15974 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:17.929657 23906 raft_consensus.cc:2749] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:17.942507 16006 raft_consensus.cc:1275] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:08:17.946812 23917 consensus_queue.cc:1048] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:08:17.967541 23906 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:17.968178 23906 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:17.970504 23906 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:17.971298 23923 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:17.971964 23923 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:08:17.978242 16533 leader_election.cc:336] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:17.979877 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:08:17.978267 15974 leader_election.cc:336] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:17.981146 23923 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:17.982788 23922 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:17.983345 23922 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:17.985594 23922 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:17.980613 15934 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 21.
I20260709 13:08:17.987200 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:17.987854 16569 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 27.
W20260709 13:08:17.996232 15900 leader_election.cc:336] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:17.998661 15976 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:08:18.002415 23906 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:08:18.003957 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:18.004112 23906 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 27 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:18.011229 23906 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:08:18.012614 15902 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:18.014511 23922 raft_consensus.cc:2749] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:17.994575 16533 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:08:18.022039 23923 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:18.024175 23923 raft_consensus.cc:493] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:18.025921 23923 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:18.064020 23906 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:18.066272 23906 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 28 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:08:18.071358 15974 leader_election.cc:336] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:18.074311 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e236a7bf118740c9b3d88227bed5670e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:18.074992 16569 raft_consensus.cc:3060] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:08:18.088338 23923 raft_consensus.cc:515] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:18.092880 23923 leader_election.cc:290] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:18.097761 16569 raft_consensus.cc:2468] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 28.
I20260709 13:08:18.103844 15976 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:08:18.119956 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "260d22f804a345ed8f055536994fcb14" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:18.107160 23917 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:18.120672 15930 raft_consensus.cc:3060] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:18.120869 23917 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:18.126860 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:18.127449 23917 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:18.129957 15930 raft_consensus.cc:2468] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 22.
I20260709 13:08:18.130956 16006 raft_consensus.cc:1275] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:08:18.108991 23906 raft_consensus.cc:2804] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 28 FOLLOWER]: Leader election won for term 28
W20260709 13:08:18.134636 15900 consensus_peers.cc:597] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.135931 16533 consensus_peers.cc:597] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:18.136339 23910 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:18.160048 23910 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:18.136821 23895 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:18.165480 23895 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:18.166532 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:18.167232 15930 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 20.
I20260709 13:08:18.168780 23895 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:18.169380 15974 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:08:18.170957 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:18.162895 23910 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:08:18.177399 15974 leader_election.cc:336] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:18.179265 23895 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:18.179791 23895 raft_consensus.cc:493] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:18.180171 23895 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:08:18.189631 15974 leader_election.cc:336] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:18.190346 15974 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:18.191351 23910 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:18.192236 23906 raft_consensus.cc:697] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 28 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:18.193339 23906 consensus_queue.cc:237] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:08:18.199170 15744 catalog_manager.cc:4755] TS 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): ChangeConfig:REMOVE_PEER RPC failed for tablet dfeaf1f6660d48848ae69710739427a0: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:18.200450 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:18.224426 16569 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 20.
I20260709 13:08:18.202203 23922 consensus_queue.cc:1048] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:08:18.206326 23924 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:18.209924 16533 leader_election.cc:304] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:18.214627 16562 raft_consensus.cc:1275] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:08:18.221397 23917 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:18.228405 23924 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:18.229432 23921 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:18.229966 23921 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:18.231148 23924 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:18.231775 23925 raft_consensus.cc:2804] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:08:18.232385 23925 raft_consensus.cc:697] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:08:18.233328 23925 consensus_queue.cc:237] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:18.229940 23917 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:18.243158 23917 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:18.243264 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:18.244778 16562 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 24.
I20260709 13:08:18.227886 23895 raft_consensus.cc:515] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:18.243744 23921 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:18.291708 15757 catalog_manager.cc:5697] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 27 to 28, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 28 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:18.333671 23938 log.cc:927] Time spent T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5: Append to log took a long time: real 0.077s	user 0.001s	sys 0.001s
I20260709 13:08:18.350867 23895 leader_election.cc:290] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:18.356241 23924 consensus_queue.cc:1048] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:08:18.358939 23906 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:18.360350 23906 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:18.373065 23906 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:18.378670 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:18.396042 23880 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:18.396457 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:18.397339 16005 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 22.
I20260709 13:08:18.397131 23880 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:08:18.398577 15974 leader_election.cc:336] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:18.400393 15974 leader_election.cc:336] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:18.380093 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:18.402663 16006 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 24.
I20260709 13:08:18.392397 15930 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 20.
I20260709 13:08:18.410557 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:18.411945 15937 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 22.
W20260709 13:08:18.406329 15974 leader_election.cc:336] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:18.411907 15976 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:08:18.414459 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65db3107b664e9c8ce9be2cd2cb36c6" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:18.415349 15936 raft_consensus.cc:3060] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:18.416054 23906 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:08:18.416729 23906 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 24 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:18.417265 23906 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:18.418841 23922 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:18.428279 15974 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:08:18.431067 23910 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:18.433370 23910 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:18.433972 23910 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:18.431942 23922 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:18.441216 23922 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:18.446527 16533 leader_election.cc:336] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:18.448395 16535 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:18.449776 23762 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:08:18.450289 23762 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:18.450738 23762 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:08:18.455274 15902 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
W20260709 13:08:18.456266 15900 leader_election.cc:336] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:18.462055 23880 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:18.458407 23924 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:18.468473 23924 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:18.469484 23924 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:08:18.487982 15900 consensus_peers.cc:597] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:18.513427 15936 raft_consensus.cc:2468] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 21.
W20260709 13:08:18.524394 15900 leader_election.cc:336] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:18.529634 23895 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:18.530948 23895 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:18.536706 23895 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:08:18.538695 15900 leader_election.cc:336] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:18.540088 15974 leader_election.cc:336] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:18.493758 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 46203.2us
W20260709 13:08:18.550169 16533 leader_election.cc:336] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:18.550699 16533 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:08:18.516731 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 78635.6us
I20260709 13:08:18.562801 23906 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:18.563844 23762 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:18.570582 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:18.572152 23762 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 25 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:18.573665 23940 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:18.580098 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:18.568555 23910 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:18.590314 23910 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 23 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:08:18.580272 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 125302us
W20260709 13:08:18.595116 15974 leader_election.cc:336] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:18.596674 15974 leader_election.cc:336] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:18.601109 23906 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 25 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:18.611774 15760 catalog_manager.cc:5697] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 21 to 22, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 22 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:18.613797 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:18.614686 16005 raft_consensus.cc:2393] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 25: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:08:18.618093 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:18.618904 16562 raft_consensus.cc:2393] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 25: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:08:18.603046 15974 leader_election.cc:304] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:08:18.629204 23945 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:18.630172 23945 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:18.633248 23924 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:18.636765 23910 raft_consensus.cc:2804] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:18.641666 15936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:18.644716 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:18.649832 15931 raft_consensus.cc:2393] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 23: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:08:18.648947 15936 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 20.
I20260709 13:08:18.654865 23910 raft_consensus.cc:697] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:18.671766 23945 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:18.672565 23910 consensus_queue.cc:237] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:18.693508 15931 raft_consensus.cc:1275] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:08:18.694223 16569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:18.689144 15902 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:18.699982 23945 raft_consensus.cc:2749] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:18.701202 15976 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:18.706336 15976 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:08:18.711360 16533 leader_election.cc:336] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:18.707183 23924 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 23 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:18.715982 23944 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:18.716887 23944 raft_consensus.cc:2749] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:08:18.700740 23880 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:18.718622 23880 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:18.721625 23880 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:18.722800 23895 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:18.723632 23895 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:18.737269 23935 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:18.738247 23935 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:18.740731 23935 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:18.744127 23895 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:08:18.776913 16533 leader_election.cc:336] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:18.777843 23917 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:18.780185 23923 consensus_queue.cc:1048] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.005s
W20260709 13:08:18.789533 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 82419.4us
I20260709 13:08:18.802300 23917 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:18.805697 23917 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:08:18.832876 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 132481us
W20260709 13:08:18.848295 15900 leader_election.cc:336] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:18.837756 15974 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:08:18.849128 15900 leader_election.cc:336] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:18.854938 15900 leader_election.cc:336] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:18.857888 23958 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:18.858659 23958 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:18.861115 23958 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:08:18.863587 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 157291us
I20260709 13:08:18.864741 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:18.865506 16006 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 22.
I20260709 13:08:18.866530 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:18.867200 16011 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 20.
I20260709 13:08:18.852217 23944 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:08:18.868505 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:18.869302 16011 raft_consensus.cc:2393] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 23: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
W20260709 13:08:18.887238 15900 leader_election.cc:336] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:18.888262 16533 consensus_peers.cc:597] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.888748 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 87756us
I20260709 13:08:18.889770 23939 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:18.890493 23939 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:18.893573 23939 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:18.901057 23923 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:18.901742 23923 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:18.904740 15760 catalog_manager.cc:5697] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 20 to 21, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 21 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:18.908177 23923 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:18.913750 16533 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 285 similar messages]
I20260709 13:08:18.920028 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:18.950290 15931 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 15.
I20260709 13:08:18.944687 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:18.951285 15934 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 20.
I20260709 13:08:18.953374 15902 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:18.957610 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:18.961947 16008 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 18.
I20260709 13:08:18.959556 16535 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:18.963518 23923 raft_consensus.cc:2749] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:08:18.964627 23945 raft_consensus.cc:2749] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:18.970883 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:18.971550 16562 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 15.
W20260709 13:08:18.976575 15974 leader_election.cc:336] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:18.977241 15974 leader_election.cc:336] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:18.977865 15974 consensus_peers.cc:597] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:18.983536 16567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:18.984287 16567 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
I20260709 13:08:18.985862 16569 raft_consensus.cc:1275] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 28 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 13:08:18.992165 16533 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:18.997294 23923 raft_consensus.cc:2804] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:18.997857 23923 raft_consensus.cc:493] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:19.006692 23921 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:19.007370 23921 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:19.009711 23921 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:19.003351 23923 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:19.023340 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:19.024092 15934 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 18.
I20260709 13:08:19.026651 23921 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:19.028496 23921 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:19.030655 23921 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:08:19.031258 16533 leader_election.cc:336] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:19.032783 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:19.032829 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:19.036986 16008 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 20.
W20260709 13:08:19.038483 16533 leader_election.cc:336] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:19.037026 16010 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 16.
W20260709 13:08:19.045068 16533 leader_election.cc:336] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:19.034014 23955 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:19.052517 23955 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:19.054703 23955 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:19.060032 23923 raft_consensus.cc:515] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:08:19.064805 16533 consensus_peers.cc:597] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:19.073731 23923 leader_election.cc:290] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:08:19.067478 16533 consensus_peers.cc:597] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:19.076367 23924 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:19.076997 23924 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:08:19.079630 23939 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:19.080339 23939 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:19.083559 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:19.102602 15931 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 21.
I20260709 13:08:19.084228 23924 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
W20260709 13:08:19.085717 15974 leader_election.cc:336] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:19.088498 16535 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:08:19.103773 23939 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:19.104332 16535 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
W20260709 13:08:19.104950 15900 leader_election.cc:336] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:19.105494 23925 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:19.105922 23925 raft_consensus.cc:493] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:19.106308 23925 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:08:19.109853 15974 leader_election.cc:336] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:19.100412 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626a74f434ad4e70bed42f8b404a6e81" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:19.110548 15934 raft_consensus.cc:3060] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:08:19.091090 16533 leader_election.cc:336] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:19.114434 15974 consensus_peers.cc:597] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:19.118575 15934 raft_consensus.cc:2468] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 16.
I20260709 13:08:19.122763 16533 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:08:19.123181 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:19.123656 23917 raft_consensus.cc:2804] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:19.123978 15931 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 15.
I20260709 13:08:19.124970 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:19.133597 23917 raft_consensus.cc:697] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:08:19.128388 15934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
W20260709 13:08:19.115062 15974 leader_election.cc:336] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:19.134994 23917 consensus_queue.cc:237] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:19.135784 15974 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:08:19.136654 23955 raft_consensus.cc:2804] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:19.137017 23955 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:19.137392 23955 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:19.104368 23939 raft_consensus.cc:2804] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:19.139963 23939 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:19.140347 23939 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:08:19.143608 15900 consensus_peers.cc:597] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:19.145179 15902 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:08:19.146994 23955 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:19.150331 23925 raft_consensus.cc:515] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:19.152004 15974 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:08:19.156643 23924 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:19.157073 23924 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:19.157456 23924 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:19.158317 15902 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:19.160336 23952 raft_consensus.cc:2749] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:08:19.160449 16533 leader_election.cc:336] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:19.160887 16533 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:19.161178 23955 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:19.163030 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:19.163750 16005 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 20.
I20260709 13:08:19.166798 15976 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:08:19.167294 23955 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:19.167816 23955 raft_consensus.cc:493] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:19.168205 23955 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:08:19.170709 16533 consensus_peers.cc:597] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:19.168385 23940 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:19.172158 23957 raft_consensus.cc:2804] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:08:19.172580 23957 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:19.172943 23957 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:08:19.173074 15902 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:08:19.174151 23956 raft_consensus.cc:2804] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:08:19.174553 23956 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:19.174901 23956 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:08:19.176940 15902 leader_election.cc:304] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:08:19.177768 23945 raft_consensus.cc:2764] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:08:19.179988 23924 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:19.182591 23957 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:19.185668 23944 consensus_queue.cc:1048] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
W20260709 13:08:19.186599 15900 leader_election.cc:336] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:19.189265 23945 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
W20260709 13:08:19.189460 15900 consensus_peers.cc:597] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:19.189895 23945 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:19.193013 23925 leader_election.cc:290] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:19.196396 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:19.197181 15937 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:19.187186 23957 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:19.210018 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:19.211483 15931 raft_consensus.cc:1275] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
W20260709 13:08:19.213428 15900 leader_election.cc:336] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:19.214598 23939 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:19.200493 23924 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 23 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:19.200407 23945 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:19.224100 15937 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 21.
I20260709 13:08:19.228336 15974 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:08:19.190900 23955 raft_consensus.cc:515] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:08:19.228878 16533 leader_election.cc:336] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:19.231190 15902 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:08:19.230420 15974 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:19.238693 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac847593d6724522bda12a2207f91c31" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:19.239287 16011 raft_consensus.cc:3060] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:19.242991 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:19.234045 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfeaf1f6660d48848ae69710739427a0" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
W20260709 13:08:19.231698 16533 consensus_peers.cc:597] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:19.245029 16010 raft_consensus.cc:3060] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:19.237953 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:19.231962 23957 raft_consensus.cc:2804] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:08:19.246963 16562 raft_consensus.cc:2393] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 17: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:08:19.248673 15902 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:19.246662 23895 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:19.246855 23935 consensus_queue.cc:1048] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.006s
I20260709 13:08:19.253486 23957 raft_consensus.cc:493] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:19.232646 23922 raft_consensus.cc:2804] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:19.254096 23957 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:19.254379 23922 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:19.254961 23922 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:19.256327 23952 raft_consensus.cc:2749] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:19.254416 15757 catalog_manager.cc:5697] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 15 to 16, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 16 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:19.224831 23956 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:19.265437 23957 raft_consensus.cc:515] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:19.266103 15930 raft_consensus.cc:2393] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 19: Already voted for candidate 8430db80a39d4868bcf7641c16de35ea in this term.
I20260709 13:08:19.267186 23956 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:19.265776 16011 raft_consensus.cc:2468] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 23.
I20260709 13:08:19.272492 23957 leader_election.cc:290] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:19.281131 15902 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:08:19.282439 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:19.285216 16008 raft_consensus.cc:2393] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8430db80a39d4868bcf7641c16de35ea in current term 19: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
W20260709 13:08:19.296562 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 45960us
I20260709 13:08:19.296196 23952 raft_consensus.cc:2804] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:08:19.297850 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:19.298771 16005 raft_consensus.cc:2393] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 17: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
I20260709 13:08:19.297940 23952 raft_consensus.cc:697] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 23 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:08:19.303856 16533 leader_election.cc:336] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:19.312796 23955 leader_election.cc:290] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:19.293246 23953 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:19.316156 23953 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:19.318581 23953 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:19.293339 23939 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:19.319077 15900 consensus_peers.cc:597] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:19.319703 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:19.320349 16008 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 19.
I20260709 13:08:19.321578 15902 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:08:19.321710 23952 consensus_queue.cc:237] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:19.322449 23945 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:19.322856 23945 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:19.305445 16535 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:08:19.303920 15974 consensus_peers.cc:597] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:19.323228 23945 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:19.327185 23939 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:08:19.327179 15974 leader_election.cc:336] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:19.332618 15900 leader_election.cc:336] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:19.338804 16010 raft_consensus.cc:2468] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 21.
W20260709 13:08:19.372323 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 45169.9us
I20260709 13:08:19.374662 15900 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:19.375895 16535 leader_election.cc:304] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:08:19.377942 23939 raft_consensus.cc:2804] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:19.378654 23939 raft_consensus.cc:697] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
W20260709 13:08:19.381423 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 48861.7us
I20260709 13:08:19.382815 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0fef632f9a4485838855b553a79163" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:19.383412 15931 raft_consensus.cc:3060] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:19.387403 15974 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:19.389003 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f65380951b7478bad13f7cbe01eece2" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:19.389638 15937 raft_consensus.cc:3060] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:08:19.379649 23939 consensus_queue.cc:237] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:19.392819 23956 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:19.393458 23956 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:19.398633 15931 raft_consensus.cc:2468] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 22.
I20260709 13:08:19.394337 23953 raft_consensus.cc:2749] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:08:19.401909 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:19.402643 16561 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 19.
I20260709 13:08:19.403873 23956 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:19.404017 15902 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:19.404883 23956 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:19.405232 23956 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:19.405601 23956 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:19.406723 23922 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
W20260709 13:08:19.398777 15900 consensus_peers.cc:597] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:19.408191 15900 leader_election.cc:336] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:19.415063 15937 raft_consensus.cc:2468] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 16.
W20260709 13:08:19.415498 15900 consensus_peers.cc:597] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:19.417577 15900 consensus_peers.cc:597] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:19.428714 15974 leader_election.cc:336] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:19.430325 15974 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:19.432451 23935 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:08:19.431458 23945 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:19.423416 23910 raft_consensus.cc:2749] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:08:19.413863 23957 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:19.441090 23922 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 election: Requested vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033)
I20260709 13:08:19.441035 23957 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:19.443609 23957 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:19.447405 16010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9577f76f40348e9b009255d441d012a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:19.448067 16010 raft_consensus.cc:3060] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:19.448482 23956 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:19.451017 23956 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:08:19.451877 15974 consensus_peers.cc:597] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:19.452811 15974 leader_election.cc:304] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:08:19.453519 23910 raft_consensus.cc:2804] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:08:19.453629 15974 leader_election.cc:304] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:08:19.454095 23910 raft_consensus.cc:697] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:19.455116 23910 consensus_queue.cc:237] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:19.455750 23945 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:19.456478 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b76daed20584023b02d5f1bc7964f58" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:19.456611 23957 raft_consensus.cc:2804] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:08:19.457259 16011 raft_consensus.cc:3060] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:19.457294 23957 raft_consensus.cc:697] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:19.458467 23957 consensus_queue.cc:237] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:19.465955 15760 catalog_manager.cc:5697] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 22 to 23, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 23 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:19.471974 16010 raft_consensus.cc:2468] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 21.
I20260709 13:08:19.473305 16011 raft_consensus.cc:2468] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 20.
I20260709 13:08:19.474877 15902 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:08:19.475690 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca01a56f016b4291b8abfddb75b74dd9" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:19.476070 15902 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:08:19.476311 16561 raft_consensus.cc:3060] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:08:19.476881 23952 raft_consensus.cc:2804] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:19.477455 23952 raft_consensus.cc:697] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 21 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
W20260709 13:08:19.479570 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 47306.8us
I20260709 13:08:19.482666 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:19.483206 15931 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 21.
I20260709 13:08:19.476923 23945 raft_consensus.cc:2804] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:19.496184 23945 raft_consensus.cc:697] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 20 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:08:19.496248 23952 consensus_queue.cc:237] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:08:19.497236 23945 consensus_queue.cc:237] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:08:19.500698 15900 leader_election.cc:336] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:19.489079 15974 consensus_peers.cc:597] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:19.514457 15900 leader_election.cc:336] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:19.515421 15900 leader_election.cc:336] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:19.516038 15900 leader_election.cc:336] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:19.508041 15974 leader_election.cc:336] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:19.523078 15974 leader_election.cc:336] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:19.524061 15974 consensus_peers.cc:597] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:19.524359 15900 consensus_peers.cc:597] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:19.526160 15974 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:08:19.528784 23935 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:19.529356 23935 raft_consensus.cc:493] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:19.529832 23935 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:19.534996 16561 raft_consensus.cc:2468] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 20.
W20260709 13:08:19.557175 15974 leader_election.cc:336] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:19.564921 15902 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:08:19.566954 23956 raft_consensus.cc:2804] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:19.567703 23956 raft_consensus.cc:697] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 20 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:08:19.568652 23956 consensus_queue.cc:237] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:08:19.579878 16533 consensus_peers.cc:597] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:19.558250 15757 catalog_manager.cc:5697] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 20 to 21, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 21 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:08:19.604830 15974 consensus_peers.cc:597] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:19.613274 15757 catalog_manager.cc:5697] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 21 to 22, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 22 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:19.618958 15757 catalog_manager.cc:5697] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 15 to 16, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 16 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:19.621351 15760 catalog_manager.cc:5697] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 20 to 21, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 21 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:19.625494 15760 catalog_manager.cc:5697] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 19 to 20, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 20 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:19.621054 23935 raft_consensus.cc:515] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:08:19.642683 15974 leader_election.cc:336] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:19.644369 23935 leader_election.cc:290] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:19.647881 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0a5e0b84774549a054f45adf4adf59" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:19.649677 15930 raft_consensus.cc:3060] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:19.658715 15930 raft_consensus.cc:2468] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 22.
I20260709 13:08:19.664621 15974 leader_election.cc:304] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:08:19.669539 23935 raft_consensus.cc:2804] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:08:19.670221 23935 raft_consensus.cc:697] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
W20260709 13:08:19.663976 16533 consensus_peers.cc:597] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:19.672650 23935 consensus_queue.cc:237] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:19.678396 15759 catalog_manager.cc:5697] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 19 to 20, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 20 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:19.690412 16533 connection.cc:769] client connection to 127.14.103.1:38033 send error: Network error: sendmsg error: Broken pipe (error 32)
W20260709 13:08:19.696038 16533 consensus_peers.cc:597] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: sendmsg error: Broken pipe (error 32). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:19.713462 15759 catalog_manager.cc:5697] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 21 to 22, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 22 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:19.740083 15937 raft_consensus.cc:1275] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:08:19.741581 23925 consensus_queue.cc:1048] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:08:19.768545 15900 consensus_peers.cc:597] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:19.802325 16533 consensus_peers.cc:597] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:19.807554 16533 consensus_peers.cc:597] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:19.843603 15974 consensus_peers.cc:597] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:19.844323 15900 consensus_peers.cc:597] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:19.867959 15900 consensus_peers.cc:597] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:19.874471 16005 raft_consensus.cc:1275] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:08:19.880352 23956 consensus_queue.cc:1048] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:08:19.914390 16011 raft_consensus.cc:1275] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:08:19.916440 23956 consensus_queue.cc:1048] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:08:19.914290 16533 consensus_peers.cc:597] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:19.931785 16533 consensus_peers.cc:597] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:20.021544 15930 raft_consensus.cc:1275] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:08:20.024008 23895 consensus_queue.cc:1048] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:08:20.042100 16561 raft_consensus.cc:1275] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:08:20.047605 23956 consensus_queue.cc:1048] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:08:20.055912 15931 raft_consensus.cc:1275] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:08:20.057659 23935 consensus_queue.cc:1048] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:08:20.066723 15900 consensus_peers.cc:597] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:20.090368 15930 raft_consensus.cc:1275] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
W20260709 13:08:20.096690 15974 consensus_peers.cc:597] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:20.099836 23910 consensus_queue.cc:1048] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:08:20.133121 16011 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 13:08:20.146992 15900 consensus_peers.cc:597] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:20.160593 15974 consensus_peers.cc:597] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:20.173581 15900 consensus_peers.cc:597] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:20.201186 16011 raft_consensus.cc:1275] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 13:08:20.211805 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 71390.9us
I20260709 13:08:20.216745 23956 consensus_queue.cc:1048] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:08:20.226025 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 111230us
I20260709 13:08:20.236734 23987 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.022s
W20260709 13:08:20.255962 15974 consensus_peers.cc:597] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:20.263334 15930 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 13:08:20.280663 15974 consensus_peers.cc:597] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:20.282030 23917 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:08:20.283109 23910 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:20.283928 23910 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:20.332880 23910 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:20.334270 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:20.335060 16561 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 20.
W20260709 13:08:20.346550 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 63934.2us
I20260709 13:08:20.360435 23921 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:20.361052 23921 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:20.369525 23921 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:20.370703 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:20.371860 16561 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 25.
I20260709 13:08:20.372370 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:20.373203 15976 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:20.374598 23921 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:08:20.375064 23921 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:20.375368 23921 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:08:20.386770 16533 consensus_peers.cc:597] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:20.374060 15937 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 20.
I20260709 13:08:20.398272 23921 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:08:20.404433 15900 consensus_peers.cc:597] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:20.414608 23921 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 26 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:20.423022 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a10bba95cd1421aa3fc332639e97319" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:20.423822 16561 raft_consensus.cc:3060] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:08:20.429215 15974 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:20.437326 23990 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:20.437791 23990 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting leader election (detected failure of leader 31882606e87447fb9a36bbf625129fe8)
I20260709 13:08:20.438195 23990 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:20.442523 16561 raft_consensus.cc:2468] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 26.
I20260709 13:08:20.444092 15976 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:20.445012 23957 raft_consensus.cc:2804] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:08:20.445565 23957 raft_consensus.cc:697] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 26 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:20.446544 23957 consensus_queue.cc:237] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:20.448016 23990 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:20.470813 23990 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:20.473806 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
W20260709 13:08:20.488416 15974 leader_election.cc:336] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
I20260709 13:08:20.475453 16561 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:20.501732 16561 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 21.
I20260709 13:08:20.506908 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:20.507495 15937 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:20.513433 15937 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 21.
W20260709 13:08:20.527601 15974 leader_election.cc:336] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:20.531370 15974 leader_election.cc:336] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
I20260709 13:08:20.524938 15760 catalog_manager.cc:5697] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 24 to 26, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 26 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:20.547194 15974 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:20.548053 23990 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:08:20.548552 23990 raft_consensus.cc:697] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:20.549523 23990 consensus_queue.cc:237] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:08:20.568488 15974 leader_election.cc:336] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:20.584690 15974 consensus_peers.cc:597] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:20.676673 23957 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:20.677222 23957 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:20.679275 23957 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:20.685137 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:20.684356 15760 catalog_manager.cc:5697] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 20 to 21, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 21 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:20.685909 15931 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 23.
I20260709 13:08:20.695823 15974 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:08:20.699757 23910 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:08:20.700477 23910 raft_consensus.cc:493] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:20.700974 23910 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:08:20.708946 15974 leader_election.cc:336] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:20.713382 14748 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 15883)
I20260709 13:08:20.714587 23910 raft_consensus.cc:515] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:20.737354 23910 leader_election.cc:290] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 24 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:20.741408 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71b32319bee4f979532b3ac6a16f2bd" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:20.742241 15931 raft_consensus.cc:3060] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:08:20.753942 15931 raft_consensus.cc:2468] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 24.
I20260709 13:08:20.763496 15974 leader_election.cc:304] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:08:20.771888 23957 raft_consensus.cc:2804] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:08:20.772480 23957 raft_consensus.cc:697] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 24 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:20.773420 23957 consensus_queue.cc:237] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:08:20.776110 15974 leader_election.cc:336] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
I20260709 13:08:20.815250 15760 catalog_manager.cc:5697] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 22 to 24, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 24 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:20.867079 16011 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5"
dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:45198
I20260709 13:08:20.867916 16011 raft_consensus.cc:606] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 LEADER]: Received request to transfer leadership
I20260709 13:08:20.874769 16011 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5"
dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:45198
I20260709 13:08:20.875386 16011 raft_consensus.cc:606] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 LEADER]: Received request to transfer leadership
I20260709 13:08:20.887816 16011 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5"
dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:45198
I20260709 13:08:20.888453 16011 raft_consensus.cc:606] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 LEADER]: Received request to transfer leadership
I20260709 13:08:20.910682 16561 raft_consensus.cc:1275] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 26 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 13:08:20.912345 23990 consensus_queue.cc:1048] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 13:08:20.956525 15974 consensus_peers.cc:597] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:20.969080 23910 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:20.970981 23910 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:20.973174 23910 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:20.976676 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:20.977399 16562 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 17.
I20260709 13:08:20.978442 15976 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:20.985145 23910 raft_consensus.cc:2804] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:08:20.985630 23910 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:20.985996 23910 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:08:21.019546 15974 consensus_peers.cc:597] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:21.020095 15974 leader_election.cc:336] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:21.042474 15808 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 13:08:21.042966 15808 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 83a1b79cdcf046c78453695c798a14a5 already in progress
W20260709 13:08:21.043274 15808 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 83a1b79cdcf046c78453695c798a14a5 already in progress
I20260709 13:08:21.051764 15931 raft_consensus.cc:1275] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:08:21.042068 23910 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:21.053399 23957 consensus_queue.cc:1048] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:08:21.055549 23910 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:21.064069 16561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15301761b2ec47fe82ae2f431c8b34ee" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:21.064839 16561 raft_consensus.cc:3060] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:08:21.083958 15974 leader_election.cc:336] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
I20260709 13:08:21.106961 16561 raft_consensus.cc:2468] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 18.
I20260709 13:08:21.108229 15976 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:08:21.109459 23910 raft_consensus.cc:2804] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:08:21.110029 23910 raft_consensus.cc:697] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 18 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:21.111047 23910 consensus_queue.cc:237] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:21.125013 23957 raft_consensus.cc:993] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee: : Instructing follower 8430db80a39d4868bcf7641c16de35ea to start an election
I20260709 13:08:21.125682 24006 raft_consensus.cc:1081] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 LEADER]: Signalling peer 8430db80a39d4868bcf7641c16de35ea to start an election
I20260709 13:08:21.129245 15931 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5"
dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
 from {username='slave'} at 127.0.0.1:58402
I20260709 13:08:21.129930 15931 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:21.130306 15931 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:21.139446 15759 catalog_manager.cc:5697] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 16 to 18, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 18 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:21.147251 16561 raft_consensus.cc:1275] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:08:21.150570 23910 consensus_queue.cc:1048] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:08:21.153949 15931 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:21.157832 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: true dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:21.158510 16011 raft_consensus.cc:3055] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:08:21.158849 16011 raft_consensus.cc:740] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:21.159727 16011 consensus_queue.cc:260] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:21.161036 16011 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:08:21.164346 15900 leader_election.cc:336] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:21.164870 16533 consensus_peers.cc:597] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:21.185707 16011 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 22.
W20260709 13:08:21.204352 15900 consensus_peers.cc:597] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:21.212154 15931 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:21.214602 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: true dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:21.215224 16562 raft_consensus.cc:3060] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:21.226961 16562 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8430db80a39d4868bcf7641c16de35ea in term 22.
I20260709 13:08:21.233773 15902 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:08:21.234612 23922 raft_consensus.cc:2804] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:08:21.235122 23922 raft_consensus.cc:697] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 22 LEADER]: Becoming Leader. State: Replica: 8430db80a39d4868bcf7641c16de35ea, State: Running, Role: LEADER
I20260709 13:08:21.236058 23922 consensus_queue.cc:237] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:21.249833 15759 catalog_manager.cc:5697] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: term changed from 21 to 22, leader changed from 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3) to 8430db80a39d4868bcf7641c16de35ea (127.14.103.2). New cstate: current_term: 22 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:21.317006 15974 consensus_peers.cc:597] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:21.332252 16533 consensus_peers.cc:597] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:21.352998 15931 raft_consensus.cc:1275] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 24 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:08:21.355832 24006 consensus_queue.cc:1048] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:08:21.449798 15900 consensus_peers.cc:597] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:21.483067 23990 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:21.483659 23990 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:21.486054 23990 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:21.490216 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea" is_pre_election: true
I20260709 13:08:21.491012 15931 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 19.
I20260709 13:08:21.494274 15974 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 
I20260709 13:08:21.496167 23957 raft_consensus.cc:2804] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:08:21.496613 23957 raft_consensus.cc:493] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:21.496959 23957 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:08:21.528067 15974 leader_election.cc:336] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:21.537828 16533 consensus_peers.cc:597] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:21.594318 23957 raft_consensus.cc:515] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:21.596614 23957 leader_election.cc:290] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 election: Requested vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:21.603386 15937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "850b6b8a2f54468fb87ea6a22c5c870f" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8430db80a39d4868bcf7641c16de35ea"
I20260709 13:08:21.604934 15937 raft_consensus.cc:3060] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:08:21.606515 15974 consensus_peers.cc:597] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:21.607023 15974 leader_election.cc:336] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
I20260709 13:08:21.618351 15937 raft_consensus.cc:2468] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 20.
W20260709 13:08:21.621169 15900 consensus_peers.cc:597] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:21.622839 15974 leader_election.cc:304] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8
I20260709 13:08:21.624562 24006 raft_consensus.cc:2804] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:08:21.625175 24006 raft_consensus.cc:697] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 20 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:21.626502 24006 consensus_queue.cc:237] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:08:21.659199 16533 consensus_peers.cc:597] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:21.658968 15759 catalog_manager.cc:5697] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 18 to 20, leader changed from 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 20 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } }
W20260709 13:08:21.677654 15974 consensus_peers.cc:597] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:21.680544 15974 consensus_peers.cc:597] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:21.685669 16562 raft_consensus.cc:1275] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:08:21.687577 23945 consensus_queue.cc:1048] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:08:21.701159 15974 consensus_peers.cc:597] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:21.713877 14748 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 15883)
I20260709 13:08:21.717303 16561 raft_consensus.cc:1275] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:08:21.719126 23910 consensus_queue.cc:1048] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:08:21.734449 15900 consensus_peers.cc:597] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:21.751900 15900 consensus_peers.cc:597] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:21.843881 15900 consensus_peers.cc:597] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:21.848419 16011 raft_consensus.cc:1275] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:08:21.850312 23922 consensus_queue.cc:1048] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:08:21.853487 16533 consensus_peers.cc:597] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:21.909077 15974 consensus_peers.cc:597] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:21.938678 15974 consensus_peers.cc:597] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:21.949350 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 52580.9us
W20260709 13:08:21.984043 15900 consensus_peers.cc:597] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:21.984541 15900 consensus_peers.cc:597] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:21.984692 15974 consensus_peers.cc:597] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:21.996134 15900 consensus_peers.cc:597] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:22.006390 16533 consensus_peers.cc:597] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:22.122721 15900 consensus_peers.cc:597] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:22.141844 15931 raft_consensus.cc:1275] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:08:22.144946 23990 consensus_queue.cc:1048] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:08:22.166569 16533 consensus_peers.cc:597] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:22.176184 15900 consensus_peers.cc:597] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:22.176184 16533 consensus_peers.cc:597] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:22.211923 16533 consensus_peers.cc:597] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:22.252830 15974 consensus_peers.cc:597] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:22.360723 15974 consensus_peers.cc:597] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:22.430656 15900 consensus_peers.cc:597] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:22.442358 15974 consensus_peers.cc:597] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:22.457499 16533 consensus_peers.cc:597] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:22.458555 15900 consensus_peers.cc:597] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:22.451396 15931 consensus_queue.cc:237] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
W20260709 13:08:22.487794 15902 consensus_peers.cc:597] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 05f069356f3b4f72a756c3367d8713b7. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:22.491181 16011 raft_consensus.cc:1275] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:08:22.493374 24020 consensus_queue.cc:1048] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:08:22.498526 16533 consensus_peers.cc:597] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:22.503985 15900 consensus_peers.cc:597] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:22.507783 15900 consensus_peers.cc:597] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:22.513453 16011 raft_consensus.cc:2955] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Committing config change with OpId 20.14: config changed from index -1 to 14, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
I20260709 13:08:22.511511 24020 raft_consensus.cc:2955] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 20 LEADER]: Committing config change with OpId 20.14: config changed from index -1 to 14, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
W20260709 13:08:22.518414 16533 consensus_peers.cc:597] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:22.536969 15931 consensus_queue.cc:237] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:08:22.549446 15759 catalog_manager.cc:5697] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea reported cstate change: config changed from index -1 to 14, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 20 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:22.607120 15900 consensus_peers.cc:597] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:22.607280 15974 consensus_peers.cc:597] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:22.629034 15902 consensus_peers.cc:597] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea -> Peer eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643): Couldn't send request to peer eb1e0493af1e46dbb6dec7e25525bab5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 975ecd3d27dd45ba9a554ac0ee591e0e. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:22.631294 15900 consensus_peers.cc:597] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:22.635007 16011 raft_consensus.cc:1275] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:08:22.639147 24017 consensus_queue.cc:1048] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:08:22.704695 24020 raft_consensus.cc:2955] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 21 LEADER]: Committing config change with OpId 21.12: config changed from index -1 to 12, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
W20260709 13:08:22.714395 14748 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 15883)
I20260709 13:08:22.713824 16008 raft_consensus.cc:2955] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Committing config change with OpId 21.12: config changed from index -1 to 12, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } } }
W20260709 13:08:22.735883 15900 consensus_peers.cc:597] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:22.757798 15759 catalog_manager.cc:5697] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea reported cstate change: config changed from index -1 to 12, NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) added. New cstate: current_term: 21 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:22.799562 15974 consensus_peers.cc:597] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:22.820227 15974 consensus_peers.cc:597] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:22.839403 15974 consensus_peers.cc:597] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:22.981117 16533 consensus_peers.cc:597] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:23.090092 24036 ts_tablet_manager.cc:933] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:23.100988 24036 tablet_copy_client.cc:323] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.2:43009
I20260709 13:08:23.103322 15946 tablet_copy_service.cc:140] P 8430db80a39d4868bcf7641c16de35ea: Received BeginTabletCopySession request for tablet 05f069356f3b4f72a756c3367d8713b7 from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:53636)
I20260709 13:08:23.103960 15946 tablet_copy_service.cc:161] P 8430db80a39d4868bcf7641c16de35ea: Beginning new tablet copy session on tablet 05f069356f3b4f72a756c3367d8713b7 from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:53636: session id = eb1e0493af1e46dbb6dec7e25525bab5-05f069356f3b4f72a756c3367d8713b7
I20260709 13:08:23.121531 15946 tablet_copy_source_session.cc:215] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:08:23.133461 24036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05f069356f3b4f72a756c3367d8713b7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:08:23.177934 15974 consensus_peers.cc:597] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:23.219774 24036 tablet_copy_client.cc:806] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:08:23.220702 24036 tablet_copy_client.cc:670] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:08:23.225492 24039 ts_tablet_manager.cc:933] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5: Initiating tablet copy from peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:23.330646 24036 tablet_copy_client.cc:538] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:08:23.347748 24036 tablet_bootstrap.cc:492] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
I20260709 13:08:23.364920 24039 tablet_copy_client.cc:323] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Beginning tablet copy session from remote peer at address 127.14.103.2:43009
I20260709 13:08:23.373131 15946 tablet_copy_service.cc:140] P 8430db80a39d4868bcf7641c16de35ea: Received BeginTabletCopySession request for tablet 975ecd3d27dd45ba9a554ac0ee591e0e from peer eb1e0493af1e46dbb6dec7e25525bab5 ({username='slave'} at 127.0.0.1:53636)
I20260709 13:08:23.373728 15946 tablet_copy_service.cc:161] P 8430db80a39d4868bcf7641c16de35ea: Beginning new tablet copy session on tablet 975ecd3d27dd45ba9a554ac0ee591e0e from peer eb1e0493af1e46dbb6dec7e25525bab5 at {username='slave'} at 127.0.0.1:53636: session id = eb1e0493af1e46dbb6dec7e25525bab5-975ecd3d27dd45ba9a554ac0ee591e0e
W20260709 13:08:23.400928 15974 consensus_peers.cc:597] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:23.415838 15946 tablet_copy_source_session.cc:215] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:08:23.438546 24039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 975ecd3d27dd45ba9a554ac0ee591e0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:23.479799 24039 tablet_copy_client.cc:806] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 0 data blocks...
I20260709 13:08:23.480512 24039 tablet_copy_client.cc:670] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Starting download of 1 WAL segments...
I20260709 13:08:23.488119 24039 tablet_copy_client.cc:538] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:08:23.510071 24039 tablet_bootstrap.cc:492] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap starting.
W20260709 13:08:23.714895 14748 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 15883)
W20260709 13:08:23.726356 15900 consensus_peers.cc:597] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:08:23.763827 24036 tablet_bootstrap.cc:492] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=1 applied=14 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:23.765188 24036 tablet_bootstrap.cc:492] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
W20260709 13:08:23.782356 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 92357.1us
W20260709 13:08:23.783473 16533 consensus_peers.cc:597] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:23.785674 24036 ts_tablet_manager.cc:1403] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.438s	user 0.053s	sys 0.019s
I20260709 13:08:23.796669 24036 raft_consensus.cc:359] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:08:23.797775 24036 raft_consensus.cc:740] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
I20260709 13:08:23.798662 24036 consensus_queue.cc:260] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
W20260709 13:08:23.828919 16533 consensus_peers.cc:597] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:23.847324 15974 consensus_peers.cc:597] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:08:23.860859 24039 tablet_bootstrap.cc:492] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=2 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:23.874346 24039 tablet_bootstrap.cc:492] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5: Bootstrap complete.
I20260709 13:08:23.909914 24039 ts_tablet_manager.cc:1403] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5: Time spent bootstrapping tablet: real 0.400s	user 0.042s	sys 0.016s
W20260709 13:08:23.911724 16533 consensus_peers.cc:597] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:08:23.933171 24036 ts_tablet_manager.cc:1434] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.147s	user 0.038s	sys 0.033s
I20260709 13:08:23.938031 15946 tablet_copy_service.cc:342] P 8430db80a39d4868bcf7641c16de35ea: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-05f069356f3b4f72a756c3367d8713b7 received from {username='slave'} at 127.0.0.1:53636
I20260709 13:08:23.938521 15946 tablet_copy_service.cc:434] P 8430db80a39d4868bcf7641c16de35ea: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-05f069356f3b4f72a756c3367d8713b7 on tablet 05f069356f3b4f72a756c3367d8713b7 with peer eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:08:23.941821 15974 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111) [suppressed 458 similar messages]
W20260709 13:08:23.951857 15974 consensus_peers.cc:597] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:23.953992 15900 consensus_peers.cc:597] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:08:23.961748 24039 raft_consensus.cc:359] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:08:23.962714 24039 raft_consensus.cc:740] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 LEARNER]: Becoming Follower/Learner. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Initialized, Role: LEARNER
W20260709 13:08:23.967469 16533 consensus_peers.cc:597] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:08:23.963400 24039 consensus_queue.cc:260] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: NON_VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: true } }
I20260709 13:08:23.993440 24039 ts_tablet_manager.cc:1434] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5: Time spent starting tablet: real 0.083s	user 0.007s	sys 0.001s
I20260709 13:08:24.044581 15946 tablet_copy_service.cc:342] P 8430db80a39d4868bcf7641c16de35ea: Request end of tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-975ecd3d27dd45ba9a554ac0ee591e0e received from {username='slave'} at 127.0.0.1:53636
I20260709 13:08:24.045068 15946 tablet_copy_service.cc:434] P 8430db80a39d4868bcf7641c16de35ea: ending tablet copy session eb1e0493af1e46dbb6dec7e25525bab5-975ecd3d27dd45ba9a554ac0ee591e0e on tablet 975ecd3d27dd45ba9a554ac0ee591e0e with peer eb1e0493af1e46dbb6dec7e25525bab5
W20260709 13:08:24.135221 15900 consensus_peers.cc:597] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:08:24.150684 16562 raft_consensus.cc:1217] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 LEARNER]: Deduplicated request from leader. Original: 20.13->[20.14-20.14]   Dedup: 20.14->[]
W20260709 13:08:24.182353 15974 consensus_peers.cc:597] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:24.266799 16562 raft_consensus.cc:1217] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 LEARNER]: Deduplicated request from leader. Original: 21.11->[21.12-21.12]   Dedup: 21.12->[]
W20260709 13:08:24.316210 15974 consensus_peers.cc:597] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:24.331061 16533 consensus_peers.cc:597] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:24.353791 15900 consensus_peers.cc:597] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:24.354423 15900 consensus_peers.cc:597] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:24.370868 15974 consensus_peers.cc:597] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:24.496196 15974 consensus_peers.cc:597] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:24.538801 15974 consensus_peers.cc:597] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:24.554909 15900 consensus_peers.cc:597] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:24.561218 16533 consensus_peers.cc:597] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:24.596043 15974 consensus_peers.cc:597] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:24.659667 15900 consensus_peers.cc:597] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:24.705108 24020 raft_consensus.cc:1064] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:08:24.708499 24020 consensus_queue.cc:237] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 12, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:08:24.715451 14748 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 15883)
W20260709 13:08:24.724356 15900 consensus_peers.cc:597] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:08:24.758810 16005 raft_consensus.cc:1275] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:08:24.759290 16561 raft_consensus.cc:1275] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 LEARNER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:08:24.760571 24033 consensus_queue.cc:1048] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:08:24.763758 24017 consensus_queue.cc:1048] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:08:24.785549 16533 consensus_peers.cc:597] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:24.789161 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 46119.3us
W20260709 13:08:24.807982 16533 consensus_peers.cc:597] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:24.814225 16533 consensus_peers.cc:597] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:24.814446 15974 consensus_peers.cc:597] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:24.841689 15900 consensus_peers.cc:597] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:24.843349 16533 consensus_peers.cc:597] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:24.846944 15900 consensus_peers.cc:597] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:24.867841 15900 consensus_peers.cc:597] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:24.871925 24042 raft_consensus.cc:1064] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea: attempting to promote NON_VOTER eb1e0493af1e46dbb6dec7e25525bab5 to VOTER
I20260709 13:08:24.875598 24042 consensus_queue.cc:237] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 10, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:08:24.947991 16533 consensus_peers.cc:597] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:08:24.951150 24033 raft_consensus.cc:2955] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 20 LEADER]: Committing config change with OpId 20.15: config changed from index 14 to 15, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:24.960932 16562 raft_consensus.cc:2955] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Committing config change with OpId 20.15: config changed from index 14 to 15, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:08:24.978046 15900 consensus_peers.cc:597] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:24.984455 16561 raft_consensus.cc:1275] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 LEARNER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:08:24.994570 16008 raft_consensus.cc:1275] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:08:25.013441 24042 consensus_queue.cc:1048] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:08:25.011623 15759 catalog_manager.cc:5697] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: config changed from index 14 to 15, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 20 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:25.032902 24020 consensus_queue.cc:1048] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:08:25.058619 16005 raft_consensus.cc:2955] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Committing config change with OpId 20.15: config changed from index 14 to 15, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:08:25.060127 15974 consensus_peers.cc:597] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:25.060709 15974 consensus_peers.cc:597] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:25.065841 15900 consensus_peers.cc:597] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:25.079699 15974 consensus_peers.cc:597] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:25.146641 24058 raft_consensus.cc:2955] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 21 LEADER]: Committing config change with OpId 21.13: config changed from index 12 to 13, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:25.169131 16562 raft_consensus.cc:2955] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Committing config change with OpId 21.13: config changed from index 12 to 13, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:08:25.179227 16533 consensus_peers.cc:597] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:08:25.183702 16005 raft_consensus.cc:2955] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Committing config change with OpId 21.13: config changed from index 12 to 13, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:08:25.199214 15900 consensus_peers.cc:597] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:25.228212 15974 consensus_peers.cc:597] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:08:25.250439 15757 catalog_manager.cc:5697] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: config changed from index 12 to 13, eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4) changed from NON_VOTER to VOTER. New cstate: current_term: 21 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } attrs { replace: true } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:08:25.266095 15900 consensus_peers.cc:597] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:25.311453 15931 consensus_queue.cc:237] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:25.319473 16005 raft_consensus.cc:1275] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:08:25.319569 16562 raft_consensus.cc:1275] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Refusing update from remote peer 8430db80a39d4868bcf7641c16de35ea: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:08:25.321579 24020 consensus_queue.cc:1048] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:08:25.327855 24020 consensus_queue.cc:1048] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.002s
I20260709 13:08:25.342235 24033 raft_consensus.cc:2955] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 21 LEADER]: Committing config change with OpId 21.14: config changed from index 13 to 14, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:08:25.343665 15900 consensus_peers.cc:597] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:25.360473 16005 raft_consensus.cc:2955] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Committing config change with OpId 21.14: config changed from index 13 to 14, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:25.372067 16562 raft_consensus.cc:2955] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Committing config change with OpId 21.14: config changed from index 13 to 14, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
W20260709 13:08:25.383138 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 70123.9us
I20260709 13:08:25.393613 15759 catalog_manager.cc:5697] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: config changed from index 13 to 14, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New cstate: current_term: 21 leader_uuid: "8430db80a39d4868bcf7641c16de35ea" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:25.394862 15744 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 975ecd3d27dd45ba9a554ac0ee591e0e with cas_config_opid_index 13: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:08:25.460126 15744 catalog_manager.cc:4755] TS 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 975ecd3d27dd45ba9a554ac0ee591e0e: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:25.526705 16533 consensus_peers.cc:597] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:25.536860 15974 consensus_peers.cc:597] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:25.589815 15974 consensus_peers.cc:597] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:25.644920 15974 consensus_peers.cc:597] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:25.716022 14748 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 15883)
W20260709 13:08:25.948565 15974 consensus_peers.cc:597] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:25.995990 16533 consensus_peers.cc:597] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:26.191396 16533 consensus_peers.cc:597] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:26.216995 15974 consensus_peers.cc:597] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:26.251304 15900 consensus_peers.cc:597] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:26.298204 16533 consensus_peers.cc:597] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:26.319919 15900 consensus_peers.cc:597] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:26.355170 15974 consensus_peers.cc:597] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:26.362517 15900 consensus_peers.cc:597] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:26.443742 16533 consensus_peers.cc:597] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:26.605554 15974 consensus_peers.cc:597] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:26.676985 15974 consensus_peers.cc:597] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:26.716648 14748 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 15883)
W20260709 13:08:26.719374 15974 consensus_peers.cc:597] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:26.738132 15744 catalog_manager.cc:4755] TS 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 975ecd3d27dd45ba9a554ac0ee591e0e: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:26.889564 16533 consensus_peers.cc:597] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:26.949326 16533 consensus_peers.cc:597] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:26.952221 15900 consensus_peers.cc:597] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:26.974254 15974 consensus_peers.cc:597] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:26.982412 15900 consensus_peers.cc:597] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:27.019999 15974 consensus_peers.cc:597] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:27.038633 15900 consensus_peers.cc:597] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:27.047612 15900 consensus_peers.cc:597] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:27.135692 15900 consensus_peers.cc:597] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:27.197847 15974 consensus_peers.cc:597] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:27.275760 16533 consensus_peers.cc:597] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:27.288879 15900 consensus_peers.cc:597] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:27.324741 15900 consensus_peers.cc:597] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:27.371905 15900 consensus_peers.cc:597] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:27.419138 16533 consensus_peers.cc:597] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:27.454473 16533 consensus_peers.cc:597] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:27.494626 15974 consensus_peers.cc:597] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:27.497902 16533 consensus_peers.cc:597] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:27.559648 15974 consensus_peers.cc:597] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:27.579224 16533 consensus_peers.cc:597] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:27.669528 15974 consensus_peers.cc:597] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:27.701088 15974 consensus_peers.cc:597] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:27.717717 14748 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 15883)
W20260709 13:08:27.719520 15974 consensus_peers.cc:597] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:27.723165 16533 consensus_peers.cc:597] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:27.748414 16533 consensus_peers.cc:597] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:27.768529 15900 consensus_peers.cc:597] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:27.820973 15744 catalog_manager.cc:4755] TS 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 975ecd3d27dd45ba9a554ac0ee591e0e: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:27.840209 15900 consensus_peers.cc:597] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:27.887318 15974 consensus_peers.cc:597] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:27.924564 15974 consensus_peers.cc:597] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:27.939787 15900 consensus_peers.cc:597] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:28.049548 15900 consensus_peers.cc:597] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:28.083348 15974 consensus_peers.cc:597] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:28.136440 15974 consensus_peers.cc:597] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:28.468868 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 135822us
W20260709 13:08:28.516440 16533 consensus_peers.cc:597] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:28.626502 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 57015.1us
W20260709 13:08:28.638806 16533 consensus_peers.cc:597] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:28.657465 15974 consensus_peers.cc:597] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:28.693449 15900 consensus_peers.cc:597] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:28.718431 14748 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 15883)
W20260709 13:08:28.735981 15900 consensus_peers.cc:597] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:28.739542 16533 consensus_peers.cc:597] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:28.874763 15974 consensus_peers.cc:597] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:28.911832 15900 consensus_peers.cc:597] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:28.913040 15974 consensus_peers.cc:597] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:28.915352 15900 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111) [suppressed 456 similar messages]
W20260709 13:08:28.996724 15974 consensus_peers.cc:597] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:29.072090 15974 consensus_peers.cc:597] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:29.172309 16533 consensus_peers.cc:597] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:29.710969 16591 debug-util.cc:398] Leaking SignalData structure 0x7b08003747a0 after lost signal to thread 14751
W20260709 13:08:29.712482 16591 debug-util.cc:398] Leaking SignalData structure 0x7b0800249160 after lost signal to thread 15794
W20260709 13:08:29.714316 16591 debug-util.cc:398] Leaking SignalData structure 0x7b08002e3a60 after lost signal to thread 15961
W20260709 13:08:29.716290 16591 debug-util.cc:398] Leaking SignalData structure 0x7b08002e3120 after lost signal to thread 16035
W20260709 13:08:29.718199 16591 debug-util.cc:398] Leaking SignalData structure 0x7b08000a93c0 after lost signal to thread 16594
W20260709 13:08:29.719002 14748 thread.cc:527] Waited for 10000ms trying to join with diag-logger (tid 15883)
W20260709 13:08:29.952008 15974 consensus_peers.cc:597] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:29.961870 15976 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.014s	sys 0.025s
W20260709 13:08:29.962306 15976 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.014s	sys 0.029s
W20260709 13:08:30.041739 16533 consensus_peers.cc:597] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:30.062104 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 80444.9us
W20260709 13:08:30.111197 15744 catalog_manager.cc:4755] TS 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 975ecd3d27dd45ba9a554ac0ee591e0e: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:30.147380 15900 consensus_peers.cc:597] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:30.149363 15900 consensus_peers.cc:597] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:30.150915 15900 consensus_peers.cc:597] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:30.153964 15900 consensus_peers.cc:597] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:30.164727 15974 consensus_peers.cc:597] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:30.180220 16533 consensus_peers.cc:597] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:30.181862 16533 consensus_peers.cc:597] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:30.203616 16533 consensus_peers.cc:597] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:30.255100 15974 consensus_peers.cc:597] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:30.255838 15974 consensus_peers.cc:597] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:30.314276 15974 consensus_peers.cc:597] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:30.320616 16533 consensus_peers.cc:597] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:30.328580 15900 consensus_peers.cc:597] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:30.376674 15900 consensus_peers.cc:597] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:30.388316 15791 debug-util.cc:398] Leaking SignalData structure 0x7b0800008fa0 after lost signal to thread 14751
W20260709 13:08:30.389479 15791 debug-util.cc:398] Leaking SignalData structure 0x7b080026ba00 after lost signal to thread 15794
W20260709 13:08:30.390882 15791 debug-util.cc:398] Leaking SignalData structure 0x7b08003788c0 after lost signal to thread 15961
W20260709 13:08:30.392167 15791 debug-util.cc:398] Leaking SignalData structure 0x7b080016b0e0 after lost signal to thread 16035
W20260709 13:08:30.399061 15974 consensus_peers.cc:597] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:30.400769 15791 debug-util.cc:398] Leaking SignalData structure 0x7b080034ad00 after lost signal to thread 16594
W20260709 13:08:30.495808 16533 consensus_peers.cc:597] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:30.501878 15900 consensus_peers.cc:597] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:30.511227 16533 consensus_peers.cc:597] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:30.532460 15900 consensus_peers.cc:597] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:30.550527 16533 consensus_peers.cc:597] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:30.578083 15974 consensus_peers.cc:597] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:30.575572 16533 consensus_peers.cc:597] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:30.652879 15900 consensus_peers.cc:597] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:30.665130 15974 consensus_peers.cc:597] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:30.719511 14748 thread.cc:527] Waited for 11000ms trying to join with diag-logger (tid 15883)
W20260709 13:08:30.740219 15974 consensus_peers.cc:597] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:30.747866 15900 consensus_peers.cc:597] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:30.758105 15974 consensus_peers.cc:597] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:30.947855 15974 consensus_peers.cc:597] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:30.980356 15900 consensus_peers.cc:597] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:31.052947 15974 consensus_peers.cc:597] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:31.194757 15900 consensus_peers.cc:597] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:31.237218 15974 consensus_peers.cc:597] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:08:31.302536 15974 consensus_peers.cc:597] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:31.354852 15900 consensus_peers.cc:597] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:08:31.371886 15900 consensus_peers.cc:597] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:08:31.405567 16533 consensus_peers.cc:597] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:08:31.430991 15974 consensus_peers.cc:597] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:31.529368 16533 consensus_peers.cc:597] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:08:31.585523 16533 consensus_peers.cc:597] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:31.720923 14748 thread.cc:527] Waited for 12000ms trying to join with diag-logger (tid 15883)
W20260709 13:08:31.794840 15974 consensus_peers.cc:597] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:31.962589 15974 consensus_peers.cc:597] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:08:32.048193 15974 consensus_peers.cc:597] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:08:32.078308 15900 consensus_peers.cc:597] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:08:32.133008 16535 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 86642.3us
W20260709 13:08:32.156008 15902 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:46251, user_credentials={real_user=slave}} blocked reactor thread for 97494.9us
W20260709 13:08:32.178960 16533 consensus_peers.cc:597] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:08:32.230844 15900 consensus_peers.cc:597] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:32.377362 16533 consensus_peers.cc:597] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:08:32.400137 15900 consensus_peers.cc:597] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:08:32.458621 16533 consensus_peers.cc:597] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:08:32.461656 16533 consensus_peers.cc:597] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:32.494390 15900 consensus_peers.cc:597] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:32.580380 15974 consensus_peers.cc:597] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:32.580833 16533 consensus_peers.cc:597] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:08:32.594785 15900 consensus_peers.cc:597] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:32.684687 15900 consensus_peers.cc:597] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:08:32.718993 15974 consensus_peers.cc:597] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:08:32.733481 15974 consensus_peers.cc:597] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:08:32.795293 16533 consensus_peers.cc:597] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:32.857472 15974 consensus_peers.cc:597] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:33.068009 15900 consensus_peers.cc:597] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:08:33.068845 15900 consensus_peers.cc:597] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:08:33.109931 15974 consensus_peers.cc:597] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:33.131321 15974 consensus_peers.cc:597] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:08:33.152494 15974 consensus_peers.cc:597] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:08:33.206146 16533 consensus_peers.cc:597] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:33.206972 16533 consensus_peers.cc:597] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:08:33.221942 15900 consensus_peers.cc:597] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:33.274564 16533 consensus_peers.cc:597] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:08:33.290311 16533 consensus_peers.cc:597] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:08:33.297263 15900 consensus_peers.cc:597] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:33.361219 15974 consensus_peers.cc:597] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:33.362337 15900 consensus_peers.cc:597] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:33.507484 15974 consensus_peers.cc:597] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:33.560866 15974 consensus_peers.cc:597] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:33.732331 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 50163.3us
W20260709 13:08:33.751482 15900 consensus_peers.cc:597] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:33.784660 15974 consensus_peers.cc:597] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:08:33.790728 15900 consensus_peers.cc:597] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:33.872073 16533 consensus_peers.cc:597] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:08:33.916265 15958 debug-util.cc:398] Leaking SignalData structure 0x7b08000b6d60 after lost signal to thread 14751
W20260709 13:08:33.917374 15958 debug-util.cc:398] Leaking SignalData structure 0x7b08002edfc0 after lost signal to thread 15794
W20260709 13:08:33.918689 15958 debug-util.cc:398] Leaking SignalData structure 0x7b0800202a60 after lost signal to thread 15961
W20260709 13:08:33.919812 15958 debug-util.cc:398] Leaking SignalData structure 0x7b08002d0e40 after lost signal to thread 16035
W20260709 13:08:33.920928 15958 debug-util.cc:398] Leaking SignalData structure 0x7b08003b7d20 after lost signal to thread 16594
W20260709 13:08:33.972291 15900 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111) [suppressed 415 similar messages]
W20260709 13:08:33.982538 15900 consensus_peers.cc:597] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:08:34.001543 15974 consensus_peers.cc:597] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:08:34.044250 15974 consensus_peers.cc:597] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:08:34.053597 14748 tablet_server.cc:196] TabletServer@127.14.103.1:0 shutdown complete.
W20260709 13:08:34.114547 15974 consensus_peers.cc:597] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:34.165382 15900 consensus_peers.cc:597] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:08:34.179876 16533 consensus_peers.cc:597] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:08:34.193784 16533 consensus_peers.cc:597] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:34.314455 15744 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 66257.4us
W20260709 13:08:34.321355 15974 consensus_peers.cc:597] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:08:34.323726 15744 catalog_manager.cc:4755] TS 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 975ecd3d27dd45ba9a554ac0ee591e0e: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:34.326328 15974 consensus_peers.cc:597] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:08:34.406901 14748 tablet_server.cc:179] TabletServer@127.14.103.2:0 shutting down...
W20260709 13:08:34.426395 16533 consensus_peers.cc:597] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:34.508864 15974 consensus_peers.cc:597] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:08:34.534737 16533 consensus_peers.cc:597] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:34.542892 16533 consensus_peers.cc:597] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:34.550448 15974 consensus_peers.cc:597] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:34.556450 15900 consensus_peers.cc:597] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:34.597899 15974 consensus_peers.cc:597] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:34.650888 16533 consensus_peers.cc:597] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:34.653921 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 44407.7us
W20260709 13:08:34.655634 15900 consensus_peers.cc:597] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:08:34.659003 14748 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:34.686183 16562 consensus_queue.cc:237] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:34.729588 14748 tablet_replica.cc:333] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:08:34.731130 14748 raft_consensus.cc:2243] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:08:34.733419 14748 raft_consensus.cc:2272] T 975ecd3d27dd45ba9a554ac0ee591e0e P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:34.743532 14748 tablet_replica.cc:333] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:08:34.744732 14748 raft_consensus.cc:2243] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:34.745769 14748 raft_consensus.cc:2272] T 7c0fef632f9a4485838855b553a79163 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:34.747399 15900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38033, user_credentials={real_user=slave}} blocked reactor thread for 91831.3us
W20260709 13:08:34.748010 16533 consensus_peers.cc:597] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:08:34.766898 16009 raft_consensus.cc:1275] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:08:34.769913 24105 consensus_queue.cc:1048] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 13:08:34.755492 16533 consensus_peers.cc:597] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:34.792906 14748 tablet_replica.cc:333] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:08:34.795660 24120 raft_consensus.cc:2955] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 LEADER]: Committing config change with OpId 21.17: config changed from index 10 to 17, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:34.801442 14748 raft_consensus.cc:2243] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:08:34.804517 16009 raft_consensus.cc:2955] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Committing config change with OpId 21.17: config changed from index 10 to 17, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } } }
I20260709 13:08:34.829843 15746 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet dfeaf1f6660d48848ae69710739427a0 with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER succeeded (attempt 12)
I20260709 13:08:34.846802 14748 raft_consensus.cc:2272] T 1d9c19e253ce4d53bb9e2f2241d3cafc P 8430db80a39d4868bcf7641c16de35ea [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:34.850760 14748 tablet_replica.cc:333] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:08:34.857228 15760 catalog_manager.cc:5697] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: config changed from index 10 to 17, VOTER 31882606e87447fb9a36bbf625129fe8 (127.14.103.1) evicted. New cstate: current_term: 21 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:08:34.862718 16533 consensus_peers.cc:597] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:08:34.867882 14748 raft_consensus.cc:2243] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:08:34.869200 14748 raft_consensus.cc:2272] T f9822eb87c7e4819af0dc0b0d2951def P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:34.892791 16533 consensus_peers.cc:597] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:34.890969 14748 tablet_replica.cc:333] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
W20260709 13:08:34.910038 16533 consensus_peers.cc:597] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:34.910480 14748 raft_consensus.cc:2243] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:34.911451 14748 raft_consensus.cc:2272] T c65db3107b664e9c8ce9be2cd2cb36c6 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:34.912242 15974 consensus_peers.cc:597] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:34.912909 15974 consensus_peers.cc:597] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:34.913544 15974 consensus_peers.cc:597] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:34.904114 15900 consensus_peers.cc:597] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:08:34.916033 14748 tablet_replica.cc:333] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
W20260709 13:08:34.916152 15900 consensus_peers.cc:597] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:34.916375 16533 consensus_peers.cc:597] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:08:34.917182 14748 raft_consensus.cc:2243] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:34.918241 14748 raft_consensus.cc:2272] T 260d22f804a345ed8f055536994fcb14 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:34.929386 15974 consensus_peers.cc:597] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:34.930164 15974 consensus_peers.cc:597] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:34.947836 14748 tablet_replica.cc:333] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
W20260709 13:08:34.951001 15974 consensus_peers.cc:597] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:34.953013 14748 raft_consensus.cc:2243] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:34.954555 14748 raft_consensus.cc:2272] T bbd61c62d0f0489291627ee496b25051 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:34.953586 16533 consensus_peers.cc:597] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:08:34.958410 15974 consensus_peers.cc:597] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:34.957665 16533 consensus_peers.cc:597] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:34.960397 15974 consensus_peers.cc:597] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:34.961581 15974 consensus_peers.cc:597] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:34.962212 16533 consensus_peers.cc:597] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:34.964745 15974 consensus_peers.cc:597] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:34.965979 15974 consensus_peers.cc:597] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:34.966773 16533 consensus_peers.cc:597] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:34.968271 15974 consensus_peers.cc:597] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:34.971266 16533 consensus_peers.cc:597] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:34.972019 16533 consensus_peers.cc:597] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:34.973680 16533 consensus_peers.cc:597] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:34.970165 14748 tablet_replica.cc:333] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:08:34.975821 14748 raft_consensus.cc:2243] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:08:34.981474 14748 raft_consensus.cc:2272] T 4fcbf678894c4a1eb80c68b5b53acaef P 8430db80a39d4868bcf7641c16de35ea [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:35.019464 14748 tablet_replica.cc:333] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:08:35.084142 14748 raft_consensus.cc:2243] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:35.085301 14748 raft_consensus.cc:2272] T f8b7d73618ae489990e21e8cc312177d P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:35.088954 14748 tablet_replica.cc:333] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
W20260709 13:08:35.100574 15976 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:43643, user_credentials={real_user=slave}} blocked reactor thread for 69695.6us
I20260709 13:08:35.137920 14748 raft_consensus.cc:2243] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:35.139753 14748 raft_consensus.cc:2272] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:35.144496 14748 tablet_replica.cc:333] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
W20260709 13:08:35.152376 15900 consensus_peers.cc:597] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:08:35.157083 14748 raft_consensus.cc:2243] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:08:35.158344 14748 raft_consensus.cc:2272] T d2be2d6f06bc47a39f26307ae9372194 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:35.161423 14748 tablet_replica.cc:333] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:08:35.170125 14748 raft_consensus.cc:2243] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:35.172374 14748 raft_consensus.cc:2272] T 6dc06813d5b7464fbdb39cde50f5ab48 P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:35.125125 16533 consensus_peers.cc:597] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:35.177721 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 52648.7us
W20260709 13:08:35.182742 15974 consensus_peers.cc:597] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:08:35.187026 14748 tablet_replica.cc:333] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:08:35.193078 14748 raft_consensus.cc:2243] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:35.193805 14748 raft_consensus.cc:2272] T eabbcb31af804f2dac6af5731e18a4dd P 8430db80a39d4868bcf7641c16de35ea [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:35.201248 14748 tablet_replica.cc:333] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:08:35.206890 14748 raft_consensus.cc:2243] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:08:35.209291 14748 raft_consensus.cc:2272] T 32207d388e0e4185898655b74ae976b1 P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:35.213450 14748 tablet_replica.cc:333] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:08:35.220379 14748 raft_consensus.cc:2243] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:35.223666 14748 raft_consensus.cc:2272] T 2f401f200ca34a14a539f8a74675eab4 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:35.215214 16533 consensus_peers.cc:597] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:08:35.228026 14748 tablet_replica.cc:333] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:08:35.233600 14748 raft_consensus.cc:2243] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:35.234375 14748 raft_consensus.cc:2272] T e71b32319bee4f979532b3ac6a16f2bd P 8430db80a39d4868bcf7641c16de35ea [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:35.215729 15974 consensus_peers.cc:597] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:08:35.252491 14748 tablet_replica.cc:333] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
W20260709 13:08:35.256393 15974 consensus_peers.cc:597] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:35.259871 15974 consensus_peers.cc:597] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:08:35.267781 14748 raft_consensus.cc:2243] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:35.274852 14748 raft_consensus.cc:2272] T 3a0a5e0b84774549a054f45adf4adf59 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:35.264506 16533 consensus_peers.cc:597] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:08:35.285045 14748 tablet_replica.cc:333] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:08:35.285957 14748 raft_consensus.cc:2243] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:35.286664 14748 raft_consensus.cc:2272] T 0b3676041725488fb6ffad37889b386b P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:35.301720 14748 tablet_replica.cc:333] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:08:35.302654 14748 raft_consensus.cc:2243] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:08:35.303995 14748 raft_consensus.cc:2272] T 91d0cc5018ee4a9fb3d196c783814c44 P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:35.324942 14748 tablet_replica.cc:333] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:08:35.325755 14748 raft_consensus.cc:2243] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:08:35.327005 14748 raft_consensus.cc:2272] T 9b72a30273dd49e883c237c7e5ad237b P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:35.338300 14748 tablet_replica.cc:333] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:08:35.339184 14748 raft_consensus.cc:2243] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:35.340051 14748 raft_consensus.cc:2272] T c04fe1420dfe4602909deef4baa3f115 P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:35.346403 14748 tablet_replica.cc:333] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:08:35.348313 14748 raft_consensus.cc:2243] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:35.349277 14748 raft_consensus.cc:2272] T b5d68c78f0d94e0eb90543b642d1f1af P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:35.353206 14748 tablet_replica.cc:333] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:08:35.354043 14748 raft_consensus.cc:2243] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:35.354772 14748 raft_consensus.cc:2272] T 78530df4cdb742eda67781671b0431c6 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:35.358574 14748 tablet_replica.cc:333] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:08:35.359422 14748 raft_consensus.cc:2243] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:35.360127 14748 raft_consensus.cc:2272] T d50c1e8d433a407aa9fae9cb9866f184 P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:35.363116 14748 tablet_replica.cc:333] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:08:35.363897 14748 raft_consensus.cc:2243] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:08:35.365199 14748 raft_consensus.cc:2272] T ac847593d6724522bda12a2207f91c31 P 8430db80a39d4868bcf7641c16de35ea [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:35.367823 14748 tablet_replica.cc:333] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:08:35.368736 14748 raft_consensus.cc:2243] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:35.369467 14748 raft_consensus.cc:2272] T dfeaf1f6660d48848ae69710739427a0 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:35.374976 14748 tablet_replica.cc:333] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:08:35.375775 14748 raft_consensus.cc:2243] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:08:35.382146 14748 raft_consensus.cc:2272] T 4e1a3c90a16e428fb2b328aff66b4ea6 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:35.385459 14748 tablet_replica.cc:333] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
W20260709 13:08:35.386181 15974 consensus_peers.cc:597] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:08:35.386631 14748 raft_consensus.cc:2243] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:08:35.392253 14748 raft_consensus.cc:2272] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:35.396349 14748 tablet_replica.cc:333] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:08:35.398619 14748 raft_consensus.cc:2243] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:08:35.399914 14748 raft_consensus.cc:2272] T 1b76daed20584023b02d5f1bc7964f58 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:35.406124 14748 tablet_replica.cc:333] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
W20260709 13:08:35.411094 15900 consensus_peers.cc:597] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:08:35.412086 14748 raft_consensus.cc:2243] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:35.412921 14748 raft_consensus.cc:2272] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:35.415707 14748 tablet_replica.cc:333] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:08:35.416575 14748 raft_consensus.cc:2243] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:35.417572 14748 raft_consensus.cc:2272] T ceaf02d91e334d5eba077b59ccce3a38 P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:35.424049 14748 tablet_replica.cc:333] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:08:35.424858 14748 raft_consensus.cc:2243] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:35.425526 14748 raft_consensus.cc:2272] T ae068fc0a8a84b148d068df2b4463b8c P 8430db80a39d4868bcf7641c16de35ea [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:35.428320 14748 tablet_replica.cc:333] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:08:35.430938 14748 raft_consensus.cc:2243] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:35.431774 14748 raft_consensus.cc:2272] T 741afa89185645afa8b1c82d31a75748 P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:35.434530 14748 tablet_replica.cc:333] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:08:35.435386 14748 raft_consensus.cc:2243] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:35.436493 14748 raft_consensus.cc:2272] T c138dd864152446f83386a4cd43c6efc P 8430db80a39d4868bcf7641c16de35ea [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:35.439280 14748 tablet_replica.cc:333] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:08:35.440143 14748 raft_consensus.cc:2243] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:08:35.441571 14748 raft_consensus.cc:2272] T ab61d516f4754d22b660c6fa5d052e3c P 8430db80a39d4868bcf7641c16de35ea [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:35.444561 14748 tablet_replica.cc:333] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:08:35.446091 14748 raft_consensus.cc:2243] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:08:35.448591 14748 raft_consensus.cc:2272] T ca01a56f016b4291b8abfddb75b74dd9 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:35.448997 15744 catalog_manager.cc:4755] TS 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet dfeaf1f6660d48848ae69710739427a0: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:35.449164 16533 consensus_peers.cc:597] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:08:35.452391 14748 tablet_replica.cc:333] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:08:35.453445 14748 raft_consensus.cc:2243] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:08:35.455180 14748 raft_consensus.cc:2272] T 05f069356f3b4f72a756c3367d8713b7 P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:35.458154 14748 tablet_replica.cc:333] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:08:35.459007 14748 raft_consensus.cc:2243] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:35.459769 14748 raft_consensus.cc:2272] T 9f65380951b7478bad13f7cbe01eece2 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:35.461896 24106 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:35.462354 24106 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:35.463162 14748 tablet_replica.cc:333] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:08:35.464129 14748 raft_consensus.cc:2243] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:08:35.464967 24106 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:35.465438 14748 raft_consensus.cc:2272] T e9577f76f40348e9b009255d441d012a P 8430db80a39d4868bcf7641c16de35ea [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:35.466240 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:35.466635 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd8e1402dafb4300911e5b0189c8fe0a" candidate_uuid: "8430db80a39d4868bcf7641c16de35ea" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:35.468971 15902 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8430db80a39d4868bcf7641c16de35ea; no voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:35.469782 24106 raft_consensus.cc:2749] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:35.470495 14748 tablet_replica.cc:333] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:08:35.471369 14748 raft_consensus.cc:2243] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:35.472311 14748 raft_consensus.cc:2272] T 3a538795cdae403fae5941538575596b P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:35.476004 14748 tablet_replica.cc:333] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:08:35.476788 14748 raft_consensus.cc:2243] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:35.482743 14748 raft_consensus.cc:2272] T 850b6b8a2f54468fb87ea6a22c5c870f P 8430db80a39d4868bcf7641c16de35ea [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:35.485790 14748 tablet_replica.cc:333] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:08:35.486779 14748 raft_consensus.cc:2243] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:35.487403 14748 raft_consensus.cc:2272] T 626a74f434ad4e70bed42f8b404a6e81 P 8430db80a39d4868bcf7641c16de35ea [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:35.491647 14748 tablet_replica.cc:333] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:08:35.492586 14748 raft_consensus.cc:2243] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:35.493330 14748 raft_consensus.cc:2272] T fd8e1402dafb4300911e5b0189c8fe0a P 8430db80a39d4868bcf7641c16de35ea [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:35.496012 14748 tablet_replica.cc:333] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:08:35.497121 14748 raft_consensus.cc:2243] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:08:35.499639 14748 raft_consensus.cc:2272] T 83a1b79cdcf046c78453695c798a14a5 P 8430db80a39d4868bcf7641c16de35ea [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:35.502410 14748 tablet_replica.cc:333] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea: stopping tablet replica
I20260709 13:08:35.503127 14748 raft_consensus.cc:2243] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:08:35.504314 14748 raft_consensus.cc:2272] T ffbfc0e667484157953a66ae2b697013 P 8430db80a39d4868bcf7641c16de35ea [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:35.551168 16533 consensus_peers.cc:597] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:08:35.618685 24116 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:35.619279 24116 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:35.623044 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:08:35.624472 16533 leader_election.cc:336] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:08:35.627234 16535 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:35.632128 24120 raft_consensus.cc:2749] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:35.645653 24116 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:35.677484 15974 consensus_peers.cc:597] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:35.760560 15974 consensus_peers.cc:597] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:08:35.769213 16533 consensus_peers.cc:597] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:08:35.796156 24120 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:35.796769 24120 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:35.798998 24120 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:08:35.813747 16533 leader_election.cc:336] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:35.816125 16533 leader_election.cc:336] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:35.816650 16533 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:35.817710 24120 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:08:35.870558 15974 consensus_peers.cc:597] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:08:35.997905 24117 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:35.998525 24117 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:36.001315 24117 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:36.001873 24117 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:36.002390 24117 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:36.009740 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:36.024614 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:36.025338 16562 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 21.
I20260709 13:08:36.026703 15976 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:36.028198 24136 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:08:36.028668 24136 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:36.029057 24136 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:08:36.044171 24117 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:36.081117 24136 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:36.084609 16562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "975ecd3d27dd45ba9a554ac0ee591e0e" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:36.085263 16562 raft_consensus.cc:3060] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:08:36.115562 15974 leader_election.cc:336] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:36.116107 15974 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:36.117120 24136 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:36.123356 24117 raft_consensus.cc:2749] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:36.124953 16562 raft_consensus.cc:2468] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 22.
W20260709 13:08:36.124966 15974 leader_election.cc:336] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:36.126971 15976 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:36.128063 24136 raft_consensus.cc:2804] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:08:36.128630 24136 raft_consensus.cc:697] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 22 LEADER]: Becoming Leader. State: Replica: 85e997ba5003439781ec4f1f8db0c3ee, State: Running, Role: LEADER
I20260709 13:08:36.129561 24136 consensus_queue.cc:237] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:08:36.132162 15974 consensus_peers.cc:597] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:36.148466 15974 leader_election.cc:336] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:36.169193 15760 catalog_manager.cc:5697] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee reported cstate change: term changed from 21 to 22, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3). New cstate: current_term: 22 leader_uuid: "85e997ba5003439781ec4f1f8db0c3ee" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:36.224475 15974 consensus_peers.cc:597] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:08:36.276567 24120 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:36.277137 24120 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:36.279138 24120 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:36.328233 24136 raft_consensus.cc:493] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:36.328766 24136 raft_consensus.cc:515] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
W20260709 13:08:36.330090 15974 consensus_peers.cc:597] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:36.330168 16533 leader_election.cc:336] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:36.331444 24136 leader_election.cc:290] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:08:36.335216 16533 leader_election.cc:336] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
I20260709 13:08:36.335660 16533 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:36.347875 24120 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:08:36.349431 24105 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:36.349918 24105 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:36.356825 24105 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:36.356972 16533 leader_election.cc:336] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:36.358458 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:36.359175 16005 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 22.
I20260709 13:08:36.361420 16535 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:36.362948 24105 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:08:36.365197 24105 raft_consensus.cc:493] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:36.366593 24105 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:08:36.382354 15974 leader_election.cc:336] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:36.387184 15974 leader_election.cc:336] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:36.388633 15974 leader_election.cc:304] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:36.389762 24120 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:36.390395 24120 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:36.394302 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
W20260709 13:08:36.398299 16533 leader_election.cc:336] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:36.399338 16533 leader_election.cc:336] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:36.399893 16533 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:36.400905 24120 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:36.402751 24120 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:36.406038 24117 raft_consensus.cc:493] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:36.406589 24117 raft_consensus.cc:515] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:36.403208 24120 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:36.408661 24117 leader_election.cc:290] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:36.410874 24117 raft_consensus.cc:2749] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:36.385309 24105 raft_consensus.cc:515] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:36.416386 24105 leader_election.cc:290] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:36.429306 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9822eb87c7e4819af0dc0b0d2951def" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:36.429965 16011 raft_consensus.cc:3060] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:08:36.432556 24105 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:36.443553 16011 raft_consensus.cc:2468] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 23.
I20260709 13:08:36.448446 16535 leader_election.cc:304] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:36.450055 24105 raft_consensus.cc:2804] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:08:36.450865 16533 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:43009, user_credentials={real_user=slave}} blocked reactor thread for 51554.3us
I20260709 13:08:36.450649 24105 raft_consensus.cc:697] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 LEADER]: Becoming Leader. State: Replica: eb1e0493af1e46dbb6dec7e25525bab5, State: Running, Role: LEADER
I20260709 13:08:36.450516 24120 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:08:36.456262 15974 leader_election.cc:336] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:36.455108 24105 consensus_queue.cc:237] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:08:36.461246 15974 consensus_peers.cc:597] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:08:36.461692 15974 leader_election.cc:336] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
I20260709 13:08:36.462137 15974 leader_election.cc:304] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:36.463012 24117 raft_consensus.cc:2749] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:08:36.465669 15974 consensus_peers.cc:597] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:36.471995 15974 consensus_peers.cc:597] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:36.480512 16533 leader_election.cc:336] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
W20260709 13:08:36.481527 16533 leader_election.cc:336] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
W20260709 13:08:36.482263 16533 leader_election.cc:336] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
I20260709 13:08:36.482774 16533 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:36.484905 24105 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:36.489907 15760 catalog_manager.cc:5697] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 reported cstate change: term changed from 22 to 23, leader changed from 8430db80a39d4868bcf7641c16de35ea (127.14.103.2) to eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4). New cstate: current_term: 23 leader_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:08:36.626533 16533 consensus_peers.cc:597] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:08:36.631717 16533 consensus_peers.cc:597] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:08:36.650722 16533 consensus_peers.cc:597] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:08:36.656519 24117 raft_consensus.cc:493] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:36.657050 24117 raft_consensus.cc:515] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:36.657744 24124 raft_consensus.cc:493] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:36.658265 24124 raft_consensus.cc:515] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:36.659011 24117 leader_election.cc:290] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:36.660231 24124 leader_election.cc:290] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:36.661105 24124 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:36.661700 24124 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:36.664669 24124 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:08:36.667907 15974 leader_election.cc:336] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
I20260709 13:08:36.668099 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
W20260709 13:08:36.668807 15974 leader_election.cc:336] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:36.669991 15974 leader_election.cc:336] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:36.670374 15974 leader_election.cc:304] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:36.671087 24105 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:36.671149 24124 raft_consensus.cc:2749] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:36.671720 24105 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:08:36.671203 15974 leader_election.cc:336] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:36.673473 15974 leader_election.cc:304] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:36.674011 24105 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:36.674130 24124 raft_consensus.cc:2749] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:08:36.674505 24105 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:36.674974 24105 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:36.676784 16011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83a1b79cdcf046c78453695c798a14a5" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:36.677383 16011 raft_consensus.cc:2468] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 22.
I20260709 13:08:36.677580 24105 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:08:36.681461 15974 leader_election.cc:336] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
I20260709 13:08:36.681922 15974 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:36.682591 24124 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:08:36.682695 15974 leader_election.cc:336] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
W20260709 13:08:36.695288 16533 leader_election.cc:336] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:36.695858 16533 leader_election.cc:336] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:36.696405 16533 leader_election.cc:336] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:36.696753 16533 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:36.699441 24105 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:08:36.700534 16533 leader_election.cc:336] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:36.700953 16533 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:36.702750 24105 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:36.704098 24124 raft_consensus.cc:493] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:36.704612 24124 raft_consensus.cc:515] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } }
I20260709 13:08:36.706624 24124 leader_election.cc:290] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:08:36.716439 15974 leader_election.cc:336] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:36.717077 15974 leader_election.cc:336] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:36.717456 15974 leader_election.cc:304] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:36.718169 24124 raft_consensus.cc:2749] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:36.721168 24105 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:36.721673 24105 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:36.723436 24105 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:08:36.736018 16533 leader_election.cc:336] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:36.736675 16533 leader_election.cc:336] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:36.737038 16533 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
W20260709 13:08:36.738847 15974 consensus_peers.cc:597] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:36.742112 24149 raft_consensus.cc:2749] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:36.746281 16565 raft_consensus.cc:1275] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Refusing update from remote peer 85e997ba5003439781ec4f1f8db0c3ee: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:08:36.747812 24117 consensus_queue.cc:1048] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:08:36.754698 24117 raft_consensus.cc:493] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:36.755177 24117 raft_consensus.cc:515] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } }
I20260709 13:08:36.761337 24117 leader_election.cc:290] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:08:36.782907 15974 leader_election.cc:336] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:36.788141 15974 leader_election.cc:336] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:36.788651 15974 leader_election.cc:304] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:36.789443 24117 raft_consensus.cc:2749] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:36.791692 24105 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:36.792259 24105 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:36.793713 24117 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:36.794238 24105 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:36.794184 24117 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:36.796003 24117 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:36.795900 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee" is_pre_election: true
I20260709 13:08:36.796634 16009 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1e0493af1e46dbb6dec7e25525bab5 in term 20.
I20260709 13:08:36.796967 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:36.797554 16565 raft_consensus.cc:2468] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 20.
I20260709 13:08:36.798683 16535 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:36.800732 15976 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 
I20260709 13:08:36.801451 24124 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:36.801813 24124 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:36.802150 24124 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:08:36.802685 24105 raft_consensus.cc:2804] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:08:36.803088 24105 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:36.803455 24105 raft_consensus.cc:3060] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:08:36.808120 16533 leader_election.cc:336] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:36.809270 24149 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:36.809804 24149 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:36.812139 24149 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:08:36.825019 15974 leader_election.cc:336] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:36.811908 24124 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:36.832255 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5"
I20260709 13:08:36.833361 24105 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:36.834652 16565 raft_consensus.cc:2393] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 85e997ba5003439781ec4f1f8db0c3ee in current term 21: Already voted for candidate eb1e0493af1e46dbb6dec7e25525bab5 in this term.
I20260709 13:08:36.836673 16009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4908a0c5a714f5e8c8f2b20cc7f2840" candidate_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "85e997ba5003439781ec4f1f8db0c3ee"
I20260709 13:08:36.836480 24124 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
I20260709 13:08:36.837404 16009 raft_consensus.cc:2393] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate eb1e0493af1e46dbb6dec7e25525bab5 in current term 21: Already voted for candidate 85e997ba5003439781ec4f1f8db0c3ee in this term.
W20260709 13:08:36.838708 16533 leader_election.cc:336] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
I20260709 13:08:36.837847 24105 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 election: Requested vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:36.843531 16533 leader_election.cc:336] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:36.843981 16533 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:36.844959 24149 raft_consensus.cc:2749] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:36.845660 24124 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:36.846225 24124 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:08:36.847905 16533 leader_election.cc:336] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:36.848467 16533 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:36.849745 24105 raft_consensus.cc:2749] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:08:36.851959 16565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05f069356f3b4f72a756c3367d8713b7" candidate_uuid: "85e997ba5003439781ec4f1f8db0c3ee" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" is_pre_election: true
I20260709 13:08:36.852550 16565 raft_consensus.cc:2468] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85e997ba5003439781ec4f1f8db0c3ee in term 20.
I20260709 13:08:36.855559 24124 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), eb1e0493af1e46dbb6dec7e25525bab5 (127.14.103.4:43643)
W20260709 13:08:36.858543 15974 leader_election.cc:336] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:36.859073 15974 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee; no voters: 8430db80a39d4868bcf7641c16de35ea, eb1e0493af1e46dbb6dec7e25525bab5
I20260709 13:08:36.859908 24124 raft_consensus.cc:2749] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:08:36.875895 15974 consensus_peers.cc:597] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:08:36.876509 15974 leader_election.cc:336] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:36.877666 15974 leader_election.cc:336] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:36.878172 15974 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 85e997ba5003439781ec4f1f8db0c3ee, eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:36.879124 24117 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:08:36.883800 15974 consensus_peers.cc:597] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:08:36.915058 24149 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:36.915815 24149 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:36.918141 24149 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:08:36.924521 16533 consensus_peers.cc:597] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:36.945250 16533 leader_election.cc:336] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
W20260709 13:08:36.952389 16533 leader_election.cc:336] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
I20260709 13:08:36.953748 16533 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:36.957327 24105 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:08:36.983469 16533 consensus_peers.cc:597] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:36.988375 16533 consensus_peers.cc:597] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:37.005582 16009 raft_consensus.cc:1275] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Refusing update from remote peer eb1e0493af1e46dbb6dec7e25525bab5: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:08:37.007982 24149 consensus_queue.cc:1048] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 13:08:37.044863 15974 consensus_peers.cc:597] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:37.064170 15974 consensus_peers.cc:597] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:37.064752 15974 consensus_peers.cc:597] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:37.065431 15974 consensus_peers.cc:597] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:08:37.091461 15974 consensus_peers.cc:597] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:37.091794 16533 consensus_peers.cc:597] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:37.092314 16533 consensus_peers.cc:597] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:37.103120 14748 tablet_server.cc:196] TabletServer@127.14.103.2:0 shutdown complete.
W20260709 13:08:37.112623 16533 consensus_peers.cc:597] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:37.132269 15974 consensus_peers.cc:597] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:37.133486 16533 consensus_peers.cc:597] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:37.149869 16533 consensus_peers.cc:597] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:08:37.180809 15974 consensus_peers.cc:597] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:37.187642 15974 consensus_peers.cc:597] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:37.190739 16533 consensus_peers.cc:597] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:37.196508 16533 consensus_peers.cc:597] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:37.196977 16533 consensus_peers.cc:597] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:37.200313 15974 consensus_peers.cc:597] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:37.209663 16533 consensus_peers.cc:597] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:37.254824 16533 consensus_peers.cc:597] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:37.258590 16533 consensus_peers.cc:597] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:37.294535 14748 tablet_server.cc:179] TabletServer@127.14.103.3:0 shutting down...
W20260709 13:08:37.315583 15974 consensus_peers.cc:597] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:37.329439 15974 consensus_peers.cc:597] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:37.356617 16533 consensus_peers.cc:597] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:08:37.377427 16533 consensus_peers.cc:597] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:37.378769 16535 consensus_peers.cc:597] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Couldn't send request to peer 85e997ba5003439781ec4f1f8db0c3ee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:37.381328 14748 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:37.388160 14748 tablet_replica.cc:333] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.389189 14748 raft_consensus.cc:2243] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:08:37.390439 14748 raft_consensus.cc:2272] T dd0dfa7599d64160b42b750a38e5c239 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.393237 14748 tablet_replica.cc:333] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
W20260709 13:08:37.398699 16535 consensus_peers.cc:597] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Couldn't send request to peer 85e997ba5003439781ec4f1f8db0c3ee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:37.400362 14748 raft_consensus.cc:2243] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:08:37.401983 14748 raft_consensus.cc:2272] T 975ecd3d27dd45ba9a554ac0ee591e0e P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:37.403182 15744 catalog_manager.cc:4755] TS 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet dfeaf1f6660d48848ae69710739427a0: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
I20260709 13:08:37.410117 14748 tablet_replica.cc:333] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.411087 14748 raft_consensus.cc:2243] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:08:37.412144 14748 raft_consensus.cc:2272] T 7b6e87097733428bb2167c6f308fe589 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.415023 14748 tablet_replica.cc:333] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.415951 14748 raft_consensus.cc:2243] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:37.416693 14748 raft_consensus.cc:2272] T 8849099536af497bae91c72da893681c P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:37.427637 15974 consensus_peers.cc:597] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:37.428272 14748 tablet_replica.cc:333] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.429240 14748 raft_consensus.cc:2243] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:08:37.430423 14748 raft_consensus.cc:2272] T 7c0fef632f9a4485838855b553a79163 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:37.428301 15974 consensus_peers.cc:597] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:37.433380 16535 consensus_peers.cc:597] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Couldn't send request to peer 85e997ba5003439781ec4f1f8db0c3ee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:37.434479 14748 tablet_replica.cc:333] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.435506 14748 raft_consensus.cc:2243] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:37.436228 14748 raft_consensus.cc:2272] T f8b7d73618ae489990e21e8cc312177d P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.439345 14748 tablet_replica.cc:333] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.440208 14748 raft_consensus.cc:2243] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:37.440816 14748 raft_consensus.cc:2272] T f9822eb87c7e4819af0dc0b0d2951def P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.443991 14748 tablet_replica.cc:333] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.444906 14748 raft_consensus.cc:2243] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:37.445488 14748 raft_consensus.cc:2272] T 73f4b066dd3d464699e6b00ebf1cdc44 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.448350 14748 tablet_replica.cc:333] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
W20260709 13:08:37.449180 16535 consensus_peers.cc:597] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Couldn't send request to peer 85e997ba5003439781ec4f1f8db0c3ee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:37.449347 14748 raft_consensus.cc:2243] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:08:37.450666 14748 raft_consensus.cc:2272] T c65db3107b664e9c8ce9be2cd2cb36c6 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.453851 14748 tablet_replica.cc:333] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.454769 14748 raft_consensus.cc:2243] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:37.455082 24156 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:37.455361 14748 raft_consensus.cc:2272] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.458564 14748 tablet_replica.cc:333] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.459498 14748 raft_consensus.cc:2243] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:08:37.460615 14748 raft_consensus.cc:2272] T 6dc06813d5b7464fbdb39cde50f5ab48 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.455648 24156 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:37.463471 14748 tablet_replica.cc:333] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.464468 14748 raft_consensus.cc:2243] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:37.465068 14748 raft_consensus.cc:2272] T 679229ffe7114facacb8e3a2c4c6d226 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.468041 14748 tablet_replica.cc:333] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.468847 14748 raft_consensus.cc:2243] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 28 LEADER]: Raft consensus shutting down.
I20260709 13:08:37.470108 14748 raft_consensus.cc:2272] T e236a7bf118740c9b3d88227bed5670e P 85e997ba5003439781ec4f1f8db0c3ee [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:37.472826 16533 consensus_peers.cc:597] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:08:37.478086 14748 tablet_replica.cc:333] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.479089 14748 raft_consensus.cc:2243] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:08:37.480098 24156 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:37.480548 14748 raft_consensus.cc:2272] T e71b32319bee4f979532b3ac6a16f2bd P 85e997ba5003439781ec4f1f8db0c3ee [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:37.482605 16535 consensus_peers.cc:597] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Couldn't send request to peer 85e997ba5003439781ec4f1f8db0c3ee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:37.486121 14748 tablet_replica.cc:333] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.488060 14748 raft_consensus.cc:2243] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:08:37.489166 14748 raft_consensus.cc:2272] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:37.490759 16533 leader_election.cc:336] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:37.491544 16533 leader_election.cc:336] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:37.492074 14748 tablet_replica.cc:333] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.491993 16533 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:37.492919 24156 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:08:37.494402 14748 raft_consensus.cc:2243] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:37.495249 14748 raft_consensus.cc:2272] T 32207d388e0e4185898655b74ae976b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.498411 14748 tablet_replica.cc:333] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.499204 14748 raft_consensus.cc:2243] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:08:37.500396 14748 raft_consensus.cc:2272] T 0a10bba95cd1421aa3fc332639e97319 P 85e997ba5003439781ec4f1f8db0c3ee [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:37.503115 15974 consensus_peers.cc:597] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:37.503286 14748 tablet_replica.cc:333] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.504914 14748 raft_consensus.cc:2243] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:08:37.506045 14748 raft_consensus.cc:2272] T bbd61c62d0f0489291627ee496b25051 P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.509771 14748 tablet_replica.cc:333] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.510667 14748 raft_consensus.cc:2243] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:37.511313 14748 raft_consensus.cc:2272] T 0b3676041725488fb6ffad37889b386b P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.514191 14748 tablet_replica.cc:333] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.514976 14748 raft_consensus.cc:2243] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:37.515606 14748 raft_consensus.cc:2272] T d50c1e8d433a407aa9fae9cb9866f184 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.518674 14748 tablet_replica.cc:333] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.519440 14748 raft_consensus.cc:2243] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:37.520041 14748 raft_consensus.cc:2272] T ac847593d6724522bda12a2207f91c31 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.522519 14748 tablet_replica.cc:333] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.523187 14748 raft_consensus.cc:2243] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:37.524446 14748 raft_consensus.cc:2272] T dfeaf1f6660d48848ae69710739427a0 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.528115 14748 tablet_replica.cc:333] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.528843 14748 raft_consensus.cc:2243] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:37.529549 14748 raft_consensus.cc:2272] T 9b72a30273dd49e883c237c7e5ad237b P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:37.530920 16535 consensus_peers.cc:597] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Couldn't send request to peer 85e997ba5003439781ec4f1f8db0c3ee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:37.532003 14748 tablet_replica.cc:333] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.532784 14748 raft_consensus.cc:2243] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:08:37.533740 14748 raft_consensus.cc:2272] T b5d68c78f0d94e0eb90543b642d1f1af P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.539256 14748 tablet_replica.cc:333] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.540030 14748 raft_consensus.cc:2243] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:37.540885 14748 raft_consensus.cc:2272] T ab61d516f4754d22b660c6fa5d052e3c P 85e997ba5003439781ec4f1f8db0c3ee [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.544682 14748 tablet_replica.cc:333] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.545485 14748 raft_consensus.cc:2243] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 18 LEADER]: Raft consensus shutting down.
W20260709 13:08:37.547524 15974 consensus_peers.cc:597] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:37.548153 14748 raft_consensus.cc:2272] T 15301761b2ec47fe82ae2f431c8b34ee P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.551203 14748 tablet_replica.cc:333] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.552027 14748 raft_consensus.cc:2243] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:37.552783 14748 raft_consensus.cc:2272] T 952bd0ea3bba4d38ad3383fdf572cd01 P 85e997ba5003439781ec4f1f8db0c3ee [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:37.553707 16535 consensus_peers.cc:597] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Couldn't send request to peer 85e997ba5003439781ec4f1f8db0c3ee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:37.562078 14748 tablet_replica.cc:333] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
W20260709 13:08:37.563071 16533 consensus_peers.cc:597] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:08:37.562965 14748 raft_consensus.cc:2243] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:08:37.565763 14748 raft_consensus.cc:2272] T ecbdcd9cd6c944e7a9277f7c8f191e96 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.568413 14748 tablet_replica.cc:333] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.569159 14748 raft_consensus.cc:2243] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:37.569796 14748 raft_consensus.cc:2272] T ceaf02d91e334d5eba077b59ccce3a38 P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.572638 14748 tablet_replica.cc:333] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.573333 14748 raft_consensus.cc:2243] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:08:37.574326 14748 raft_consensus.cc:2272] T ae068fc0a8a84b148d068df2b4463b8c P 85e997ba5003439781ec4f1f8db0c3ee [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.577005 14748 tablet_replica.cc:333] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.577976 14748 raft_consensus.cc:2243] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:08:37.579033 14748 raft_consensus.cc:2272] T 741afa89185645afa8b1c82d31a75748 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.581626 14748 tablet_replica.cc:333] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.582332 14748 raft_consensus.cc:2243] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:08:37.583348 14748 raft_consensus.cc:2272] T c138dd864152446f83386a4cd43c6efc P 85e997ba5003439781ec4f1f8db0c3ee [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:37.583787 16533 consensus_peers.cc:597] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:08:37.587330 14748 tablet_replica.cc:333] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.589496 14748 raft_consensus.cc:2243] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:08:37.591950 14748 raft_consensus.cc:2272] T 3a0a5e0b84774549a054f45adf4adf59 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.594561 14748 tablet_replica.cc:333] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.595306 14748 raft_consensus.cc:2243] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:37.595955 14748 raft_consensus.cc:2272] T 1b76daed20584023b02d5f1bc7964f58 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.598568 14748 tablet_replica.cc:333] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.599291 14748 raft_consensus.cc:2243] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:37.600042 14748 raft_consensus.cc:2272] T 888cce851f074f3c969de10b02ff95ad P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.602569 14748 tablet_replica.cc:333] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.603240 14748 raft_consensus.cc:2243] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:37.603765 14748 raft_consensus.cc:2272] T f4908a0c5a714f5e8c8f2b20cc7f2840 P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.606297 14748 tablet_replica.cc:333] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.607049 14748 raft_consensus.cc:2243] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:08:37.608167 14748 raft_consensus.cc:2272] T 850b6b8a2f54468fb87ea6a22c5c870f P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.610836 14748 tablet_replica.cc:333] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.611811 14748 raft_consensus.cc:2243] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:37.612622 14748 raft_consensus.cc:2272] T 05f069356f3b4f72a756c3367d8713b7 P 85e997ba5003439781ec4f1f8db0c3ee [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.615482 14748 tablet_replica.cc:333] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.616245 14748 raft_consensus.cc:2243] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:08:37.617359 14748 raft_consensus.cc:2272] T c766f90082d54f299b03f3e9d75d2db0 P 85e997ba5003439781ec4f1f8db0c3ee [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.620451 14748 tablet_replica.cc:333] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.621140 14748 raft_consensus.cc:2243] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:37.621714 14748 raft_consensus.cc:2272] T e9577f76f40348e9b009255d441d012a P 85e997ba5003439781ec4f1f8db0c3ee [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.624511 14748 tablet_replica.cc:333] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.625174 14748 raft_consensus.cc:2243] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:08:37.626363 14748 raft_consensus.cc:2272] T 9f65380951b7478bad13f7cbe01eece2 P 85e997ba5003439781ec4f1f8db0c3ee [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.628823 14748 tablet_replica.cc:333] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.629495 14748 raft_consensus.cc:2243] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 24 LEADER]: Raft consensus shutting down.
W20260709 13:08:37.629992 16535 consensus_peers.cc:597] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Couldn't send request to peer 85e997ba5003439781ec4f1f8db0c3ee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:37.630550 14748 raft_consensus.cc:2272] T 471dbd2e16264fe5837e8b7ea3601f94 P 85e997ba5003439781ec4f1f8db0c3ee [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:37.633267 16535 consensus_peers.cc:597] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Couldn't send request to peer 85e997ba5003439781ec4f1f8db0c3ee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:37.633278 14748 tablet_replica.cc:333] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.634476 14748 raft_consensus.cc:2243] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:37.635049 14748 raft_consensus.cc:2272] T 40e7e6076e8d475da059aad457e0e8b1 P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.638394 14748 tablet_replica.cc:333] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.639025 14748 raft_consensus.cc:2243] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:08:37.640105 14748 raft_consensus.cc:2272] T fd8e1402dafb4300911e5b0189c8fe0a P 85e997ba5003439781ec4f1f8db0c3ee [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:37.654155 16535 consensus_peers.cc:597] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Couldn't send request to peer 85e997ba5003439781ec4f1f8db0c3ee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:37.658531 14748 tablet_replica.cc:333] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.659334 14748 raft_consensus.cc:2243] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:37.660408 14748 raft_consensus.cc:2272] T 83a1b79cdcf046c78453695c798a14a5 P 85e997ba5003439781ec4f1f8db0c3ee [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:37.663209 14748 tablet_replica.cc:333] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee: stopping tablet replica
I20260709 13:08:37.663900 14748 raft_consensus.cc:2243] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:37.664489 14748 raft_consensus.cc:2272] T 3a538795cdae403fae5941538575596b P 85e997ba5003439781ec4f1f8db0c3ee [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:37.665095 16535 consensus_peers.cc:597] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Couldn't send request to peer 85e997ba5003439781ec4f1f8db0c3ee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:37.669904 16535 consensus_peers.cc:597] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Couldn't send request to peer 85e997ba5003439781ec4f1f8db0c3ee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:37.670643 16535 consensus_peers.cc:597] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Couldn't send request to peer 85e997ba5003439781ec4f1f8db0c3ee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:37.675493 16535 consensus_peers.cc:597] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Couldn't send request to peer 85e997ba5003439781ec4f1f8db0c3ee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:37.740832 16533 consensus_peers.cc:597] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:08:37.966833 16533 consensus_peers.cc:597] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:08:38.038159 24160 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:38.038700 24160 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:38.041309 24160 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:08:38.046072 16533 leader_election.cc:336] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:38.046839 16533 leader_election.cc:336] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:38.047232 16533 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:38.047976 24160 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:08:38.161952 16533 consensus_peers.cc:597] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:08:38.170487 16533 consensus_peers.cc:597] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:08:38.316691 24149 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:38.317327 24149 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:38.319727 24149 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:08:38.324465 16533 leader_election.cc:336] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:38.325249 16533 leader_election.cc:336] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:38.325726 16533 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:38.326552 24149 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:38.459144 24149 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:38.459736 24149 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:38.462015 24149 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:08:38.468537 16533 leader_election.cc:336] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:38.475246 16533 leader_election.cc:336] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:38.475800 16533 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:38.476606 24156 raft_consensus.cc:2749] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:08:38.478921 24156 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:38.479454 24156 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:38.481613 24156 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:08:38.490093 16533 leader_election.cc:336] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:38.490971 16533 leader_election.cc:336] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:38.491389 16533 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:38.492136 24156 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:38.546155 24156 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:38.546679 24156 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:38.547472 24149 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:38.548079 24149 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:38.549247 24156 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:38.549747 24156 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:38.550251 24156 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:38.552721 24156 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:38.550727 24149 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:38.555935 24149 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:38.557801 24149 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:38.560353 24149 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:38.562273 16535 leader_election.cc:336] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111)
W20260709 13:08:38.562849 16535 leader_election.cc:336] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111)
W20260709 13:08:38.563336 16535 leader_election.cc:336] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111)
W20260709 13:08:38.567075 16535 leader_election.cc:336] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111)
W20260709 13:08:38.570880 16533 leader_election.cc:336] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
I20260709 13:08:38.571393 16533 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:38.572184 24149 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:08:38.574210 16533 leader_election.cc:336] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:38.574680 16533 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:38.575470 24156 raft_consensus.cc:2749] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:08:38.576243 16533 leader_election.cc:336] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
W20260709 13:08:38.576993 16533 leader_election.cc:336] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:38.577360 16533 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:38.578051 24156 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:08:38.581905 16533 leader_election.cc:336] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
I20260709 13:08:38.582477 16533 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:38.583196 24156 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:38.652679 24156 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:38.653192 24156 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:38.655223 24156 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:38.660403 16533 leader_election.cc:336] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
W20260709 13:08:38.661106 16535 leader_election.cc:336] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111)
I20260709 13:08:38.661707 16535 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:38.694682 24149 raft_consensus.cc:2749] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:38.698114 24149 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:38.699036 24149 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:38.702344 24149 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:38.705041 24149 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:38.712680 24149 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:38.718374 24149 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:38.722498 16535 leader_election.cc:336] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111)
W20260709 13:08:38.729140 16535 leader_election.cc:336] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111)
W20260709 13:08:38.733901 16533 leader_election.cc:336] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
I20260709 13:08:38.734396 16533 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:38.736004 24160 raft_consensus.cc:2749] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 13:08:38.738160 16533 leader_election.cc:336] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:38.738716 16533 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:38.739961 24149 raft_consensus.cc:2749] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:38.751123 24149 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:38.752254 24149 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:38.756489 24163 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:38.757642 24163 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:08:38.768481 16535 leader_election.cc:336] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111)
W20260709 13:08:38.771862 16535 leader_election.cc:336] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111)
I20260709 13:08:38.772680 24163 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:38.774297 24149 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:38.779230 14748 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 16032)
W20260709 13:08:38.781759 16533 leader_election.cc:336] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:38.782199 16533 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:38.782806 24149 raft_consensus.cc:2749] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:08:38.782995 16533 leader_election.cc:336] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:38.783475 16533 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:38.784230 24149 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:08:38.784641 16533 leader_election.cc:336] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
I20260709 13:08:38.793977 24149 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:38.794822 24149 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:38.798799 24149 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:38.801119 16535 leader_election.cc:336] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111)
W20260709 13:08:38.802060 16533 leader_election.cc:336] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
I20260709 13:08:38.802456 16533 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:38.803231 24163 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:38.844153 24149 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:38.845052 24149 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:38.847079 24149 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:38.852365 24163 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:38.853276 24163 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:08:38.855158 16533 leader_election.cc:336] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
I20260709 13:08:38.855856 24163 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:38.857720 16535 leader_election.cc:336] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111)
I20260709 13:08:38.858240 16535 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:38.860512 24163 raft_consensus.cc:2749] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:08:38.867239 16533 leader_election.cc:336] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:38.867810 16535 leader_election.cc:336] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111)
I20260709 13:08:38.868836 16535 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:08:38.869680 16533 consensus_peers.cc:597] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:08:38.870476 24149 raft_consensus.cc:2749] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:08:38.912103 24149 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:38.912918 24149 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:08:38.915164 16591 debug-util.cc:398] Leaking SignalData structure 0x7b08003bece0 after lost signal to thread 14751
W20260709 13:08:38.917520 16591 debug-util.cc:398] Leaking SignalData structure 0x7b080032cdc0 after lost signal to thread 15794
W20260709 13:08:38.921137 16591 debug-util.cc:398] Leaking SignalData structure 0x7b0800370b80 after lost signal to thread 16594
I20260709 13:08:38.915169 24149 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:38.928402 16535 leader_election.cc:336] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111)
W20260709 13:08:38.932184 16533 leader_election.cc:336] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:38.932714 16533 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:38.933282 24149 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:38.933516 24163 raft_consensus.cc:2749] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:38.934418 24149 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:38.942679 24149 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:38.947935 16535 leader_election.cc:336] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111)
W20260709 13:08:38.955097 16533 leader_election.cc:336] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:38.957089 16533 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:38.959867 24149 raft_consensus.cc:2749] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:08:38.972432 16533 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111) [suppressed 641 similar messages]
W20260709 13:08:38.974099 16533 consensus_peers.cc:597] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:08:38.997162 24163 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:38.997884 24163 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:39.000195 24163 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:39.005230 16535 leader_election.cc:336] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111)
W20260709 13:08:39.009794 16533 leader_election.cc:336] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
I20260709 13:08:39.010429 16533 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:39.011341 24163 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:39.014971 24163 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:39.015893 24163 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:39.018411 24163 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:39.025130 16535 leader_election.cc:336] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111)
I20260709 13:08:39.042483 24163 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:39.043531 24163 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:39.046397 24163 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
I20260709 13:08:39.049091 24149 raft_consensus.cc:493] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:39.050091 24149 raft_consensus.cc:515] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:39.052595 24149 leader_election.cc:290] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:08:39.074736 16533 leader_election.cc:336] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
I20260709 13:08:39.075394 16533 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:39.076363 24163 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:08:39.077824 16533 leader_election.cc:336] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:39.090123 16533 leader_election.cc:336] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:39.090766 16533 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:39.091596 24163 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:39.092686 24163 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:39.093662 24163 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:08:39.102038 16535 leader_election.cc:336] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111)
I20260709 13:08:39.103008 24163 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:39.108160 16533 leader_election.cc:336] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:39.116583 16533 consensus_peers.cc:597] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:08:39.122495 16533 leader_election.cc:336] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:39.123121 16533 leader_election.cc:304] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:39.124037 24163 raft_consensus.cc:2749] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:08:39.132253 16533 leader_election.cc:336] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
I20260709 13:08:39.132884 16533 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:39.133817 24163 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:08:39.361763 16533 consensus_peers.cc:597] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:39.368683 16533 consensus_peers.cc:597] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:39.384066 16533 consensus_peers.cc:597] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:39.412830 24149 raft_consensus.cc:493] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:39.413902 24149 raft_consensus.cc:515] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:39.417526 24149 leader_election.cc:290] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:08:39.456269 16533 leader_election.cc:336] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
W20260709 13:08:39.459604 16533 consensus_peers.cc:597] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:39.460529 16533 leader_election.cc:336] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
I20260709 13:08:39.461054 16533 leader_election.cc:304] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:39.461974 24149 raft_consensus.cc:2749] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:08:39.477695 16533 consensus_peers.cc:597] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:39.511358 16533 consensus_peers.cc:597] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:39.548652 16533 consensus_peers.cc:597] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:39.554380 15744 catalog_manager.cc:4755] TS 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet dfeaf1f6660d48848ae69710739427a0: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:39.564100 16533 consensus_peers.cc:597] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:39.564821 16533 consensus_peers.cc:597] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:39.670073 16533 consensus_peers.cc:597] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:39.703522 16533 consensus_peers.cc:597] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:39.734652 16533 consensus_peers.cc:597] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:08:39.779743 14748 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 16032)
W20260709 13:08:39.801630 16535 consensus_peers.cc:597] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Couldn't send request to peer 85e997ba5003439781ec4f1f8db0c3ee. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:39.814776 16533 consensus_peers.cc:597] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:39.815510 16533 consensus_peers.cc:597] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:39.859287 16533 consensus_peers.cc:597] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:08:39.939277 16535 consensus_peers.cc:597] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Couldn't send request to peer 85e997ba5003439781ec4f1f8db0c3ee. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:39.970677 16535 consensus_peers.cc:597] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Couldn't send request to peer 85e997ba5003439781ec4f1f8db0c3ee. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:39.996835 16535 consensus_peers.cc:597] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Couldn't send request to peer 85e997ba5003439781ec4f1f8db0c3ee. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:39.998265 16533 consensus_peers.cc:597] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:40.014297 16533 consensus_peers.cc:597] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:08:40.017625 24149 raft_consensus.cc:493] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:40.018464 24149 raft_consensus.cc:515] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:08:40.020180 16535 consensus_peers.cc:597] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Couldn't send request to peer 85e997ba5003439781ec4f1f8db0c3ee. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:40.020498 24149 leader_election.cc:290] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:08:40.026295 16533 leader_election.cc:336] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
W20260709 13:08:40.030179 16533 leader_election.cc:336] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
I20260709 13:08:40.030648 16533 leader_election.cc:304] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:40.031433 24149 raft_consensus.cc:2749] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:08:40.047535 24149 raft_consensus.cc:493] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:40.048280 24149 raft_consensus.cc:515] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:40.050536 24149 leader_election.cc:290] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:08:40.060284 16533 leader_election.cc:336] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
W20260709 13:08:40.064320 16533 leader_election.cc:336] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
I20260709 13:08:40.064833 16533 leader_election.cc:304] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:40.065668 24149 raft_consensus.cc:2749] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:08:40.096601 16535 consensus_peers.cc:597] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Couldn't send request to peer 85e997ba5003439781ec4f1f8db0c3ee. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:40.101898 16533 consensus_peers.cc:597] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:08:40.105738 16535 consensus_peers.cc:597] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Couldn't send request to peer 85e997ba5003439781ec4f1f8db0c3ee. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:40.163236 16535 consensus_peers.cc:597] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Couldn't send request to peer 85e997ba5003439781ec4f1f8db0c3ee. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:40.163772 16535 consensus_peers.cc:597] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Couldn't send request to peer 85e997ba5003439781ec4f1f8db0c3ee. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:40.168453 24149 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:40.169397 24149 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:40.172055 24149 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:40.176904 16533 leader_election.cc:336] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:40.178871 16535 leader_election.cc:336] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111)
I20260709 13:08:40.179404 16535 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:40.180624 24149 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:08:40.191371 16535 consensus_peers.cc:597] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Couldn't send request to peer 85e997ba5003439781ec4f1f8db0c3ee. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:40.218782 16535 consensus_peers.cc:597] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Couldn't send request to peer 85e997ba5003439781ec4f1f8db0c3ee. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:40.220336 16533 consensus_peers.cc:597] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:08:40.234084 16535 consensus_peers.cc:597] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Couldn't send request to peer 85e997ba5003439781ec4f1f8db0c3ee. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:40.236562 24149 raft_consensus.cc:493] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:40.237356 24149 raft_consensus.cc:515] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
W20260709 13:08:40.238380 16533 consensus_peers.cc:597] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:08:40.239538 24149 leader_election.cc:290] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:40.244508 16533 leader_election.cc:336] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
W20260709 13:08:40.245451 16535 leader_election.cc:336] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111)
I20260709 13:08:40.245857 16535 leader_election.cc:304] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:40.246572 24149 raft_consensus.cc:2749] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:08:40.298523 16535 consensus_peers.cc:597] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Couldn't send request to peer 85e997ba5003439781ec4f1f8db0c3ee. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:40.338165 24149 raft_consensus.cc:493] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:40.338716 24149 raft_consensus.cc:515] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:40.340873 24149 leader_election.cc:290] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:40.345853 16533 leader_election.cc:336] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
W20260709 13:08:40.346447 16535 leader_election.cc:336] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111)
I20260709 13:08:40.346861 16535 leader_election.cc:304] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:40.347544 24149 raft_consensus.cc:2749] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:08:40.360092 24149 raft_consensus.cc:493] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:40.360661 24149 raft_consensus.cc:515] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:40.362849 24149 leader_election.cc:290] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:40.367611 16535 leader_election.cc:336] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111)
W20260709 13:08:40.368929 16533 leader_election.cc:336] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:40.369462 16533 leader_election.cc:304] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:40.370167 24149 raft_consensus.cc:2749] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:40.403308 24149 raft_consensus.cc:493] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:40.404158 24149 raft_consensus.cc:515] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:40.408648 24163 raft_consensus.cc:493] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:40.409178 24163 raft_consensus.cc:515] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:40.411163 24149 leader_election.cc:290] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:40.411269 24163 leader_election.cc:290] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:40.413919 16535 leader_election.cc:336] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111)
W20260709 13:08:40.417255 16535 leader_election.cc:336] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111)
W20260709 13:08:40.417780 16533 leader_election.cc:336] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:40.419425 16533 leader_election.cc:304] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:40.421169 24149 raft_consensus.cc:2749] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:08:40.427101 16533 leader_election.cc:336] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
I20260709 13:08:40.427708 16533 leader_election.cc:304] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:40.428506 24163 raft_consensus.cc:2749] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 13:08:40.460879 16533 consensus_peers.cc:597] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:08:40.475874 16535 consensus_peers.cc:597] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Couldn't send request to peer 85e997ba5003439781ec4f1f8db0c3ee. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:40.602270 16533 consensus_peers.cc:597] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:08:40.685406 24149 raft_consensus.cc:493] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:40.685992 24149 raft_consensus.cc:515] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:40.688228 24149 leader_election.cc:290] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:40.692631 16533 leader_election.cc:336] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:40.694536 16535 leader_election.cc:336] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111)
I20260709 13:08:40.695010 16535 leader_election.cc:304] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:40.695775 24163 raft_consensus.cc:2749] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:08:40.748940 16533 consensus_peers.cc:597] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:08:40.780328 14748 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 16032)
I20260709 13:08:40.816551 24163 raft_consensus.cc:493] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:40.817188 24163 raft_consensus.cc:515] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:40.819533 24163 leader_election.cc:290] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:40.824967 16533 leader_election.cc:336] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
W20260709 13:08:40.826550 16535 leader_election.cc:336] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111)
I20260709 13:08:40.826973 16535 leader_election.cc:304] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:40.827793 24163 raft_consensus.cc:2749] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:08:40.859544 24163 raft_consensus.cc:493] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:40.860191 24163 raft_consensus.cc:515] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:40.862635 24163 leader_election.cc:290] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:40.869189 16535 leader_election.cc:336] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111)
W20260709 13:08:40.870739 16533 leader_election.cc:336] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
I20260709 13:08:40.871287 16533 leader_election.cc:304] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:40.872164 24149 raft_consensus.cc:2749] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:08:40.883919 24149 raft_consensus.cc:493] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:40.884475 24149 raft_consensus.cc:515] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:40.886689 24149 leader_election.cc:290] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:40.890762 16533 leader_election.cc:336] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:40.893174 16535 leader_election.cc:336] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111)
I20260709 13:08:40.893708 16535 leader_election.cc:304] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:40.894446 24149 raft_consensus.cc:2749] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:40.963922 24163 raft_consensus.cc:493] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:40.964025 24149 raft_consensus.cc:493] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:40.964645 24163 raft_consensus.cc:515] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:40.964771 24149 raft_consensus.cc:515] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:40.967235 24163 leader_election.cc:290] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:40.967379 24149 leader_election.cc:290] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:40.975031 16535 leader_election.cc:336] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111)
W20260709 13:08:40.975610 16535 leader_election.cc:336] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111)
W20260709 13:08:40.977790 16533 leader_election.cc:336] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
I20260709 13:08:40.978317 16533 leader_election.cc:304] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:40.979063 24163 raft_consensus.cc:2749] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:08:40.979266 16533 leader_election.cc:336] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
I20260709 13:08:40.979812 16533 leader_election.cc:304] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:40.980532 24163 raft_consensus.cc:2749] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:08:40.980746 16533 leader_election.cc:336] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:41.004688 24163 raft_consensus.cc:493] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:41.005232 24163 raft_consensus.cc:515] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:41.007476 24163 leader_election.cc:290] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
I20260709 13:08:41.011153 24163 raft_consensus.cc:493] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:41.011698 24163 raft_consensus.cc:515] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:41.013863 24163 leader_election.cc:290] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:41.014438 16533 leader_election.cc:336] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:41.015833 16535 leader_election.cc:336] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111)
I20260709 13:08:41.016363 16535 leader_election.cc:304] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:41.017879 24163 raft_consensus.cc:2749] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:08:41.020447 16533 leader_election.cc:336] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
W20260709 13:08:41.020982 16535 leader_election.cc:336] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111)
I20260709 13:08:41.021464 16535 leader_election.cc:304] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:41.022233 24163 raft_consensus.cc:2749] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:08:41.151746 24163 raft_consensus.cc:493] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:41.152381 24163 raft_consensus.cc:515] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:41.155027 24163 leader_election.cc:290] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:41.160348 16533 leader_election.cc:336] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:41.160907 16535 leader_election.cc:336] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111)
I20260709 13:08:41.161394 16535 leader_election.cc:304] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:41.162267 24163 raft_consensus.cc:2749] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:08:41.227890 24163 raft_consensus.cc:493] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:41.228847 24163 raft_consensus.cc:515] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } attrs { replace: true } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:41.232355 24163 leader_election.cc:290] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:41.239152 16533 leader_election.cc:336] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:41.240797 16535 leader_election.cc:336] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111)
I20260709 13:08:41.241302 16535 leader_election.cc:304] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
W20260709 13:08:41.242460 16533 leader_election.cc:336] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:41.242884 24163 raft_consensus.cc:2749] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:08:41.281312 24163 raft_consensus.cc:493] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:41.282332 24163 raft_consensus.cc:515] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:41.284940 24163 leader_election.cc:290] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:08:41.294692 16533 leader_election.cc:336] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:41.296002 16533 leader_election.cc:336] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:41.296487 16533 leader_election.cc:304] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:41.297477 24149 raft_consensus.cc:2749] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:08:41.388873 16533 consensus_peers.cc:597] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:08:41.549665 24163 raft_consensus.cc:493] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:41.550477 24163 raft_consensus.cc:515] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:41.553022 24163 leader_election.cc:290] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:08:41.558329 16533 leader_election.cc:336] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
W20260709 13:08:41.559165 16533 leader_election.cc:336] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
I20260709 13:08:41.559813 16533 leader_election.cc:304] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:41.560984 24163 raft_consensus.cc:2749] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:08:41.581651 16533 consensus_peers.cc:597] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:08:41.593195 16533 consensus_peers.cc:597] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:08:41.811213 16533 consensus_peers.cc:597] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:41.870059 24168 raft_consensus.cc:493] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.870647 24168 raft_consensus.cc:515] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:41.872696 24168 leader_election.cc:290] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:41.876052 16533 consensus_peers.cc:597] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:41.877506 16535 leader_election.cc:336] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111)
W20260709 13:08:41.879717 16533 leader_election.cc:336] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:08:41.880144 16533 leader_election.cc:304] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 8430db80a39d4868bcf7641c16de35ea, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:41.882591 24169 raft_consensus.cc:2749] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:08:41.923341 16533 consensus_peers.cc:597] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:41.941032 16533 consensus_peers.cc:597] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:41.948962 16533 consensus_peers.cc:597] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:41.960799 24168 raft_consensus.cc:493] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8430db80a39d4868bcf7641c16de35ea)
I20260709 13:08:41.961378 24168 raft_consensus.cc:515] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "8430db80a39d4868bcf7641c16de35ea" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 43009 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:41.963605 24168 leader_election.cc:290] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009)
W20260709 13:08:41.965472 16533 consensus_peers.cc:597] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:41.968483 16533 leader_election.cc:336] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
W20260709 13:08:41.969089 16533 leader_election.cc:336] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
I20260709 13:08:41.969451 16533 leader_election.cc:304] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 8430db80a39d4868bcf7641c16de35ea
I20260709 13:08:41.970134 24168 raft_consensus.cc:2749] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:08:42.035148 14748 tablet_server.cc:196] TabletServer@127.14.103.3:0 shutdown complete.
W20260709 13:08:42.066648 16533 consensus_peers.cc:597] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:42.067274 16533 consensus_peers.cc:597] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:42.137169 16533 consensus_peers.cc:597] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:42.139555 16533 consensus_peers.cc:597] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:42.147912 24168 raft_consensus.cc:493] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 85e997ba5003439781ec4f1f8db0c3ee)
I20260709 13:08:42.148557 24168 raft_consensus.cc:515] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "31882606e87447fb9a36bbf625129fe8" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38033 } } peers { permanent_uuid: "85e997ba5003439781ec4f1f8db0c3ee" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46251 } } peers { permanent_uuid: "eb1e0493af1e46dbb6dec7e25525bab5" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 43643 } attrs { promote: false } }
I20260709 13:08:42.157138 24168 leader_election.cc:290] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033), 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251)
W20260709 13:08:42.157402 16535 leader_election.cc:336] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111)
W20260709 13:08:42.158143 16533 leader_election.cc:336] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
I20260709 13:08:42.158695 16533 leader_election.cc:304] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb1e0493af1e46dbb6dec7e25525bab5; no voters: 31882606e87447fb9a36bbf625129fe8, 85e997ba5003439781ec4f1f8db0c3ee
I20260709 13:08:42.159575 24168 raft_consensus.cc:2749] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:08:42.178835 16533 consensus_peers.cc:597] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): Couldn't send request to peer 31882606e87447fb9a36bbf625129fe8. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:08:42.204555 14748 tablet_server.cc:179] TabletServer@127.14.103.4:0 shutting down...
W20260709 13:08:42.204502 16533 consensus_peers.cc:597] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:42.225281 16533 consensus_peers.cc:597] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:42.237812 16535 consensus_peers.cc:597] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Couldn't send request to peer 85e997ba5003439781ec4f1f8db0c3ee. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:42.254558 14748 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:42.255450 14748 tablet_replica.cc:333] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.256407 14748 raft_consensus.cc:2243] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:42.257266 14748 raft_consensus.cc:2272] T 83a1b79cdcf046c78453695c798a14a5 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.260023 14748 tablet_replica.cc:333] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.260788 14748 raft_consensus.cc:2243] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:08:42.262033 14748 raft_consensus.cc:2272] T dfeaf1f6660d48848ae69710739427a0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.264848 14748 tablet_replica.cc:333] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.265508 14748 raft_consensus.cc:2243] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:08:42.266602 14748 raft_consensus.cc:2272] T 679229ffe7114facacb8e3a2c4c6d226 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.270458 14748 tablet_replica.cc:333] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.271324 14748 raft_consensus.cc:2243] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:42.272217 14748 raft_consensus.cc:2272] T 4fcbf678894c4a1eb80c68b5b53acaef P eb1e0493af1e46dbb6dec7e25525bab5 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.274906 14748 tablet_replica.cc:333] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.275537 14748 raft_consensus.cc:2243] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:42.276357 14748 raft_consensus.cc:2272] T ca01a56f016b4291b8abfddb75b74dd9 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.279575 14748 tablet_replica.cc:333] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.280640 14748 raft_consensus.cc:2243] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:08:42.281996 14748 raft_consensus.cc:2272] T 2f401f200ca34a14a539f8a74675eab4 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.284628 14748 tablet_replica.cc:333] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.285383 14748 raft_consensus.cc:2243] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:42.286253 14748 raft_consensus.cc:2272] T 0a10bba95cd1421aa3fc332639e97319 P eb1e0493af1e46dbb6dec7e25525bab5 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.288663 14748 tablet_replica.cc:333] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.289253 14748 raft_consensus.cc:2243] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:08:42.290284 14748 raft_consensus.cc:2272] T 73f4b066dd3d464699e6b00ebf1cdc44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.292584 14748 tablet_replica.cc:333] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.293179 14748 raft_consensus.cc:2243] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:08:42.294221 14748 raft_consensus.cc:2272] T 952bd0ea3bba4d38ad3383fdf572cd01 P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.296546 14748 tablet_replica.cc:333] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.297184 14748 raft_consensus.cc:2243] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:42.297734 14748 raft_consensus.cc:2272] T 1d9c19e253ce4d53bb9e2f2241d3cafc P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.300060 14748 tablet_replica.cc:333] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.300809 14748 raft_consensus.cc:2243] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:42.301990 14748 raft_consensus.cc:2272] T 975ecd3d27dd45ba9a554ac0ee591e0e P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.304962 14748 tablet_replica.cc:333] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.305616 14748 raft_consensus.cc:2243] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:08:42.306974 14748 raft_consensus.cc:2272] T 40e7e6076e8d475da059aad457e0e8b1 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.309348 14748 tablet_replica.cc:333] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.310009 14748 raft_consensus.cc:2243] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:42.310729 14748 raft_consensus.cc:2272] T 471dbd2e16264fe5837e8b7ea3601f94 P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.313961 14748 tablet_replica.cc:333] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.314815 14748 raft_consensus.cc:2243] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:08:42.316596 14748 raft_consensus.cc:2272] T 626a74f434ad4e70bed42f8b404a6e81 P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:42.317277 16533 consensus_peers.cc:597] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): Couldn't send request to peer 8430db80a39d4868bcf7641c16de35ea. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:42.319947 14748 tablet_replica.cc:333] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.320609 14748 raft_consensus.cc:2243] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:08:42.321695 14748 raft_consensus.cc:2272] T 260d22f804a345ed8f055536994fcb14 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.324970 14748 tablet_replica.cc:333] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.325688 14748 raft_consensus.cc:2243] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:08:42.327167 14748 raft_consensus.cc:2272] T 0b3676041725488fb6ffad37889b386b P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:42.330148 16535 consensus_peers.cc:597] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 -> Peer 85e997ba5003439781ec4f1f8db0c3ee (127.14.103.3:46251): Couldn't send request to peer 85e997ba5003439781ec4f1f8db0c3ee. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.3:46251: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:42.330374 14748 tablet_replica.cc:333] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.331131 14748 raft_consensus.cc:2243] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:08:42.332345 14748 raft_consensus.cc:2272] T 6e422bf1c2eb46fcb2fe33e4eb00f888 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.334878 14748 tablet_replica.cc:333] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.335513 14748 raft_consensus.cc:2243] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:42.336472 14748 raft_consensus.cc:2272] T ffbfc0e667484157953a66ae2b697013 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.339828 14748 tablet_replica.cc:333] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.340675 14748 raft_consensus.cc:2243] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:42.341305 14748 raft_consensus.cc:2272] T fd8e1402dafb4300911e5b0189c8fe0a P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.343617 14748 tablet_replica.cc:333] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.344302 14748 raft_consensus.cc:2243] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:08:42.345359 14748 raft_consensus.cc:2272] T 8849099536af497bae91c72da893681c P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.347522 14748 tablet_replica.cc:333] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.348084 14748 raft_consensus.cc:2243] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:08:42.349046 14748 raft_consensus.cc:2272] T 888cce851f074f3c969de10b02ff95ad P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.351349 14748 tablet_replica.cc:333] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.352067 14748 raft_consensus.cc:2243] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:42.352746 14748 raft_consensus.cc:2272] T f4908a0c5a714f5e8c8f2b20cc7f2840 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.354961 14748 tablet_replica.cc:333] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.355497 14748 raft_consensus.cc:2243] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:08:42.356734 14748 raft_consensus.cc:2272] T f9822eb87c7e4819af0dc0b0d2951def P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.358982 14748 tablet_replica.cc:333] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.359501 14748 raft_consensus.cc:2243] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:42.360097 14748 raft_consensus.cc:2272] T d2be2d6f06bc47a39f26307ae9372194 P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.362684 14748 tablet_replica.cc:333] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.363463 14748 raft_consensus.cc:2243] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:08:42.364665 14748 raft_consensus.cc:2272] T 3a538795cdae403fae5941538575596b P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.366889 14748 tablet_replica.cc:333] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.367475 14748 raft_consensus.cc:2243] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:08:42.368564 14748 raft_consensus.cc:2272] T 78530df4cdb742eda67781671b0431c6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.370643 14748 tablet_replica.cc:333] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.371265 14748 raft_consensus.cc:2243] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:42.372097 14748 raft_consensus.cc:2272] T 05f069356f3b4f72a756c3367d8713b7 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.374801 14748 tablet_replica.cc:333] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.375391 14748 raft_consensus.cc:2243] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:42.376149 14748 raft_consensus.cc:2272] T c766f90082d54f299b03f3e9d75d2db0 P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.378182 14748 tablet_replica.cc:333] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.378727 14748 raft_consensus.cc:2243] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:08:42.379897 14748 raft_consensus.cc:2272] T ceaf02d91e334d5eba077b59ccce3a38 P eb1e0493af1e46dbb6dec7e25525bab5 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.382150 14748 tablet_replica.cc:333] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.382730 14748 raft_consensus.cc:2243] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:08:42.383772 14748 raft_consensus.cc:2272] T f8b7d73618ae489990e21e8cc312177d P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.385842 14748 tablet_replica.cc:333] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.386350 14748 raft_consensus.cc:2243] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:42.386967 14748 raft_consensus.cc:2272] T 7b6e87097733428bb2167c6f308fe589 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.388923 14748 tablet_replica.cc:333] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.389569 14748 raft_consensus.cc:2243] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:08:42.390705 14748 raft_consensus.cc:2272] T d50c1e8d433a407aa9fae9cb9866f184 P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.393272 14748 tablet_replica.cc:333] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.393780 14748 raft_consensus.cc:2243] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:42.394397 14748 raft_consensus.cc:2272] T 3b3e05ffd87b4bbcb04b315d3bfdf8ed P eb1e0493af1e46dbb6dec7e25525bab5 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.396548 14748 tablet_replica.cc:333] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.397092 14748 raft_consensus.cc:2243] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:42.397691 14748 raft_consensus.cc:2272] T b5d68c78f0d94e0eb90543b642d1f1af P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.399798 14748 tablet_replica.cc:333] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.400440 14748 raft_consensus.cc:2243] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:42.401197 14748 raft_consensus.cc:2272] T 91d0cc5018ee4a9fb3d196c783814c44 P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.403847 14748 tablet_replica.cc:333] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.404431 14748 raft_consensus.cc:2243] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:08:42.405439 14748 raft_consensus.cc:2272] T c04fe1420dfe4602909deef4baa3f115 P eb1e0493af1e46dbb6dec7e25525bab5 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.407510 14748 tablet_replica.cc:333] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.408072 14748 raft_consensus.cc:2243] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:42.408931 14748 raft_consensus.cc:2272] T 15301761b2ec47fe82ae2f431c8b34ee P eb1e0493af1e46dbb6dec7e25525bab5 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.411449 14748 tablet_replica.cc:333] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.412007 14748 raft_consensus.cc:2243] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:42.412618 14748 raft_consensus.cc:2272] T 4e1a3c90a16e428fb2b328aff66b4ea6 P eb1e0493af1e46dbb6dec7e25525bab5 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.414496 14748 tablet_replica.cc:333] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.414987 14748 raft_consensus.cc:2243] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:42.415472 14748 raft_consensus.cc:2272] T ae068fc0a8a84b148d068df2b4463b8c P eb1e0493af1e46dbb6dec7e25525bab5 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.417470 14748 tablet_replica.cc:333] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.417977 14748 raft_consensus.cc:2243] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:42.418495 14748 raft_consensus.cc:2272] T c138dd864152446f83386a4cd43c6efc P eb1e0493af1e46dbb6dec7e25525bab5 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.420560 14748 tablet_replica.cc:333] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.421017 14748 raft_consensus.cc:2243] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:42.421494 14748 raft_consensus.cc:2272] T ecbdcd9cd6c944e7a9277f7c8f191e96 P eb1e0493af1e46dbb6dec7e25525bab5 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.423470 14748 tablet_replica.cc:333] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.423957 14748 raft_consensus.cc:2243] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:42.424525 14748 raft_consensus.cc:2272] T dd0dfa7599d64160b42b750a38e5c239 P eb1e0493af1e46dbb6dec7e25525bab5 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.426546 14748 tablet_replica.cc:333] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.427001 14748 raft_consensus.cc:2243] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:42.427793 14748 raft_consensus.cc:2272] T e236a7bf118740c9b3d88227bed5670e P eb1e0493af1e46dbb6dec7e25525bab5 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:42.430008 14748 tablet_replica.cc:333] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5: stopping tablet replica
I20260709 13:08:42.430459 14748 raft_consensus.cc:2243] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:08:42.431555 14748 raft_consensus.cc:2272] T eabbcb31af804f2dac6af5731e18a4dd P eb1e0493af1e46dbb6dec7e25525bab5 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:42.535430 15744 catalog_manager.cc:4755] TS 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 975ecd3d27dd45ba9a554ac0ee591e0e: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
I20260709 13:08:42.809670 14748 tablet_server.cc:196] TabletServer@127.14.103.4:0 shutdown complete.
I20260709 13:08:42.893047 14748 master.cc:562] Master@127.14.103.62:42985 shutting down...
W20260709 13:08:42.926967 15808 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 13:08:42.927208 15808 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:08:42.927377 15808 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:08:43.691911 15744 catalog_manager.cc:4755] TS 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet dfeaf1f6660d48848ae69710739427a0: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:08:58.930953 15744 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111) [suppressed 331 similar messages]
W20260709 13:08:58.935284 15744 catalog_manager.cc:4755] TS 31882606e87447fb9a36bbf625129fe8 (127.14.103.1:38033): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 975ecd3d27dd45ba9a554ac0ee591e0e: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38033: connect: Connection refused (error 111)
W20260709 13:09:02.477680 15744 catalog_manager.cc:4755] TS 8430db80a39d4868bcf7641c16de35ea (127.14.103.2:43009): ChangeConfig:REMOVE_PEER RPC failed for tablet 83a1b79cdcf046c78453695c798a14a5: Network error: Client connection negotiation failed: client connection to 127.14.103.2:43009: connect: Connection refused (error 111)
I20260709 13:09:04.181880 14748 raft_consensus.cc:2243] T 00000000000000000000000000000000 P d33cebb189b14a989836214fdbafb898 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:04.183149 14748 raft_consensus.cc:2272] T 00000000000000000000000000000000 P d33cebb189b14a989836214fdbafb898 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:04.183568 14748 tablet_replica.cc:333] T 00000000000000000000000000000000 P d33cebb189b14a989836214fdbafb898: stopping tablet replica
I20260709 13:09:04.413664 14748 master.cc:584] Master@127.14.103.62:42985 shutdown complete.
I20260709 13:09:04.476785 14748 test_util.cc:182] -----------------------------------------------
I20260709 13:09:04.477006 14748 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602247531521-14748-0
[  FAILED  ] LeaderRebalancerTest.AddTserver (272663 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 13:09:04.480814 14748 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.103.62:41611
I20260709 13:09:04.481964 14748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:04.488865 24180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:04.488997 24181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:04.490082 24183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:04.490554 14748 server_base.cc:1061] running on GCE node
I20260709 13:09:04.495527 14748 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:04.495888 14748 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:04.496018 14748 hybrid_clock.cc:648] HybridClock initialized: now 1783602544496006 us; error 0 us; skew 500 ppm
I20260709 13:09:04.496567 14748 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:04.499459 14748 webserver.cc:533] Webserver started at http://127.14.103.62:34479/ using document root <none> and password file <none>
I20260709 13:09:04.500064 14748 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:04.500242 14748 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:04.500469 14748 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:04.502115 14748 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602247531521-14748-0/minicluster-data/master-0-root/instance:
uuid: "a20e5c2ad87d4af086eb0cf554c1d829"
format_stamp: "Formatted at 2026-07-09 13:09:04 on dist-test-slave-fpp7"
I20260709 13:09:04.507781 14748 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.001s
I20260709 13:09:04.511970 24188 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:04.512936 14748 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:09:04.513746 14748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602247531521-14748-0/minicluster-data/master-0-root
uuid: "a20e5c2ad87d4af086eb0cf554c1d829"
format_stamp: "Formatted at 2026-07-09 13:09:04 on dist-test-slave-fpp7"
I20260709 13:09:04.514107 14748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602247531521-14748-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602247531521-14748-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602247531521-14748-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:04.535909 14748 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:04.537413 14748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:04.585606 14748 rpc_server.cc:307] RPC server started. Bound to: 127.14.103.62:41611
I20260709 13:09:04.585708 24239 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.103.62:41611 every 8 connection(s)
I20260709 13:09:04.589982 24240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:04.602943 24240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a20e5c2ad87d4af086eb0cf554c1d829: Bootstrap starting.
I20260709 13:09:04.607965 24240 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a20e5c2ad87d4af086eb0cf554c1d829: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:04.612399 24240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a20e5c2ad87d4af086eb0cf554c1d829: No bootstrap required, opened a new log
I20260709 13:09:04.614770 24240 raft_consensus.cc:359] T 00000000000000000000000000000000 P a20e5c2ad87d4af086eb0cf554c1d829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a20e5c2ad87d4af086eb0cf554c1d829" member_type: VOTER }
I20260709 13:09:04.615218 24240 raft_consensus.cc:385] T 00000000000000000000000000000000 P a20e5c2ad87d4af086eb0cf554c1d829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:04.615485 24240 raft_consensus.cc:740] T 00000000000000000000000000000000 P a20e5c2ad87d4af086eb0cf554c1d829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a20e5c2ad87d4af086eb0cf554c1d829, State: Initialized, Role: FOLLOWER
I20260709 13:09:04.616288 24240 consensus_queue.cc:260] T 00000000000000000000000000000000 P a20e5c2ad87d4af086eb0cf554c1d829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a20e5c2ad87d4af086eb0cf554c1d829" member_type: VOTER }
I20260709 13:09:04.616781 24240 raft_consensus.cc:399] T 00000000000000000000000000000000 P a20e5c2ad87d4af086eb0cf554c1d829 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:09:04.617041 24240 raft_consensus.cc:493] T 00000000000000000000000000000000 P a20e5c2ad87d4af086eb0cf554c1d829 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:09:04.617354 24240 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a20e5c2ad87d4af086eb0cf554c1d829 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:04.622977 24240 raft_consensus.cc:515] T 00000000000000000000000000000000 P a20e5c2ad87d4af086eb0cf554c1d829 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a20e5c2ad87d4af086eb0cf554c1d829" member_type: VOTER }
I20260709 13:09:04.623545 24240 leader_election.cc:304] T 00000000000000000000000000000000 P a20e5c2ad87d4af086eb0cf554c1d829 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a20e5c2ad87d4af086eb0cf554c1d829; no voters: 
I20260709 13:09:04.624782 24240 leader_election.cc:290] T 00000000000000000000000000000000 P a20e5c2ad87d4af086eb0cf554c1d829 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:09:04.625149 24243 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a20e5c2ad87d4af086eb0cf554c1d829 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:04.626550 24243 raft_consensus.cc:697] T 00000000000000000000000000000000 P a20e5c2ad87d4af086eb0cf554c1d829 [term 1 LEADER]: Becoming Leader. State: Replica: a20e5c2ad87d4af086eb0cf554c1d829, State: Running, Role: LEADER
I20260709 13:09:04.627287 24243 consensus_queue.cc:237] T 00000000000000000000000000000000 P a20e5c2ad87d4af086eb0cf554c1d829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a20e5c2ad87d4af086eb0cf554c1d829" member_type: VOTER }
I20260709 13:09:04.627825 24240 sys_catalog.cc:565] T 00000000000000000000000000000000 P a20e5c2ad87d4af086eb0cf554c1d829 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:09:04.633904 24244 sys_catalog.cc:455] T 00000000000000000000000000000000 P a20e5c2ad87d4af086eb0cf554c1d829 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a20e5c2ad87d4af086eb0cf554c1d829" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a20e5c2ad87d4af086eb0cf554c1d829" member_type: VOTER } }
I20260709 13:09:04.634567 24245 sys_catalog.cc:455] T 00000000000000000000000000000000 P a20e5c2ad87d4af086eb0cf554c1d829 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a20e5c2ad87d4af086eb0cf554c1d829. Latest consensus state: current_term: 1 leader_uuid: "a20e5c2ad87d4af086eb0cf554c1d829" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a20e5c2ad87d4af086eb0cf554c1d829" member_type: VOTER } }
I20260709 13:09:04.634927 24244 sys_catalog.cc:458] T 00000000000000000000000000000000 P a20e5c2ad87d4af086eb0cf554c1d829 [sys.catalog]: This master's current role is: LEADER
I20260709 13:09:04.635300 24245 sys_catalog.cc:458] T 00000000000000000000000000000000 P a20e5c2ad87d4af086eb0cf554c1d829 [sys.catalog]: This master's current role is: LEADER
I20260709 13:09:04.637089 24252 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:09:04.644026 24252 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:09:04.645067 14748 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:09:04.653376 24252 catalog_manager.cc:1383] Generated new cluster ID: 7cb56ba0b8fa4d8d8d25f96ea7a058fa
I20260709 13:09:04.653712 24252 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:09:04.668377 24252 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:09:04.669718 24252 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:09:04.687057 24252 catalog_manager.cc:6070] T 00000000000000000000000000000000 P a20e5c2ad87d4af086eb0cf554c1d829: Generated new TSK 0
I20260709 13:09:04.687732 24252 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:09:04.712091 14748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:04.719462 24261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:04.720232 24262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:04.722426 24264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:04.724433 14748 server_base.cc:1061] running on GCE node
I20260709 13:09:04.727102 14748 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:04.727385 14748 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:04.727573 14748 hybrid_clock.cc:648] HybridClock initialized: now 1783602544727543 us; error 0 us; skew 500 ppm
I20260709 13:09:04.728176 14748 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:04.736089 14748 webserver.cc:533] Webserver started at http://127.14.103.1:39811/ using document root <none> and password file <none>
I20260709 13:09:04.736616 14748 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:04.736840 14748 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:04.737124 14748 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:04.738327 14748 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602247531521-14748-0/minicluster-data/ts-0-root/instance:
uuid: "9c2a14143eb744cf83bfa5f46284a381"
format_stamp: "Formatted at 2026-07-09 13:09:04 on dist-test-slave-fpp7"
I20260709 13:09:04.743330 14748 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.003s
I20260709 13:09:04.746951 24269 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:04.747776 14748 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:09:04.748243 14748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602247531521-14748-0/minicluster-data/ts-0-root
uuid: "9c2a14143eb744cf83bfa5f46284a381"
format_stamp: "Formatted at 2026-07-09 13:09:04 on dist-test-slave-fpp7"
I20260709 13:09:04.748581 14748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602247531521-14748-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602247531521-14748-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602247531521-14748-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:04.786048 14748 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:04.787317 14748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:04.789073 14748 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:09:04.791399 14748 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:09:04.791617 14748 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:04.791908 14748 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:09:04.792109 14748 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:04.836670 14748 rpc_server.cc:307] RPC server started. Bound to: 127.14.103.1:45351
I20260709 13:09:04.836786 24331 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.103.1:45351 every 8 connection(s)
I20260709 13:09:04.842715 14748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:04.852854 24336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:04.853590 24337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:04.866044 24339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:04.867200 14748 server_base.cc:1061] running on GCE node
I20260709 13:09:04.867463 24332 heartbeater.cc:344] Connected to a master server at 127.14.103.62:41611
I20260709 13:09:04.867991 24332 heartbeater.cc:461] Registering TS with master...
I20260709 13:09:04.868912 24332 heartbeater.cc:507] Master 127.14.103.62:41611 requested a full tablet report, sending...
I20260709 13:09:04.871836 24205 ts_manager.cc:194] Registered new tserver with Master: 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:04.872900 14748 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:04.873311 14748 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:04.873507 14748 hybrid_clock.cc:648] HybridClock initialized: now 1783602544873491 us; error 0 us; skew 500 ppm
I20260709 13:09:04.874452 14748 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:04.874621 24205 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34384
I20260709 13:09:04.878975 14748 webserver.cc:533] Webserver started at http://127.14.103.2:41873/ using document root <none> and password file <none>
I20260709 13:09:04.879755 14748 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:04.880017 14748 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:04.880326 14748 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:04.881812 14748 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602247531521-14748-0/minicluster-data/ts-1-root/instance:
uuid: "c40e7ba102224d24a22ccd1f68aaae45"
format_stamp: "Formatted at 2026-07-09 13:09:04 on dist-test-slave-fpp7"
I20260709 13:09:04.887946 14748 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.002s	sys 0.005s
I20260709 13:09:04.893245 24344 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:04.894485 14748 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.001s
I20260709 13:09:04.894886 14748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602247531521-14748-0/minicluster-data/ts-1-root
uuid: "c40e7ba102224d24a22ccd1f68aaae45"
format_stamp: "Formatted at 2026-07-09 13:09:04 on dist-test-slave-fpp7"
I20260709 13:09:04.895259 14748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602247531521-14748-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602247531521-14748-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602247531521-14748-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:04.921365 14748 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:04.922703 14748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:04.924592 14748 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:09:04.926985 14748 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:09:04.927165 14748 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:04.927471 14748 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:09:04.927623 14748 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:04.974395 14748 rpc_server.cc:307] RPC server started. Bound to: 127.14.103.2:46177
I20260709 13:09:04.974524 24406 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.103.2:46177 every 8 connection(s)
I20260709 13:09:04.979247 14748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:04.987264 24410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:04.988695 24411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:04.990681 14748 server_base.cc:1061] running on GCE node
W20260709 13:09:04.991295 24413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:04.992429 14748 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:04.992731 14748 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:04.992945 14748 hybrid_clock.cc:648] HybridClock initialized: now 1783602544992926 us; error 0 us; skew 500 ppm
I20260709 13:09:04.993703 14748 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:04.993742 24407 heartbeater.cc:344] Connected to a master server at 127.14.103.62:41611
I20260709 13:09:04.994247 24407 heartbeater.cc:461] Registering TS with master...
I20260709 13:09:04.995096 24407 heartbeater.cc:507] Master 127.14.103.62:41611 requested a full tablet report, sending...
I20260709 13:09:04.997658 14748 webserver.cc:533] Webserver started at http://127.14.103.3:44167/ using document root <none> and password file <none>
I20260709 13:09:04.998346 14748 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:04.997753 24205 ts_manager.cc:194] Registered new tserver with Master: c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:05.001160 14748 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:05.001600 14748 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:05.003077 24205 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34396
I20260709 13:09:05.003726 14748 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602247531521-14748-0/minicluster-data/ts-2-root/instance:
uuid: "32088c942fbb48239e0eed952d4df449"
format_stamp: "Formatted at 2026-07-09 13:09:04 on dist-test-slave-fpp7"
I20260709 13:09:05.010041 14748 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:09:05.013775 24418 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:05.014853 14748 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:09:05.015188 14748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602247531521-14748-0/minicluster-data/ts-2-root
uuid: "32088c942fbb48239e0eed952d4df449"
format_stamp: "Formatted at 2026-07-09 13:09:04 on dist-test-slave-fpp7"
I20260709 13:09:05.015553 14748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602247531521-14748-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602247531521-14748-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602247531521-14748-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:05.040920 14748 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:05.042182 14748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:05.044291 14748 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:09:05.046851 14748 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:09:05.047103 14748 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:05.047365 14748 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:09:05.047545 14748 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:05.089319 14748 rpc_server.cc:307] RPC server started. Bound to: 127.14.103.3:39679
I20260709 13:09:05.089468 24480 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.103.3:39679 every 8 connection(s)
I20260709 13:09:05.094372 14748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:05.102092 24484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:05.104624 24485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:05.106331 14748 server_base.cc:1061] running on GCE node
W20260709 13:09:05.107501 24487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:05.108530 24481 heartbeater.cc:344] Connected to a master server at 127.14.103.62:41611
I20260709 13:09:05.108660 14748 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:05.108992 14748 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:05.108982 24481 heartbeater.cc:461] Registering TS with master...
I20260709 13:09:05.109309 14748 hybrid_clock.cc:648] HybridClock initialized: now 1783602545109292 us; error 0 us; skew 500 ppm
I20260709 13:09:05.110030 14748 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:05.110060 24481 heartbeater.cc:507] Master 127.14.103.62:41611 requested a full tablet report, sending...
I20260709 13:09:05.112494 24205 ts_manager.cc:194] Registered new tserver with Master: 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:05.113260 14748 webserver.cc:533] Webserver started at http://127.14.103.4:46697/ using document root <none> and password file <none>
I20260709 13:09:05.113915 14748 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:05.114158 14748 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:05.114473 14748 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:05.114394 24205 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34412
I20260709 13:09:05.116165 14748 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602247531521-14748-0/minicluster-data/ts-3-root/instance:
uuid: "838cb9997b0f437989d3f4e3e9904390"
format_stamp: "Formatted at 2026-07-09 13:09:05 on dist-test-slave-fpp7"
I20260709 13:09:05.121199 14748 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:09:05.124830 24492 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:05.125604 14748 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:09:05.125910 14748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602247531521-14748-0/minicluster-data/ts-3-root
uuid: "838cb9997b0f437989d3f4e3e9904390"
format_stamp: "Formatted at 2026-07-09 13:09:05 on dist-test-slave-fpp7"
I20260709 13:09:05.126200 14748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602247531521-14748-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602247531521-14748-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602247531521-14748-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:05.144431 14748 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:05.145705 14748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:05.146975 14748 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:09:05.149056 14748 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:09:05.149284 14748 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:05.149652 14748 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:09:05.149859 14748 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:05.191933 14748 rpc_server.cc:307] RPC server started. Bound to: 127.14.103.4:33251
I20260709 13:09:05.192049 24554 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.103.4:33251 every 8 connection(s)
I20260709 13:09:05.206981 24555 heartbeater.cc:344] Connected to a master server at 127.14.103.62:41611
I20260709 13:09:05.207458 24555 heartbeater.cc:461] Registering TS with master...
I20260709 13:09:05.208288 24555 heartbeater.cc:507] Master 127.14.103.62:41611 requested a full tablet report, sending...
I20260709 13:09:05.210300 24205 ts_manager.cc:194] Registered new tserver with Master: 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:05.210445 14748 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.015340109s
I20260709 13:09:05.212224 24205 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34424
I20260709 13:09:05.239977 24205 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34434:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 13:09:05.494663 24445 tablet_service.cc:1511] Processing CreateTablet for tablet e4dd51cda55249b48d7bfa39d5feb398 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:09:05.496590 24445 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4dd51cda55249b48d7bfa39d5feb398. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.500059 24446 tablet_service.cc:1511] Processing CreateTablet for tablet 8775bc34e96d48ae8f5598affd26efdc (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:09:05.500726 24443 tablet_service.cc:1511] Processing CreateTablet for tablet 373185ca8ab04fdb91c7947ec0668808 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:09:05.501648 24446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8775bc34e96d48ae8f5598affd26efdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.502256 24443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 373185ca8ab04fdb91c7947ec0668808. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.504936 24442 tablet_service.cc:1511] Processing CreateTablet for tablet 958ccd87b65945468e9247c40ca27c33 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:09:05.506415 24442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 958ccd87b65945468e9247c40ca27c33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.508996 24441 tablet_service.cc:1511] Processing CreateTablet for tablet 48aea62663ac40998d7f09389b93d586 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:09:05.510524 24441 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48aea62663ac40998d7f09389b93d586. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.512805 24520 tablet_service.cc:1511] Processing CreateTablet for tablet 4af7fc369d124452beb5fd07cf8592ec (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:09:05.512925 24444 tablet_service.cc:1511] Processing CreateTablet for tablet db9e8fa567ab481c9b80ae8e30acd628 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:09:05.513108 24440 tablet_service.cc:1511] Processing CreateTablet for tablet 54fa63b278394f569a9ecffed4dec233 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:09:05.514437 24520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4af7fc369d124452beb5fd07cf8592ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.514624 24444 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db9e8fa567ab481c9b80ae8e30acd628. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.515319 24440 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54fa63b278394f569a9ecffed4dec233. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.528517 24372 tablet_service.cc:1511] Processing CreateTablet for tablet 8775bc34e96d48ae8f5598affd26efdc (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:09:05.530121 24372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8775bc34e96d48ae8f5598affd26efdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.533214 24371 tablet_service.cc:1511] Processing CreateTablet for tablet 4af7fc369d124452beb5fd07cf8592ec (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:09:05.534757 24371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4af7fc369d124452beb5fd07cf8592ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.536959 24370 tablet_service.cc:1511] Processing CreateTablet for tablet db9e8fa567ab481c9b80ae8e30acd628 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:09:05.538416 24370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db9e8fa567ab481c9b80ae8e30acd628. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.541311 24369 tablet_service.cc:1511] Processing CreateTablet for tablet 373185ca8ab04fdb91c7947ec0668808 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:09:05.547403 24367 tablet_service.cc:1511] Processing CreateTablet for tablet 48aea62663ac40998d7f09389b93d586 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:09:05.549721 24368 tablet_service.cc:1511] Processing CreateTablet for tablet 42bcfb18b77e41afa60057ccfd718fa9 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:09:05.551072 24366 tablet_service.cc:1511] Processing CreateTablet for tablet 54fa63b278394f569a9ecffed4dec233 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:09:05.560159 24369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 373185ca8ab04fdb91c7947ec0668808. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.565270 24575 tablet_bootstrap.cc:492] T 4af7fc369d124452beb5fd07cf8592ec P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:05.566498 24365 tablet_service.cc:1511] Processing CreateTablet for tablet 9711acbe7d044647bf774ee849c8a032 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:09:05.565745 24368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42bcfb18b77e41afa60057ccfd718fa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.571229 24365 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9711acbe7d044647bf774ee849c8a032. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.572096 24366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54fa63b278394f569a9ecffed4dec233. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.575481 24367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48aea62663ac40998d7f09389b93d586. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.587193 24364 tablet_service.cc:1511] Processing CreateTablet for tablet aac1b62e27a44fb5b9a23cfc42a5d00b (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:09:05.588696 24364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aac1b62e27a44fb5b9a23cfc42a5d00b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.594839 24520 tablet_service.cc:1511] Processing CreateTablet for tablet e4dd51cda55249b48d7bfa39d5feb398 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:09:05.597151 24520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4dd51cda55249b48d7bfa39d5feb398. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.599192 24446 tablet_service.cc:1511] Processing CreateTablet for tablet c60b42ac6d7e48bfae725dc113758659 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:09:05.603358 24370 tablet_service.cc:1511] Processing CreateTablet for tablet a3b35ea0300f4b9e9afa80cf1dc9ba0f (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:09:05.604646 24444 tablet_service.cc:1511] Processing CreateTablet for tablet aac1b62e27a44fb5b9a23cfc42a5d00b (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:09:05.607864 24519 tablet_service.cc:1511] Processing CreateTablet for tablet db9e8fa567ab481c9b80ae8e30acd628 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:09:05.608909 24297 tablet_service.cc:1511] Processing CreateTablet for tablet 8775bc34e96d48ae8f5598affd26efdc (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:09:05.609767 24518 tablet_service.cc:1511] Processing CreateTablet for tablet 42bcfb18b77e41afa60057ccfd718fa9 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:09:05.614512 24575 tablet_bootstrap.cc:654] T 4af7fc369d124452beb5fd07cf8592ec P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:05.613977 24513 tablet_service.cc:1511] Processing CreateTablet for tablet aac1b62e27a44fb5b9a23cfc42a5d00b (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:09:05.616447 24514 tablet_service.cc:1511] Processing CreateTablet for tablet c60b42ac6d7e48bfae725dc113758659 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:09:05.621176 24516 tablet_service.cc:1511] Processing CreateTablet for tablet 48aea62663ac40998d7f09389b93d586 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:09:05.618914 24296 tablet_service.cc:1511] Processing CreateTablet for tablet 4af7fc369d124452beb5fd07cf8592ec (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:09:05.615216 24576 tablet_bootstrap.cc:492] T 54fa63b278394f569a9ecffed4dec233 P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:05.629650 24444 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aac1b62e27a44fb5b9a23cfc42a5d00b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.630052 24516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48aea62663ac40998d7f09389b93d586. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.630301 24295 tablet_service.cc:1511] Processing CreateTablet for tablet e4dd51cda55249b48d7bfa39d5feb398 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:09:05.632689 24514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c60b42ac6d7e48bfae725dc113758659. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.632773 24517 tablet_service.cc:1511] Processing CreateTablet for tablet 958ccd87b65945468e9247c40ca27c33 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:09:05.635322 24519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db9e8fa567ab481c9b80ae8e30acd628. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.639658 24294 tablet_service.cc:1511] Processing CreateTablet for tablet 373185ca8ab04fdb91c7947ec0668808 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:09:05.640964 24291 tablet_service.cc:1511] Processing CreateTablet for tablet 54fa63b278394f569a9ecffed4dec233 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:09:05.642082 24288 tablet_service.cc:1511] Processing CreateTablet for tablet a3b35ea0300f4b9e9afa80cf1dc9ba0f (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:09:05.644997 24290 tablet_service.cc:1511] Processing CreateTablet for tablet 9711acbe7d044647bf774ee849c8a032 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:09:05.648159 24518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42bcfb18b77e41afa60057ccfd718fa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.642076 24293 tablet_service.cc:1511] Processing CreateTablet for tablet 42bcfb18b77e41afa60057ccfd718fa9 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:09:05.649547 24363 tablet_service.cc:1511] Processing CreateTablet for tablet 1adea9b113494bed8c9d57fdbee6e22b (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:09:05.650008 24441 tablet_service.cc:1511] Processing CreateTablet for tablet 861acffd0d804c8fbd1a63771d7937c7 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:09:05.651635 24441 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 861acffd0d804c8fbd1a63771d7937c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.623487 24515 tablet_service.cc:1511] Processing CreateTablet for tablet 9711acbe7d044647bf774ee849c8a032 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
W20260709 13:09:05.652029 24253 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:05.653013 24513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aac1b62e27a44fb5b9a23cfc42a5d00b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.647603 24289 tablet_service.cc:1511] Processing CreateTablet for tablet c60b42ac6d7e48bfae725dc113758659 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:09:05.643276 24292 tablet_service.cc:1511] Processing CreateTablet for tablet 958ccd87b65945468e9247c40ca27c33 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:09:05.656121 24517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 958ccd87b65945468e9247c40ca27c33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.657227 24512 tablet_service.cc:1511] Processing CreateTablet for tablet 1adea9b113494bed8c9d57fdbee6e22b (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:09:05.658632 24512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1adea9b113494bed8c9d57fdbee6e22b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.660496 24295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4dd51cda55249b48d7bfa39d5feb398. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.661932 24297 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8775bc34e96d48ae8f5598affd26efdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.664006 24291 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54fa63b278394f569a9ecffed4dec233. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.668073 24288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3b35ea0300f4b9e9afa80cf1dc9ba0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.669593 24511 tablet_service.cc:1511] Processing CreateTablet for tablet 861acffd0d804c8fbd1a63771d7937c7 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:09:05.671000 24511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 861acffd0d804c8fbd1a63771d7937c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.672019 24294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 373185ca8ab04fdb91c7947ec0668808. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.673022 24296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4af7fc369d124452beb5fd07cf8592ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.673673 24292 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 958ccd87b65945468e9247c40ca27c33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.676692 24290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9711acbe7d044647bf774ee849c8a032. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.680105 24577 tablet_bootstrap.cc:492] T 8775bc34e96d48ae8f5598affd26efdc P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:05.684445 24515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9711acbe7d044647bf774ee849c8a032. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.692384 24363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1adea9b113494bed8c9d57fdbee6e22b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.711330 24370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3b35ea0300f4b9e9afa80cf1dc9ba0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.711948 24293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42bcfb18b77e41afa60057ccfd718fa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.719034 24289 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c60b42ac6d7e48bfae725dc113758659. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.734788 24577 tablet_bootstrap.cc:654] T 8775bc34e96d48ae8f5598affd26efdc P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:05.755841 24440 tablet_service.cc:1511] Processing CreateTablet for tablet a3b35ea0300f4b9e9afa80cf1dc9ba0f (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:09:05.756687 24297 tablet_service.cc:1511] Processing CreateTablet for tablet 1adea9b113494bed8c9d57fdbee6e22b (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:09:05.757382 24440 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3b35ea0300f4b9e9afa80cf1dc9ba0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.758121 24297 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1adea9b113494bed8c9d57fdbee6e22b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.758724 24446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c60b42ac6d7e48bfae725dc113758659. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.761157 24576 tablet_bootstrap.cc:654] T 54fa63b278394f569a9ecffed4dec233 P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:05.784307 24371 tablet_service.cc:1511] Processing CreateTablet for tablet 4f085ebc2da04fbb87d48e4f206e61f0 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:09:05.785755 24371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f085ebc2da04fbb87d48e4f206e61f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.785991 24442 tablet_service.cc:1511] Processing CreateTablet for tablet 4f085ebc2da04fbb87d48e4f206e61f0 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:09:05.787721 24442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f085ebc2da04fbb87d48e4f206e61f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.789364 24291 tablet_service.cc:1511] Processing CreateTablet for tablet 861acffd0d804c8fbd1a63771d7937c7 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:09:05.790714 24291 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 861acffd0d804c8fbd1a63771d7937c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.797331 24511 tablet_service.cc:1511] Processing CreateTablet for tablet 4f085ebc2da04fbb87d48e4f206e61f0 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:09:05.801016 24511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f085ebc2da04fbb87d48e4f206e61f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.801484 24367 tablet_service.cc:1511] Processing CreateTablet for tablet 4f32a46373be4a29b6ac72efa35494b4 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:09:05.841018 24293 tablet_service.cc:1511] Processing CreateTablet for tablet 4f32a46373be4a29b6ac72efa35494b4 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:09:05.842549 24367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f32a46373be4a29b6ac72efa35494b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.847615 24577 tablet_bootstrap.cc:492] T 8775bc34e96d48ae8f5598affd26efdc P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:05.848184 24577 ts_tablet_manager.cc:1403] T 8775bc34e96d48ae8f5598affd26efdc P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.168s	user 0.013s	sys 0.011s
I20260709 13:09:05.848523 24575 tablet_bootstrap.cc:492] T 4af7fc369d124452beb5fd07cf8592ec P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:05.849031 24575 ts_tablet_manager.cc:1403] T 4af7fc369d124452beb5fd07cf8592ec P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.284s	user 0.018s	sys 0.013s
I20260709 13:09:05.851280 24577 raft_consensus.cc:359] T 8775bc34e96d48ae8f5598affd26efdc P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:05.851974 24579 tablet_bootstrap.cc:492] T 958ccd87b65945468e9247c40ca27c33 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:05.852566 24577 raft_consensus.cc:385] T 8775bc34e96d48ae8f5598affd26efdc P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:05.852979 24577 raft_consensus.cc:740] T 8775bc34e96d48ae8f5598affd26efdc P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:05.853945 24577 consensus_queue.cc:260] T 8775bc34e96d48ae8f5598affd26efdc P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:05.858736 24293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f32a46373be4a29b6ac72efa35494b4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:09:05.864343 24189 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.14.103.4:33251, user_credentials={real_user=slave}} blocked reactor thread for 43556us
I20260709 13:09:05.878774 24332 heartbeater.cc:499] Master 127.14.103.62:41611 was elected leader, sending a full tablet report...
I20260709 13:09:05.879703 24295 tablet_service.cc:1511] Processing CreateTablet for tablet 53b7256c3dd84f5b952d1f9d47691f4e (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:09:05.881093 24295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53b7256c3dd84f5b952d1f9d47691f4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.886632 24576 tablet_bootstrap.cc:492] T 54fa63b278394f569a9ecffed4dec233 P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:05.887468 24576 ts_tablet_manager.cc:1403] T 54fa63b278394f569a9ecffed4dec233 P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.273s	user 0.014s	sys 0.005s
I20260709 13:09:05.890726 24576 raft_consensus.cc:359] T 54fa63b278394f569a9ecffed4dec233 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:05.891533 24576 raft_consensus.cc:385] T 54fa63b278394f569a9ecffed4dec233 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:05.894606 24407 heartbeater.cc:499] Master 127.14.103.62:41611 was elected leader, sending a full tablet report...
I20260709 13:09:05.894932 24577 ts_tablet_manager.cc:1434] T 8775bc34e96d48ae8f5598affd26efdc P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.002s
I20260709 13:09:05.895990 24577 tablet_bootstrap.cc:492] T db9e8fa567ab481c9b80ae8e30acd628 P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:05.900260 24579 tablet_bootstrap.cc:654] T 958ccd87b65945468e9247c40ca27c33 P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:05.903148 24577 tablet_bootstrap.cc:654] T db9e8fa567ab481c9b80ae8e30acd628 P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:05.897125 24575 raft_consensus.cc:359] T 4af7fc369d124452beb5fd07cf8592ec P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:05.903620 24575 raft_consensus.cc:385] T 4af7fc369d124452beb5fd07cf8592ec P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:05.904160 24575 raft_consensus.cc:740] T 4af7fc369d124452beb5fd07cf8592ec P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:05.905084 24575 consensus_queue.cc:260] T 4af7fc369d124452beb5fd07cf8592ec P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:05.912358 24294 tablet_service.cc:1511] Processing CreateTablet for tablet 93b6310306174b4ca68b467ea7f5d23e (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:09:05.913779 24294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93b6310306174b4ca68b467ea7f5d23e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.915757 24444 tablet_service.cc:1511] Processing CreateTablet for tablet 4f32a46373be4a29b6ac72efa35494b4 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:09:05.917018 24444 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f32a46373be4a29b6ac72efa35494b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.893690 24576 raft_consensus.cc:740] T 54fa63b278394f569a9ecffed4dec233 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:05.920895 24576 consensus_queue.cc:260] T 54fa63b278394f569a9ecffed4dec233 P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:05.932624 24370 tablet_service.cc:1511] Processing CreateTablet for tablet 53b7256c3dd84f5b952d1f9d47691f4e (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:09:05.933991 24370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53b7256c3dd84f5b952d1f9d47691f4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.934407 24514 tablet_service.cc:1511] Processing CreateTablet for tablet 53b7256c3dd84f5b952d1f9d47691f4e (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:09:05.936123 24514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53b7256c3dd84f5b952d1f9d47691f4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.936663 24368 tablet_service.cc:1511] Processing CreateTablet for tablet ae6c1a13de5442c5bb4e58e7ce36e741 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:09:05.937984 24368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae6c1a13de5442c5bb4e58e7ce36e741. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.932677 24512 tablet_service.cc:1511] Processing CreateTablet for tablet 93b6310306174b4ca68b467ea7f5d23e (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:09:05.941385 24512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93b6310306174b4ca68b467ea7f5d23e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.965227 24297 tablet_service.cc:1511] Processing CreateTablet for tablet 9b7cd923257143648cfe6a296e18b642 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:09:05.966674 24297 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b7cd923257143648cfe6a296e18b642. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.979009 24582 raft_consensus.cc:493] T 8775bc34e96d48ae8f5598affd26efdc P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:05.979642 24582 raft_consensus.cc:515] T 8775bc34e96d48ae8f5598affd26efdc P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
W20260709 13:09:05.979990 24408 tablet.cc:2406] T 8775bc34e96d48ae8f5598affd26efdc P c40e7ba102224d24a22ccd1f68aaae45: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:09:05.982601 24582 leader_election.cc:290] T 8775bc34e96d48ae8f5598affd26efdc P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:05.984513 24555 heartbeater.cc:499] Master 127.14.103.62:41611 was elected leader, sending a full tablet report...
I20260709 13:09:05.953461 24442 tablet_service.cc:1511] Processing CreateTablet for tablet ae6c1a13de5442c5bb4e58e7ce36e741 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:09:05.987157 24442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae6c1a13de5442c5bb4e58e7ce36e741. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.990777 24440 tablet_service.cc:1511] Processing CreateTablet for tablet 9b7cd923257143648cfe6a296e18b642 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:09:05.992142 24440 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b7cd923257143648cfe6a296e18b642. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.993294 24443 tablet_service.cc:1511] Processing CreateTablet for tablet 72d2665a42a94a82b1ceec249960fdf6 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:09:05.994558 24443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72d2665a42a94a82b1ceec249960fdf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:05.995461 24289 tablet_service.cc:1511] Processing CreateTablet for tablet 72d2665a42a94a82b1ceec249960fdf6 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:09:05.996827 24289 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72d2665a42a94a82b1ceec249960fdf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.006834 24511 tablet_service.cc:1511] Processing CreateTablet for tablet 72d2665a42a94a82b1ceec249960fdf6 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:09:05.958588 24446 tablet_service.cc:1511] Processing CreateTablet for tablet 93b6310306174b4ca68b467ea7f5d23e (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:09:06.008334 24511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72d2665a42a94a82b1ceec249960fdf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.008706 24513 tablet_service.cc:1511] Processing CreateTablet for tablet 5eaa917e65d9403dbb241f84ee96ba9d (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:09:06.009208 24446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93b6310306174b4ca68b467ea7f5d23e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.012622 24575 ts_tablet_manager.cc:1434] T 4af7fc369d124452beb5fd07cf8592ec P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.163s	user 0.017s	sys 0.034s
I20260709 13:09:06.010036 24513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5eaa917e65d9403dbb241f84ee96ba9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.013655 24575 tablet_bootstrap.cc:492] T e4dd51cda55249b48d7bfa39d5feb398 P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:06.016516 24517 tablet_service.cc:1511] Processing CreateTablet for tablet ae6c1a13de5442c5bb4e58e7ce36e741 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:09:06.031011 24445 tablet_service.cc:1511] Processing CreateTablet for tablet 05840bb250764e02998cdddc290e5139 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:09:06.032394 24445 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05840bb250764e02998cdddc290e5139. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.023239 24576 ts_tablet_manager.cc:1434] T 54fa63b278394f569a9ecffed4dec233 P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.135s	user 0.018s	sys 0.004s
I20260709 13:09:06.040675 24577 tablet_bootstrap.cc:492] T db9e8fa567ab481c9b80ae8e30acd628 P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:06.041580 24577 ts_tablet_manager.cc:1403] T db9e8fa567ab481c9b80ae8e30acd628 P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.146s	user 0.011s	sys 0.006s
I20260709 13:09:06.048794 24577 raft_consensus.cc:359] T db9e8fa567ab481c9b80ae8e30acd628 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:06.049589 24577 raft_consensus.cc:385] T db9e8fa567ab481c9b80ae8e30acd628 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:06.049914 24577 raft_consensus.cc:740] T db9e8fa567ab481c9b80ae8e30acd628 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:06.050710 24577 consensus_queue.cc:260] T db9e8fa567ab481c9b80ae8e30acd628 P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:06.051518 24579 tablet_bootstrap.cc:492] T 958ccd87b65945468e9247c40ca27c33 P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:06.052054 24579 ts_tablet_manager.cc:1403] T 958ccd87b65945468e9247c40ca27c33 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.207s	user 0.007s	sys 0.011s
I20260709 13:09:06.052867 24291 tablet_service.cc:1511] Processing CreateTablet for tablet 5eaa917e65d9403dbb241f84ee96ba9d (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:09:06.054384 24291 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5eaa917e65d9403dbb241f84ee96ba9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.054936 24579 raft_consensus.cc:359] T 958ccd87b65945468e9247c40ca27c33 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:06.055649 24579 raft_consensus.cc:385] T 958ccd87b65945468e9247c40ca27c33 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:06.056252 24579 raft_consensus.cc:740] T 958ccd87b65945468e9247c40ca27c33 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:06.057169 24579 consensus_queue.cc:260] T 958ccd87b65945468e9247c40ca27c33 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:06.058156 24515 tablet_service.cc:1511] Processing CreateTablet for tablet 05840bb250764e02998cdddc290e5139 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:09:06.059964 24481 heartbeater.cc:499] Master 127.14.103.62:41611 was elected leader, sending a full tablet report...
I20260709 13:09:06.061810 24295 tablet_service.cc:1511] Processing CreateTablet for tablet b147866f4e5f4596b0b3989418cc4b0a (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:09:06.038573 24517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae6c1a13de5442c5bb4e58e7ce36e741. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.066974 24295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b147866f4e5f4596b0b3989418cc4b0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.073874 24515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05840bb250764e02998cdddc290e5139. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.075202 24575 tablet_bootstrap.cc:654] T e4dd51cda55249b48d7bfa39d5feb398 P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:06.079228 24576 tablet_bootstrap.cc:492] T db9e8fa567ab481c9b80ae8e30acd628 P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:06.084900 24444 tablet_service.cc:1511] Processing CreateTablet for tablet b147866f4e5f4596b0b3989418cc4b0a (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:09:06.086222 24444 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b147866f4e5f4596b0b3989418cc4b0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.088152 24583 raft_consensus.cc:493] T 4af7fc369d124452beb5fd07cf8592ec P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:06.088676 24583 raft_consensus.cc:515] T 4af7fc369d124452beb5fd07cf8592ec P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:06.088888 24516 tablet_service.cc:1511] Processing CreateTablet for tablet e2b4c67e0f6b4a1e9cfb3703e43e988f (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:09:06.090174 24516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2b4c67e0f6b4a1e9cfb3703e43e988f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.091749 24583 leader_election.cc:290] T 4af7fc369d124452beb5fd07cf8592ec P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:06.097376 24579 ts_tablet_manager.cc:1434] T 958ccd87b65945468e9247c40ca27c33 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.045s	user 0.006s	sys 0.017s
I20260709 13:09:06.122552 24577 ts_tablet_manager.cc:1434] T db9e8fa567ab481c9b80ae8e30acd628 P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.081s	user 0.006s	sys 0.001s
I20260709 13:09:06.123457 24293 tablet_service.cc:1511] Processing CreateTablet for tablet e2b4c67e0f6b4a1e9cfb3703e43e988f (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:09:06.133273 24293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2b4c67e0f6b4a1e9cfb3703e43e988f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.156535 24577 tablet_bootstrap.cc:492] T 4af7fc369d124452beb5fd07cf8592ec P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:06.168573 24368 tablet_service.cc:1511] Processing CreateTablet for tablet 9b7cd923257143648cfe6a296e18b642 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:09:06.169951 24368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b7cd923257143648cfe6a296e18b642. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.181306 24371 tablet_service.cc:1511] Processing CreateTablet for tablet 5eaa917e65d9403dbb241f84ee96ba9d (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:09:06.181625 24367 tablet_service.cc:1511] Processing CreateTablet for tablet 05840bb250764e02998cdddc290e5139 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:09:06.182940 24371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5eaa917e65d9403dbb241f84ee96ba9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.183796 24363 tablet_service.cc:1511] Processing CreateTablet for tablet b147866f4e5f4596b0b3989418cc4b0a (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:09:06.185876 24369 tablet_service.cc:1511] Processing CreateTablet for tablet e2b4c67e0f6b4a1e9cfb3703e43e988f (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:09:06.186996 24518 tablet_service.cc:1511] Processing CreateTablet for tablet 722754340deb4df18b716dc801c35c34 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:09:06.157812 24579 tablet_bootstrap.cc:492] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:06.188277 24518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 722754340deb4df18b716dc801c35c34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.192158 24296 tablet_service.cc:1511] Processing CreateTablet for tablet 722754340deb4df18b716dc801c35c34 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:09:06.193451 24296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 722754340deb4df18b716dc801c35c34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.193769 24366 tablet_service.cc:1511] Processing CreateTablet for tablet 2039124a6d8848149440a70100af8755 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:09:06.198254 24443 tablet_service.cc:1511] Processing CreateTablet for tablet 722754340deb4df18b716dc801c35c34 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:09:06.199695 24443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 722754340deb4df18b716dc801c35c34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.201504 24456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8775bc34e96d48ae8f5598affd26efdc" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:06.204967 24363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b147866f4e5f4596b0b3989418cc4b0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.163137 24576 tablet_bootstrap.cc:654] T db9e8fa567ab481c9b80ae8e30acd628 P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:06.211108 24366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2039124a6d8848149440a70100af8755. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.212344 24446 tablet_service.cc:1511] Processing CreateTablet for tablet 2039124a6d8848149440a70100af8755 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:09:06.215781 24446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2039124a6d8848149440a70100af8755. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.216180 24367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05840bb250764e02998cdddc290e5139. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:09:06.204993 24345 leader_election.cc:343] T 8775bc34e96d48ae8f5598affd26efdc P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:06.219453 24369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2b4c67e0f6b4a1e9cfb3703e43e988f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.242365 24579 tablet_bootstrap.cc:654] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:06.246177 24577 tablet_bootstrap.cc:654] T 4af7fc369d124452beb5fd07cf8592ec P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:06.247198 24382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af7fc369d124452beb5fd07cf8592ec" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
W20260709 13:09:06.249187 24493 leader_election.cc:343] T 4af7fc369d124452beb5fd07cf8592ec P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:06.252593 24575 tablet_bootstrap.cc:492] T e4dd51cda55249b48d7bfa39d5feb398 P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:06.253176 24575 ts_tablet_manager.cc:1403] T e4dd51cda55249b48d7bfa39d5feb398 P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.240s	user 0.020s	sys 0.039s
I20260709 13:09:06.256395 24575 raft_consensus.cc:359] T e4dd51cda55249b48d7bfa39d5feb398 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:06.257112 24575 raft_consensus.cc:385] T e4dd51cda55249b48d7bfa39d5feb398 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:06.257419 24575 raft_consensus.cc:740] T e4dd51cda55249b48d7bfa39d5feb398 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:06.258270 24575 consensus_queue.cc:260] T e4dd51cda55249b48d7bfa39d5feb398 P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:06.259310 24511 tablet_service.cc:1511] Processing CreateTablet for tablet 2039124a6d8848149440a70100af8755 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:09:06.260613 24511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2039124a6d8848149440a70100af8755. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.269829 24583 raft_consensus.cc:493] T e4dd51cda55249b48d7bfa39d5feb398 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:06.270375 24583 raft_consensus.cc:515] T e4dd51cda55249b48d7bfa39d5feb398 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:06.271482 24307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8775bc34e96d48ae8f5598affd26efdc" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
W20260709 13:09:06.273084 24346 leader_election.cc:343] T 8775bc34e96d48ae8f5598affd26efdc P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:06.273510 24583 leader_election.cc:290] T e4dd51cda55249b48d7bfa39d5feb398 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:06.273538 24346 leader_election.cc:304] T 8775bc34e96d48ae8f5598affd26efdc P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:06.274521 24582 raft_consensus.cc:2749] T 8775bc34e96d48ae8f5598affd26efdc P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:06.282182 24575 ts_tablet_manager.cc:1434] T e4dd51cda55249b48d7bfa39d5feb398 P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.029s	user 0.002s	sys 0.004s
I20260709 13:09:06.283219 24575 tablet_bootstrap.cc:492] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:06.285317 24363 tablet_service.cc:1511] Processing CreateTablet for tablet 90d9e64f97544895b9b97dfdc5ceb359 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:09:06.290108 24456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dd51cda55249b48d7bfa39d5feb398" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:06.293483 24363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90d9e64f97544895b9b97dfdc5ceb359. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.292129 24511 tablet_service.cc:1511] Processing CreateTablet for tablet 5bb19353ba334eac831ae2cce277298a (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:09:06.293754 24297 tablet_service.cc:1511] Processing CreateTablet for tablet 5bb19353ba334eac831ae2cce277298a (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:09:06.294977 24511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bb19353ba334eac831ae2cce277298a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.294978 24297 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bb19353ba334eac831ae2cce277298a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:09:06.295257 24493 leader_election.cc:343] T e4dd51cda55249b48d7bfa39d5feb398 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:06.298997 24307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af7fc369d124452beb5fd07cf8592ec" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:06.299536 24306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dd51cda55249b48d7bfa39d5feb398" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
W20260709 13:09:06.300593 24494 leader_election.cc:343] T 4af7fc369d124452beb5fd07cf8592ec P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:06.301056 24494 leader_election.cc:304] T 4af7fc369d124452beb5fd07cf8592ec P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390; no voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:06.301823 24583 raft_consensus.cc:2749] T 4af7fc369d124452beb5fd07cf8592ec P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:09:06.302662 24494 leader_election.cc:343] T e4dd51cda55249b48d7bfa39d5feb398 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:06.303148 24494 leader_election.cc:304] T e4dd51cda55249b48d7bfa39d5feb398 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390; no voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:06.303930 24583 raft_consensus.cc:2749] T e4dd51cda55249b48d7bfa39d5feb398 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:06.306957 24371 tablet_service.cc:1511] Processing CreateTablet for tablet 5bb19353ba334eac831ae2cce277298a (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:09:06.308485 24371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bb19353ba334eac831ae2cce277298a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.296941 24291 tablet_service.cc:1511] Processing CreateTablet for tablet 90d9e64f97544895b9b97dfdc5ceb359 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:09:06.316323 24291 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90d9e64f97544895b9b97dfdc5ceb359. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.321557 24442 tablet_service.cc:1511] Processing CreateTablet for tablet 6427fa143f574eaba2ee4c979e5485fc (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:09:06.323328 24442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6427fa143f574eaba2ee4c979e5485fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.295589 24445 tablet_service.cc:1511] Processing CreateTablet for tablet 90d9e64f97544895b9b97dfdc5ceb359 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:09:06.327234 24445 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90d9e64f97544895b9b97dfdc5ceb359. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.334288 24518 tablet_service.cc:1511] Processing CreateTablet for tablet 6427fa143f574eaba2ee4c979e5485fc (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:09:06.335702 24575 tablet_bootstrap.cc:654] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:06.335819 24518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6427fa143f574eaba2ee4c979e5485fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.336143 24296 tablet_service.cc:1511] Processing CreateTablet for tablet 6427fa143f574eaba2ee4c979e5485fc (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:09:06.337441 24296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6427fa143f574eaba2ee4c979e5485fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.343844 24584 raft_consensus.cc:493] T 54fa63b278394f569a9ecffed4dec233 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:06.344401 24366 tablet_service.cc:1511] Processing CreateTablet for tablet 72e12f99e2084675ab3f53ffb836e464 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:09:06.344400 24584 raft_consensus.cc:515] T 54fa63b278394f569a9ecffed4dec233 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:06.345822 24366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72e12f99e2084675ab3f53ffb836e464. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.348968 24584 leader_election.cc:290] T 54fa63b278394f569a9ecffed4dec233 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:06.354408 24576 tablet_bootstrap.cc:492] T db9e8fa567ab481c9b80ae8e30acd628 P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:06.354957 24576 ts_tablet_manager.cc:1403] T db9e8fa567ab481c9b80ae8e30acd628 P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.276s	user 0.017s	sys 0.006s
I20260709 13:09:06.358047 24577 tablet_bootstrap.cc:492] T 4af7fc369d124452beb5fd07cf8592ec P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:06.358222 24576 raft_consensus.cc:359] T db9e8fa567ab481c9b80ae8e30acd628 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:06.358669 24577 ts_tablet_manager.cc:1403] T 4af7fc369d124452beb5fd07cf8592ec P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.230s	user 0.010s	sys 0.014s
I20260709 13:09:06.358992 24576 raft_consensus.cc:385] T db9e8fa567ab481c9b80ae8e30acd628 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:06.359369 24576 raft_consensus.cc:740] T db9e8fa567ab481c9b80ae8e30acd628 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:06.363111 24579 tablet_bootstrap.cc:492] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:06.362947 24577 raft_consensus.cc:359] T 4af7fc369d124452beb5fd07cf8592ec P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:06.369937 24443 tablet_service.cc:1511] Processing CreateTablet for tablet 72e12f99e2084675ab3f53ffb836e464 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:09:06.371290 24443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72e12f99e2084675ab3f53ffb836e464. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.367239 24579 ts_tablet_manager.cc:1403] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.236s	user 0.007s	sys 0.015s
I20260709 13:09:06.376542 24576 consensus_queue.cc:260] T db9e8fa567ab481c9b80ae8e30acd628 P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:06.366977 24577 raft_consensus.cc:385] T 4af7fc369d124452beb5fd07cf8592ec P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:06.377866 24577 raft_consensus.cc:740] T 4af7fc369d124452beb5fd07cf8592ec P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:06.390908 24576 ts_tablet_manager.cc:1434] T db9e8fa567ab481c9b80ae8e30acd628 P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.036s	user 0.004s	sys 0.004s
I20260709 13:09:06.392031 24576 tablet_bootstrap.cc:492] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:06.404130 24576 tablet_bootstrap.cc:654] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:06.409899 24291 tablet_service.cc:1511] Processing CreateTablet for tablet 6ae3dc25b8484b6f99d8f41685229cb8 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:09:06.411230 24291 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ae3dc25b8484b6f99d8f41685229cb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.413143 24579 raft_consensus.cc:359] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:06.419039 24579 raft_consensus.cc:385] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:06.417124 24371 tablet_service.cc:1511] Processing CreateTablet for tablet 6ae3dc25b8484b6f99d8f41685229cb8 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:09:06.387882 24577 consensus_queue.cc:260] T 4af7fc369d124452beb5fd07cf8592ec P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:06.426258 24582 raft_consensus.cc:493] T 4af7fc369d124452beb5fd07cf8592ec P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:06.426518 24306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fa63b278394f569a9ecffed4dec233" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:06.424160 24512 tablet_service.cc:1511] Processing CreateTablet for tablet 3e71fd85d9424a08aed292991bccf5ee (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:09:06.426687 24371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ae3dc25b8484b6f99d8f41685229cb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.426719 24582 raft_consensus.cc:515] T 4af7fc369d124452beb5fd07cf8592ec P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:06.427763 24512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e71fd85d9424a08aed292991bccf5ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.428776 24446 tablet_service.cc:1511] Processing CreateTablet for tablet 6ae3dc25b8484b6f99d8f41685229cb8 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:09:06.430124 24446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ae3dc25b8484b6f99d8f41685229cb8. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:09:06.430852 24420 leader_election.cc:343] T 54fa63b278394f569a9ecffed4dec233 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:06.431491 24582 leader_election.cc:290] T 4af7fc369d124452beb5fd07cf8592ec P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:06.434513 24306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af7fc369d124452beb5fd07cf8592ec" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
W20260709 13:09:06.436372 24346 leader_election.cc:343] T 4af7fc369d124452beb5fd07cf8592ec P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:06.441218 24577 ts_tablet_manager.cc:1434] T 4af7fc369d124452beb5fd07cf8592ec P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.082s	user 0.013s	sys 0.001s
I20260709 13:09:06.442266 24577 tablet_bootstrap.cc:492] T aac1b62e27a44fb5b9a23cfc42a5d00b P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:06.447003 24579 raft_consensus.cc:740] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:06.449242 24579 consensus_queue.cc:260] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:06.451881 24575 tablet_bootstrap.cc:492] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:06.453483 24575 ts_tablet_manager.cc:1403] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.170s	user 0.015s	sys 0.001s
I20260709 13:09:06.453979 24582 raft_consensus.cc:493] T db9e8fa567ab481c9b80ae8e30acd628 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:06.454486 24582 raft_consensus.cc:515] T db9e8fa567ab481c9b80ae8e30acd628 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:06.455963 24589 raft_consensus.cc:493] T 958ccd87b65945468e9247c40ca27c33 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:06.456638 24582 leader_election.cc:290] T db9e8fa567ab481c9b80ae8e30acd628 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:06.456449 24589 raft_consensus.cc:515] T 958ccd87b65945468e9247c40ca27c33 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:06.425271 24297 tablet_service.cc:1511] Processing CreateTablet for tablet 3e71fd85d9424a08aed292991bccf5ee (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:09:06.456722 24575 raft_consensus.cc:359] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:06.457590 24575 raft_consensus.cc:385] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:06.457937 24297 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e71fd85d9424a08aed292991bccf5ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.457903 24575 raft_consensus.cc:740] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:06.459101 24456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9e8fa567ab481c9b80ae8e30acd628" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:06.459002 24575 consensus_queue.cc:260] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:06.459759 24456 raft_consensus.cc:2468] T db9e8fa567ab481c9b80ae8e30acd628 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 0.
I20260709 13:09:06.461416 24589 leader_election.cc:290] T 958ccd87b65945468e9247c40ca27c33 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:06.461580 24575 ts_tablet_manager.cc:1434] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:09:06.461984 24345 leader_election.cc:304] T db9e8fa567ab481c9b80ae8e30acd628 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:06.463776 24582 raft_consensus.cc:2804] T db9e8fa567ab481c9b80ae8e30acd628 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:06.464141 24582 raft_consensus.cc:493] T db9e8fa567ab481c9b80ae8e30acd628 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:06.464493 24582 raft_consensus.cc:3060] T db9e8fa567ab481c9b80ae8e30acd628 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:06.466358 24575 tablet_bootstrap.cc:492] T db9e8fa567ab481c9b80ae8e30acd628 P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:06.479828 24583 raft_consensus.cc:493] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:06.480365 24583 raft_consensus.cc:515] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:06.482513 24583 leader_election.cc:290] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:06.488358 24306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42bcfb18b77e41afa60057ccfd718fa9" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
W20260709 13:09:06.490278 24494 leader_election.cc:343] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:06.490774 24382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42bcfb18b77e41afa60057ccfd718fa9" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:06.490833 24363 tablet_service.cc:1511] Processing CreateTablet for tablet 3e71fd85d9424a08aed292991bccf5ee (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:09:06.492014 24518 tablet_service.cc:1511] Processing CreateTablet for tablet 2deba793363c4afca7d75b2fc3814304 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:09:06.493088 24363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e71fd85d9424a08aed292991bccf5ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.493196 24518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2deba793363c4afca7d75b2fc3814304. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.494127 24381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fa63b278394f569a9ecffed4dec233" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
W20260709 13:09:06.494871 24493 leader_election.cc:343] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:06.495378 24493 leader_election.cc:304] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390; no voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45
W20260709 13:09:06.495847 24419 leader_election.cc:343] T 54fa63b278394f569a9ecffed4dec233 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:06.496335 24419 leader_election.cc:304] T 54fa63b278394f569a9ecffed4dec233 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32088c942fbb48239e0eed952d4df449; no voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:06.496603 24583 raft_consensus.cc:2749] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:06.495208 24575 tablet_bootstrap.cc:654] T db9e8fa567ab481c9b80ae8e30acd628 P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:06.497558 24584 raft_consensus.cc:2749] T 54fa63b278394f569a9ecffed4dec233 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:06.508916 24443 tablet_service.cc:1511] Processing CreateTablet for tablet 2deba793363c4afca7d75b2fc3814304 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:09:06.510959 24443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2deba793363c4afca7d75b2fc3814304. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.512998 24456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958ccd87b65945468e9247c40ca27c33" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
W20260709 13:09:06.514827 24270 leader_election.cc:343] T 958ccd87b65945468e9247c40ca27c33 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:06.512985 24511 tablet_service.cc:1511] Processing CreateTablet for tablet 72e12f99e2084675ab3f53ffb836e464 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:09:06.516822 24577 tablet_bootstrap.cc:654] T aac1b62e27a44fb5b9a23cfc42a5d00b P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:06.516299 24582 raft_consensus.cc:515] T db9e8fa567ab481c9b80ae8e30acd628 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:06.523784 24582 leader_election.cc:290] T db9e8fa567ab481c9b80ae8e30acd628 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 election: Requested vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:06.524420 24456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9e8fa567ab481c9b80ae8e30acd628" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:06.525034 24456 raft_consensus.cc:3060] T db9e8fa567ab481c9b80ae8e30acd628 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:06.532626 24456 raft_consensus.cc:2468] T db9e8fa567ab481c9b80ae8e30acd628 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 1.
I20260709 13:09:06.533810 24345 leader_election.cc:304] T db9e8fa567ab481c9b80ae8e30acd628 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:06.538838 24296 tablet_service.cc:1511] Processing CreateTablet for tablet 2deba793363c4afca7d75b2fc3814304 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:09:06.541778 24579 ts_tablet_manager.cc:1434] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.165s	user 0.006s	sys 0.007s
I20260709 13:09:06.541853 24296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2deba793363c4afca7d75b2fc3814304. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.544832 24579 tablet_bootstrap.cc:492] T 54fa63b278394f569a9ecffed4dec233 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:06.547101 24371 tablet_service.cc:1511] Processing CreateTablet for tablet f4e53ea5cf694db697fa0e474da54423 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:09:06.548069 24511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72e12f99e2084675ab3f53ffb836e464. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.548595 24371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4e53ea5cf694db697fa0e474da54423. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.539476 24582 raft_consensus.cc:2804] T db9e8fa567ab481c9b80ae8e30acd628 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:06.552270 24582 raft_consensus.cc:697] T db9e8fa567ab481c9b80ae8e30acd628 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 LEADER]: Becoming Leader. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Running, Role: LEADER
I20260709 13:09:06.552524 24442 tablet_service.cc:1511] Processing CreateTablet for tablet f4e53ea5cf694db697fa0e474da54423 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:09:06.553280 24582 consensus_queue.cc:237] T db9e8fa567ab481c9b80ae8e30acd628 P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:06.553898 24442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4e53ea5cf694db697fa0e474da54423. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.554749 24517 tablet_service.cc:1511] Processing CreateTablet for tablet f4e53ea5cf694db697fa0e474da54423 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:09:06.556113 24517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4e53ea5cf694db697fa0e474da54423. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.574251 24291 tablet_service.cc:1511] Processing CreateTablet for tablet 72ac163496084a3e977344fa9228a3c9 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:09:06.574932 24530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958ccd87b65945468e9247c40ca27c33" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:06.575719 24291 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72ac163496084a3e977344fa9228a3c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.576790 24528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9e8fa567ab481c9b80ae8e30acd628" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:06.578467 24529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af7fc369d124452beb5fd07cf8592ec" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:06.579066 24527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9e8fa567ab481c9b80ae8e30acd628" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:06.579327 24529 raft_consensus.cc:2468] T 4af7fc369d124452beb5fd07cf8592ec P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 0.
W20260709 13:09:06.580622 24270 leader_election.cc:343] T 958ccd87b65945468e9247c40ca27c33 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:06.581202 24270 leader_election.cc:304] T 958ccd87b65945468e9247c40ca27c33 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381; no voters: 32088c942fbb48239e0eed952d4df449, 838cb9997b0f437989d3f4e3e9904390
I20260709 13:09:06.582146 24589 raft_consensus.cc:2749] T 958ccd87b65945468e9247c40ca27c33 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:06.583175 24366 tablet_service.cc:1511] Processing CreateTablet for tablet 72ac163496084a3e977344fa9228a3c9 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:09:06.585223 24366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72ac163496084a3e977344fa9228a3c9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:09:06.606618 24345 leader_election.cc:343] T db9e8fa567ab481c9b80ae8e30acd628 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:06.607182 24446 tablet_service.cc:1511] Processing CreateTablet for tablet 72ac163496084a3e977344fa9228a3c9 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:09:06.610795 24446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72ac163496084a3e977344fa9228a3c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.613057 24297 tablet_service.cc:1511] Processing CreateTablet for tablet 40dd44ec1fb347f49c31a8b650ab1639 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:09:06.613111 24516 tablet_service.cc:1511] Processing CreateTablet for tablet 40dd44ec1fb347f49c31a8b650ab1639 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:09:06.614487 24297 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40dd44ec1fb347f49c31a8b650ab1639. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.614566 24516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40dd44ec1fb347f49c31a8b650ab1639. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:09:06.615989 24345 leader_election.cc:343] T db9e8fa567ab481c9b80ae8e30acd628 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:06.622139 24345 leader_election.cc:304] T 4af7fc369d124452beb5fd07cf8592ec P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:06.623072 24577 tablet_bootstrap.cc:492] T aac1b62e27a44fb5b9a23cfc42a5d00b P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:06.623512 24603 raft_consensus.cc:2804] T 4af7fc369d124452beb5fd07cf8592ec P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:06.623646 24577 ts_tablet_manager.cc:1403] T aac1b62e27a44fb5b9a23cfc42a5d00b P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.182s	user 0.013s	sys 0.007s
I20260709 13:09:06.624050 24603 raft_consensus.cc:493] T 4af7fc369d124452beb5fd07cf8592ec P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:06.624423 24603 raft_consensus.cc:3060] T 4af7fc369d124452beb5fd07cf8592ec P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:06.626986 24577 raft_consensus.cc:359] T aac1b62e27a44fb5b9a23cfc42a5d00b P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:06.627806 24577 raft_consensus.cc:385] T aac1b62e27a44fb5b9a23cfc42a5d00b P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:06.628094 24577 raft_consensus.cc:740] T aac1b62e27a44fb5b9a23cfc42a5d00b P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:06.629019 24577 consensus_queue.cc:260] T aac1b62e27a44fb5b9a23cfc42a5d00b P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:06.636865 24576 tablet_bootstrap.cc:492] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:06.637501 24576 ts_tablet_manager.cc:1403] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.246s	user 0.028s	sys 0.011s
I20260709 13:09:06.641135 24576 raft_consensus.cc:359] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:06.641263 24369 tablet_service.cc:1511] Processing CreateTablet for tablet 40dd44ec1fb347f49c31a8b650ab1639 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:09:06.641911 24576 raft_consensus.cc:385] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:06.642220 24576 raft_consensus.cc:740] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:06.642653 24369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40dd44ec1fb347f49c31a8b650ab1639. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.643105 24443 tablet_service.cc:1511] Processing CreateTablet for tablet 836ed40e5bb8485cb93cb03bc2508265 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:09:06.644457 24443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 836ed40e5bb8485cb93cb03bc2508265. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.651737 24517 tablet_service.cc:1511] Processing CreateTablet for tablet 836ed40e5bb8485cb93cb03bc2508265 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:09:06.653275 24517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 836ed40e5bb8485cb93cb03bc2508265. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:09:06.654630 24253 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:06.660038 24575 tablet_bootstrap.cc:492] T db9e8fa567ab481c9b80ae8e30acd628 P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:06.660625 24575 ts_tablet_manager.cc:1403] T db9e8fa567ab481c9b80ae8e30acd628 P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.194s	user 0.012s	sys 0.015s
I20260709 13:09:06.661036 24293 tablet_service.cc:1511] Processing CreateTablet for tablet 836ed40e5bb8485cb93cb03bc2508265 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:09:06.662362 24293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 836ed40e5bb8485cb93cb03bc2508265. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.663883 24575 raft_consensus.cc:359] T db9e8fa567ab481c9b80ae8e30acd628 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:06.664099 24368 tablet_service.cc:1511] Processing CreateTablet for tablet a1b91f42722c4116950ce80bc02eccd0 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:09:06.664819 24575 raft_consensus.cc:385] T db9e8fa567ab481c9b80ae8e30acd628 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:06.665212 24575 raft_consensus.cc:740] T db9e8fa567ab481c9b80ae8e30acd628 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:06.665453 24368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1b91f42722c4116950ce80bc02eccd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.666122 24575 consensus_queue.cc:260] T db9e8fa567ab481c9b80ae8e30acd628 P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:06.670400 24589 raft_consensus.cc:493] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:06.671011 24589 raft_consensus.cc:515] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:06.672899 24579 tablet_bootstrap.cc:654] T 54fa63b278394f569a9ecffed4dec233 P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:06.673432 24589 leader_election.cc:290] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:06.670499 24575 ts_tablet_manager.cc:1434] T db9e8fa567ab481c9b80ae8e30acd628 P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:09:06.677875 24610 raft_consensus.cc:493] T aac1b62e27a44fb5b9a23cfc42a5d00b P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:06.678237 24575 tablet_bootstrap.cc:492] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:06.678357 24610 raft_consensus.cc:515] T aac1b62e27a44fb5b9a23cfc42a5d00b P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:06.680760 24610 leader_election.cc:290] T aac1b62e27a44fb5b9a23cfc42a5d00b P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:06.681427 24456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aac1b62e27a44fb5b9a23cfc42a5d00b" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
W20260709 13:09:06.682909 24345 leader_election.cc:343] T aac1b62e27a44fb5b9a23cfc42a5d00b P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:06.676126 24603 raft_consensus.cc:515] T 4af7fc369d124452beb5fd07cf8592ec P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:06.663914 24576 consensus_queue.cc:260] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:06.686831 24527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af7fc369d124452beb5fd07cf8592ec" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:06.687144 24306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af7fc369d124452beb5fd07cf8592ec" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:06.687481 24529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aac1b62e27a44fb5b9a23cfc42a5d00b" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:06.691354 24577 ts_tablet_manager.cc:1434] T aac1b62e27a44fb5b9a23cfc42a5d00b P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.067s	user 0.003s	sys 0.007s
I20260709 13:09:06.691465 24456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8775bc34e96d48ae8f5598affd26efdc" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:06.692258 24456 raft_consensus.cc:2468] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 0.
I20260709 13:09:06.692763 24577 tablet_bootstrap.cc:492] T 54fa63b278394f569a9ecffed4dec233 P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
W20260709 13:09:06.693747 24346 leader_election.cc:343] T 4af7fc369d124452beb5fd07cf8592ec P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:06.694566 24575 tablet_bootstrap.cc:654] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:06.689775 24527 raft_consensus.cc:3060] T 4af7fc369d124452beb5fd07cf8592ec P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:06.685609 24603 leader_election.cc:290] T 4af7fc369d124452beb5fd07cf8592ec P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 election: Requested vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
W20260709 13:09:06.708056 24345 leader_election.cc:343] T aac1b62e27a44fb5b9a23cfc42a5d00b P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:06.708648 24345 leader_election.cc:304] T aac1b62e27a44fb5b9a23cfc42a5d00b P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449, 838cb9997b0f437989d3f4e3e9904390
I20260709 13:09:06.709488 24603 raft_consensus.cc:2749] T aac1b62e27a44fb5b9a23cfc42a5d00b P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:06.718966 24576 ts_tablet_manager.cc:1434] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.081s	user 0.007s	sys 0.001s
I20260709 13:09:06.722527 24577 tablet_bootstrap.cc:654] T 54fa63b278394f569a9ecffed4dec233 P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:06.713979 24270 leader_election.cc:304] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381; no voters: 
I20260709 13:09:06.715579 24527 raft_consensus.cc:2468] T 4af7fc369d124452beb5fd07cf8592ec P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 1.
I20260709 13:09:06.725497 24576 tablet_bootstrap.cc:492] T 48aea62663ac40998d7f09389b93d586 P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:06.725356 24589 raft_consensus.cc:2804] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:06.726876 24589 raft_consensus.cc:493] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:06.727370 24589 raft_consensus.cc:3060] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:06.735728 24345 leader_election.cc:304] T 4af7fc369d124452beb5fd07cf8592ec P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:06.734552 24443 tablet_service.cc:1511] Processing CreateTablet for tablet a1b91f42722c4116950ce80bc02eccd0 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:09:06.737903 24582 raft_consensus.cc:2804] T 4af7fc369d124452beb5fd07cf8592ec P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:06.738469 24443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1b91f42722c4116950ce80bc02eccd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.740087 24205 catalog_manager.cc:5697] T db9e8fa567ab481c9b80ae8e30acd628 P c40e7ba102224d24a22ccd1f68aaae45 reported cstate change: term changed from 0 to 1, leader changed from <none> to c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2). New cstate: current_term: 1 leader_uuid: "c40e7ba102224d24a22ccd1f68aaae45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:06.744282 24589 raft_consensus.cc:515] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:06.745368 24381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8775bc34e96d48ae8f5598affd26efdc" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:06.743357 24582 raft_consensus.cc:697] T 4af7fc369d124452beb5fd07cf8592ec P c40e7ba102224d24a22ccd1f68aaae45 [term 1 LEADER]: Becoming Leader. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Running, Role: LEADER
I20260709 13:09:06.746037 24381 raft_consensus.cc:2468] T 8775bc34e96d48ae8f5598affd26efdc P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 0.
I20260709 13:09:06.746502 24589 leader_election.cc:290] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Requested vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:06.746579 24582 consensus_queue.cc:237] T 4af7fc369d124452beb5fd07cf8592ec P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:06.747795 24456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8775bc34e96d48ae8f5598affd26efdc" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:06.748431 24456 raft_consensus.cc:3060] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:06.756922 24516 tablet_service.cc:1511] Processing CreateTablet for tablet a1b91f42722c4116950ce80bc02eccd0 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:09:06.761256 24516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1b91f42722c4116950ce80bc02eccd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.761456 24381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8775bc34e96d48ae8f5598affd26efdc" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:06.762398 24381 raft_consensus.cc:3060] T 8775bc34e96d48ae8f5598affd26efdc P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:06.768954 24456 raft_consensus.cc:2468] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 1.
I20260709 13:09:06.770406 24270 leader_election.cc:304] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381; no voters: 
I20260709 13:09:06.771337 24589 raft_consensus.cc:2804] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:06.774967 24579 tablet_bootstrap.cc:492] T 54fa63b278394f569a9ecffed4dec233 P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:06.775918 24579 ts_tablet_manager.cc:1403] T 54fa63b278394f569a9ecffed4dec233 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.231s	user 0.030s	sys 0.012s
I20260709 13:09:06.781431 24579 raft_consensus.cc:359] T 54fa63b278394f569a9ecffed4dec233 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:06.782517 24579 raft_consensus.cc:385] T 54fa63b278394f569a9ecffed4dec233 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:06.782904 24579 raft_consensus.cc:740] T 54fa63b278394f569a9ecffed4dec233 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:06.790506 24293 tablet_service.cc:1511] Processing CreateTablet for tablet 2f7b6652eba4485ca0632f9fefc2e326 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:09:06.792404 24293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f7b6652eba4485ca0632f9fefc2e326. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.800509 24368 tablet_service.cc:1511] Processing CreateTablet for tablet 2f7b6652eba4485ca0632f9fefc2e326 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:09:06.802435 24368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f7b6652eba4485ca0632f9fefc2e326. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.776600 24576 tablet_bootstrap.cc:654] T 48aea62663ac40998d7f09389b93d586 P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:06.816464 24589 raft_consensus.cc:697] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Becoming Leader. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Running, Role: LEADER
I20260709 13:09:06.816906 24579 consensus_queue.cc:260] T 54fa63b278394f569a9ecffed4dec233 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:06.817590 24589 consensus_queue.cc:237] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:06.818339 24577 tablet_bootstrap.cc:492] T 54fa63b278394f569a9ecffed4dec233 P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:06.818900 24577 ts_tablet_manager.cc:1403] T 54fa63b278394f569a9ecffed4dec233 P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.126s	user 0.040s	sys 0.034s
I20260709 13:09:06.819413 24579 ts_tablet_manager.cc:1434] T 54fa63b278394f569a9ecffed4dec233 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.043s	user 0.008s	sys 0.000s
I20260709 13:09:06.820547 24579 tablet_bootstrap.cc:492] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:06.822254 24577 raft_consensus.cc:359] T 54fa63b278394f569a9ecffed4dec233 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:06.823014 24577 raft_consensus.cc:385] T 54fa63b278394f569a9ecffed4dec233 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:06.823324 24577 raft_consensus.cc:740] T 54fa63b278394f569a9ecffed4dec233 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:06.824046 24577 consensus_queue.cc:260] T 54fa63b278394f569a9ecffed4dec233 P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:06.825732 24381 raft_consensus.cc:2468] T 8775bc34e96d48ae8f5598affd26efdc P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 1.
I20260709 13:09:06.826654 24577 ts_tablet_manager.cc:1434] T 54fa63b278394f569a9ecffed4dec233 P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:09:06.827715 24577 tablet_bootstrap.cc:492] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:06.829501 24443 tablet_service.cc:1511] Processing CreateTablet for tablet 2f7b6652eba4485ca0632f9fefc2e326 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:09:06.830865 24443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f7b6652eba4485ca0632f9fefc2e326. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.834010 24577 tablet_bootstrap.cc:654] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:06.844027 24516 tablet_service.cc:1511] Processing CreateTablet for tablet 40cf5b30d13f4c3e86a9aa657c20c650 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:09:06.845434 24516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40cf5b30d13f4c3e86a9aa657c20c650. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.845539 24577 tablet_bootstrap.cc:492] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:06.846153 24577 ts_tablet_manager.cc:1403] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.004s
I20260709 13:09:06.849310 24577 raft_consensus.cc:359] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:06.850107 24577 raft_consensus.cc:385] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:06.850411 24577 raft_consensus.cc:740] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:06.849162 24203 catalog_manager.cc:5697] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "9c2a14143eb744cf83bfa5f46284a381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:06.851390 24577 consensus_queue.cc:260] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:06.854121 24577 ts_tablet_manager.cc:1434] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:09:06.855274 24577 tablet_bootstrap.cc:492] T 48aea62663ac40998d7f09389b93d586 P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:06.870452 24297 tablet_service.cc:1511] Processing CreateTablet for tablet 40cf5b30d13f4c3e86a9aa657c20c650 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:09:06.871881 24297 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40cf5b30d13f4c3e86a9aa657c20c650. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.876427 24577 tablet_bootstrap.cc:654] T 48aea62663ac40998d7f09389b93d586 P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:06.876989 24368 tablet_service.cc:1511] Processing CreateTablet for tablet 40cf5b30d13f4c3e86a9aa657c20c650 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:09:06.878407 24368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40cf5b30d13f4c3e86a9aa657c20c650. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.879446 24576 tablet_bootstrap.cc:492] T 48aea62663ac40998d7f09389b93d586 P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:06.880020 24576 ts_tablet_manager.cc:1403] T 48aea62663ac40998d7f09389b93d586 P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.155s	user 0.020s	sys 0.012s
I20260709 13:09:06.883147 24576 raft_consensus.cc:359] T 48aea62663ac40998d7f09389b93d586 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:06.885465 24575 tablet_bootstrap.cc:492] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:06.896347 24575 ts_tablet_manager.cc:1403] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.218s	user 0.012s	sys 0.008s
I20260709 13:09:06.896607 24443 tablet_service.cc:1511] Processing CreateTablet for tablet 363cde7d4cb84353a74a18cbd9701fa7 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:09:06.897029 24369 tablet_service.cc:1511] Processing CreateTablet for tablet 84af43749e474087b0a5857a95dfb31d (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:09:06.898044 24443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 363cde7d4cb84353a74a18cbd9701fa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.898309 24369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84af43749e474087b0a5857a95dfb31d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.901722 24575 raft_consensus.cc:359] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:06.906797 24575 raft_consensus.cc:385] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:06.907719 24575 raft_consensus.cc:740] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:06.908565 24575 consensus_queue.cc:260] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:06.888273 24517 tablet_service.cc:1511] Processing CreateTablet for tablet 363cde7d4cb84353a74a18cbd9701fa7 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:09:06.924305 24205 catalog_manager.cc:5697] T 4af7fc369d124452beb5fd07cf8592ec P c40e7ba102224d24a22ccd1f68aaae45 reported cstate change: term changed from 0 to 1, leader changed from <none> to c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2). New cstate: current_term: 1 leader_uuid: "c40e7ba102224d24a22ccd1f68aaae45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:06.925129 24517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 363cde7d4cb84353a74a18cbd9701fa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.888501 24296 tablet_service.cc:1511] Processing CreateTablet for tablet 363cde7d4cb84353a74a18cbd9701fa7 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:09:06.927892 24296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 363cde7d4cb84353a74a18cbd9701fa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.904001 24577 tablet_bootstrap.cc:492] T 48aea62663ac40998d7f09389b93d586 P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:06.929391 24577 ts_tablet_manager.cc:1403] T 48aea62663ac40998d7f09389b93d586 P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.074s	user 0.004s	sys 0.023s
I20260709 13:09:06.930822 24518 tablet_service.cc:1511] Processing CreateTablet for tablet 142bbcbf57124963866a3967c8adebed (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:09:06.932145 24518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 142bbcbf57124963866a3967c8adebed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.932283 24293 tablet_service.cc:1511] Processing CreateTablet for tablet 142bbcbf57124963866a3967c8adebed (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:09:06.933660 24293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 142bbcbf57124963866a3967c8adebed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.937029 24577 raft_consensus.cc:359] T 48aea62663ac40998d7f09389b93d586 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:06.938674 24291 tablet_service.cc:1511] Processing CreateTablet for tablet 9c88d903166c4812a88653e389ab5776 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:09:06.940012 24291 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c88d903166c4812a88653e389ab5776. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.883881 24576 raft_consensus.cc:385] T 48aea62663ac40998d7f09389b93d586 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:06.943929 24576 raft_consensus.cc:740] T 48aea62663ac40998d7f09389b93d586 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:06.915055 24446 tablet_service.cc:1511] Processing CreateTablet for tablet 84af43749e474087b0a5857a95dfb31d (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:09:06.944808 24294 tablet_service.cc:1511] Processing CreateTablet for tablet dd5ac0d1debc4695a406617ec06f07a0 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:09:06.945592 24446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84af43749e474087b0a5857a95dfb31d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.946143 24294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd5ac0d1debc4695a406617ec06f07a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.944803 24576 consensus_queue.cc:260] T 48aea62663ac40998d7f09389b93d586 P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:06.950736 24289 tablet_service.cc:1511] Processing CreateTablet for tablet c6af3489742e4539866e5a3b10761ca2 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:09:06.952199 24289 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6af3489742e4539866e5a3b10761ca2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.956562 24576 ts_tablet_manager.cc:1434] T 48aea62663ac40998d7f09389b93d586 P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.076s	user 0.007s	sys 0.000s
I20260709 13:09:06.893364 24579 tablet_bootstrap.cc:654] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:06.918974 24516 tablet_service.cc:1511] Processing CreateTablet for tablet 84af43749e474087b0a5857a95dfb31d (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:09:06.960374 24516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84af43749e474087b0a5857a95dfb31d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.962221 24576 tablet_bootstrap.cc:492] T 861acffd0d804c8fbd1a63771d7937c7 P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:06.922506 24368 tablet_service.cc:1511] Processing CreateTablet for tablet 915d8224f8b64cdab90dff1689046548 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:09:06.971108 24517 tablet_service.cc:1511] Processing CreateTablet for tablet 9c88d903166c4812a88653e389ab5776 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:09:06.972260 24368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 915d8224f8b64cdab90dff1689046548. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.973081 24442 tablet_service.cc:1511] Processing CreateTablet for tablet 9c88d903166c4812a88653e389ab5776 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:09:06.973428 24445 tablet_service.cc:1511] Processing CreateTablet for tablet a3e3b771ce2c4165aad02954773c58d7 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:09:06.974529 24442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c88d903166c4812a88653e389ab5776. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.975608 24440 tablet_service.cc:1511] Processing CreateTablet for tablet b88feffc0859413482b0daf562d2f86a (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:09:06.976989 24440 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b88feffc0859413482b0daf562d2f86a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.977589 24366 tablet_service.cc:1511] Processing CreateTablet for tablet a3e3b771ce2c4165aad02954773c58d7 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:09:06.977880 24363 tablet_service.cc:1511] Processing CreateTablet for tablet dd5ac0d1debc4695a406617ec06f07a0 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:09:06.978204 24371 tablet_service.cc:1511] Processing CreateTablet for tablet c6af3489742e4539866e5a3b10761ca2 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:09:06.978879 24366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3e3b771ce2c4165aad02954773c58d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.923138 24443 tablet_service.cc:1511] Processing CreateTablet for tablet 915d8224f8b64cdab90dff1689046548 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:09:06.971411 24511 tablet_service.cc:1511] Processing CreateTablet for tablet a3e3b771ce2c4165aad02954773c58d7 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:09:06.980082 24363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd5ac0d1debc4695a406617ec06f07a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.980355 24443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 915d8224f8b64cdab90dff1689046548. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.980863 24511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3e3b771ce2c4165aad02954773c58d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.980993 24445 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3e3b771ce2c4165aad02954773c58d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.920459 24297 tablet_service.cc:1511] Processing CreateTablet for tablet 915d8224f8b64cdab90dff1689046548 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:09:06.982036 24297 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 915d8224f8b64cdab90dff1689046548. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.979475 24444 tablet_service.cc:1511] Processing CreateTablet for tablet dd5ac0d1debc4695a406617ec06f07a0 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:09:06.966871 24575 ts_tablet_manager.cc:1434] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.070s	user 0.008s	sys 0.000s
I20260709 13:09:06.984028 24444 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd5ac0d1debc4695a406617ec06f07a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.984741 24575 tablet_bootstrap.cc:492] T 48aea62663ac40998d7f09389b93d586 P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:06.974138 24512 tablet_service.cc:1511] Processing CreateTablet for tablet c6af3489742e4539866e5a3b10761ca2 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:09:06.988456 24512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6af3489742e4539866e5a3b10761ca2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.989734 24371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6af3489742e4539866e5a3b10761ca2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.937844 24577 raft_consensus.cc:385] T 48aea62663ac40998d7f09389b93d586 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:06.991837 24577 raft_consensus.cc:740] T 48aea62663ac40998d7f09389b93d586 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:06.992044 24517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c88d903166c4812a88653e389ab5776. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.992705 24577 consensus_queue.cc:260] T 48aea62663ac40998d7f09389b93d586 P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:06.993613 24515 tablet_service.cc:1511] Processing CreateTablet for tablet b88feffc0859413482b0daf562d2f86a (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:09:06.994913 24515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b88feffc0859413482b0daf562d2f86a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:06.995042 24577 ts_tablet_manager.cc:1434] T 48aea62663ac40998d7f09389b93d586 P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.065s	user 0.002s	sys 0.002s
I20260709 13:09:06.996153 24577 tablet_bootstrap.cc:492] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:07.003209 24577 tablet_bootstrap.cc:654] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:07.004666 24582 raft_consensus.cc:493] T 54fa63b278394f569a9ecffed4dec233 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:07.005205 24582 raft_consensus.cc:515] T 54fa63b278394f569a9ecffed4dec233 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:07.007390 24582 leader_election.cc:290] T 54fa63b278394f569a9ecffed4dec233 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:07.008911 24306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fa63b278394f569a9ecffed4dec233" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:07.012147 24306 raft_consensus.cc:2468] T 54fa63b278394f569a9ecffed4dec233 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 0.
I20260709 13:09:07.009618 24456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fa63b278394f569a9ecffed4dec233" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:07.013401 24456 raft_consensus.cc:2468] T 54fa63b278394f569a9ecffed4dec233 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 0.
I20260709 13:09:07.018472 24345 leader_election.cc:304] T 54fa63b278394f569a9ecffed4dec233 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:07.019335 24582 raft_consensus.cc:2804] T 54fa63b278394f569a9ecffed4dec233 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:07.019737 24582 raft_consensus.cc:493] T 54fa63b278394f569a9ecffed4dec233 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:07.020112 24582 raft_consensus.cc:3060] T 54fa63b278394f569a9ecffed4dec233 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:07.023600 24575 tablet_bootstrap.cc:654] T 48aea62663ac40998d7f09389b93d586 P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:07.036474 24369 tablet_service.cc:1511] Processing CreateTablet for tablet 142bbcbf57124963866a3967c8adebed (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:09:07.037878 24369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 142bbcbf57124963866a3967c8adebed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:07.038408 24297 tablet_service.cc:1511] Processing CreateTablet for tablet b88feffc0859413482b0daf562d2f86a (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:09:07.039528 24297 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b88feffc0859413482b0daf562d2f86a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:07.039856 24603 consensus_queue.cc:1048] T db9e8fa567ab481c9b80ae8e30acd628 P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:07.041882 24441 tablet_service.cc:1511] Processing CreateTablet for tablet 624f99e3165f49babbfce35effc9d627 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:09:07.043107 24441 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 624f99e3165f49babbfce35effc9d627. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:07.046095 24293 tablet_service.cc:1511] Processing CreateTablet for tablet bd607d60354e4864ae5499ee1c1a05c9 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:09:07.047781 24576 tablet_bootstrap.cc:654] T 861acffd0d804c8fbd1a63771d7937c7 P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:07.049074 24513 tablet_service.cc:1511] Processing CreateTablet for tablet 624f99e3165f49babbfce35effc9d627 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:09:07.072708 24513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 624f99e3165f49babbfce35effc9d627. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:07.049656 24293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd607d60354e4864ae5499ee1c1a05c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:07.060997 24527 raft_consensus.cc:3060] T db9e8fa567ab481c9b80ae8e30acd628 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:07.061244 24370 tablet_service.cc:1511] Processing CreateTablet for tablet bd607d60354e4864ae5499ee1c1a05c9 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:09:07.061246 24367 tablet_service.cc:1511] Processing CreateTablet for tablet 624f99e3165f49babbfce35effc9d627 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:09:07.078344 24370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd607d60354e4864ae5499ee1c1a05c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:07.081560 24367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 624f99e3165f49babbfce35effc9d627. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:07.039243 24582 raft_consensus.cc:515] T 54fa63b278394f569a9ecffed4dec233 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:07.104970 24582 leader_election.cc:290] T 54fa63b278394f569a9ecffed4dec233 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 election: Requested vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:07.106205 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fa63b278394f569a9ecffed4dec233" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:07.106639 24306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54fa63b278394f569a9ecffed4dec233" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:07.106899 24455 raft_consensus.cc:3060] T 54fa63b278394f569a9ecffed4dec233 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:07.107259 24306 raft_consensus.cc:3060] T 54fa63b278394f569a9ecffed4dec233 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:07.115010 24515 tablet_service.cc:1511] Processing CreateTablet for tablet 0f795acd16fd472b8c9360fb14e7e5d7 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:09:07.116668 24515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f795acd16fd472b8c9360fb14e7e5d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:07.122118 24289 tablet_service.cc:1511] Processing CreateTablet for tablet 0f795acd16fd472b8c9360fb14e7e5d7 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:09:07.124513 24289 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f795acd16fd472b8c9360fb14e7e5d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:07.146320 24440 tablet_service.cc:1511] Processing CreateTablet for tablet bd607d60354e4864ae5499ee1c1a05c9 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:09:07.147748 24440 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd607d60354e4864ae5499ee1c1a05c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:07.147853 24455 raft_consensus.cc:2468] T 54fa63b278394f569a9ecffed4dec233 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 1.
I20260709 13:09:07.149142 24345 leader_election.cc:304] T 54fa63b278394f569a9ecffed4dec233 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:07.150409 24582 raft_consensus.cc:2804] T 54fa63b278394f569a9ecffed4dec233 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:07.151239 24582 raft_consensus.cc:697] T 54fa63b278394f569a9ecffed4dec233 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 LEADER]: Becoming Leader. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Running, Role: LEADER
I20260709 13:09:07.153044 24603 raft_consensus.cc:493] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:07.153556 24603 raft_consensus.cc:515] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:07.152848 24582 consensus_queue.cc:237] T 54fa63b278394f569a9ecffed4dec233 P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:07.160089 24297 tablet_service.cc:1511] Processing CreateTablet for tablet edd24afe4d7847e3980c6e586b9dac5e (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:09:07.161711 24297 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edd24afe4d7847e3980c6e586b9dac5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:07.156936 24529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9711acbe7d044647bf774ee849c8a032" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:07.162497 24305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9711acbe7d044647bf774ee849c8a032" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:07.159272 24579 tablet_bootstrap.cc:492] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:07.163736 24579 ts_tablet_manager.cc:1403] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.343s	user 0.011s	sys 0.013s
W20260709 13:09:07.165042 24345 leader_election.cc:343] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:07.166285 24346 leader_election.cc:343] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:07.166706 24346 leader_election.cc:304] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:07.166849 24579 raft_consensus.cc:359] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:07.167600 24609 raft_consensus.cc:2749] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:09:07.168656 24346 consensus_peers.cc:597] T 4af7fc369d124452beb5fd07cf8592ec P c40e7ba102224d24a22ccd1f68aaae45 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:07.167605 24579 raft_consensus.cc:385] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:07.169652 24579 raft_consensus.cc:740] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:07.170934 24516 tablet_service.cc:1511] Processing CreateTablet for tablet edd24afe4d7847e3980c6e586b9dac5e (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:09:07.170993 24445 tablet_service.cc:1511] Processing CreateTablet for tablet edd24afe4d7847e3980c6e586b9dac5e (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:09:07.171577 24579 consensus_queue.cc:260] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:07.155651 24603 leader_election.cc:290] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:07.172626 24516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edd24afe4d7847e3980c6e586b9dac5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:07.182634 24583 raft_consensus.cc:493] T db9e8fa567ab481c9b80ae8e30acd628 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:07.184355 24583 raft_consensus.cc:515] T db9e8fa567ab481c9b80ae8e30acd628 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:07.190412 24583 leader_election.cc:290] T db9e8fa567ab481c9b80ae8e30acd628 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:07.191479 24456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9e8fa567ab481c9b80ae8e30acd628" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:07.191933 24381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db9e8fa567ab481c9b80ae8e30acd628" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:07.192521 24603 consensus_queue.cc:1048] T db9e8fa567ab481c9b80ae8e30acd628 P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:07.175118 24367 tablet_service.cc:1511] Processing CreateTablet for tablet 0f795acd16fd472b8c9360fb14e7e5d7 (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:09:07.199265 24493 leader_election.cc:304] T db9e8fa567ab481c9b80ae8e30acd628 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390; no voters: 32088c942fbb48239e0eed952d4df449, c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:07.199479 24445 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edd24afe4d7847e3980c6e586b9dac5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:07.206084 24579 ts_tablet_manager.cc:1434] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.042s	user 0.002s	sys 0.008s
I20260709 13:09:07.207237 24579 tablet_bootstrap.cc:492] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:07.207540 24367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f795acd16fd472b8c9360fb14e7e5d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:07.213090 24575 tablet_bootstrap.cc:492] T 48aea62663ac40998d7f09389b93d586 P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:07.213654 24575 ts_tablet_manager.cc:1403] T 48aea62663ac40998d7f09389b93d586 P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.229s	user 0.006s	sys 0.006s
I20260709 13:09:07.215618 24306 raft_consensus.cc:2468] T 54fa63b278394f569a9ecffed4dec233 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 1.
I20260709 13:09:07.222730 24577 tablet_bootstrap.cc:492] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:07.223538 24577 ts_tablet_manager.cc:1403] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.228s	user 0.007s	sys 0.006s
I20260709 13:09:07.228948 24576 tablet_bootstrap.cc:492] T 861acffd0d804c8fbd1a63771d7937c7 P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:07.237190 24369 tablet_service.cc:1511] Processing CreateTablet for tablet 3cf2c3be9fd244d4b9d4703d424d3c5d (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:09:07.238989 24577 raft_consensus.cc:359] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:07.239866 24577 raft_consensus.cc:385] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:07.240181 24577 raft_consensus.cc:740] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:07.240679 24369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cf2c3be9fd244d4b9d4703d424d3c5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:07.241053 24577 consensus_queue.cc:260] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:07.242823 24577 ts_tablet_manager.cc:1434] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.019s	user 0.003s	sys 0.004s
I20260709 13:09:07.243346 24576 ts_tablet_manager.cc:1403] T 861acffd0d804c8fbd1a63771d7937c7 P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.281s	user 0.018s	sys 0.012s
I20260709 13:09:07.245977 24577 tablet_bootstrap.cc:492] T 42bcfb18b77e41afa60057ccfd718fa9 P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:07.250993 24442 tablet_service.cc:1511] Processing CreateTablet for tablet 3cf2c3be9fd244d4b9d4703d424d3c5d (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:09:07.251322 24205 catalog_manager.cc:5697] T 54fa63b278394f569a9ecffed4dec233 P c40e7ba102224d24a22ccd1f68aaae45 reported cstate change: term changed from 0 to 1, leader changed from <none> to c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2). New cstate: current_term: 1 leader_uuid: "c40e7ba102224d24a22ccd1f68aaae45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:07.249847 24291 tablet_service.cc:1511] Processing CreateTablet for tablet e219ba07753f49a5abe454b27cd1494e (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:09:07.253209 24291 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e219ba07753f49a5abe454b27cd1494e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:07.253871 24442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cf2c3be9fd244d4b9d4703d424d3c5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:07.242537 24575 raft_consensus.cc:359] T 48aea62663ac40998d7f09389b93d586 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:07.257507 24576 raft_consensus.cc:359] T 861acffd0d804c8fbd1a63771d7937c7 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:07.260753 24576 raft_consensus.cc:385] T 861acffd0d804c8fbd1a63771d7937c7 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:07.261072 24576 raft_consensus.cc:740] T 861acffd0d804c8fbd1a63771d7937c7 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:07.261948 24576 consensus_queue.cc:260] T 861acffd0d804c8fbd1a63771d7937c7 P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:07.262605 24297 tablet_service.cc:1511] Processing CreateTablet for tablet 5b8d15473a964a0c8b8c283c4552706e (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:09:07.263800 24297 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b8d15473a964a0c8b8c283c4552706e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:07.268879 24584 raft_consensus.cc:493] T 48aea62663ac40998d7f09389b93d586 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:07.269398 24584 raft_consensus.cc:515] T 48aea62663ac40998d7f09389b93d586 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:07.280016 24381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48aea62663ac40998d7f09389b93d586" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:07.280205 24589 consensus_queue.cc:1048] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:07.280733 24381 raft_consensus.cc:2468] T 48aea62663ac40998d7f09389b93d586 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 0.
I20260709 13:09:07.281613 24584 leader_election.cc:290] T 48aea62663ac40998d7f09389b93d586 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:07.276598 24575 raft_consensus.cc:385] T 48aea62663ac40998d7f09389b93d586 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:07.298764 24363 tablet_service.cc:1511] Processing CreateTablet for tablet e219ba07753f49a5abe454b27cd1494e (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:09:07.298805 24441 tablet_service.cc:1511] Processing CreateTablet for tablet e219ba07753f49a5abe454b27cd1494e (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:09:07.299804 24579 tablet_bootstrap.cc:654] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:07.300356 24363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e219ba07753f49a5abe454b27cd1494e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:07.307453 24609 raft_consensus.cc:493] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:07.308063 24609 raft_consensus.cc:515] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:07.310113 24609 leader_election.cc:290] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:07.312142 24456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "373185ca8ab04fdb91c7947ec0668808" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:07.312742 24576 ts_tablet_manager.cc:1434] T 861acffd0d804c8fbd1a63771d7937c7 P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.069s	user 0.003s	sys 0.004s
I20260709 13:09:07.313867 24576 tablet_bootstrap.cc:492] T e4dd51cda55249b48d7bfa39d5feb398 P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:07.291795 24575 raft_consensus.cc:740] T 48aea62663ac40998d7f09389b93d586 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:07.314643 24306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "373185ca8ab04fdb91c7947ec0668808" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:07.314810 24575 consensus_queue.cc:260] T 48aea62663ac40998d7f09389b93d586 P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
W20260709 13:09:07.316460 24346 leader_election.cc:343] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:07.294881 24419 leader_election.cc:304] T 48aea62663ac40998d7f09389b93d586 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:07.300356 24441 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e219ba07753f49a5abe454b27cd1494e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:09:07.318427 24345 leader_election.cc:343] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:07.321298 24584 raft_consensus.cc:2804] T 48aea62663ac40998d7f09389b93d586 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:07.323836 24589 raft_consensus.cc:493] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:07.321324 24345 leader_election.cc:304] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:07.324505 24589 raft_consensus.cc:515] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:07.325107 24582 raft_consensus.cc:2749] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:07.321465 24609 consensus_queue.cc:1048] T 4af7fc369d124452beb5fd07cf8592ec P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:09:07.323920 24584 raft_consensus.cc:493] T 48aea62663ac40998d7f09389b93d586 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:07.331925 24584 raft_consensus.cc:3060] T 48aea62663ac40998d7f09389b93d586 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:07.332504 24575 ts_tablet_manager.cc:1434] T 48aea62663ac40998d7f09389b93d586 P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.119s	user 0.005s	sys 0.005s
I20260709 13:09:07.333463 24577 tablet_bootstrap.cc:654] T 42bcfb18b77e41afa60057ccfd718fa9 P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:07.333617 24575 tablet_bootstrap.cc:492] T 9711acbe7d044647bf774ee849c8a032 P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:07.326776 24589 leader_election.cc:290] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:07.346247 24369 tablet_service.cc:1511] Processing CreateTablet for tablet 5b8d15473a964a0c8b8c283c4552706e (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:09:07.346879 24382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42bcfb18b77e41afa60057ccfd718fa9" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:07.347579 24369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b8d15473a964a0c8b8c283c4552706e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:09:07.348371 24270 leader_election.cc:343] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:07.348551 24528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42bcfb18b77e41afa60057ccfd718fa9" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:07.349224 24528 raft_consensus.cc:2468] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 0.
I20260709 13:09:07.341135 24513 tablet_service.cc:1511] Processing CreateTablet for tablet 3cf2c3be9fd244d4b9d4703d424d3c5d (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:09:07.355420 24513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cf2c3be9fd244d4b9d4703d424d3c5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:07.352510 24297 tablet_service.cc:1511] Processing CreateTablet for tablet 86262701775a456dacd47fcdc1a5d13b (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:09:07.357182 24297 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86262701775a456dacd47fcdc1a5d13b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:07.365156 24584 raft_consensus.cc:515] T 48aea62663ac40998d7f09389b93d586 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:07.367347 24584 leader_election.cc:290] T 48aea62663ac40998d7f09389b93d586 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 election: Requested vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:07.368388 24382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48aea62663ac40998d7f09389b93d586" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:07.369055 24382 raft_consensus.cc:3060] T 48aea62663ac40998d7f09389b93d586 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:07.368795 24440 tablet_service.cc:1511] Processing CreateTablet for tablet 86262701775a456dacd47fcdc1a5d13b (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:09:07.368884 24518 tablet_service.cc:1511] Processing CreateTablet for tablet 5b8d15473a964a0c8b8c283c4552706e (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:09:07.362217 24270 leader_election.cc:304] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381; no voters: c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:07.370141 24440 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86262701775a456dacd47fcdc1a5d13b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:07.370440 24518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b8d15473a964a0c8b8c283c4552706e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:07.371112 24589 raft_consensus.cc:2804] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:07.371498 24589 raft_consensus.cc:493] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:07.372385 24589 raft_consensus.cc:3060] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:07.376878 24382 raft_consensus.cc:2468] T 48aea62663ac40998d7f09389b93d586 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 1.
I20260709 13:09:07.378150 24419 leader_election.cc:304] T 48aea62663ac40998d7f09389b93d586 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:07.386000 24583 raft_consensus.cc:2749] T db9e8fa567ab481c9b80ae8e30acd628 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:07.418140 24583 raft_consensus.cc:493] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:07.418757 24583 raft_consensus.cc:515] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:07.385524 24584 raft_consensus.cc:493] T 861acffd0d804c8fbd1a63771d7937c7 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:07.419914 24584 raft_consensus.cc:515] T 861acffd0d804c8fbd1a63771d7937c7 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:07.432760 24456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861acffd0d804c8fbd1a63771d7937c7" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:07.433507 24456 raft_consensus.cc:2468] T 861acffd0d804c8fbd1a63771d7937c7 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 0.
I20260709 13:09:07.434930 24493 leader_election.cc:304] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 838cb9997b0f437989d3f4e3e9904390; no voters: 
I20260709 13:09:07.435966 24630 raft_consensus.cc:2804] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:07.436335 24630 raft_consensus.cc:493] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:07.436638 24630 raft_consensus.cc:3060] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:07.444767 24642 raft_consensus.cc:2804] T 48aea62663ac40998d7f09389b93d586 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:07.445199 24583 leader_election.cc:290] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:07.445562 24642 raft_consensus.cc:697] T 48aea62663ac40998d7f09389b93d586 P 32088c942fbb48239e0eed952d4df449 [term 1 LEADER]: Becoming Leader. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Running, Role: LEADER
I20260709 13:09:07.446817 24642 consensus_queue.cc:237] T 48aea62663ac40998d7f09389b93d586 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:07.448529 24579 tablet_bootstrap.cc:492] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:07.449045 24579 ts_tablet_manager.cc:1403] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.242s	user 0.018s	sys 0.001s
I20260709 13:09:07.449844 24516 tablet_service.cc:1511] Processing CreateTablet for tablet 86262701775a456dacd47fcdc1a5d13b (DEFAULT_TABLE table=test-workload [id=295a88c3e4ee45f5a79be5512651e022]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:09:07.451285 24516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86262701775a456dacd47fcdc1a5d13b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:07.459939 24577 tablet_bootstrap.cc:492] T 42bcfb18b77e41afa60057ccfd718fa9 P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:07.460623 24577 ts_tablet_manager.cc:1403] T 42bcfb18b77e41afa60057ccfd718fa9 P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.216s	user 0.010s	sys 0.008s
I20260709 13:09:07.462106 24579 raft_consensus.cc:359] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:07.463042 24579 raft_consensus.cc:385] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:07.463361 24579 raft_consensus.cc:740] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:07.464112 24579 consensus_queue.cc:260] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:07.465384 24630 raft_consensus.cc:515] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:07.468338 24456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861acffd0d804c8fbd1a63771d7937c7" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:07.468899 24456 raft_consensus.cc:3060] T 861acffd0d804c8fbd1a63771d7937c7 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:07.454591 24576 tablet_bootstrap.cc:654] T e4dd51cda55249b48d7bfa39d5feb398 P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
W20260709 13:09:07.453423 24419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:46177, user_credentials={real_user=slave}} blocked reactor thread for 75365.9us
I20260709 13:09:07.478736 24577 raft_consensus.cc:359] T 42bcfb18b77e41afa60057ccfd718fa9 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:07.481314 24577 raft_consensus.cc:385] T 42bcfb18b77e41afa60057ccfd718fa9 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:07.481562 24577 raft_consensus.cc:740] T 42bcfb18b77e41afa60057ccfd718fa9 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:07.482273 24577 consensus_queue.cc:260] T 42bcfb18b77e41afa60057ccfd718fa9 P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:07.480338 24575 tablet_bootstrap.cc:654] T 9711acbe7d044647bf774ee849c8a032 P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:07.481837 24630 leader_election.cc:290] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Requested vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:07.487619 24306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861acffd0d804c8fbd1a63771d7937c7" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
W20260709 13:09:07.489347 24494 leader_election.cc:343] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:07.491178 24577 ts_tablet_manager.cc:1434] T 42bcfb18b77e41afa60057ccfd718fa9 P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.000s
I20260709 13:09:07.492306 24577 tablet_bootstrap.cc:492] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:07.493273 24527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48aea62663ac40998d7f09389b93d586" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:07.493943 24527 raft_consensus.cc:2468] T 48aea62663ac40998d7f09389b93d586 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 0.
I20260709 13:09:07.495729 24529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48aea62663ac40998d7f09389b93d586" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:07.496383 24529 raft_consensus.cc:3060] T 48aea62663ac40998d7f09389b93d586 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:07.475518 24589 raft_consensus.cc:515] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:07.497561 24456 raft_consensus.cc:2468] T 861acffd0d804c8fbd1a63771d7937c7 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 1.
I20260709 13:09:07.500463 24205 catalog_manager.cc:5697] T 48aea62663ac40998d7f09389b93d586 P 32088c942fbb48239e0eed952d4df449 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32088c942fbb48239e0eed952d4df449 (127.14.103.3). New cstate: current_term: 1 leader_uuid: "32088c942fbb48239e0eed952d4df449" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:07.503937 24577 tablet_bootstrap.cc:654] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:07.506124 24530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861acffd0d804c8fbd1a63771d7937c7" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:07.507262 24530 raft_consensus.cc:2393] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 32088c942fbb48239e0eed952d4df449 in current term 1: Already voted for candidate 838cb9997b0f437989d3f4e3e9904390 in this term.
I20260709 13:09:07.517482 24584 leader_election.cc:290] T 861acffd0d804c8fbd1a63771d7937c7 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:07.518944 24306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861acffd0d804c8fbd1a63771d7937c7" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:07.519985 24529 raft_consensus.cc:2468] T 48aea62663ac40998d7f09389b93d586 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 1.
I20260709 13:09:07.519402 24589 leader_election.cc:290] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Requested vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
W20260709 13:09:07.522065 24420 leader_election.cc:343] T 861acffd0d804c8fbd1a63771d7937c7 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:07.524463 24306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861acffd0d804c8fbd1a63771d7937c7" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:07.524506 24493 leader_election.cc:304] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 838cb9997b0f437989d3f4e3e9904390; no voters: 
I20260709 13:09:07.525674 24630 raft_consensus.cc:2804] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:07.526170 24630 raft_consensus.cc:697] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Becoming Leader. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Running, Role: LEADER
I20260709 13:09:07.526301 24381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42bcfb18b77e41afa60057ccfd718fa9" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:07.526983 24381 raft_consensus.cc:3060] T 42bcfb18b77e41afa60057ccfd718fa9 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:07.527738 24646 consensus_queue.cc:1048] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:07.527065 24630 consensus_queue.cc:237] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:07.520327 24579 ts_tablet_manager.cc:1434] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.071s	user 0.012s	sys 0.001s
I20260709 13:09:07.532864 24579 tablet_bootstrap.cc:492] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:07.530522 24419 leader_election.cc:304] T 861acffd0d804c8fbd1a63771d7937c7 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32088c942fbb48239e0eed952d4df449; no voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:07.536234 24584 raft_consensus.cc:2749] T 861acffd0d804c8fbd1a63771d7937c7 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:09:07.542970 24494 leader_election.cc:343] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:07.548623 24381 raft_consensus.cc:2468] T 42bcfb18b77e41afa60057ccfd718fa9 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 1.
I20260709 13:09:07.549175 24529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42bcfb18b77e41afa60057ccfd718fa9" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:07.561647 24270 leader_election.cc:304] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:07.561985 24529 raft_consensus.cc:3060] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:07.563513 24650 raft_consensus.cc:2804] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:07.564409 24650 raft_consensus.cc:697] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Becoming Leader. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Running, Role: LEADER
I20260709 13:09:07.565966 24650 consensus_queue.cc:237] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:07.548466 24579 tablet_bootstrap.cc:654] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:07.573369 24576 tablet_bootstrap.cc:492] T e4dd51cda55249b48d7bfa39d5feb398 P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:07.573967 24576 ts_tablet_manager.cc:1403] T e4dd51cda55249b48d7bfa39d5feb398 P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.260s	user 0.005s	sys 0.010s
I20260709 13:09:07.577239 24576 raft_consensus.cc:359] T e4dd51cda55249b48d7bfa39d5feb398 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:07.577992 24576 raft_consensus.cc:385] T e4dd51cda55249b48d7bfa39d5feb398 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:07.578291 24576 raft_consensus.cc:740] T e4dd51cda55249b48d7bfa39d5feb398 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:07.580844 24529 raft_consensus.cc:2468] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 1.
I20260709 13:09:07.580449 24576 consensus_queue.cc:260] T e4dd51cda55249b48d7bfa39d5feb398 P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:07.583164 24576 ts_tablet_manager.cc:1434] T e4dd51cda55249b48d7bfa39d5feb398 P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 13:09:07.584161 24576 tablet_bootstrap.cc:492] T 373185ca8ab04fdb91c7947ec0668808 P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:07.584184 24575 tablet_bootstrap.cc:492] T 9711acbe7d044647bf774ee849c8a032 P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:07.584863 24575 ts_tablet_manager.cc:1403] T 9711acbe7d044647bf774ee849c8a032 P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.251s	user 0.018s	sys 0.013s
I20260709 13:09:07.588071 24577 tablet_bootstrap.cc:492] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:07.588711 24577 ts_tablet_manager.cc:1403] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.097s	user 0.019s	sys 0.009s
I20260709 13:09:07.591514 24577 raft_consensus.cc:359] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:07.592217 24576 tablet_bootstrap.cc:654] T 373185ca8ab04fdb91c7947ec0668808 P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:07.592830 24577 raft_consensus.cc:385] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:07.593505 24577 raft_consensus.cc:740] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:07.594693 24577 consensus_queue.cc:260] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:07.598289 24577 ts_tablet_manager.cc:1434] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:09:07.587942 24575 raft_consensus.cc:359] T 9711acbe7d044647bf774ee849c8a032 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:07.600742 24575 raft_consensus.cc:385] T 9711acbe7d044647bf774ee849c8a032 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:07.601058 24575 raft_consensus.cc:740] T 9711acbe7d044647bf774ee849c8a032 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:07.601747 24577 tablet_bootstrap.cc:492] T 4f32a46373be4a29b6ac72efa35494b4 P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:07.601893 24575 consensus_queue.cc:260] T 9711acbe7d044647bf774ee849c8a032 P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:07.604638 24575 ts_tablet_manager.cc:1434] T 9711acbe7d044647bf774ee849c8a032 P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 13:09:07.605661 24575 tablet_bootstrap.cc:492] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:07.609874 24579 tablet_bootstrap.cc:492] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:07.610545 24579 ts_tablet_manager.cc:1403] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.078s	user 0.012s	sys 0.004s
I20260709 13:09:07.611021 24577 tablet_bootstrap.cc:654] T 4f32a46373be4a29b6ac72efa35494b4 P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:07.613824 24579 raft_consensus.cc:359] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:07.614596 24579 raft_consensus.cc:385] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:07.613672 24204 catalog_manager.cc:5697] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "9c2a14143eb744cf83bfa5f46284a381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:07.614952 24579 raft_consensus.cc:740] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:07.615883 24579 consensus_queue.cc:260] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:07.617646 24579 ts_tablet_manager.cc:1434] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 13:09:07.618661 24579 tablet_bootstrap.cc:492] T e4dd51cda55249b48d7bfa39d5feb398 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:07.628379 24603 consensus_queue.cc:1048] T 54fa63b278394f569a9ecffed4dec233 P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:07.642284 24579 tablet_bootstrap.cc:654] T e4dd51cda55249b48d7bfa39d5feb398 P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:07.640969 24201 catalog_manager.cc:5697] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 reported cstate change: term changed from 0 to 1, leader changed from <none> to 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4). New cstate: current_term: 1 leader_uuid: "838cb9997b0f437989d3f4e3e9904390" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:07.661412 24610 consensus_queue.cc:1048] T 54fa63b278394f569a9ecffed4dec233 P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:07.663319 24584 raft_consensus.cc:493] T e4dd51cda55249b48d7bfa39d5feb398 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:07.663918 24584 raft_consensus.cc:515] T e4dd51cda55249b48d7bfa39d5feb398 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:07.666008 24584 leader_election.cc:290] T e4dd51cda55249b48d7bfa39d5feb398 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:07.668408 24529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dd51cda55249b48d7bfa39d5feb398" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
W20260709 13:09:07.668967 24253 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:07.669062 24529 raft_consensus.cc:2468] T e4dd51cda55249b48d7bfa39d5feb398 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 0.
I20260709 13:09:07.669234 24307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dd51cda55249b48d7bfa39d5feb398" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:07.670514 24419 leader_election.cc:304] T e4dd51cda55249b48d7bfa39d5feb398 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 838cb9997b0f437989d3f4e3e9904390; no voters: 
I20260709 13:09:07.671344 24575 tablet_bootstrap.cc:654] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
W20260709 13:09:07.671487 24420 leader_election.cc:343] T e4dd51cda55249b48d7bfa39d5feb398 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:07.672497 24584 raft_consensus.cc:2804] T e4dd51cda55249b48d7bfa39d5feb398 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:07.672924 24584 raft_consensus.cc:493] T e4dd51cda55249b48d7bfa39d5feb398 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:07.673377 24584 raft_consensus.cc:3060] T e4dd51cda55249b48d7bfa39d5feb398 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:07.681356 24576 tablet_bootstrap.cc:492] T 373185ca8ab04fdb91c7947ec0668808 P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:07.681915 24576 ts_tablet_manager.cc:1403] T 373185ca8ab04fdb91c7947ec0668808 P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.098s	user 0.014s	sys 0.008s
I20260709 13:09:07.683202 24577 tablet_bootstrap.cc:492] T 4f32a46373be4a29b6ac72efa35494b4 P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:07.683658 24577 ts_tablet_manager.cc:1403] T 4f32a46373be4a29b6ac72efa35494b4 P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.082s	user 0.020s	sys 0.051s
I20260709 13:09:07.685034 24576 raft_consensus.cc:359] T 373185ca8ab04fdb91c7947ec0668808 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:07.685801 24576 raft_consensus.cc:385] T 373185ca8ab04fdb91c7947ec0668808 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:07.686090 24576 raft_consensus.cc:740] T 373185ca8ab04fdb91c7947ec0668808 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:07.686877 24576 consensus_queue.cc:260] T 373185ca8ab04fdb91c7947ec0668808 P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:07.689740 24576 ts_tablet_manager.cc:1434] T 373185ca8ab04fdb91c7947ec0668808 P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:09:07.691084 24576 tablet_bootstrap.cc:492] T 958ccd87b65945468e9247c40ca27c33 P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:07.692902 24579 tablet_bootstrap.cc:492] T e4dd51cda55249b48d7bfa39d5feb398 P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:07.693457 24579 ts_tablet_manager.cc:1403] T e4dd51cda55249b48d7bfa39d5feb398 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.075s	user 0.021s	sys 0.001s
I20260709 13:09:07.695061 24584 raft_consensus.cc:515] T e4dd51cda55249b48d7bfa39d5feb398 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:07.696260 24577 raft_consensus.cc:359] T 4f32a46373be4a29b6ac72efa35494b4 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:07.699996 24577 raft_consensus.cc:385] T 4f32a46373be4a29b6ac72efa35494b4 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:07.700314 24577 raft_consensus.cc:740] T 4f32a46373be4a29b6ac72efa35494b4 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:07.700994 24577 consensus_queue.cc:260] T 4f32a46373be4a29b6ac72efa35494b4 P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:07.702308 24529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dd51cda55249b48d7bfa39d5feb398" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:07.702899 24529 raft_consensus.cc:3060] T e4dd51cda55249b48d7bfa39d5feb398 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:07.703064 24577 ts_tablet_manager.cc:1434] T 4f32a46373be4a29b6ac72efa35494b4 P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.001s
I20260709 13:09:07.704146 24577 tablet_bootstrap.cc:492] T 1adea9b113494bed8c9d57fdbee6e22b P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:07.696605 24579 raft_consensus.cc:359] T e4dd51cda55249b48d7bfa39d5feb398 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:07.709057 24579 raft_consensus.cc:385] T e4dd51cda55249b48d7bfa39d5feb398 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:07.709381 24579 raft_consensus.cc:740] T e4dd51cda55249b48d7bfa39d5feb398 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:07.710129 24647 raft_consensus.cc:493] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:07.710134 24579 consensus_queue.cc:260] T e4dd51cda55249b48d7bfa39d5feb398 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:07.697923 24576 tablet_bootstrap.cc:654] T 958ccd87b65945468e9247c40ca27c33 P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:07.710762 24647 raft_consensus.cc:515] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:07.712831 24647 leader_election.cc:290] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:07.714289 24381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3b35ea0300f4b9e9afa80cf1dc9ba0f" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:07.714996 24381 raft_consensus.cc:2468] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 0.
I20260709 13:09:07.715580 24579 ts_tablet_manager.cc:1434] T e4dd51cda55249b48d7bfa39d5feb398 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.000s
I20260709 13:09:07.716917 24270 leader_election.cc:304] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:07.717672 24579 tablet_bootstrap.cc:492] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:07.717880 24647 raft_consensus.cc:2804] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:07.718290 24647 raft_consensus.cc:493] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:07.718618 24647 raft_consensus.cc:3060] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:07.722185 24577 tablet_bootstrap.cc:654] T 1adea9b113494bed8c9d57fdbee6e22b P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:07.722256 24456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3b35ea0300f4b9e9afa80cf1dc9ba0f" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
W20260709 13:09:07.724233 24270 leader_election.cc:343] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:07.726209 24647 raft_consensus.cc:515] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:07.727910 24576 tablet_bootstrap.cc:492] T 958ccd87b65945468e9247c40ca27c33 P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:07.728428 24576 ts_tablet_manager.cc:1403] T 958ccd87b65945468e9247c40ca27c33 P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.038s	user 0.007s	sys 0.008s
I20260709 13:09:07.729422 24381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3b35ea0300f4b9e9afa80cf1dc9ba0f" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:07.730001 24381 raft_consensus.cc:3060] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:07.731189 24576 raft_consensus.cc:359] T 958ccd87b65945468e9247c40ca27c33 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:07.731894 24576 raft_consensus.cc:385] T 958ccd87b65945468e9247c40ca27c33 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:07.732152 24576 raft_consensus.cc:740] T 958ccd87b65945468e9247c40ca27c33 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:07.732683 24579 tablet_bootstrap.cc:654] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:07.733033 24576 consensus_queue.cc:260] T 958ccd87b65945468e9247c40ca27c33 P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:07.736518 24576 ts_tablet_manager.cc:1434] T 958ccd87b65945468e9247c40ca27c33 P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:09:07.737468 24576 tablet_bootstrap.cc:492] T aac1b62e27a44fb5b9a23cfc42a5d00b P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:07.741124 24577 tablet_bootstrap.cc:492] T 1adea9b113494bed8c9d57fdbee6e22b P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:07.741612 24577 ts_tablet_manager.cc:1403] T 1adea9b113494bed8c9d57fdbee6e22b P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.038s	user 0.009s	sys 0.003s
I20260709 13:09:07.742347 24529 raft_consensus.cc:2468] T e4dd51cda55249b48d7bfa39d5feb398 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 1.
I20260709 13:09:07.744307 24307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dd51cda55249b48d7bfa39d5feb398" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:07.744503 24577 raft_consensus.cc:359] T 1adea9b113494bed8c9d57fdbee6e22b P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:07.745153 24577 raft_consensus.cc:385] T 1adea9b113494bed8c9d57fdbee6e22b P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:07.745199 24307 raft_consensus.cc:3060] T e4dd51cda55249b48d7bfa39d5feb398 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:07.745560 24577 raft_consensus.cc:740] T 1adea9b113494bed8c9d57fdbee6e22b P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:07.746812 24577 consensus_queue.cc:260] T 1adea9b113494bed8c9d57fdbee6e22b P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:07.750279 24577 ts_tablet_manager.cc:1434] T 1adea9b113494bed8c9d57fdbee6e22b P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:09:07.746779 24381 raft_consensus.cc:2468] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 1.
I20260709 13:09:07.751346 24576 tablet_bootstrap.cc:654] T aac1b62e27a44fb5b9a23cfc42a5d00b P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:07.752281 24270 leader_election.cc:304] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:07.753165 24650 raft_consensus.cc:2804] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:07.753625 24650 raft_consensus.cc:697] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Becoming Leader. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Running, Role: LEADER
I20260709 13:09:07.754463 24650 consensus_queue.cc:237] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:07.751205 24577 tablet_bootstrap.cc:492] T 4f085ebc2da04fbb87d48e4f206e61f0 P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:07.762816 24647 leader_election.cc:290] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Requested vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:07.764271 24456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3b35ea0300f4b9e9afa80cf1dc9ba0f" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
W20260709 13:09:07.766094 24270 leader_election.cc:343] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:07.766182 24419 leader_election.cc:304] T e4dd51cda55249b48d7bfa39d5feb398 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 838cb9997b0f437989d3f4e3e9904390; no voters: 
I20260709 13:09:07.767545 24642 raft_consensus.cc:2804] T e4dd51cda55249b48d7bfa39d5feb398 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:07.768188 24642 raft_consensus.cc:697] T e4dd51cda55249b48d7bfa39d5feb398 P 32088c942fbb48239e0eed952d4df449 [term 1 LEADER]: Becoming Leader. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Running, Role: LEADER
I20260709 13:09:07.768802 24642 consensus_queue.cc:237] T e4dd51cda55249b48d7bfa39d5feb398 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:07.770462 24579 tablet_bootstrap.cc:492] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:07.770957 24579 ts_tablet_manager.cc:1403] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.053s	user 0.025s	sys 0.003s
I20260709 13:09:07.774331 24584 leader_election.cc:290] T e4dd51cda55249b48d7bfa39d5feb398 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 election: Requested vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:07.774561 24579 raft_consensus.cc:359] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:07.776153 24579 raft_consensus.cc:385] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:07.776518 24579 raft_consensus.cc:740] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:07.777328 24579 consensus_queue.cc:260] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:07.778965 24575 tablet_bootstrap.cc:492] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:07.779557 24575 ts_tablet_manager.cc:1403] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.174s	user 0.019s	sys 0.017s
I20260709 13:09:07.780922 24576 tablet_bootstrap.cc:492] T aac1b62e27a44fb5b9a23cfc42a5d00b P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:07.781504 24307 raft_consensus.cc:2468] T e4dd51cda55249b48d7bfa39d5feb398 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 1.
I20260709 13:09:07.787077 24576 ts_tablet_manager.cc:1403] T aac1b62e27a44fb5b9a23cfc42a5d00b P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.050s	user 0.017s	sys 0.008s
I20260709 13:09:07.790012 24576 raft_consensus.cc:359] T aac1b62e27a44fb5b9a23cfc42a5d00b P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:07.790671 24576 raft_consensus.cc:385] T aac1b62e27a44fb5b9a23cfc42a5d00b P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:07.783254 24579 ts_tablet_manager.cc:1434] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.006s
I20260709 13:09:07.790971 24576 raft_consensus.cc:740] T aac1b62e27a44fb5b9a23cfc42a5d00b P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:07.791939 24579 tablet_bootstrap.cc:492] T 4af7fc369d124452beb5fd07cf8592ec P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:07.791265 24201 catalog_manager.cc:5697] T e4dd51cda55249b48d7bfa39d5feb398 P 32088c942fbb48239e0eed952d4df449 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32088c942fbb48239e0eed952d4df449 (127.14.103.3). New cstate: current_term: 1 leader_uuid: "32088c942fbb48239e0eed952d4df449" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:07.792748 24650 raft_consensus.cc:493] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:07.793219 24650 raft_consensus.cc:515] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:07.794489 24205 catalog_manager.cc:5697] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "9c2a14143eb744cf83bfa5f46284a381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:07.795994 24381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "373185ca8ab04fdb91c7947ec0668808" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:07.796622 24381 raft_consensus.cc:2468] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 0.
I20260709 13:09:07.797904 24270 leader_election.cc:304] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:07.782617 24575 raft_consensus.cc:359] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:07.798744 24647 raft_consensus.cc:2804] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:07.799201 24575 raft_consensus.cc:385] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:07.799245 24647 raft_consensus.cc:493] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:07.800486 24575 raft_consensus.cc:740] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:07.800659 24647 raft_consensus.cc:3060] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:07.791786 24576 consensus_queue.cc:260] T aac1b62e27a44fb5b9a23cfc42a5d00b P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:07.800187 24579 tablet_bootstrap.cc:654] T 4af7fc369d124452beb5fd07cf8592ec P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:07.801530 24575 consensus_queue.cc:260] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:07.802966 24456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "373185ca8ab04fdb91c7947ec0668808" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:07.803665 24456 raft_consensus.cc:2468] T 373185ca8ab04fdb91c7947ec0668808 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 0.
I20260709 13:09:07.806777 24576 ts_tablet_manager.cc:1434] T aac1b62e27a44fb5b9a23cfc42a5d00b P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 13:09:07.807197 24575 ts_tablet_manager.cc:1434] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.002s
I20260709 13:09:07.807782 24576 tablet_bootstrap.cc:492] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:07.808310 24575 tablet_bootstrap.cc:492] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:07.808765 24647 raft_consensus.cc:515] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:07.811743 24381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "373185ca8ab04fdb91c7947ec0668808" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:07.812311 24381 raft_consensus.cc:3060] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:07.815126 24575 tablet_bootstrap.cc:654] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:07.815793 24650 leader_election.cc:290] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:07.817966 24576 tablet_bootstrap.cc:654] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:07.818266 24603 raft_consensus.cc:493] T 4f32a46373be4a29b6ac72efa35494b4 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:07.820250 24603 raft_consensus.cc:515] T 4f32a46373be4a29b6ac72efa35494b4 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:07.822468 24603 leader_election.cc:290] T 4f32a46373be4a29b6ac72efa35494b4 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:07.824258 24307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f32a46373be4a29b6ac72efa35494b4" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:07.824713 24456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f32a46373be4a29b6ac72efa35494b4" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
W20260709 13:09:07.825817 24346 leader_election.cc:343] T 4f32a46373be4a29b6ac72efa35494b4 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:07.826432 24345 leader_election.cc:343] T 4f32a46373be4a29b6ac72efa35494b4 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:07.826869 24345 leader_election.cc:304] T 4f32a46373be4a29b6ac72efa35494b4 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:07.784265 24577 tablet_bootstrap.cc:654] T 4f085ebc2da04fbb87d48e4f206e61f0 P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:07.828197 24603 raft_consensus.cc:2749] T 4f32a46373be4a29b6ac72efa35494b4 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:07.832120 24575 tablet_bootstrap.cc:492] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:07.832737 24575 ts_tablet_manager.cc:1403] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.025s	user 0.009s	sys 0.008s
I20260709 13:09:07.834371 24579 tablet_bootstrap.cc:492] T 4af7fc369d124452beb5fd07cf8592ec P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:07.834852 24579 ts_tablet_manager.cc:1403] T 4af7fc369d124452beb5fd07cf8592ec P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.043s	user 0.009s	sys 0.012s
I20260709 13:09:07.836375 24647 leader_election.cc:290] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Requested vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:07.827502 24381 raft_consensus.cc:2468] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 1.
I20260709 13:09:07.837638 24579 raft_consensus.cc:359] T 4af7fc369d124452beb5fd07cf8592ec P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:07.838383 24270 leader_election.cc:304] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:07.838420 24579 raft_consensus.cc:385] T 4af7fc369d124452beb5fd07cf8592ec P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:07.839015 24579 raft_consensus.cc:740] T 4af7fc369d124452beb5fd07cf8592ec P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:07.838980 24456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "373185ca8ab04fdb91c7947ec0668808" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:07.839395 24647 raft_consensus.cc:2804] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:07.839865 24456 raft_consensus.cc:3060] T 373185ca8ab04fdb91c7947ec0668808 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:07.840165 24647 raft_consensus.cc:697] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Becoming Leader. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Running, Role: LEADER
I20260709 13:09:07.840009 24579 consensus_queue.cc:260] T 4af7fc369d124452beb5fd07cf8592ec P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:07.840581 24577 tablet_bootstrap.cc:492] T 4f085ebc2da04fbb87d48e4f206e61f0 P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:07.841079 24577 ts_tablet_manager.cc:1403] T 4f085ebc2da04fbb87d48e4f206e61f0 P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.090s	user 0.013s	sys 0.000s
I20260709 13:09:07.841234 24647 consensus_queue.cc:237] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:07.843453 24576 tablet_bootstrap.cc:492] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:07.841974 24575 raft_consensus.cc:359] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:07.843959 24575 raft_consensus.cc:385] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:07.844184 24577 raft_consensus.cc:359] T 4f085ebc2da04fbb87d48e4f206e61f0 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:07.846043 24577 raft_consensus.cc:385] T 4f085ebc2da04fbb87d48e4f206e61f0 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:07.846511 24577 raft_consensus.cc:740] T 4f085ebc2da04fbb87d48e4f206e61f0 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:07.847203 24577 consensus_queue.cc:260] T 4f085ebc2da04fbb87d48e4f206e61f0 P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:07.849316 24577 ts_tablet_manager.cc:1434] T 4f085ebc2da04fbb87d48e4f206e61f0 P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:09:07.844002 24576 ts_tablet_manager.cc:1403] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.036s	user 0.011s	sys 0.005s
I20260709 13:09:07.845993 24575 raft_consensus.cc:740] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:07.850404 24575 consensus_queue.cc:260] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:07.845597 24579 ts_tablet_manager.cc:1434] T 4af7fc369d124452beb5fd07cf8592ec P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:09:07.852088 24642 consensus_queue.cc:1048] T 48aea62663ac40998d7f09389b93d586 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:07.852933 24579 tablet_bootstrap.cc:492] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:07.853021 24575 ts_tablet_manager.cc:1434] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.020s	user 0.003s	sys 0.004s
I20260709 13:09:07.857187 24575 tablet_bootstrap.cc:492] T 958ccd87b65945468e9247c40ca27c33 P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:07.859419 24579 tablet_bootstrap.cc:654] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:07.860425 24577 tablet_bootstrap.cc:492] T ae6c1a13de5442c5bb4e58e7ce36e741 P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:07.863313 24575 tablet_bootstrap.cc:654] T 958ccd87b65945468e9247c40ca27c33 P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:07.865198 24642 raft_consensus.cc:493] T 958ccd87b65945468e9247c40ca27c33 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:07.865795 24642 raft_consensus.cc:515] T 958ccd87b65945468e9247c40ca27c33 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:07.866951 24577 tablet_bootstrap.cc:654] T ae6c1a13de5442c5bb4e58e7ce36e741 P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:07.869422 24307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958ccd87b65945468e9247c40ca27c33" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:07.870078 24307 raft_consensus.cc:2468] T 958ccd87b65945468e9247c40ca27c33 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 0.
I20260709 13:09:07.871322 24420 leader_election.cc:304] T 958ccd87b65945468e9247c40ca27c33 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381; no voters: 
I20260709 13:09:07.872233 24584 raft_consensus.cc:2804] T 958ccd87b65945468e9247c40ca27c33 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:07.872649 24584 raft_consensus.cc:493] T 958ccd87b65945468e9247c40ca27c33 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:07.873013 24584 raft_consensus.cc:3060] T 958ccd87b65945468e9247c40ca27c33 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:07.873438 24642 leader_election.cc:290] T 958ccd87b65945468e9247c40ca27c33 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:07.876080 24528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958ccd87b65945468e9247c40ca27c33" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
W20260709 13:09:07.877945 24419 leader_election.cc:343] T 958ccd87b65945468e9247c40ca27c33 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:07.880365 24575 tablet_bootstrap.cc:492] T 958ccd87b65945468e9247c40ca27c33 P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:07.881987 24575 ts_tablet_manager.cc:1403] T 958ccd87b65945468e9247c40ca27c33 P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.025s	user 0.007s	sys 0.006s
I20260709 13:09:07.852116 24576 raft_consensus.cc:359] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:07.883373 24576 raft_consensus.cc:385] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:07.883870 24576 raft_consensus.cc:740] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:07.884728 24576 consensus_queue.cc:260] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:07.859434 24456 raft_consensus.cc:2468] T 373185ca8ab04fdb91c7947ec0668808 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 1.
I20260709 13:09:07.887866 24575 raft_consensus.cc:359] T 958ccd87b65945468e9247c40ca27c33 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:07.890578 24575 raft_consensus.cc:385] T 958ccd87b65945468e9247c40ca27c33 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:07.891330 24575 raft_consensus.cc:740] T 958ccd87b65945468e9247c40ca27c33 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:07.888480 24584 raft_consensus.cc:515] T 958ccd87b65945468e9247c40ca27c33 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:07.894130 24584 leader_election.cc:290] T 958ccd87b65945468e9247c40ca27c33 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 election: Requested vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:07.896669 24307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958ccd87b65945468e9247c40ca27c33" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:07.894963 24575 consensus_queue.cc:260] T 958ccd87b65945468e9247c40ca27c33 P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:07.897542 24307 raft_consensus.cc:3060] T 958ccd87b65945468e9247c40ca27c33 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:07.899562 24575 ts_tablet_manager.cc:1434] T 958ccd87b65945468e9247c40ca27c33 P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:09:07.902894 24575 tablet_bootstrap.cc:492] T 4f085ebc2da04fbb87d48e4f206e61f0 P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:07.906340 24576 ts_tablet_manager.cc:1434] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.057s	user 0.006s	sys 0.000s
I20260709 13:09:07.915992 24575 tablet_bootstrap.cc:654] T 4f085ebc2da04fbb87d48e4f206e61f0 P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:07.917763 24529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "958ccd87b65945468e9247c40ca27c33" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:07.918639 24529 raft_consensus.cc:3060] T 958ccd87b65945468e9247c40ca27c33 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:07.919083 24642 consensus_queue.cc:1048] T 48aea62663ac40998d7f09389b93d586 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:07.921816 24576 tablet_bootstrap.cc:492] T 4f085ebc2da04fbb87d48e4f206e61f0 P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:07.933099 24307 raft_consensus.cc:2468] T 958ccd87b65945468e9247c40ca27c33 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 1.
I20260709 13:09:07.934737 24420 leader_election.cc:304] T 958ccd87b65945468e9247c40ca27c33 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381; no voters: 
I20260709 13:09:07.936180 24642 raft_consensus.cc:2804] T 958ccd87b65945468e9247c40ca27c33 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:07.936776 24642 raft_consensus.cc:697] T 958ccd87b65945468e9247c40ca27c33 P 32088c942fbb48239e0eed952d4df449 [term 1 LEADER]: Becoming Leader. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Running, Role: LEADER
I20260709 13:09:07.937930 24642 consensus_queue.cc:237] T 958ccd87b65945468e9247c40ca27c33 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:07.951777 24630 raft_consensus.cc:493] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:07.952546 24630 raft_consensus.cc:515] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:07.953289 24649 raft_consensus.cc:493] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:07.953814 24649 raft_consensus.cc:515] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:07.955843 24649 leader_election.cc:290] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:07.956940 24381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1adea9b113494bed8c9d57fdbee6e22b" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:07.957906 24381 raft_consensus.cc:2468] T 1adea9b113494bed8c9d57fdbee6e22b P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 0.
I20260709 13:09:07.958951 24603 raft_consensus.cc:493] T 4f085ebc2da04fbb87d48e4f206e61f0 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:07.959616 24603 raft_consensus.cc:515] T 4f085ebc2da04fbb87d48e4f206e61f0 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:07.962239 24603 leader_election.cc:290] T 4f085ebc2da04fbb87d48e4f206e61f0 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:07.963336 24647 raft_consensus.cc:493] T 4af7fc369d124452beb5fd07cf8592ec P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:07.963939 24647 raft_consensus.cc:515] T 4af7fc369d124452beb5fd07cf8592ec P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:07.966187 24647 leader_election.cc:290] T 4af7fc369d124452beb5fd07cf8592ec P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:07.967218 24204 catalog_manager.cc:5697] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "9c2a14143eb744cf83bfa5f46284a381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:07.967873 24306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1adea9b113494bed8c9d57fdbee6e22b" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:07.968312 24305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c60b42ac6d7e48bfae725dc113758659" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:07.969385 24630 leader_election.cc:290] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
W20260709 13:09:07.970635 24494 leader_election.cc:343] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:07.971388 24494 leader_election.cc:343] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:07.969528 24528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af7fc369d124452beb5fd07cf8592ec" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:07.976246 24644 raft_consensus.cc:493] T aac1b62e27a44fb5b9a23cfc42a5d00b P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:07.976915 24644 raft_consensus.cc:515] T aac1b62e27a44fb5b9a23cfc42a5d00b P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:07.980703 24644 leader_election.cc:290] T aac1b62e27a44fb5b9a23cfc42a5d00b P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:07.981459 24647 raft_consensus.cc:493] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:07.981690 24576 tablet_bootstrap.cc:654] T 4f085ebc2da04fbb87d48e4f206e61f0 P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:07.982087 24647 raft_consensus.cc:515] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:07.984311 24647 leader_election.cc:290] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:07.985090 24528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aac1b62e27a44fb5b9a23cfc42a5d00b" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:07.984969 24530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f085ebc2da04fbb87d48e4f206e61f0" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
W20260709 13:09:07.986922 24419 leader_election.cc:343] T aac1b62e27a44fb5b9a23cfc42a5d00b P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:07.988152 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f085ebc2da04fbb87d48e4f206e61f0" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:07.990223 24493 leader_election.cc:304] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:07.992781 24530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9711acbe7d044647bf774ee849c8a032" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:07.993432 24530 raft_consensus.cc:2468] T 9711acbe7d044647bf774ee849c8a032 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 0.
I20260709 13:09:07.995927 24270 leader_election.cc:304] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381; no voters: 
I20260709 13:09:07.996804 24647 raft_consensus.cc:2804] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:07.997220 24647 raft_consensus.cc:493] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:07.997591 24647 raft_consensus.cc:3060] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:07.997370 24381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af7fc369d124452beb5fd07cf8592ec" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
W20260709 13:09:08.000200 24345 leader_election.cc:343] T 4f085ebc2da04fbb87d48e4f206e61f0 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:08.003357 24345 leader_election.cc:343] T 4f085ebc2da04fbb87d48e4f206e61f0 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:08.004081 24649 raft_consensus.cc:2804] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:08.004627 24649 raft_consensus.cc:493] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:08.005066 24649 raft_consensus.cc:3060] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:08.004859 24345 leader_election.cc:304] T 4f085ebc2da04fbb87d48e4f206e61f0 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449, 838cb9997b0f437989d3f4e3e9904390
I20260709 13:09:08.006377 24630 consensus_queue.cc:1048] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:08.001174 24270 leader_election.cc:304] T 4af7fc369d124452beb5fd07cf8592ec P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381; no voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:07.981374 24382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aac1b62e27a44fb5b9a23cfc42a5d00b" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:08.010082 24382 raft_consensus.cc:2468] T aac1b62e27a44fb5b9a23cfc42a5d00b P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 0.
I20260709 13:09:08.011554 24419 leader_election.cc:304] T aac1b62e27a44fb5b9a23cfc42a5d00b P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, c40e7ba102224d24a22ccd1f68aaae45; no voters: 838cb9997b0f437989d3f4e3e9904390
I20260709 13:09:08.012066 24650 raft_consensus.cc:2749] T 4af7fc369d124452beb5fd07cf8592ec P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:08.012846 24644 raft_consensus.cc:2804] T aac1b62e27a44fb5b9a23cfc42a5d00b P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:08.013269 24644 raft_consensus.cc:493] T aac1b62e27a44fb5b9a23cfc42a5d00b P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:08.013600 24644 raft_consensus.cc:3060] T aac1b62e27a44fb5b9a23cfc42a5d00b P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:08.013913 24456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c60b42ac6d7e48bfae725dc113758659" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:08.014174 24649 raft_consensus.cc:515] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
W20260709 13:09:08.015870 24493 leader_election.cc:343] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:08.017402 24306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1adea9b113494bed8c9d57fdbee6e22b" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:08.016484 24493 leader_election.cc:304] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390; no voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381
W20260709 13:09:08.018949 24494 leader_election.cc:343] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:08.020300 24630 raft_consensus.cc:2749] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:08.021415 24647 raft_consensus.cc:515] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:08.023445 24603 raft_consensus.cc:2749] T 4f085ebc2da04fbb87d48e4f206e61f0 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:08.024526 24530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9711acbe7d044647bf774ee849c8a032" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:08.025254 24530 raft_consensus.cc:3060] T 9711acbe7d044647bf774ee849c8a032 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:08.027581 24382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9711acbe7d044647bf774ee849c8a032" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:08.029008 24382 raft_consensus.cc:2468] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 0.
I20260709 13:09:08.037374 24647 leader_election.cc:290] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Requested vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:08.037653 24644 raft_consensus.cc:515] T aac1b62e27a44fb5b9a23cfc42a5d00b P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:08.035305 24650 consensus_queue.cc:1048] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:08.041180 24530 raft_consensus.cc:2468] T 9711acbe7d044647bf774ee849c8a032 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 1.
I20260709 13:09:08.042852 24579 tablet_bootstrap.cc:492] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:08.043412 24579 ts_tablet_manager.cc:1403] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.191s	user 0.026s	sys 0.019s
I20260709 13:09:08.046852 24577 tablet_bootstrap.cc:492] T ae6c1a13de5442c5bb4e58e7ce36e741 P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:08.047401 24577 ts_tablet_manager.cc:1403] T ae6c1a13de5442c5bb4e58e7ce36e741 P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.187s	user 0.071s	sys 0.052s
I20260709 13:09:08.050199 24577 raft_consensus.cc:359] T ae6c1a13de5442c5bb4e58e7ce36e741 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:08.050884 24577 raft_consensus.cc:385] T ae6c1a13de5442c5bb4e58e7ce36e741 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.051149 24577 raft_consensus.cc:740] T ae6c1a13de5442c5bb4e58e7ce36e741 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.051797 24577 consensus_queue.cc:260] T ae6c1a13de5442c5bb4e58e7ce36e741 P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:08.053268 24579 raft_consensus.cc:359] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:08.053789 24577 ts_tablet_manager.cc:1434] T ae6c1a13de5442c5bb4e58e7ce36e741 P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:09:08.053954 24579 raft_consensus.cc:385] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.054224 24579 raft_consensus.cc:740] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.054880 24577 tablet_bootstrap.cc:492] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:08.054823 24579 consensus_queue.cc:260] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:08.062958 24579 ts_tablet_manager.cc:1434] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.001s
I20260709 13:09:08.063823 24579 tablet_bootstrap.cc:492] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:08.065871 24381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aac1b62e27a44fb5b9a23cfc42a5d00b" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:08.066505 24381 raft_consensus.cc:3060] T aac1b62e27a44fb5b9a23cfc42a5d00b P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:08.067798 24576 tablet_bootstrap.cc:492] T 4f085ebc2da04fbb87d48e4f206e61f0 P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:08.068274 24576 ts_tablet_manager.cc:1403] T 4f085ebc2da04fbb87d48e4f206e61f0 P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.147s	user 0.016s	sys 0.004s
I20260709 13:09:08.071122 24380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1adea9b113494bed8c9d57fdbee6e22b" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:08.071808 24380 raft_consensus.cc:3060] T 1adea9b113494bed8c9d57fdbee6e22b P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:08.072134 24530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aac1b62e27a44fb5b9a23cfc42a5d00b" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
W20260709 13:09:08.073609 24419 leader_election.cc:343] T aac1b62e27a44fb5b9a23cfc42a5d00b P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:08.073002 24201 catalog_manager.cc:5697] T 958ccd87b65945468e9247c40ca27c33 P 32088c942fbb48239e0eed952d4df449 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32088c942fbb48239e0eed952d4df449 (127.14.103.3). New cstate: current_term: 1 leader_uuid: "32088c942fbb48239e0eed952d4df449" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:08.074237 24529 raft_consensus.cc:2468] T 958ccd87b65945468e9247c40ca27c33 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 1.
I20260709 13:09:08.061419 24649 leader_election.cc:290] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Requested vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:08.074358 24644 leader_election.cc:290] T aac1b62e27a44fb5b9a23cfc42a5d00b P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 election: Requested vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
W20260709 13:09:08.092762 24494 consensus_peers.cc:597] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:08.102034 24577 tablet_bootstrap.cc:654] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.087281 24576 raft_consensus.cc:359] T 4f085ebc2da04fbb87d48e4f206e61f0 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:08.104444 24576 raft_consensus.cc:385] T 4f085ebc2da04fbb87d48e4f206e61f0 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.104843 24576 raft_consensus.cc:740] T 4f085ebc2da04fbb87d48e4f206e61f0 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.083081 24382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9711acbe7d044647bf774ee849c8a032" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:08.106545 24382 raft_consensus.cc:3060] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:08.109056 24575 tablet_bootstrap.cc:492] T 4f085ebc2da04fbb87d48e4f206e61f0 P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:08.109714 24575 ts_tablet_manager.cc:1403] T 4f085ebc2da04fbb87d48e4f206e61f0 P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.207s	user 0.030s	sys 0.045s
I20260709 13:09:08.111017 24579 tablet_bootstrap.cc:654] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.113354 24575 raft_consensus.cc:359] T 4f085ebc2da04fbb87d48e4f206e61f0 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:08.114701 24575 raft_consensus.cc:385] T 4f085ebc2da04fbb87d48e4f206e61f0 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.115131 24575 raft_consensus.cc:740] T 4f085ebc2da04fbb87d48e4f206e61f0 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.118332 24575 consensus_queue.cc:260] T 4f085ebc2da04fbb87d48e4f206e61f0 P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:08.117372 24576 consensus_queue.cc:260] T 4f085ebc2da04fbb87d48e4f206e61f0 P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:08.114769 24270 leader_election.cc:304] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381; no voters: 
I20260709 13:09:08.130298 24307 raft_consensus.cc:3060] T 4af7fc369d124452beb5fd07cf8592ec P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:08.132059 24650 raft_consensus.cc:2804] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:08.132477 24575 ts_tablet_manager.cc:1434] T 4f085ebc2da04fbb87d48e4f206e61f0 P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.022s	user 0.003s	sys 0.005s
I20260709 13:09:08.132632 24650 raft_consensus.cc:697] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Becoming Leader. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Running, Role: LEADER
I20260709 13:09:08.133503 24575 tablet_bootstrap.cc:492] T aac1b62e27a44fb5b9a23cfc42a5d00b P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:08.133603 24650 consensus_queue.cc:237] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:08.136632 24576 ts_tablet_manager.cc:1434] T 4f085ebc2da04fbb87d48e4f206e61f0 P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.068s	user 0.005s	sys 0.001s
W20260709 13:09:08.137782 24270 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:33251, user_credentials={real_user=slave}} blocked reactor thread for 51302.1us
I20260709 13:09:08.137787 24382 raft_consensus.cc:2468] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 1.
I20260709 13:09:08.140678 24577 tablet_bootstrap.cc:492] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:08.141151 24577 ts_tablet_manager.cc:1403] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.086s	user 0.012s	sys 0.003s
I20260709 13:09:08.141454 24645 raft_consensus.cc:493] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.142053 24576 tablet_bootstrap.cc:492] T c60b42ac6d7e48bfae725dc113758659 P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:08.141947 24645 raft_consensus.cc:515] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:08.142689 24647 consensus_queue.cc:1048] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:08.144099 24645 leader_election.cc:290] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:08.146193 24382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f32a46373be4a29b6ac72efa35494b4" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:08.146912 24382 raft_consensus.cc:2468] T 4f32a46373be4a29b6ac72efa35494b4 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 0.
I20260709 13:09:08.144526 24577 raft_consensus.cc:359] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:08.155642 24577 raft_consensus.cc:385] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.155697 24380 raft_consensus.cc:2468] T 1adea9b113494bed8c9d57fdbee6e22b P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 1.
I20260709 13:09:08.156569 24577 raft_consensus.cc:740] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.158188 24493 leader_election.cc:304] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:08.158428 24577 consensus_queue.cc:260] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:08.159654 24630 raft_consensus.cc:2804] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:08.160182 24630 raft_consensus.cc:697] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Becoming Leader. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Running, Role: LEADER
I20260709 13:09:08.161127 24577 ts_tablet_manager.cc:1434] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.020s	user 0.000s	sys 0.009s
I20260709 13:09:08.163095 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f32a46373be4a29b6ac72efa35494b4" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:08.164060 24270 leader_election.cc:304] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:08.165439 24576 tablet_bootstrap.cc:654] T c60b42ac6d7e48bfae725dc113758659 P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.153775 24575 tablet_bootstrap.cc:654] T aac1b62e27a44fb5b9a23cfc42a5d00b P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.164947 24577 tablet_bootstrap.cc:492] T 05840bb250764e02998cdddc290e5139 P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:08.172137 24381 raft_consensus.cc:2468] T aac1b62e27a44fb5b9a23cfc42a5d00b P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 1.
I20260709 13:09:08.161394 24630 consensus_queue.cc:237] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:08.181818 24419 leader_election.cc:304] T aac1b62e27a44fb5b9a23cfc42a5d00b P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, c40e7ba102224d24a22ccd1f68aaae45; no voters: 838cb9997b0f437989d3f4e3e9904390
I20260709 13:09:08.183570 24644 raft_consensus.cc:2804] T aac1b62e27a44fb5b9a23cfc42a5d00b P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:08.184273 24644 raft_consensus.cc:697] T aac1b62e27a44fb5b9a23cfc42a5d00b P 32088c942fbb48239e0eed952d4df449 [term 1 LEADER]: Becoming Leader. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Running, Role: LEADER
I20260709 13:09:08.185359 24644 consensus_queue.cc:237] T aac1b62e27a44fb5b9a23cfc42a5d00b P 32088c942fbb48239e0eed952d4df449 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:08.187300 24577 tablet_bootstrap.cc:654] T 05840bb250764e02998cdddc290e5139 P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
W20260709 13:09:08.191915 24270 leader_election.cc:343] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:08.193473 24647 consensus_queue.cc:1048] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:08.196895 24642 raft_consensus.cc:493] T 4f085ebc2da04fbb87d48e4f206e61f0 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.197651 24579 tablet_bootstrap.cc:492] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:08.198282 24579 ts_tablet_manager.cc:1403] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.135s	user 0.013s	sys 0.021s
I20260709 13:09:08.197587 24642 raft_consensus.cc:515] T 4f085ebc2da04fbb87d48e4f206e61f0 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:08.171989 24690 raft_consensus.cc:2804] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:08.201918 24642 leader_election.cc:290] T 4f085ebc2da04fbb87d48e4f206e61f0 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:08.201772 24579 raft_consensus.cc:359] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:08.202448 24579 raft_consensus.cc:385] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.202729 24579 raft_consensus.cc:740] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.203783 24530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f085ebc2da04fbb87d48e4f206e61f0" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:08.204540 24530 raft_consensus.cc:2468] T 4f085ebc2da04fbb87d48e4f206e61f0 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 0.
I20260709 13:09:08.205230 24579 consensus_queue.cc:260] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:08.206269 24690 raft_consensus.cc:493] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:08.207227 24690 raft_consensus.cc:3060] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:08.209264 24579 ts_tablet_manager.cc:1434] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 13:09:08.210271 24579 tablet_bootstrap.cc:492] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:08.213052 24647 raft_consensus.cc:493] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.213551 24647 raft_consensus.cc:515] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:08.216701 24204 catalog_manager.cc:5697] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "9c2a14143eb744cf83bfa5f46284a381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:08.220258 24647 leader_election.cc:290] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:08.220968 24419 leader_election.cc:304] T 4f085ebc2da04fbb87d48e4f206e61f0 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 838cb9997b0f437989d3f4e3e9904390; no voters: 
I20260709 13:09:08.221860 24642 raft_consensus.cc:2804] T 4f085ebc2da04fbb87d48e4f206e61f0 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:08.222232 24642 raft_consensus.cc:493] T 4f085ebc2da04fbb87d48e4f206e61f0 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:08.222543 24642 raft_consensus.cc:3060] T 4f085ebc2da04fbb87d48e4f206e61f0 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:08.224004 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c60b42ac6d7e48bfae725dc113758659" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:08.224462 24201 catalog_manager.cc:5697] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 reported cstate change: term changed from 0 to 1, leader changed from <none> to 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4). New cstate: current_term: 1 leader_uuid: "838cb9997b0f437989d3f4e3e9904390" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } }
W20260709 13:09:08.228080 24270 leader_election.cc:343] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:08.228940 24530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c60b42ac6d7e48bfae725dc113758659" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:08.229640 24530 raft_consensus.cc:2468] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 0.
I20260709 13:09:08.218385 24381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f085ebc2da04fbb87d48e4f206e61f0" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:08.231531 24270 leader_election.cc:304] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381; no voters: 32088c942fbb48239e0eed952d4df449
I20260709 13:09:08.232385 24381 raft_consensus.cc:2468] T 4f085ebc2da04fbb87d48e4f206e61f0 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 0.
I20260709 13:09:08.223961 24579 tablet_bootstrap.cc:654] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.232781 24647 raft_consensus.cc:2804] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:08.234819 24680 consensus_queue.cc:1048] T e4dd51cda55249b48d7bfa39d5feb398 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:08.238454 24647 raft_consensus.cc:493] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:08.275493 24647 raft_consensus.cc:3060] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:08.290256 24609 raft_consensus.cc:493] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.290864 24609 raft_consensus.cc:515] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:08.293102 24609 leader_election.cc:290] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:08.294051 24529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b7256c3dd84f5b952d1f9d47691f4e" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:08.294621 24306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b7256c3dd84f5b952d1f9d47691f4e" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
W20260709 13:09:08.295918 24345 leader_election.cc:343] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:08.296444 24346 leader_election.cc:343] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:08.297312 24346 leader_election.cc:304] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:08.299002 24609 raft_consensus.cc:2749] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:08.305876 24680 raft_consensus.cc:493] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.314479 24577 tablet_bootstrap.cc:492] T 05840bb250764e02998cdddc290e5139 P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:08.315062 24577 ts_tablet_manager.cc:1403] T 05840bb250764e02998cdddc290e5139 P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.153s	user 0.006s	sys 0.017s
I20260709 13:09:08.316367 24576 tablet_bootstrap.cc:492] T c60b42ac6d7e48bfae725dc113758659 P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:08.316847 24576 ts_tablet_manager.cc:1403] T c60b42ac6d7e48bfae725dc113758659 P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.178s	user 0.008s	sys 0.007s
I20260709 13:09:08.317178 24647 raft_consensus.cc:515] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:08.296607 24690 raft_consensus.cc:515] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:08.318399 24577 raft_consensus.cc:359] T 05840bb250764e02998cdddc290e5139 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:08.319247 24577 raft_consensus.cc:385] T 05840bb250764e02998cdddc290e5139 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.320271 24690 leader_election.cc:290] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Requested vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:08.320622 24576 raft_consensus.cc:359] T c60b42ac6d7e48bfae725dc113758659 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:08.320990 24381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f32a46373be4a29b6ac72efa35494b4" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:08.324222 24576 raft_consensus.cc:385] T c60b42ac6d7e48bfae725dc113758659 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.324590 24381 raft_consensus.cc:3060] T 4f32a46373be4a29b6ac72efa35494b4 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:08.324689 24576 raft_consensus.cc:740] T c60b42ac6d7e48bfae725dc113758659 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.325579 24576 consensus_queue.cc:260] T c60b42ac6d7e48bfae725dc113758659 P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:08.327746 24576 ts_tablet_manager.cc:1434] T c60b42ac6d7e48bfae725dc113758659 P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:09:08.322117 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f32a46373be4a29b6ac72efa35494b4" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:08.328285 24644 consensus_queue.cc:1048] T e4dd51cda55249b48d7bfa39d5feb398 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:08.330116 24680 raft_consensus.cc:515] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:08.334481 24530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c60b42ac6d7e48bfae725dc113758659" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:08.335106 24530 raft_consensus.cc:3060] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:08.335345 24680 leader_election.cc:290] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:08.332011 24576 tablet_bootstrap.cc:492] T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:08.323213 24642 raft_consensus.cc:515] T 4f085ebc2da04fbb87d48e4f206e61f0 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:08.319542 24577 raft_consensus.cc:740] T 05840bb250764e02998cdddc290e5139 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.329236 24647 leader_election.cc:290] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Requested vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:08.339299 24577 consensus_queue.cc:260] T 05840bb250764e02998cdddc290e5139 P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:08.341060 24575 tablet_bootstrap.cc:492] T aac1b62e27a44fb5b9a23cfc42a5d00b P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:08.341694 24575 ts_tablet_manager.cc:1403] T aac1b62e27a44fb5b9a23cfc42a5d00b P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.208s	user 0.018s	sys 0.005s
I20260709 13:09:08.352712 24577 ts_tablet_manager.cc:1434] T 05840bb250764e02998cdddc290e5139 P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.000s
I20260709 13:09:08.354470 24379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3b35ea0300f4b9e9afa80cf1dc9ba0f" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:08.355090 24380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f085ebc2da04fbb87d48e4f206e61f0" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:08.353839 24577 tablet_bootstrap.cc:492] T 9b7cd923257143648cfe6a296e18b642 P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
W20260709 13:09:08.342816 24270 leader_election.cc:343] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:08.350195 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c60b42ac6d7e48bfae725dc113758659" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:08.360536 24455 raft_consensus.cc:3060] T c60b42ac6d7e48bfae725dc113758659 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:08.360729 24528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f085ebc2da04fbb87d48e4f206e61f0" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:08.361299 24528 raft_consensus.cc:3060] T 4f085ebc2da04fbb87d48e4f206e61f0 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:08.362747 24306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3b35ea0300f4b9e9afa80cf1dc9ba0f" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:08.365379 24420 leader_election.cc:304] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32088c942fbb48239e0eed952d4df449; no voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:08.366714 24680 raft_consensus.cc:2749] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:08.367049 24609 raft_consensus.cc:493] T ae6c1a13de5442c5bb4e58e7ce36e741 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.344441 24201 catalog_manager.cc:5697] T aac1b62e27a44fb5b9a23cfc42a5d00b P 32088c942fbb48239e0eed952d4df449 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32088c942fbb48239e0eed952d4df449 (127.14.103.3). New cstate: current_term: 1 leader_uuid: "32088c942fbb48239e0eed952d4df449" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:08.356998 24380 raft_consensus.cc:3060] T 4f085ebc2da04fbb87d48e4f206e61f0 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:08.367645 24609 raft_consensus.cc:515] T ae6c1a13de5442c5bb4e58e7ce36e741 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:08.357329 24642 leader_election.cc:290] T 4f085ebc2da04fbb87d48e4f206e61f0 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 election: Requested vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:08.372380 24577 tablet_bootstrap.cc:654] T 9b7cd923257143648cfe6a296e18b642 P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.373564 24576 tablet_bootstrap.cc:654] T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.377404 24609 leader_election.cc:290] T ae6c1a13de5442c5bb4e58e7ce36e741 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:08.379745 24456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae6c1a13de5442c5bb4e58e7ce36e741" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:08.380151 24575 raft_consensus.cc:359] T aac1b62e27a44fb5b9a23cfc42a5d00b P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:08.380848 24575 raft_consensus.cc:385] T aac1b62e27a44fb5b9a23cfc42a5d00b P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.381099 24575 raft_consensus.cc:740] T aac1b62e27a44fb5b9a23cfc42a5d00b P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.381443 24527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae6c1a13de5442c5bb4e58e7ce36e741" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:08.381184 24579 tablet_bootstrap.cc:492] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:08.383020 24579 ts_tablet_manager.cc:1403] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.173s	user 0.006s	sys 0.017s
I20260709 13:09:08.382903 24575 consensus_queue.cc:260] T aac1b62e27a44fb5b9a23cfc42a5d00b P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
W20260709 13:09:08.385782 24345 leader_election.cc:343] T ae6c1a13de5442c5bb4e58e7ce36e741 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:08.389011 24575 ts_tablet_manager.cc:1434] T aac1b62e27a44fb5b9a23cfc42a5d00b P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.047s	user 0.007s	sys 0.000s
W20260709 13:09:08.386560 24345 leader_election.cc:343] T ae6c1a13de5442c5bb4e58e7ce36e741 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:08.398319 24575 tablet_bootstrap.cc:492] T 72d2665a42a94a82b1ceec249960fdf6 P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:08.400043 24579 raft_consensus.cc:359] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:08.400836 24579 raft_consensus.cc:385] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.401083 24579 raft_consensus.cc:740] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.398658 24345 leader_election.cc:304] T ae6c1a13de5442c5bb4e58e7ce36e741 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449, 838cb9997b0f437989d3f4e3e9904390
I20260709 13:09:08.405790 24609 raft_consensus.cc:2749] T ae6c1a13de5442c5bb4e58e7ce36e741 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:08.407323 24381 raft_consensus.cc:2468] T 4f32a46373be4a29b6ac72efa35494b4 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 1.
I20260709 13:09:08.405794 24380 raft_consensus.cc:2468] T 4f085ebc2da04fbb87d48e4f206e61f0 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 1.
I20260709 13:09:08.408725 24419 leader_election.cc:304] T 4f085ebc2da04fbb87d48e4f206e61f0 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:08.409714 24644 raft_consensus.cc:2804] T 4f085ebc2da04fbb87d48e4f206e61f0 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:08.410267 24644 raft_consensus.cc:697] T 4f085ebc2da04fbb87d48e4f206e61f0 P 32088c942fbb48239e0eed952d4df449 [term 1 LEADER]: Becoming Leader. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Running, Role: LEADER
I20260709 13:09:08.411113 24644 consensus_queue.cc:237] T 4f085ebc2da04fbb87d48e4f206e61f0 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:08.412259 24579 consensus_queue.cc:260] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:08.416802 24530 raft_consensus.cc:2468] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 1.
I20260709 13:09:08.421293 24455 raft_consensus.cc:2468] T c60b42ac6d7e48bfae725dc113758659 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 1.
I20260709 13:09:08.425767 24456 raft_consensus.cc:3060] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:08.426690 24575 tablet_bootstrap.cc:654] T 72d2665a42a94a82b1ceec249960fdf6 P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.430850 24270 leader_election.cc:304] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449
I20260709 13:09:08.431906 24650 raft_consensus.cc:2804] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:08.432420 24650 raft_consensus.cc:697] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Becoming Leader. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Running, Role: LEADER
I20260709 13:09:08.433233 24650 consensus_queue.cc:237] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:08.438194 24690 raft_consensus.cc:493] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.438762 24690 raft_consensus.cc:515] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:08.419603 24528 raft_consensus.cc:2468] T 4f085ebc2da04fbb87d48e4f206e61f0 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 1.
I20260709 13:09:08.472128 24630 raft_consensus.cc:493] T aac1b62e27a44fb5b9a23cfc42a5d00b P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.473022 24577 tablet_bootstrap.cc:492] T 9b7cd923257143648cfe6a296e18b642 P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:08.473307 24579 ts_tablet_manager.cc:1434] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.090s	user 0.012s	sys 0.004s
I20260709 13:09:08.473556 24577 ts_tablet_manager.cc:1403] T 9b7cd923257143648cfe6a296e18b642 P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.120s	user 0.016s	sys 0.012s
I20260709 13:09:08.472734 24630 raft_consensus.cc:515] T aac1b62e27a44fb5b9a23cfc42a5d00b P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:08.477545 24576 tablet_bootstrap.cc:492] T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:08.477707 24630 leader_election.cc:290] T aac1b62e27a44fb5b9a23cfc42a5d00b P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:08.478065 24576 ts_tablet_manager.cc:1403] T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.146s	user 0.014s	sys 0.001s
I20260709 13:09:08.479831 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aac1b62e27a44fb5b9a23cfc42a5d00b" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:08.480850 24576 raft_consensus.cc:359] T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:08.480970 24696 consensus_queue.cc:1048] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:08.482404 24576 raft_consensus.cc:385] T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.482745 24576 raft_consensus.cc:740] T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.483780 24579 tablet_bootstrap.cc:492] T 861acffd0d804c8fbd1a63771d7937c7 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:08.483448 24576 consensus_queue.cc:260] T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:08.485769 24576 ts_tablet_manager.cc:1434] T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:09:08.486680 24576 tablet_bootstrap.cc:492] T 93b6310306174b4ca68b467ea7f5d23e P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:08.487723 24584 consensus_queue.cc:1048] T 958ccd87b65945468e9247c40ca27c33 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:08.489151 24690 leader_election.cc:290] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:08.490998 24579 tablet_bootstrap.cc:654] T 861acffd0d804c8fbd1a63771d7937c7 P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.497489 24577 raft_consensus.cc:359] T 9b7cd923257143648cfe6a296e18b642 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:08.502213 24577 raft_consensus.cc:385] T 9b7cd923257143648cfe6a296e18b642 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.502979 24577 raft_consensus.cc:740] T 9b7cd923257143648cfe6a296e18b642 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.504086 24577 consensus_queue.cc:260] T 9b7cd923257143648cfe6a296e18b642 P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:08.508973 24579 tablet_bootstrap.cc:492] T 861acffd0d804c8fbd1a63771d7937c7 P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:08.512761 24579 ts_tablet_manager.cc:1403] T 861acffd0d804c8fbd1a63771d7937c7 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.039s	user 0.014s	sys 0.001s
I20260709 13:09:08.512898 24575 tablet_bootstrap.cc:492] T 72d2665a42a94a82b1ceec249960fdf6 P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:08.513561 24575 ts_tablet_manager.cc:1403] T 72d2665a42a94a82b1ceec249960fdf6 P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.117s	user 0.011s	sys 0.003s
I20260709 13:09:08.515774 24579 raft_consensus.cc:359] T 861acffd0d804c8fbd1a63771d7937c7 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:08.517954 24577 ts_tablet_manager.cc:1434] T 9b7cd923257143648cfe6a296e18b642 P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.044s	user 0.008s	sys 0.000s
I20260709 13:09:08.516577 24579 raft_consensus.cc:385] T 861acffd0d804c8fbd1a63771d7937c7 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.518796 24579 raft_consensus.cc:740] T 861acffd0d804c8fbd1a63771d7937c7 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.519515 24577 tablet_bootstrap.cc:492] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:08.519711 24579 consensus_queue.cc:260] T 861acffd0d804c8fbd1a63771d7937c7 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:08.516408 24575 raft_consensus.cc:359] T 72d2665a42a94a82b1ceec249960fdf6 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:08.520714 24575 raft_consensus.cc:385] T 72d2665a42a94a82b1ceec249960fdf6 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.521096 24575 raft_consensus.cc:740] T 72d2665a42a94a82b1ceec249960fdf6 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.521962 24381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aac1b62e27a44fb5b9a23cfc42a5d00b" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:08.523135 24381 raft_consensus.cc:2393] T aac1b62e27a44fb5b9a23cfc42a5d00b P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 838cb9997b0f437989d3f4e3e9904390 in current term 1: Already voted for candidate 32088c942fbb48239e0eed952d4df449 in this term.
I20260709 13:09:08.522084 24575 consensus_queue.cc:260] T 72d2665a42a94a82b1ceec249960fdf6 P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:08.521973 24201 catalog_manager.cc:5697] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "9c2a14143eb744cf83bfa5f46284a381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:08.527153 24575 ts_tablet_manager.cc:1434] T 72d2665a42a94a82b1ceec249960fdf6 P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:09:08.530668 24577 tablet_bootstrap.cc:654] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.532699 24493 leader_election.cc:304] T aac1b62e27a44fb5b9a23cfc42a5d00b P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390; no voters: 32088c942fbb48239e0eed952d4df449, c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:08.533519 24630 raft_consensus.cc:3060] T aac1b62e27a44fb5b9a23cfc42a5d00b P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:08.482225 24270 leader_election.cc:304] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381; no voters: 
W20260709 13:09:08.534633 24270 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:33251, user_credentials={real_user=slave}} blocked reactor thread for 52506.1us
I20260709 13:09:08.534736 24650 raft_consensus.cc:2804] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:08.535312 24650 raft_consensus.cc:697] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Becoming Leader. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Running, Role: LEADER
I20260709 13:09:08.536147 24650 consensus_queue.cc:237] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:08.537978 24690 consensus_queue.cc:1048] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:08.539357 24645 consensus_queue.cc:1048] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:08.539588 24528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1adea9b113494bed8c9d57fdbee6e22b" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:08.544566 24575 tablet_bootstrap.cc:492] T 53b7256c3dd84f5b952d1f9d47691f4e P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:08.547178 24380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1adea9b113494bed8c9d57fdbee6e22b" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:08.548079 24380 raft_consensus.cc:2393] T 1adea9b113494bed8c9d57fdbee6e22b P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9c2a14143eb744cf83bfa5f46284a381 in current term 1: Already voted for candidate 838cb9997b0f437989d3f4e3e9904390 in this term.
I20260709 13:09:08.548357 24204 catalog_manager.cc:5697] T 4f085ebc2da04fbb87d48e4f206e61f0 P 32088c942fbb48239e0eed952d4df449 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32088c942fbb48239e0eed952d4df449 (127.14.103.3). New cstate: current_term: 1 leader_uuid: "32088c942fbb48239e0eed952d4df449" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:08.549525 24577 tablet_bootstrap.cc:492] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:08.550004 24577 ts_tablet_manager.cc:1403] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.031s	user 0.012s	sys 0.004s
I20260709 13:09:08.550794 24575 tablet_bootstrap.cc:654] T 53b7256c3dd84f5b952d1f9d47691f4e P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.553189 24577 raft_consensus.cc:359] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:08.555123 24577 raft_consensus.cc:385] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.555408 24577 raft_consensus.cc:740] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.556265 24577 consensus_queue.cc:260] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:08.558941 24680 consensus_queue.cc:1048] T 958ccd87b65945468e9247c40ca27c33 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:08.560402 24577 ts_tablet_manager.cc:1434] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 13:09:08.561434 24577 tablet_bootstrap.cc:492] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:08.564540 24630 raft_consensus.cc:2749] T aac1b62e27a44fb5b9a23cfc42a5d00b P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:08.522439 24579 ts_tablet_manager.cc:1434] T 861acffd0d804c8fbd1a63771d7937c7 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:09:08.567939 24579 tablet_bootstrap.cc:492] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:08.571201 24576 tablet_bootstrap.cc:654] T 93b6310306174b4ca68b467ea7f5d23e P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.576891 24577 tablet_bootstrap.cc:654] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.572494 24270 leader_election.cc:304] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381; no voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:08.577783 24690 raft_consensus.cc:3060] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:08.579231 24306 raft_consensus.cc:3060] T 861acffd0d804c8fbd1a63771d7937c7 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:08.579452 24696 consensus_queue.cc:1048] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:08.598906 24690 raft_consensus.cc:2749] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:08.616618 24649 consensus_queue.cc:1048] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:08.633301 24610 raft_consensus.cc:493] T 05840bb250764e02998cdddc290e5139 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.633898 24610 raft_consensus.cc:515] T 05840bb250764e02998cdddc290e5139 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:08.636360 24610 leader_election.cc:290] T 05840bb250764e02998cdddc290e5139 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:08.639501 24456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05840bb250764e02998cdddc290e5139" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:08.642555 24530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05840bb250764e02998cdddc290e5139" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:08.643736 24579 tablet_bootstrap.cc:654] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
W20260709 13:09:08.645823 24345 leader_election.cc:343] T 05840bb250764e02998cdddc290e5139 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:08.650547 24642 consensus_queue.cc:1048] T aac1b62e27a44fb5b9a23cfc42a5d00b P 32088c942fbb48239e0eed952d4df449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:09:08.652359 24345 leader_election.cc:343] T 05840bb250764e02998cdddc290e5139 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:08.652784 24345 leader_election.cc:304] T 05840bb250764e02998cdddc290e5139 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449, 838cb9997b0f437989d3f4e3e9904390
I20260709 13:09:08.653894 24610 raft_consensus.cc:2749] T 05840bb250764e02998cdddc290e5139 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:08.656183 24575 tablet_bootstrap.cc:492] T 53b7256c3dd84f5b952d1f9d47691f4e P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:08.656693 24575 ts_tablet_manager.cc:1403] T 53b7256c3dd84f5b952d1f9d47691f4e P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.113s	user 0.024s	sys 0.011s
I20260709 13:09:08.660619 24575 raft_consensus.cc:359] T 53b7256c3dd84f5b952d1f9d47691f4e P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:08.661345 24575 raft_consensus.cc:385] T 53b7256c3dd84f5b952d1f9d47691f4e P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.661617 24575 raft_consensus.cc:740] T 53b7256c3dd84f5b952d1f9d47691f4e P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.667106 24575 consensus_queue.cc:260] T 53b7256c3dd84f5b952d1f9d47691f4e P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:08.658564 24650 consensus_queue.cc:1048] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:09:08.673326 24575 ts_tablet_manager.cc:1434] T 53b7256c3dd84f5b952d1f9d47691f4e P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:09:08.674744 24575 tablet_bootstrap.cc:492] T 5eaa917e65d9403dbb241f84ee96ba9d P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:08.676250 24577 tablet_bootstrap.cc:492] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:08.676771 24577 ts_tablet_manager.cc:1403] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.116s	user 0.011s	sys 0.023s
I20260709 13:09:08.677171 24579 tablet_bootstrap.cc:492] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:08.677733 24579 ts_tablet_manager.cc:1403] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.110s	user 0.011s	sys 0.007s
I20260709 13:09:08.692451 24575 tablet_bootstrap.cc:654] T 5eaa917e65d9403dbb241f84ee96ba9d P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.680744 24649 consensus_queue.cc:1048] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 13:09:08.697711 24579 raft_consensus.cc:359] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:08.698485 24579 raft_consensus.cc:385] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.699302 24579 raft_consensus.cc:740] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.700039 24579 consensus_queue.cc:260] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:08.708000 24577 raft_consensus.cc:359] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:08.708915 24584 consensus_queue.cc:1048] T aac1b62e27a44fb5b9a23cfc42a5d00b P 32088c942fbb48239e0eed952d4df449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:08.717530 24579 ts_tablet_manager.cc:1434] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.000s
I20260709 13:09:08.718573 24579 tablet_bootstrap.cc:492] T 93b6310306174b4ca68b467ea7f5d23e P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:08.719352 24577 raft_consensus.cc:385] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.720544 24577 raft_consensus.cc:740] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.721956 24577 consensus_queue.cc:260] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:08.722910 24576 tablet_bootstrap.cc:492] T 93b6310306174b4ca68b467ea7f5d23e P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:08.722199 24201 catalog_manager.cc:5697] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "9c2a14143eb744cf83bfa5f46284a381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:08.723850 24576 ts_tablet_manager.cc:1403] T 93b6310306174b4ca68b467ea7f5d23e P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.237s	user 0.004s	sys 0.041s
I20260709 13:09:08.726660 24576 raft_consensus.cc:359] T 93b6310306174b4ca68b467ea7f5d23e P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:08.727463 24576 raft_consensus.cc:385] T 93b6310306174b4ca68b467ea7f5d23e P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:09:08.727850 24253 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:08.734302 24579 tablet_bootstrap.cc:654] T 93b6310306174b4ca68b467ea7f5d23e P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.738575 24575 tablet_bootstrap.cc:492] T 5eaa917e65d9403dbb241f84ee96ba9d P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:08.739105 24575 ts_tablet_manager.cc:1403] T 5eaa917e65d9403dbb241f84ee96ba9d P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.065s	user 0.012s	sys 0.005s
I20260709 13:09:08.741740 24575 raft_consensus.cc:359] T 5eaa917e65d9403dbb241f84ee96ba9d P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:08.742372 24575 raft_consensus.cc:385] T 5eaa917e65d9403dbb241f84ee96ba9d P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.742638 24575 raft_consensus.cc:740] T 5eaa917e65d9403dbb241f84ee96ba9d P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.727969 24576 raft_consensus.cc:740] T 93b6310306174b4ca68b467ea7f5d23e P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.743448 24575 consensus_queue.cc:260] T 5eaa917e65d9403dbb241f84ee96ba9d P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:08.743863 24576 consensus_queue.cc:260] T 93b6310306174b4ca68b467ea7f5d23e P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:08.746202 24576 ts_tablet_manager.cc:1434] T 93b6310306174b4ca68b467ea7f5d23e P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.000s
I20260709 13:09:08.747079 24576 tablet_bootstrap.cc:492] T 9b7cd923257143648cfe6a296e18b642 P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:08.749954 24610 raft_consensus.cc:493] T 9b7cd923257143648cfe6a296e18b642 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.750514 24610 raft_consensus.cc:515] T 9b7cd923257143648cfe6a296e18b642 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:08.752779 24610 leader_election.cc:290] T 9b7cd923257143648cfe6a296e18b642 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:08.753252 24577 ts_tablet_manager.cc:1434] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.076s	user 0.007s	sys 0.000s
I20260709 13:09:08.754243 24577 tablet_bootstrap.cc:492] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:08.755057 24306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b7cd923257143648cfe6a296e18b642" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:08.755522 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b7cd923257143648cfe6a296e18b642" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
W20260709 13:09:08.756955 24345 leader_election.cc:343] T 9b7cd923257143648cfe6a296e18b642 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:08.757592 24346 leader_election.cc:343] T 9b7cd923257143648cfe6a296e18b642 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:08.758040 24346 leader_election.cc:304] T 9b7cd923257143648cfe6a296e18b642 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:08.759405 24610 raft_consensus.cc:2749] T 9b7cd923257143648cfe6a296e18b642 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:08.763981 24575 ts_tablet_manager.cc:1434] T 5eaa917e65d9403dbb241f84ee96ba9d P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 13:09:08.765113 24575 tablet_bootstrap.cc:492] T 05840bb250764e02998cdddc290e5139 P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:08.772032 24576 tablet_bootstrap.cc:654] T 9b7cd923257143648cfe6a296e18b642 P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.775044 24577 tablet_bootstrap.cc:654] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.780431 24575 tablet_bootstrap.cc:654] T 05840bb250764e02998cdddc290e5139 P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.780479 24579 tablet_bootstrap.cc:492] T 93b6310306174b4ca68b467ea7f5d23e P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:08.781579 24579 ts_tablet_manager.cc:1403] T 93b6310306174b4ca68b467ea7f5d23e P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.063s	user 0.022s	sys 0.024s
I20260709 13:09:08.784497 24579 raft_consensus.cc:359] T 93b6310306174b4ca68b467ea7f5d23e P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:08.785151 24579 raft_consensus.cc:385] T 93b6310306174b4ca68b467ea7f5d23e P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.785434 24579 raft_consensus.cc:740] T 93b6310306174b4ca68b467ea7f5d23e P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.786186 24579 consensus_queue.cc:260] T 93b6310306174b4ca68b467ea7f5d23e P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:08.788249 24576 tablet_bootstrap.cc:492] T 9b7cd923257143648cfe6a296e18b642 P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:08.788491 24579 ts_tablet_manager.cc:1434] T 93b6310306174b4ca68b467ea7f5d23e P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:09:08.788923 24576 ts_tablet_manager.cc:1403] T 9b7cd923257143648cfe6a296e18b642 P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.042s	user 0.014s	sys 0.000s
I20260709 13:09:08.789557 24579 tablet_bootstrap.cc:492] T b147866f4e5f4596b0b3989418cc4b0a P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:08.792101 24577 tablet_bootstrap.cc:492] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:08.792793 24577 ts_tablet_manager.cc:1403] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.039s	user 0.006s	sys 0.008s
I20260709 13:09:08.796931 24579 tablet_bootstrap.cc:654] T b147866f4e5f4596b0b3989418cc4b0a P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.798480 24576 raft_consensus.cc:359] T 9b7cd923257143648cfe6a296e18b642 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:08.799400 24576 raft_consensus.cc:385] T 9b7cd923257143648cfe6a296e18b642 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.799748 24576 raft_consensus.cc:740] T 9b7cd923257143648cfe6a296e18b642 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.804684 24577 raft_consensus.cc:359] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:08.805491 24577 raft_consensus.cc:385] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.805171 24584 consensus_queue.cc:1048] T 4f085ebc2da04fbb87d48e4f206e61f0 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:08.805888 24577 raft_consensus.cc:740] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.806308 24575 tablet_bootstrap.cc:492] T 05840bb250764e02998cdddc290e5139 P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:08.806813 24575 ts_tablet_manager.cc:1403] T 05840bb250764e02998cdddc290e5139 P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.042s	user 0.018s	sys 0.003s
I20260709 13:09:08.806710 24577 consensus_queue.cc:260] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:08.808269 24576 consensus_queue.cc:260] T 9b7cd923257143648cfe6a296e18b642 P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:08.810602 24576 ts_tablet_manager.cc:1434] T 9b7cd923257143648cfe6a296e18b642 P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.003s
I20260709 13:09:08.811347 24575 raft_consensus.cc:359] T 05840bb250764e02998cdddc290e5139 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:08.812063 24575 raft_consensus.cc:385] T 05840bb250764e02998cdddc290e5139 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.812327 24575 raft_consensus.cc:740] T 05840bb250764e02998cdddc290e5139 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.816743 24575 consensus_queue.cc:260] T 05840bb250764e02998cdddc290e5139 P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:08.820781 24576 tablet_bootstrap.cc:492] T 72d2665a42a94a82b1ceec249960fdf6 P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:08.821548 24577 ts_tablet_manager.cc:1434] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.002s
I20260709 13:09:08.828951 24577 tablet_bootstrap.cc:492] T 2039124a6d8848149440a70100af8755 P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:08.832969 24575 ts_tablet_manager.cc:1434] T 05840bb250764e02998cdddc290e5139 P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 13:09:08.833997 24575 tablet_bootstrap.cc:492] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:08.834299 24579 tablet_bootstrap.cc:492] T b147866f4e5f4596b0b3989418cc4b0a P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:08.834868 24579 ts_tablet_manager.cc:1403] T b147866f4e5f4596b0b3989418cc4b0a P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.046s	user 0.025s	sys 0.016s
I20260709 13:09:08.835245 24576 tablet_bootstrap.cc:654] T 72d2665a42a94a82b1ceec249960fdf6 P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.839470 24575 tablet_bootstrap.cc:654] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.840528 24577 tablet_bootstrap.cc:654] T 2039124a6d8848149440a70100af8755 P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.843943 24630 raft_consensus.cc:493] T 72d2665a42a94a82b1ceec249960fdf6 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.844451 24630 raft_consensus.cc:515] T 72d2665a42a94a82b1ceec249960fdf6 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:08.844916 24579 raft_consensus.cc:359] T b147866f4e5f4596b0b3989418cc4b0a P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:08.845548 24579 raft_consensus.cc:385] T b147866f4e5f4596b0b3989418cc4b0a P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.845793 24579 raft_consensus.cc:740] T b147866f4e5f4596b0b3989418cc4b0a P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.846465 24579 consensus_queue.cc:260] T b147866f4e5f4596b0b3989418cc4b0a P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:08.847898 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d2665a42a94a82b1ceec249960fdf6" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:08.848570 24579 ts_tablet_manager.cc:1434] T b147866f4e5f4596b0b3989418cc4b0a P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:09:08.849534 24579 tablet_bootstrap.cc:492] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:08.850714 24630 leader_election.cc:290] T 72d2665a42a94a82b1ceec249960fdf6 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:08.856398 24579 tablet_bootstrap.cc:654] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
W20260709 13:09:08.865096 24493 leader_election.cc:343] T 72d2665a42a94a82b1ceec249960fdf6 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:08.865283 24644 consensus_queue.cc:1048] T 4f085ebc2da04fbb87d48e4f206e61f0 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:08.866274 24575 tablet_bootstrap.cc:492] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:08.866786 24575 ts_tablet_manager.cc:1403] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.033s	user 0.012s	sys 0.004s
I20260709 13:09:08.869365 24576 tablet_bootstrap.cc:492] T 72d2665a42a94a82b1ceec249960fdf6 P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:08.869421 24577 tablet_bootstrap.cc:492] T 2039124a6d8848149440a70100af8755 P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:08.869944 24576 ts_tablet_manager.cc:1403] T 72d2665a42a94a82b1ceec249960fdf6 P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.049s	user 0.007s	sys 0.003s
I20260709 13:09:08.869628 24575 raft_consensus.cc:359] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:08.870213 24577 ts_tablet_manager.cc:1403] T 2039124a6d8848149440a70100af8755 P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.041s	user 0.005s	sys 0.004s
I20260709 13:09:08.870411 24575 raft_consensus.cc:385] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.870786 24575 raft_consensus.cc:740] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.871543 24575 consensus_queue.cc:260] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:08.873876 24575 ts_tablet_manager.cc:1434] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:09:08.873764 24577 raft_consensus.cc:359] T 2039124a6d8848149440a70100af8755 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:08.874800 24577 raft_consensus.cc:385] T 2039124a6d8848149440a70100af8755 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.875082 24575 tablet_bootstrap.cc:492] T ae6c1a13de5442c5bb4e58e7ce36e741 P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:08.875165 24577 raft_consensus.cc:740] T 2039124a6d8848149440a70100af8755 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.876906 24577 consensus_queue.cc:260] T 2039124a6d8848149440a70100af8755 P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:08.880204 24577 ts_tablet_manager.cc:1434] T 2039124a6d8848149440a70100af8755 P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.010s	user 0.010s	sys 0.000s
I20260709 13:09:08.881577 24577 tablet_bootstrap.cc:492] T 90d9e64f97544895b9b97dfdc5ceb359 P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:08.885142 24575 tablet_bootstrap.cc:654] T ae6c1a13de5442c5bb4e58e7ce36e741 P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.892459 24577 tablet_bootstrap.cc:654] T 90d9e64f97544895b9b97dfdc5ceb359 P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.892030 24576 raft_consensus.cc:359] T 72d2665a42a94a82b1ceec249960fdf6 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:08.893052 24576 raft_consensus.cc:385] T 72d2665a42a94a82b1ceec249960fdf6 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.893425 24576 raft_consensus.cc:740] T 72d2665a42a94a82b1ceec249960fdf6 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.894258 24576 consensus_queue.cc:260] T 72d2665a42a94a82b1ceec249960fdf6 P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:08.938452 24306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d2665a42a94a82b1ceec249960fdf6" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:08.953032 24642 raft_consensus.cc:493] T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.954167 24642 raft_consensus.cc:515] T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:08.956709 24579 tablet_bootstrap.cc:492] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:08.957999 24579 ts_tablet_manager.cc:1403] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.109s	user 0.018s	sys 0.029s
I20260709 13:09:08.958828 24645 raft_consensus.cc:493] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.959124 24576 ts_tablet_manager.cc:1434] T 72d2665a42a94a82b1ceec249960fdf6 P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.089s	user 0.008s	sys 0.000s
I20260709 13:09:08.952646 24610 raft_consensus.cc:493] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.959879 24645 raft_consensus.cc:515] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:08.960984 24610 raft_consensus.cc:515] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:08.963189 24645 leader_election.cc:290] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:08.964727 24530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b7256c3dd84f5b952d1f9d47691f4e" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:08.964808 24610 leader_election.cc:290] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:08.965845 24530 raft_consensus.cc:2468] T 53b7256c3dd84f5b952d1f9d47691f4e P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 0.
I20260709 13:09:08.966755 24610 raft_consensus.cc:493] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:08.968034 24494 leader_election.cc:343] T 72d2665a42a94a82b1ceec249960fdf6 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:08.968804 24494 leader_election.cc:304] T 72d2665a42a94a82b1ceec249960fdf6 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390; no voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:08.968894 24270 leader_election.cc:304] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381; no voters: 
I20260709 13:09:08.970333 24696 raft_consensus.cc:2804] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:08.970983 24696 raft_consensus.cc:493] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:08.971413 24696 raft_consensus.cc:3060] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:08.974445 24579 raft_consensus.cc:359] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:08.967758 24610 raft_consensus.cc:515] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:08.980708 24610 leader_election.cc:290] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:08.986088 24644 raft_consensus.cc:493] T 93b6310306174b4ca68b467ea7f5d23e P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.986922 24644 raft_consensus.cc:515] T 93b6310306174b4ca68b467ea7f5d23e P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:08.987251 24630 raft_consensus.cc:2749] T 72d2665a42a94a82b1ceec249960fdf6 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:08.991443 24644 leader_election.cc:290] T 93b6310306174b4ca68b467ea7f5d23e P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:08.975664 24579 raft_consensus.cc:385] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.995731 24528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93b6310306174b4ca68b467ea7f5d23e" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:08.997254 24575 tablet_bootstrap.cc:492] T ae6c1a13de5442c5bb4e58e7ce36e741 P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:09.001224 24579 raft_consensus.cc:740] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
W20260709 13:09:09.008462 24419 leader_election.cc:343] T 93b6310306174b4ca68b467ea7f5d23e P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:09.008486 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b147866f4e5f4596b0b3989418cc4b0a" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:09.009652 24528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2b4c67e0f6b4a1e9cfb3703e43e988f" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:09.010780 24528 raft_consensus.cc:2468] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 0.
I20260709 13:09:09.015667 24575 ts_tablet_manager.cc:1403] T ae6c1a13de5442c5bb4e58e7ce36e741 P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.141s	user 0.012s	sys 0.009s
I20260709 13:09:09.013566 24306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93b6310306174b4ca68b467ea7f5d23e" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:09.018961 24306 raft_consensus.cc:2468] T 93b6310306174b4ca68b467ea7f5d23e P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 0.
I20260709 13:09:09.020608 24420 leader_election.cc:304] T 93b6310306174b4ca68b467ea7f5d23e P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381; no voters: 838cb9997b0f437989d3f4e3e9904390
I20260709 13:09:09.020478 24575 raft_consensus.cc:359] T ae6c1a13de5442c5bb4e58e7ce36e741 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:09.021371 24575 raft_consensus.cc:385] T ae6c1a13de5442c5bb4e58e7ce36e741 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:09.021677 24575 raft_consensus.cc:740] T ae6c1a13de5442c5bb4e58e7ce36e741 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:09.022222 24680 raft_consensus.cc:2804] T 93b6310306174b4ca68b467ea7f5d23e P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:09.022763 24680 raft_consensus.cc:493] T 93b6310306174b4ca68b467ea7f5d23e P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:09.023234 24680 raft_consensus.cc:3060] T 93b6310306174b4ca68b467ea7f5d23e P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.026080 24577 tablet_bootstrap.cc:492] T 90d9e64f97544895b9b97dfdc5ceb359 P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:09.028836 24642 leader_election.cc:290] T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:09.029263 24577 ts_tablet_manager.cc:1403] T 90d9e64f97544895b9b97dfdc5ceb359 P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.147s	user 0.018s	sys 0.009s
I20260709 13:09:09.030556 24576 tablet_bootstrap.cc:492] T b147866f4e5f4596b0b3989418cc4b0a P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:09.029820 24307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f32a46373be4a29b6ac72efa35494b4" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:09.030930 24696 raft_consensus.cc:515] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:09.033138 24696 leader_election.cc:290] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Requested vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:09.034683 24530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b7256c3dd84f5b952d1f9d47691f4e" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:09.038046 24530 raft_consensus.cc:3060] T 53b7256c3dd84f5b952d1f9d47691f4e P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.041072 24647 consensus_queue.cc:1048] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:09.046077 24380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b7256c3dd84f5b952d1f9d47691f4e" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:09.047207 24380 raft_consensus.cc:2468] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 0.
I20260709 13:09:09.048664 24378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b7256c3dd84f5b952d1f9d47691f4e" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:09.049429 24378 raft_consensus.cc:3060] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.049532 24579 consensus_queue.cc:260] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:09.056142 24306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b147866f4e5f4596b0b3989418cc4b0a" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:09.057147 24306 raft_consensus.cc:2468] T b147866f4e5f4596b0b3989418cc4b0a P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 0.
I20260709 13:09:09.056286 24307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2b4c67e0f6b4a1e9cfb3703e43e988f" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:09.060669 24680 raft_consensus.cc:515] T 93b6310306174b4ca68b467ea7f5d23e P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:09.062932 24680 leader_election.cc:290] T 93b6310306174b4ca68b467ea7f5d23e P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 election: Requested vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:09.063716 24529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93b6310306174b4ca68b467ea7f5d23e" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:09.064512 24576 tablet_bootstrap.cc:654] T b147866f4e5f4596b0b3989418cc4b0a P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
W20260709 13:09:09.065279 24419 leader_election.cc:343] T 93b6310306174b4ca68b467ea7f5d23e P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:09.062134 24345 leader_election.cc:304] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:09.022643 24575 consensus_queue.cc:260] T ae6c1a13de5442c5bb4e58e7ce36e741 P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:09.070935 24575 ts_tablet_manager.cc:1434] T ae6c1a13de5442c5bb4e58e7ce36e741 P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.055s	user 0.001s	sys 0.007s
I20260709 13:09:09.057456 24579 ts_tablet_manager.cc:1434] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.099s	user 0.009s	sys 0.005s
I20260709 13:09:09.075981 24610 raft_consensus.cc:2804] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:09.076489 24610 raft_consensus.cc:493] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:09.076553 24744 raft_consensus.cc:493] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.076948 24610 raft_consensus.cc:3060] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.077203 24744 raft_consensus.cc:515] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:09.078188 24381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f32a46373be4a29b6ac72efa35494b4" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:09.079041 24381 raft_consensus.cc:2393] T 4f32a46373be4a29b6ac72efa35494b4 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 32088c942fbb48239e0eed952d4df449 in current term 1: Already voted for candidate 9c2a14143eb744cf83bfa5f46284a381 in this term.
W20260709 13:09:09.081074 24345 leader_election.cc:343] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:09.081364 24744 leader_election.cc:290] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:09.083976 24419 leader_election.cc:304] T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32088c942fbb48239e0eed952d4df449; no voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:09.086683 24642 raft_consensus.cc:3060] T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:09:09.088047 24346 leader_election.cc:343] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:09.090286 24575 tablet_bootstrap.cc:492] T 93b6310306174b4ca68b467ea7f5d23e P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:09.090909 24307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93b6310306174b4ca68b467ea7f5d23e" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:09.091698 24307 raft_consensus.cc:3060] T 93b6310306174b4ca68b467ea7f5d23e P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.096777 24529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eaa917e65d9403dbb241f84ee96ba9d" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:09.097590 24529 raft_consensus.cc:2468] T 5eaa917e65d9403dbb241f84ee96ba9d P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 0.
I20260709 13:09:09.098344 24577 raft_consensus.cc:359] T 90d9e64f97544895b9b97dfdc5ceb359 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:09.099140 24577 raft_consensus.cc:385] T 90d9e64f97544895b9b97dfdc5ceb359 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:09.099400 24577 raft_consensus.cc:740] T 90d9e64f97544895b9b97dfdc5ceb359 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:09.099701 24345 leader_election.cc:304] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:09.102440 24579 tablet_bootstrap.cc:492] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:09.111114 24577 consensus_queue.cc:260] T 90d9e64f97544895b9b97dfdc5ceb359 P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:09.116139 24609 raft_consensus.cc:2804] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:09.116370 24378 raft_consensus.cc:2468] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 1.
I20260709 13:09:09.116712 24609 raft_consensus.cc:493] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:09.117177 24609 raft_consensus.cc:3060] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.118811 24647 consensus_queue.cc:1048] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:09.123276 24577 ts_tablet_manager.cc:1434] T 90d9e64f97544895b9b97dfdc5ceb359 P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.094s	user 0.003s	sys 0.004s
I20260709 13:09:09.104147 24346 leader_election.cc:304] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449
I20260709 13:09:09.124395 24744 raft_consensus.cc:2804] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:09.124907 24744 raft_consensus.cc:493] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:09.125312 24744 raft_consensus.cc:3060] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.105786 24575 tablet_bootstrap.cc:654] T 93b6310306174b4ca68b467ea7f5d23e P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:09.106611 24690 consensus_queue.cc:1048] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:09.132192 24530 raft_consensus.cc:2468] T 53b7256c3dd84f5b952d1f9d47691f4e P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 1.
I20260709 13:09:09.132333 24306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eaa917e65d9403dbb241f84ee96ba9d" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:09.133309 24270 leader_election.cc:304] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:09.109200 24642 raft_consensus.cc:2749] T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:09.127594 24609 raft_consensus.cc:515] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:09.142338 24630 raft_consensus.cc:493] T 05840bb250764e02998cdddc290e5139 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.142848 24630 raft_consensus.cc:515] T 05840bb250764e02998cdddc290e5139 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:09.143078 24529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eaa917e65d9403dbb241f84ee96ba9d" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:09.143685 24529 raft_consensus.cc:3060] T 5eaa917e65d9403dbb241f84ee96ba9d P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.144060 24647 raft_consensus.cc:2804] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:09.144614 24647 raft_consensus.cc:697] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Becoming Leader. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Running, Role: LEADER
I20260709 13:09:09.145287 24630 leader_election.cc:290] T 05840bb250764e02998cdddc290e5139 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:09.145785 24647 consensus_queue.cc:237] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:09.148027 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05840bb250764e02998cdddc290e5139" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
W20260709 13:09:09.149526 24493 leader_election.cc:343] T 05840bb250764e02998cdddc290e5139 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:09.151721 24305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eaa917e65d9403dbb241f84ee96ba9d" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:09.134878 24577 tablet_bootstrap.cc:492] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:09.153074 24529 raft_consensus.cc:2468] T 5eaa917e65d9403dbb241f84ee96ba9d P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 1.
I20260709 13:09:09.159137 24696 consensus_queue.cc:1048] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:09.163026 24382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05840bb250764e02998cdddc290e5139" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:09.163998 24382 raft_consensus.cc:2468] T 05840bb250764e02998cdddc290e5139 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 0.
W20260709 13:09:09.164129 24346 leader_election.cc:343] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:09.164330 24680 raft_consensus.cc:493] T 9b7cd923257143648cfe6a296e18b642 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.164855 24680 raft_consensus.cc:515] T 9b7cd923257143648cfe6a296e18b642 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
W20260709 13:09:09.165520 24346 leader_election.cc:343] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:09.166988 24680 leader_election.cc:290] T 9b7cd923257143648cfe6a296e18b642 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:09.169276 24307 raft_consensus.cc:2468] T 93b6310306174b4ca68b467ea7f5d23e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 1.
I20260709 13:09:09.171175 24420 leader_election.cc:304] T 93b6310306174b4ca68b467ea7f5d23e P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381; no voters: 838cb9997b0f437989d3f4e3e9904390
I20260709 13:09:09.167883 24493 leader_election.cc:304] T 05840bb250764e02998cdddc290e5139 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449
I20260709 13:09:09.173849 24577 tablet_bootstrap.cc:654] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:09.175814 24649 raft_consensus.cc:2804] T 05840bb250764e02998cdddc290e5139 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:09.176232 24649 raft_consensus.cc:493] T 05840bb250764e02998cdddc290e5139 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:09.176607 24649 raft_consensus.cc:3060] T 05840bb250764e02998cdddc290e5139 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.180197 24680 raft_consensus.cc:2804] T 93b6310306174b4ca68b467ea7f5d23e P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:09.180775 24680 raft_consensus.cc:697] T 93b6310306174b4ca68b467ea7f5d23e P 32088c942fbb48239e0eed952d4df449 [term 1 LEADER]: Becoming Leader. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Running, Role: LEADER
I20260709 13:09:09.182034 24680 consensus_queue.cc:237] T 93b6310306174b4ca68b467ea7f5d23e P 32088c942fbb48239e0eed952d4df449 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:09.169287 24579 tablet_bootstrap.cc:654] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:09.190356 24382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b7cd923257143648cfe6a296e18b642" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:09.191533 24382 raft_consensus.cc:2468] T 9b7cd923257143648cfe6a296e18b642 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 0.
I20260709 13:09:09.193023 24419 leader_election.cc:304] T 9b7cd923257143648cfe6a296e18b642 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:09.195798 24644 raft_consensus.cc:2804] T 9b7cd923257143648cfe6a296e18b642 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:09.198161 24644 raft_consensus.cc:493] T 9b7cd923257143648cfe6a296e18b642 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:09.198529 24644 raft_consensus.cc:3060] T 9b7cd923257143648cfe6a296e18b642 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.200006 24649 raft_consensus.cc:515] T 05840bb250764e02998cdddc290e5139 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:09.201042 24650 raft_consensus.cc:493] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.141691 24609 leader_election.cc:290] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 election: Requested vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:09.213769 24650 raft_consensus.cc:515] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:09.210896 24610 raft_consensus.cc:515] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:09.222028 24575 tablet_bootstrap.cc:492] T 93b6310306174b4ca68b467ea7f5d23e P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:09.222631 24575 ts_tablet_manager.cc:1403] T 93b6310306174b4ca68b467ea7f5d23e P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.140s	user 0.016s	sys 0.003s
I20260709 13:09:09.223090 24610 leader_election.cc:290] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 election: Requested vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:09.223758 24576 tablet_bootstrap.cc:492] T b147866f4e5f4596b0b3989418cc4b0a P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:09.224404 24576 ts_tablet_manager.cc:1403] T b147866f4e5f4596b0b3989418cc4b0a P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.194s	user 0.016s	sys 0.045s
I20260709 13:09:09.227166 24305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2b4c67e0f6b4a1e9cfb3703e43e988f" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:09.228010 24650 leader_election.cc:290] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
W20260709 13:09:09.229027 24346 leader_election.cc:343] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:09.230273 24744 raft_consensus.cc:515] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:09.230991 24528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d2665a42a94a82b1ceec249960fdf6" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:09.231909 24528 raft_consensus.cc:2468] T 72d2665a42a94a82b1ceec249960fdf6 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 0.
I20260709 13:09:09.232753 24744 leader_election.cc:290] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 election: Requested vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:09.223199 24307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b7cd923257143648cfe6a296e18b642" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:09.245002 24345 leader_election.cc:304] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 9c2a14143eb744cf83bfa5f46284a381
W20260709 13:09:09.246440 24345 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:33251, user_credentials={real_user=slave}} blocked reactor thread for 80846.5us
I20260709 13:09:09.229146 24454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d2665a42a94a82b1ceec249960fdf6" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:09.254115 24528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2b4c67e0f6b4a1e9cfb3703e43e988f" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:09.254990 24528 raft_consensus.cc:3060] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.248091 24305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b147866f4e5f4596b0b3989418cc4b0a" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:09.258579 24454 raft_consensus.cc:2468] T 72d2665a42a94a82b1ceec249960fdf6 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 0.
I20260709 13:09:09.260628 24305 raft_consensus.cc:3060] T b147866f4e5f4596b0b3989418cc4b0a P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.262069 24270 leader_election.cc:304] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381; no voters: 
I20260709 13:09:09.263605 24647 raft_consensus.cc:2804] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:09.269071 24647 raft_consensus.cc:493] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:09.269505 24647 raft_consensus.cc:3060] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.269551 24381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05840bb250764e02998cdddc290e5139" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:09.270506 24381 raft_consensus.cc:3060] T 05840bb250764e02998cdddc290e5139 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.252033 24744 raft_consensus.cc:2804] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:09.274991 24744 raft_consensus.cc:697] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45 [term 1 LEADER]: Becoming Leader. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Running, Role: LEADER
I20260709 13:09:09.266664 24575 raft_consensus.cc:359] T 93b6310306174b4ca68b467ea7f5d23e P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:09.277556 24575 raft_consensus.cc:385] T 93b6310306174b4ca68b467ea7f5d23e P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:09.277803 24575 raft_consensus.cc:740] T 93b6310306174b4ca68b467ea7f5d23e P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:09.266642 24204 catalog_manager.cc:5697] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "9c2a14143eb744cf83bfa5f46284a381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:09.291899 24575 consensus_queue.cc:260] T 93b6310306174b4ca68b467ea7f5d23e P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:09.293709 24454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b147866f4e5f4596b0b3989418cc4b0a" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
W20260709 13:09:09.294229 24420 leader_election.cc:343] T 9b7cd923257143648cfe6a296e18b642 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:09.295420 24345 leader_election.cc:343] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:09.298470 24576 raft_consensus.cc:359] T b147866f4e5f4596b0b3989418cc4b0a P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:09.303153 24576 raft_consensus.cc:385] T b147866f4e5f4596b0b3989418cc4b0a P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:09.303407 24576 raft_consensus.cc:740] T b147866f4e5f4596b0b3989418cc4b0a P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:09.312384 24579 tablet_bootstrap.cc:492] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:09.313149 24579 ts_tablet_manager.cc:1403] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.211s	user 0.021s	sys 0.004s
I20260709 13:09:09.314627 24575 ts_tablet_manager.cc:1434] T 93b6310306174b4ca68b467ea7f5d23e P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.092s	user 0.009s	sys 0.001s
W20260709 13:09:09.329605 24753 log.cc:927] Time spent T 4f32a46373be4a29b6ac72efa35494b4 P c40e7ba102224d24a22ccd1f68aaae45: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
W20260709 13:09:09.330231 24749 log.cc:927] Time spent T c60b42ac6d7e48bfae725dc113758659 P 32088c942fbb48239e0eed952d4df449: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:09:09.296944 24744 consensus_queue.cc:237] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:09.318632 24305 raft_consensus.cc:2468] T b147866f4e5f4596b0b3989418cc4b0a P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 1.
I20260709 13:09:09.336086 24575 tablet_bootstrap.cc:492] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:09.336436 24346 leader_election.cc:304] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449
I20260709 13:09:09.337848 24609 raft_consensus.cc:2804] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:09.338382 24609 raft_consensus.cc:697] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45 [term 1 LEADER]: Becoming Leader. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Running, Role: LEADER
I20260709 13:09:09.339404 24609 consensus_queue.cc:237] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:09.343070 24575 tablet_bootstrap.cc:654] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:09.316282 24644 raft_consensus.cc:515] T 9b7cd923257143648cfe6a296e18b642 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:09.347162 24528 raft_consensus.cc:2468] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 1.
I20260709 13:09:09.347478 24382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b7cd923257143648cfe6a296e18b642" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:09.348407 24644 leader_election.cc:290] T 9b7cd923257143648cfe6a296e18b642 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 election: Requested vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:09.351305 24381 raft_consensus.cc:2468] T 05840bb250764e02998cdddc290e5139 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 1.
I20260709 13:09:09.363332 24382 raft_consensus.cc:3060] T 9b7cd923257143648cfe6a296e18b642 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.353432 24649 leader_election.cc:290] T 05840bb250764e02998cdddc290e5139 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Requested vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:09.354996 24305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b7cd923257143648cfe6a296e18b642" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:09.333374 24576 consensus_queue.cc:260] T b147866f4e5f4596b0b3989418cc4b0a P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:09.366086 24576 ts_tablet_manager.cc:1434] T b147866f4e5f4596b0b3989418cc4b0a P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.141s	user 0.008s	sys 0.000s
I20260709 13:09:09.367224 24576 tablet_bootstrap.cc:492] T 722754340deb4df18b716dc801c35c34 P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
W20260709 13:09:09.368813 24420 leader_election.cc:343] T 9b7cd923257143648cfe6a296e18b642 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:09.355072 24579 raft_consensus.cc:359] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:09.355862 24577 tablet_bootstrap.cc:492] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:09.372223 24579 raft_consensus.cc:385] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:09.372603 24577 ts_tablet_manager.cc:1403] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.238s	user 0.035s	sys 0.009s
I20260709 13:09:09.372561 24579 raft_consensus.cc:740] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:09.384722 24609 raft_consensus.cc:493] T 2039124a6d8848149440a70100af8755 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.385319 24609 raft_consensus.cc:515] T 2039124a6d8848149440a70100af8755 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:09.384124 24579 consensus_queue.cc:260] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:09.390864 24579 ts_tablet_manager.cc:1434] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.077s	user 0.008s	sys 0.003s
I20260709 13:09:09.390671 24577 raft_consensus.cc:359] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:09.351573 24647 raft_consensus.cc:515] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:09.392354 24579 tablet_bootstrap.cc:492] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:09.393782 24647 leader_election.cc:290] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Requested vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:09.396297 24345 leader_election.cc:304] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:09.398329 24758 raft_consensus.cc:493] T ae6c1a13de5442c5bb4e58e7ce36e741 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.399153 24579 tablet_bootstrap.cc:654] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:09.398875 24758 raft_consensus.cc:515] T ae6c1a13de5442c5bb4e58e7ce36e741 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:09.400250 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2039124a6d8848149440a70100af8755" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:09.401292 24758 leader_election.cc:290] T ae6c1a13de5442c5bb4e58e7ce36e741 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:09.402338 24530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2039124a6d8848149440a70100af8755" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:09.398907 24746 raft_consensus.cc:2804] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:09.404964 24746 raft_consensus.cc:697] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45 [term 1 LEADER]: Becoming Leader. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Running, Role: LEADER
I20260709 13:09:09.406026 24746 consensus_queue.cc:237] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:09.391611 24577 raft_consensus.cc:385] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:09.410873 24577 raft_consensus.cc:740] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:09.387459 24609 leader_election.cc:290] T 2039124a6d8848149440a70100af8755 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:09.418077 24577 consensus_queue.cc:260] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:09.404978 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d2665a42a94a82b1ceec249960fdf6" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:09.425189 24530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d2665a42a94a82b1ceec249960fdf6" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
W20260709 13:09:09.415771 24345 leader_election.cc:343] T 2039124a6d8848149440a70100af8755 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:09.431061 24455 raft_consensus.cc:3060] T 72d2665a42a94a82b1ceec249960fdf6 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.432397 24530 raft_consensus.cc:3060] T 72d2665a42a94a82b1ceec249960fdf6 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.433769 24456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05840bb250764e02998cdddc290e5139" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:09.428665 24493 leader_election.cc:304] T 05840bb250764e02998cdddc290e5139 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
W20260709 13:09:09.441104 24345 leader_election.cc:343] T 2039124a6d8848149440a70100af8755 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:09.447055 24758 raft_consensus.cc:2804] T 05840bb250764e02998cdddc290e5139 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:09:09.451406 24761 log.cc:927] Time spent T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45: Append to log took a long time: real 0.074s	user 0.001s	sys 0.001s
I20260709 13:09:09.451344 24758 raft_consensus.cc:697] T 05840bb250764e02998cdddc290e5139 P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Becoming Leader. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Running, Role: LEADER
I20260709 13:09:09.453208 24758 consensus_queue.cc:237] T 05840bb250764e02998cdddc290e5139 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:09.454420 24579 tablet_bootstrap.cc:492] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:09.454929 24579 ts_tablet_manager.cc:1403] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.063s	user 0.012s	sys 0.010s
I20260709 13:09:09.452198 24345 leader_election.cc:304] T 2039124a6d8848149440a70100af8755 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449, 838cb9997b0f437989d3f4e3e9904390
I20260709 13:09:09.452104 24577 ts_tablet_manager.cc:1434] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.079s	user 0.005s	sys 0.007s
I20260709 13:09:09.458252 24744 raft_consensus.cc:2749] T 2039124a6d8848149440a70100af8755 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:09.466646 24576 tablet_bootstrap.cc:654] T 722754340deb4df18b716dc801c35c34 P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:09.468808 24382 raft_consensus.cc:2468] T 9b7cd923257143648cfe6a296e18b642 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 1.
I20260709 13:09:09.471060 24419 leader_election.cc:304] T 9b7cd923257143648cfe6a296e18b642 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, c40e7ba102224d24a22ccd1f68aaae45; no voters: 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:09.472410 24644 raft_consensus.cc:2804] T 9b7cd923257143648cfe6a296e18b642 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:09.472579 24649 raft_consensus.cc:493] T 93b6310306174b4ca68b467ea7f5d23e P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.472899 24644 raft_consensus.cc:697] T 9b7cd923257143648cfe6a296e18b642 P 32088c942fbb48239e0eed952d4df449 [term 1 LEADER]: Becoming Leader. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Running, Role: LEADER
I20260709 13:09:09.473151 24649 raft_consensus.cc:515] T 93b6310306174b4ca68b467ea7f5d23e P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:09.473621 24456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae6c1a13de5442c5bb4e58e7ce36e741" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:09.474236 24644 consensus_queue.cc:237] T 9b7cd923257143648cfe6a296e18b642 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:09.482125 24579 raft_consensus.cc:359] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:09.484555 24579 raft_consensus.cc:385] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:09.484910 24579 raft_consensus.cc:740] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:09.490317 24381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae6c1a13de5442c5bb4e58e7ce36e741" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:09.491109 24381 raft_consensus.cc:2468] T ae6c1a13de5442c5bb4e58e7ce36e741 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 0.
W20260709 13:09:09.498368 24493 leader_election.cc:343] T 05840bb250764e02998cdddc290e5139 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:09.499446 24493 leader_election.cc:343] T ae6c1a13de5442c5bb4e58e7ce36e741 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:09.500639 24456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93b6310306174b4ca68b467ea7f5d23e" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:09.509577 24493 leader_election.cc:304] T ae6c1a13de5442c5bb4e58e7ce36e741 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449
I20260709 13:09:09.511055 24758 raft_consensus.cc:2804] T ae6c1a13de5442c5bb4e58e7ce36e741 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:09.511638 24758 raft_consensus.cc:493] T ae6c1a13de5442c5bb4e58e7ce36e741 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:09.499508 24579 consensus_queue.cc:260] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:09.512145 24758 raft_consensus.cc:3060] T ae6c1a13de5442c5bb4e58e7ce36e741 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.509511 24305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93b6310306174b4ca68b467ea7f5d23e" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:09.516561 24579 ts_tablet_manager.cc:1434] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.061s	user 0.011s	sys 0.004s
I20260709 13:09:09.519956 24577 tablet_bootstrap.cc:492] T 72e12f99e2084675ab3f53ffb836e464 P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:09.523514 24305 raft_consensus.cc:2393] T 93b6310306174b4ca68b467ea7f5d23e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 838cb9997b0f437989d3f4e3e9904390 in current term 1: Already voted for candidate 32088c942fbb48239e0eed952d4df449 in this term.
I20260709 13:09:09.508345 24649 leader_election.cc:290] T 93b6310306174b4ca68b467ea7f5d23e P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:09.531272 24744 raft_consensus.cc:493] T 90d9e64f97544895b9b97dfdc5ceb359 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.547230 24494 leader_election.cc:304] T 93b6310306174b4ca68b467ea7f5d23e P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390; no voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:09.531888 24744 raft_consensus.cc:515] T 90d9e64f97544895b9b97dfdc5ceb359 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:09.548977 24649 raft_consensus.cc:3060] T 93b6310306174b4ca68b467ea7f5d23e P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.550812 24744 leader_election.cc:290] T 90d9e64f97544895b9b97dfdc5ceb359 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:09.552497 24305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90d9e64f97544895b9b97dfdc5ceb359" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
W20260709 13:09:09.554389 24346 leader_election.cc:343] T 90d9e64f97544895b9b97dfdc5ceb359 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:09.559908 24455 raft_consensus.cc:2468] T 72d2665a42a94a82b1ceec249960fdf6 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 1.
I20260709 13:09:09.564603 24456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90d9e64f97544895b9b97dfdc5ceb359" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:09.565977 24530 raft_consensus.cc:2468] T 72d2665a42a94a82b1ceec249960fdf6 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 1.
I20260709 13:09:09.565627 24758 raft_consensus.cc:515] T ae6c1a13de5442c5bb4e58e7ce36e741 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
W20260709 13:09:09.567215 24345 leader_election.cc:343] T 90d9e64f97544895b9b97dfdc5ceb359 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:09.568161 24758 leader_election.cc:290] T ae6c1a13de5442c5bb4e58e7ce36e741 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Requested vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:09.567919 24345 leader_election.cc:304] T 90d9e64f97544895b9b97dfdc5ceb359 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:09.571277 24382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae6c1a13de5442c5bb4e58e7ce36e741" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:09.571991 24382 raft_consensus.cc:3060] T ae6c1a13de5442c5bb4e58e7ce36e741 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.575384 24577 tablet_bootstrap.cc:654] T 72e12f99e2084675ab3f53ffb836e464 P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:09.579753 24647 raft_consensus.cc:493] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.581171 24579 tablet_bootstrap.cc:492] T 9b7cd923257143648cfe6a296e18b642 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:09.570981 24744 raft_consensus.cc:2749] T 90d9e64f97544895b9b97dfdc5ceb359 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:09.585091 24270 leader_election.cc:304] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381; no voters: 
I20260709 13:09:09.580489 24647 raft_consensus.cc:515] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:09.573352 24456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae6c1a13de5442c5bb4e58e7ce36e741" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:09.587030 24746 raft_consensus.cc:493] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.602248 24746 raft_consensus.cc:515] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
W20260709 13:09:09.603330 24493 leader_election.cc:343] T ae6c1a13de5442c5bb4e58e7ce36e741 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:09.606292 24756 raft_consensus.cc:2804] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:09.607124 24756 raft_consensus.cc:697] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Becoming Leader. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Running, Role: LEADER
I20260709 13:09:09.608281 24756 consensus_queue.cc:237] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:09.618199 24576 tablet_bootstrap.cc:492] T 722754340deb4df18b716dc801c35c34 P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:09.618813 24576 ts_tablet_manager.cc:1403] T 722754340deb4df18b716dc801c35c34 P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.252s	user 0.006s	sys 0.020s
I20260709 13:09:09.612569 24746 leader_election.cc:290] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:09.635178 24575 tablet_bootstrap.cc:492] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:09.635718 24575 ts_tablet_manager.cc:1403] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.300s	user 0.026s	sys 0.002s
I20260709 13:09:09.638722 24577 tablet_bootstrap.cc:492] T 72e12f99e2084675ab3f53ffb836e464 P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:09.639215 24577 ts_tablet_manager.cc:1403] T 72e12f99e2084675ab3f53ffb836e464 P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.122s	user 0.027s	sys 0.015s
I20260709 13:09:09.639379 24575 raft_consensus.cc:359] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:09.640115 24575 raft_consensus.cc:385] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:09.634537 24307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bb19353ba334eac831ae2cce277298a" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:09.642143 24577 raft_consensus.cc:359] T 72e12f99e2084675ab3f53ffb836e464 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:09.643756 24577 raft_consensus.cc:385] T 72e12f99e2084675ab3f53ffb836e464 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:09.634756 24203 catalog_manager.cc:5697] T 93b6310306174b4ca68b467ea7f5d23e P 32088c942fbb48239e0eed952d4df449 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32088c942fbb48239e0eed952d4df449 (127.14.103.3). New cstate: current_term: 1 leader_uuid: "32088c942fbb48239e0eed952d4df449" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:09.646803 24579 tablet_bootstrap.cc:654] T 9b7cd923257143648cfe6a296e18b642 P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:09.644078 24577 raft_consensus.cc:740] T 72e12f99e2084675ab3f53ffb836e464 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:09.647776 24528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bb19353ba334eac831ae2cce277298a" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:09.649307 24647 leader_election.cc:290] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:09.652364 24528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2b4c67e0f6b4a1e9cfb3703e43e988f" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
W20260709 13:09:09.652599 24346 leader_election.cc:343] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:09.653234 24528 raft_consensus.cc:2393] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9c2a14143eb744cf83bfa5f46284a381 in current term 1: Already voted for candidate c40e7ba102224d24a22ccd1f68aaae45 in this term.
I20260709 13:09:09.640395 24575 raft_consensus.cc:740] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:09.648146 24577 consensus_queue.cc:260] T 72e12f99e2084675ab3f53ffb836e464 P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:09.632747 24649 raft_consensus.cc:2749] T 93b6310306174b4ca68b467ea7f5d23e P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:09.658215 24575 consensus_queue.cc:260] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:09.656224 24642 consensus_queue.cc:1048] T 93b6310306174b4ca68b467ea7f5d23e P 32088c942fbb48239e0eed952d4df449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:09:09.661108 24577 ts_tablet_manager.cc:1434] T 72e12f99e2084675ab3f53ffb836e464 P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.002s
I20260709 13:09:09.662256 24577 tablet_bootstrap.cc:492] T 6ae3dc25b8484b6f99d8f41685229cb8 P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:09.665644 24378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2b4c67e0f6b4a1e9cfb3703e43e988f" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
W20260709 13:09:09.665856 24345 leader_election.cc:343] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:09.666338 24345 leader_election.cc:304] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:09.667616 24609 raft_consensus.cc:2749] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:09.668769 24382 raft_consensus.cc:2468] T ae6c1a13de5442c5bb4e58e7ce36e741 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 1.
I20260709 13:09:09.674115 24270 leader_election.cc:304] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381; no voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:09.675069 24696 raft_consensus.cc:3060] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.674422 24202 catalog_manager.cc:5697] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45 reported cstate change: term changed from 0 to 1, leader changed from <none> to c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2). New cstate: current_term: 1 leader_uuid: "c40e7ba102224d24a22ccd1f68aaae45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:09.676642 24202 catalog_manager.cc:5697] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45 reported cstate change: term changed from 0 to 1, leader changed from <none> to c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2). New cstate: current_term: 1 leader_uuid: "c40e7ba102224d24a22ccd1f68aaae45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:09.679711 24577 tablet_bootstrap.cc:654] T 6ae3dc25b8484b6f99d8f41685229cb8 P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:09.684664 24746 raft_consensus.cc:493] T 72e12f99e2084675ab3f53ffb836e464 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.685245 24746 raft_consensus.cc:515] T 72e12f99e2084675ab3f53ffb836e464 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:09.693795 24575 ts_tablet_manager.cc:1434] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.058s	user 0.008s	sys 0.000s
I20260709 13:09:09.695094 24575 tablet_bootstrap.cc:492] T 2039124a6d8848149440a70100af8755 P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:09.701480 24575 tablet_bootstrap.cc:654] T 2039124a6d8848149440a70100af8755 P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:09.709424 24493 leader_election.cc:304] T ae6c1a13de5442c5bb4e58e7ce36e741 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449
I20260709 13:09:09.711524 24649 raft_consensus.cc:2804] T ae6c1a13de5442c5bb4e58e7ce36e741 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:09.712333 24649 raft_consensus.cc:697] T ae6c1a13de5442c5bb4e58e7ce36e741 P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Becoming Leader. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Running, Role: LEADER
I20260709 13:09:09.712847 24696 raft_consensus.cc:2749] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:09.713343 24690 consensus_queue.cc:1048] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:09.714676 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e12f99e2084675ab3f53ffb836e464" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:09.715636 24576 raft_consensus.cc:359] T 722754340deb4df18b716dc801c35c34 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:09.717414 24649 consensus_queue.cc:237] T ae6c1a13de5442c5bb4e58e7ce36e741 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:09.718989 24530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e12f99e2084675ab3f53ffb836e464" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:09.716931 24576 raft_consensus.cc:385] T 722754340deb4df18b716dc801c35c34 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:09.707127 24746 leader_election.cc:290] T 72e12f99e2084675ab3f53ffb836e464 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:09.721216 24576 raft_consensus.cc:740] T 722754340deb4df18b716dc801c35c34 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:09.723507 24576 consensus_queue.cc:260] T 722754340deb4df18b716dc801c35c34 P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:09.724576 24644 raft_consensus.cc:493] T b147866f4e5f4596b0b3989418cc4b0a P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.736033 24575 tablet_bootstrap.cc:492] T 2039124a6d8848149440a70100af8755 P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:09.736632 24575 ts_tablet_manager.cc:1403] T 2039124a6d8848149440a70100af8755 P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.042s	user 0.011s	sys 0.000s
W20260709 13:09:09.732048 24345 leader_election.cc:343] T 72e12f99e2084675ab3f53ffb836e464 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:09.744115 24680 consensus_queue.cc:1048] T 93b6310306174b4ca68b467ea7f5d23e P 32088c942fbb48239e0eed952d4df449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:09:09.746291 24576 ts_tablet_manager.cc:1434] T 722754340deb4df18b716dc801c35c34 P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.127s	user 0.008s	sys 0.000s
I20260709 13:09:09.738350 24644 raft_consensus.cc:515] T b147866f4e5f4596b0b3989418cc4b0a P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:09.749570 24201 catalog_manager.cc:5697] T 05840bb250764e02998cdddc290e5139 P 838cb9997b0f437989d3f4e3e9904390 reported cstate change: term changed from 0 to 1, leader changed from <none> to 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4). New cstate: current_term: 1 leader_uuid: "838cb9997b0f437989d3f4e3e9904390" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:09.756932 24644 leader_election.cc:290] T b147866f4e5f4596b0b3989418cc4b0a P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:09.757993 24577 tablet_bootstrap.cc:492] T 6ae3dc25b8484b6f99d8f41685229cb8 P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:09.758512 24577 ts_tablet_manager.cc:1403] T 6ae3dc25b8484b6f99d8f41685229cb8 P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.096s	user 0.012s	sys 0.009s
I20260709 13:09:09.758919 24307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b147866f4e5f4596b0b3989418cc4b0a" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:09.759788 24307 raft_consensus.cc:2393] T b147866f4e5f4596b0b3989418cc4b0a P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 32088c942fbb48239e0eed952d4df449 in current term 1: Already voted for candidate c40e7ba102224d24a22ccd1f68aaae45 in this term.
I20260709 13:09:09.761430 24577 raft_consensus.cc:359] T 6ae3dc25b8484b6f99d8f41685229cb8 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:09.762149 24577 raft_consensus.cc:385] T 6ae3dc25b8484b6f99d8f41685229cb8 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:09.762398 24577 raft_consensus.cc:740] T 6ae3dc25b8484b6f99d8f41685229cb8 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:09.763538 24696 consensus_queue.cc:1048] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:09.762660 24575 raft_consensus.cc:359] T 2039124a6d8848149440a70100af8755 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:09.766192 24575 raft_consensus.cc:385] T 2039124a6d8848149440a70100af8755 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:09.763058 24577 consensus_queue.cc:260] T 6ae3dc25b8484b6f99d8f41685229cb8 P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
W20260709 13:09:09.748831 24345 leader_election.cc:343] T 72e12f99e2084675ab3f53ffb836e464 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:09.752964 24579 tablet_bootstrap.cc:492] T 9b7cd923257143648cfe6a296e18b642 P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:09.767998 24579 ts_tablet_manager.cc:1403] T 9b7cd923257143648cfe6a296e18b642 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.222s	user 0.021s	sys 0.004s
I20260709 13:09:09.770470 24577 ts_tablet_manager.cc:1434] T 6ae3dc25b8484b6f99d8f41685229cb8 P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 13:09:09.766495 24575 raft_consensus.cc:740] T 2039124a6d8848149440a70100af8755 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:09.775115 24579 raft_consensus.cc:359] T 9b7cd923257143648cfe6a296e18b642 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:09.777845 24579 raft_consensus.cc:385] T 9b7cd923257143648cfe6a296e18b642 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:09.778501 24772 raft_consensus.cc:493] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.779009 24772 raft_consensus.cc:515] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:09.781222 24772 leader_election.cc:290] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:09.781847 24576 tablet_bootstrap.cc:492] T 05840bb250764e02998cdddc290e5139 P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:09.778159 24579 raft_consensus.cc:740] T 9b7cd923257143648cfe6a296e18b642 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:09.783520 24456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "722754340deb4df18b716dc801c35c34" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:09.783943 24579 consensus_queue.cc:260] T 9b7cd923257143648cfe6a296e18b642 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:09.784379 24456 raft_consensus.cc:2468] T 722754340deb4df18b716dc801c35c34 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 0.
I20260709 13:09:09.785888 24493 leader_election.cc:304] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 838cb9997b0f437989d3f4e3e9904390; no voters: 
I20260709 13:09:09.765697 24203 catalog_manager.cc:5697] T 9b7cd923257143648cfe6a296e18b642 P 32088c942fbb48239e0eed952d4df449 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32088c942fbb48239e0eed952d4df449 (127.14.103.3). New cstate: current_term: 1 leader_uuid: "32088c942fbb48239e0eed952d4df449" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:09.786738 24772 raft_consensus.cc:2804] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:09.787132 24772 raft_consensus.cc:493] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:09.787457 24772 raft_consensus.cc:3060] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.786260 24579 ts_tablet_manager.cc:1434] T 9b7cd923257143648cfe6a296e18b642 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.002s
I20260709 13:09:09.789499 24575 consensus_queue.cc:260] T 2039124a6d8848149440a70100af8755 P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:09.774704 24378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b147866f4e5f4596b0b3989418cc4b0a" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:09.789557 24307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "722754340deb4df18b716dc801c35c34" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:09.798206 24419 leader_election.cc:304] T b147866f4e5f4596b0b3989418cc4b0a P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32088c942fbb48239e0eed952d4df449; no voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45
W20260709 13:09:09.800674 24494 leader_election.cc:343] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:09.800267 24204 catalog_manager.cc:5697] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "9c2a14143eb744cf83bfa5f46284a381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:09.803892 24644 raft_consensus.cc:3060] T b147866f4e5f4596b0b3989418cc4b0a P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.814025 24756 raft_consensus.cc:493] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.816071 24696 raft_consensus.cc:493] T 9b7cd923257143648cfe6a296e18b642 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.816742 24696 raft_consensus.cc:515] T 9b7cd923257143648cfe6a296e18b642 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:09.819093 24696 leader_election.cc:290] T 9b7cd923257143648cfe6a296e18b642 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:09.771821 24345 leader_election.cc:304] T 72e12f99e2084675ab3f53ffb836e464 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449, 838cb9997b0f437989d3f4e3e9904390
I20260709 13:09:09.832269 24609 raft_consensus.cc:2749] T 72e12f99e2084675ab3f53ffb836e464 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:09.825696 24772 raft_consensus.cc:515] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:09.826910 24746 raft_consensus.cc:493] T 6ae3dc25b8484b6f99d8f41685229cb8 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.815109 24756 raft_consensus.cc:515] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:09.832683 24456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b7cd923257143648cfe6a296e18b642" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:09.836542 24746 raft_consensus.cc:515] T 6ae3dc25b8484b6f99d8f41685229cb8 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:09.838676 24746 leader_election.cc:290] T 6ae3dc25b8484b6f99d8f41685229cb8 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:09.839468 24307 raft_consensus.cc:3060] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.840508 24756 leader_election.cc:290] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:09.842343 24529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eaa917e65d9403dbb241f84ee96ba9d" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:09.843128 24529 raft_consensus.cc:2393] T 5eaa917e65d9403dbb241f84ee96ba9d P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9c2a14143eb744cf83bfa5f46284a381 in current term 1: Already voted for candidate c40e7ba102224d24a22ccd1f68aaae45 in this term.
W20260709 13:09:09.844393 24345 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:33251, user_credentials={real_user=slave}} blocked reactor thread for 96233.6us
I20260709 13:09:09.845021 24579 tablet_bootstrap.cc:492] T 722754340deb4df18b716dc801c35c34 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:09.844729 24306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ae3dc25b8484b6f99d8f41685229cb8" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:09.846822 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ae3dc25b8484b6f99d8f41685229cb8" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
W20260709 13:09:09.848194 24346 leader_election.cc:343] T 6ae3dc25b8484b6f99d8f41685229cb8 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:09.852121 24609 consensus_queue.cc:1048] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:09.854164 24575 ts_tablet_manager.cc:1434] T 2039124a6d8848149440a70100af8755 P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.117s	user 0.008s	sys 0.001s
I20260709 13:09:09.859253 24577 tablet_bootstrap.cc:492] T f4e53ea5cf694db697fa0e474da54423 P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:09.852758 24649 raft_consensus.cc:493] T 2039124a6d8848149440a70100af8755 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.872892 24649 raft_consensus.cc:515] T 2039124a6d8848149440a70100af8755 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:09.875725 24744 consensus_queue.cc:1048] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:09.876422 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "722754340deb4df18b716dc801c35c34" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:09.877094 24455 raft_consensus.cc:3060] T 722754340deb4df18b716dc801c35c34 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.877735 24649 leader_election.cc:290] T 2039124a6d8848149440a70100af8755 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:09.878397 24772 leader_election.cc:290] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Requested vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:09.879901 24304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "722754340deb4df18b716dc801c35c34" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
W20260709 13:09:09.881461 24494 leader_election.cc:343] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:09.881599 24456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2039124a6d8848149440a70100af8755" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:09.882710 24382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eaa917e65d9403dbb241f84ee96ba9d" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
W20260709 13:09:09.883322 24493 leader_election.cc:343] T 2039124a6d8848149440a70100af8755 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:09.861219 24575 tablet_bootstrap.cc:492] T 5bb19353ba334eac831ae2cce277298a P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:09.870635 24378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b7cd923257143648cfe6a296e18b642" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:09.890811 24378 raft_consensus.cc:2393] T 9b7cd923257143648cfe6a296e18b642 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9c2a14143eb744cf83bfa5f46284a381 in current term 1: Already voted for candidate 32088c942fbb48239e0eed952d4df449 in this term.
W20260709 13:09:09.872339 24345 leader_election.cc:343] T 6ae3dc25b8484b6f99d8f41685229cb8 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:09.896037 24345 leader_election.cc:304] T 6ae3dc25b8484b6f99d8f41685229cb8 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:09.901579 24609 raft_consensus.cc:2749] T 6ae3dc25b8484b6f99d8f41685229cb8 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:09.892966 24644 raft_consensus.cc:2749] T b147866f4e5f4596b0b3989418cc4b0a P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:09.903612 24270 leader_election.cc:304] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381; no voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:09.905081 24270 leader_election.cc:304] T 9b7cd923257143648cfe6a296e18b642 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381; no voters: 32088c942fbb48239e0eed952d4df449, c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:09.909475 24579 tablet_bootstrap.cc:654] T 722754340deb4df18b716dc801c35c34 P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:09.914824 24576 tablet_bootstrap.cc:654] T 05840bb250764e02998cdddc290e5139 P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:09.914693 24696 raft_consensus.cc:3060] T 9b7cd923257143648cfe6a296e18b642 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.919783 24690 raft_consensus.cc:2749] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:09.920177 24577 tablet_bootstrap.cc:654] T f4e53ea5cf694db697fa0e474da54423 P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:09.923775 24378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2039124a6d8848149440a70100af8755" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:09.924523 24378 raft_consensus.cc:2468] T 2039124a6d8848149440a70100af8755 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 0.
I20260709 13:09:09.929121 24205 catalog_manager.cc:5697] T ae6c1a13de5442c5bb4e58e7ce36e741 P 838cb9997b0f437989d3f4e3e9904390 reported cstate change: term changed from 0 to 1, leader changed from <none> to 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4). New cstate: current_term: 1 leader_uuid: "838cb9997b0f437989d3f4e3e9904390" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:09.937227 24493 leader_election.cc:304] T 2039124a6d8848149440a70100af8755 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449
I20260709 13:09:09.943709 24649 raft_consensus.cc:2804] T 2039124a6d8848149440a70100af8755 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:09.944145 24649 raft_consensus.cc:493] T 2039124a6d8848149440a70100af8755 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:09.944501 24649 raft_consensus.cc:3060] T 2039124a6d8848149440a70100af8755 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.948086 24575 tablet_bootstrap.cc:654] T 5bb19353ba334eac831ae2cce277298a P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:09.949649 24455 raft_consensus.cc:2468] T 722754340deb4df18b716dc801c35c34 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 1.
I20260709 13:09:09.952145 24649 raft_consensus.cc:515] T 2039124a6d8848149440a70100af8755 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:09.960078 24696 raft_consensus.cc:2749] T 9b7cd923257143648cfe6a296e18b642 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:09.953302 24609 consensus_queue.cc:1048] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:09:09.963833 24609 consensus_queue.cc:1048] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:09.956315 24746 consensus_queue.cc:1048] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
W20260709 13:09:09.970487 24493 consensus_peers.cc:597] T 05840bb250764e02998cdddc290e5139 P 838cb9997b0f437989d3f4e3e9904390 -> Peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Couldn't send request to peer 32088c942fbb48239e0eed952d4df449. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:09.974763 24493 leader_election.cc:304] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 838cb9997b0f437989d3f4e3e9904390; no voters: 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:09.976720 24772 raft_consensus.cc:2804] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:09.977488 24772 raft_consensus.cc:697] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Becoming Leader. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Running, Role: LEADER
I20260709 13:09:09.978430 24772 consensus_queue.cc:237] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:09.980593 24201 catalog_manager.cc:5697] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45 reported cstate change: term changed from 0 to 1, leader changed from <none> to c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2). New cstate: current_term: 1 leader_uuid: "c40e7ba102224d24a22ccd1f68aaae45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:09.988401 24746 consensus_queue.cc:1048] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:09:09.989292 24253 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:09.999361 24649 leader_election.cc:290] T 2039124a6d8848149440a70100af8755 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Requested vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:09.999989 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2039124a6d8848149440a70100af8755" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:10.018347 24579 tablet_bootstrap.cc:492] T 722754340deb4df18b716dc801c35c34 P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:10.018944 24579 ts_tablet_manager.cc:1403] T 722754340deb4df18b716dc801c35c34 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.190s	user 0.006s	sys 0.014s
I20260709 13:09:10.019429 24577 tablet_bootstrap.cc:492] T f4e53ea5cf694db697fa0e474da54423 P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
W20260709 13:09:10.020512 24493 leader_election.cc:343] T 2039124a6d8848149440a70100af8755 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:10.020810 24577 ts_tablet_manager.cc:1403] T f4e53ea5cf694db697fa0e474da54423 P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.166s	user 0.015s	sys 0.004s
I20260709 13:09:10.025359 24576 tablet_bootstrap.cc:492] T 05840bb250764e02998cdddc290e5139 P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:10.025928 24576 ts_tablet_manager.cc:1403] T 05840bb250764e02998cdddc290e5139 P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.254s	user 0.016s	sys 0.006s
I20260709 13:09:10.034356 24577 raft_consensus.cc:359] T f4e53ea5cf694db697fa0e474da54423 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:10.035166 24577 raft_consensus.cc:385] T f4e53ea5cf694db697fa0e474da54423 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.035440 24577 raft_consensus.cc:740] T f4e53ea5cf694db697fa0e474da54423 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.036478 24680 consensus_queue.cc:1048] T 9b7cd923257143648cfe6a296e18b642 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:10.044328 24576 raft_consensus.cc:359] T 05840bb250764e02998cdddc290e5139 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:10.050657 24576 raft_consensus.cc:385] T 05840bb250764e02998cdddc290e5139 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.051102 24576 raft_consensus.cc:740] T 05840bb250764e02998cdddc290e5139 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.052109 24576 consensus_queue.cc:260] T 05840bb250764e02998cdddc290e5139 P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:10.054615 24576 ts_tablet_manager.cc:1434] T 05840bb250764e02998cdddc290e5139 P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.028s	user 0.003s	sys 0.003s
W20260709 13:09:10.031044 24345 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:33251, user_credentials={real_user=slave}} blocked reactor thread for 77702.2us
I20260709 13:09:10.055518 24579 raft_consensus.cc:359] T 722754340deb4df18b716dc801c35c34 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:10.058965 24575 tablet_bootstrap.cc:492] T 5bb19353ba334eac831ae2cce277298a P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:10.059474 24575 ts_tablet_manager.cc:1403] T 5bb19353ba334eac831ae2cce277298a P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.199s	user 0.011s	sys 0.021s
I20260709 13:09:10.061007 24577 consensus_queue.cc:260] T f4e53ea5cf694db697fa0e474da54423 P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:10.062283 24575 raft_consensus.cc:359] T 5bb19353ba334eac831ae2cce277298a P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:10.062969 24575 raft_consensus.cc:385] T 5bb19353ba334eac831ae2cce277298a P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.063238 24575 raft_consensus.cc:740] T 5bb19353ba334eac831ae2cce277298a P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.063369 24577 ts_tablet_manager.cc:1434] T f4e53ea5cf694db697fa0e474da54423 P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.042s	user 0.004s	sys 0.002s
I20260709 13:09:10.063660 24378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2039124a6d8848149440a70100af8755" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:10.064272 24378 raft_consensus.cc:3060] T 2039124a6d8848149440a70100af8755 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.064479 24577 tablet_bootstrap.cc:492] T 3e71fd85d9424a08aed292991bccf5ee P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:10.056272 24579 raft_consensus.cc:385] T 722754340deb4df18b716dc801c35c34 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.067862 24579 raft_consensus.cc:740] T 722754340deb4df18b716dc801c35c34 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.067950 24576 tablet_bootstrap.cc:492] T ae6c1a13de5442c5bb4e58e7ce36e741 P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:10.068604 24579 consensus_queue.cc:260] T 722754340deb4df18b716dc801c35c34 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:10.071707 24579 ts_tablet_manager.cc:1434] T 722754340deb4df18b716dc801c35c34 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.052s	user 0.002s	sys 0.003s
I20260709 13:09:10.072798 24579 tablet_bootstrap.cc:492] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:10.074096 24577 tablet_bootstrap.cc:654] T 3e71fd85d9424a08aed292991bccf5ee P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:10.078434 24575 consensus_queue.cc:260] T 5bb19353ba334eac831ae2cce277298a P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:10.080120 24576 tablet_bootstrap.cc:654] T ae6c1a13de5442c5bb4e58e7ce36e741 P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:10.080639 24575 ts_tablet_manager.cc:1434] T 5bb19353ba334eac831ae2cce277298a P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:09:10.081794 24575 tablet_bootstrap.cc:492] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:10.083850 24630 consensus_queue.cc:1048] T 05840bb250764e02998cdddc290e5139 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:10.095495 24579 tablet_bootstrap.cc:654] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:10.099489 24680 consensus_queue.cc:1048] T 9b7cd923257143648cfe6a296e18b642 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:10.108757 24575 tablet_bootstrap.cc:654] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:10.118218 24378 raft_consensus.cc:2468] T 2039124a6d8848149440a70100af8755 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 1.
I20260709 13:09:10.122274 24579 tablet_bootstrap.cc:492] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:10.122804 24579 ts_tablet_manager.cc:1403] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.050s	user 0.008s	sys 0.003s
I20260709 13:09:10.126075 24579 raft_consensus.cc:359] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:10.126760 24579 raft_consensus.cc:385] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.127024 24579 raft_consensus.cc:740] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.133719 24577 tablet_bootstrap.cc:492] T 3e71fd85d9424a08aed292991bccf5ee P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:10.133868 24579 consensus_queue.cc:260] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:10.134371 24577 ts_tablet_manager.cc:1403] T 3e71fd85d9424a08aed292991bccf5ee P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.070s	user 0.016s	sys 0.018s
I20260709 13:09:10.133620 24493 leader_election.cc:304] T 2039124a6d8848149440a70100af8755 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449
I20260709 13:09:10.137418 24696 raft_consensus.cc:493] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.138940 24772 raft_consensus.cc:2804] T 2039124a6d8848149440a70100af8755 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:10.138839 24696 raft_consensus.cc:515] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:10.139473 24772 raft_consensus.cc:697] T 2039124a6d8848149440a70100af8755 P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Becoming Leader. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Running, Role: LEADER
I20260709 13:09:10.140681 24575 tablet_bootstrap.cc:492] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:10.141156 24575 ts_tablet_manager.cc:1403] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.060s	user 0.011s	sys 0.000s
I20260709 13:09:10.140379 24772 consensus_queue.cc:237] T 2039124a6d8848149440a70100af8755 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:10.140978 24201 catalog_manager.cc:5697] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 reported cstate change: term changed from 0 to 1, leader changed from <none> to 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4). New cstate: current_term: 1 leader_uuid: "838cb9997b0f437989d3f4e3e9904390" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:10.146636 24696 leader_election.cc:290] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:10.153069 24456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90d9e64f97544895b9b97dfdc5ceb359" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:10.153682 24576 tablet_bootstrap.cc:492] T ae6c1a13de5442c5bb4e58e7ce36e741 P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:10.153363 24575 raft_consensus.cc:359] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:10.154103 24575 raft_consensus.cc:385] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.154233 24576 ts_tablet_manager.cc:1403] T ae6c1a13de5442c5bb4e58e7ce36e741 P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.099s	user 0.023s	sys 0.015s
I20260709 13:09:10.154495 24575 raft_consensus.cc:740] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.157471 24576 raft_consensus.cc:359] T ae6c1a13de5442c5bb4e58e7ce36e741 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:10.158614 24579 ts_tablet_manager.cc:1434] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.000s
W20260709 13:09:10.159056 24270 leader_election.cc:343] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:10.158619 24576 raft_consensus.cc:385] T ae6c1a13de5442c5bb4e58e7ce36e741 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.159898 24576 raft_consensus.cc:740] T ae6c1a13de5442c5bb4e58e7ce36e741 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.160949 24576 consensus_queue.cc:260] T ae6c1a13de5442c5bb4e58e7ce36e741 P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:10.162124 24579 tablet_bootstrap.cc:492] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:10.164213 24575 consensus_queue.cc:260] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:10.169452 24575 ts_tablet_manager.cc:1434] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.000s
I20260709 13:09:10.170519 24575 tablet_bootstrap.cc:492] T 3e71fd85d9424a08aed292991bccf5ee P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:10.173067 24579 tablet_bootstrap.cc:654] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:10.178673 24378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90d9e64f97544895b9b97dfdc5ceb359" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:10.179397 24378 raft_consensus.cc:2468] T 90d9e64f97544895b9b97dfdc5ceb359 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 0.
I20260709 13:09:10.185503 24576 ts_tablet_manager.cc:1434] T ae6c1a13de5442c5bb4e58e7ce36e741 P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.031s	user 0.009s	sys 0.001s
I20260709 13:09:10.186445 24577 raft_consensus.cc:359] T 3e71fd85d9424a08aed292991bccf5ee P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:10.187323 24577 raft_consensus.cc:385] T 3e71fd85d9424a08aed292991bccf5ee P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.187599 24577 raft_consensus.cc:740] T 3e71fd85d9424a08aed292991bccf5ee P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.188282 24577 consensus_queue.cc:260] T 3e71fd85d9424a08aed292991bccf5ee P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:10.190519 24577 ts_tablet_manager.cc:1434] T 3e71fd85d9424a08aed292991bccf5ee P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.056s	user 0.004s	sys 0.000s
I20260709 13:09:10.191251 24576 tablet_bootstrap.cc:492] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:10.191505 24577 tablet_bootstrap.cc:492] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:10.192997 24205 catalog_manager.cc:5697] T 2039124a6d8848149440a70100af8755 P 838cb9997b0f437989d3f4e3e9904390 reported cstate change: term changed from 0 to 1, leader changed from <none> to 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4). New cstate: current_term: 1 leader_uuid: "838cb9997b0f437989d3f4e3e9904390" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:10.198338 24577 tablet_bootstrap.cc:654] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:10.199609 24576 tablet_bootstrap.cc:654] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:10.196358 24680 raft_consensus.cc:493] T ae6c1a13de5442c5bb4e58e7ce36e741 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.200284 24644 raft_consensus.cc:493] T 05840bb250764e02998cdddc290e5139 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.200451 24680 raft_consensus.cc:515] T ae6c1a13de5442c5bb4e58e7ce36e741 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:10.200811 24644 raft_consensus.cc:515] T 05840bb250764e02998cdddc290e5139 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:10.203205 24644 leader_election.cc:290] T 05840bb250764e02998cdddc290e5139 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:10.206033 24530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05840bb250764e02998cdddc290e5139" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:10.207768 24382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05840bb250764e02998cdddc290e5139" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:10.207823 24528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae6c1a13de5442c5bb4e58e7ce36e741" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:10.205910 24381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae6c1a13de5442c5bb4e58e7ce36e741" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:10.210834 24381 raft_consensus.cc:2393] T ae6c1a13de5442c5bb4e58e7ce36e741 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 32088c942fbb48239e0eed952d4df449 in current term 1: Already voted for candidate 838cb9997b0f437989d3f4e3e9904390 in this term.
I20260709 13:09:10.213320 24579 tablet_bootstrap.cc:492] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:10.213816 24579 ts_tablet_manager.cc:1403] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.054s	user 0.013s	sys 0.001s
I20260709 13:09:10.216791 24579 raft_consensus.cc:359] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:10.217463 24579 raft_consensus.cc:385] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.217733 24579 raft_consensus.cc:740] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.218379 24419 leader_election.cc:304] T 05840bb250764e02998cdddc290e5139 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32088c942fbb48239e0eed952d4df449; no voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:10.218395 24579 consensus_queue.cc:260] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:10.220263 24419 leader_election.cc:304] T ae6c1a13de5442c5bb4e58e7ce36e741 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32088c942fbb48239e0eed952d4df449; no voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:10.220712 24642 raft_consensus.cc:2749] T 05840bb250764e02998cdddc290e5139 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:10.221412 24642 raft_consensus.cc:3060] T ae6c1a13de5442c5bb4e58e7ce36e741 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.221541 24579 ts_tablet_manager.cc:1434] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.005s
I20260709 13:09:10.222712 24579 tablet_bootstrap.cc:492] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:10.223124 24270 leader_election.cc:304] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449
I20260709 13:09:10.225261 24647 raft_consensus.cc:2804] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:10.225641 24647 raft_consensus.cc:493] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:10.226017 24647 raft_consensus.cc:3060] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.231773 24642 raft_consensus.cc:2749] T ae6c1a13de5442c5bb4e58e7ce36e741 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:10.236367 24647 raft_consensus.cc:515] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:10.237385 24579 tablet_bootstrap.cc:654] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:10.237751 24630 consensus_queue.cc:1048] T ae6c1a13de5442c5bb4e58e7ce36e741 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:09:10.269395 24647 leader_election.cc:290] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Requested vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:10.269902 24381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90d9e64f97544895b9b97dfdc5ceb359" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:10.270533 24381 raft_consensus.cc:3060] T 90d9e64f97544895b9b97dfdc5ceb359 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.197402 24575 tablet_bootstrap.cc:654] T 3e71fd85d9424a08aed292991bccf5ee P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:10.203364 24680 leader_election.cc:290] T ae6c1a13de5442c5bb4e58e7ce36e741 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:10.274376 24772 consensus_queue.cc:1048] T ae6c1a13de5442c5bb4e58e7ce36e741 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:10.250692 24756 consensus_queue.cc:1048] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:10.276628 24690 consensus_queue.cc:1048] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:10.263445 24609 raft_consensus.cc:493] T f4e53ea5cf694db697fa0e474da54423 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.278733 24609 raft_consensus.cc:515] T f4e53ea5cf694db697fa0e474da54423 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:10.278287 24576 tablet_bootstrap.cc:492] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:10.280493 24576 ts_tablet_manager.cc:1403] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.089s	user 0.019s	sys 0.023s
I20260709 13:09:10.283133 24454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4e53ea5cf694db697fa0e474da54423" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
W20260709 13:09:10.285408 24345 leader_election.cc:343] T f4e53ea5cf694db697fa0e474da54423 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:10.287230 24528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4e53ea5cf694db697fa0e474da54423" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:10.281056 24609 leader_election.cc:290] T f4e53ea5cf694db697fa0e474da54423 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
W20260709 13:09:10.289400 24345 leader_election.cc:343] T f4e53ea5cf694db697fa0e474da54423 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:10.290030 24345 leader_election.cc:304] T f4e53ea5cf694db697fa0e474da54423 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449, 838cb9997b0f437989d3f4e3e9904390
I20260709 13:09:10.291406 24609 raft_consensus.cc:2749] T f4e53ea5cf694db697fa0e474da54423 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:10.296387 24381 raft_consensus.cc:2468] T 90d9e64f97544895b9b97dfdc5ceb359 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 1.
W20260709 13:09:10.297508 24419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:46177, user_credentials={real_user=slave}} blocked reactor thread for 76403.2us
I20260709 13:09:10.306627 24270 leader_election.cc:304] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:10.285112 24576 raft_consensus.cc:359] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:10.315294 24576 raft_consensus.cc:385] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.316465 24576 raft_consensus.cc:740] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.323029 24454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90d9e64f97544895b9b97dfdc5ceb359" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:10.319274 24696 raft_consensus.cc:2804] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:10.326351 24696 raft_consensus.cc:697] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Becoming Leader. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Running, Role: LEADER
I20260709 13:09:10.328960 24696 consensus_queue.cc:237] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:10.331619 24576 consensus_queue.cc:260] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:10.334843 24454 raft_consensus.cc:3060] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.349788 24454 raft_consensus.cc:2468] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 1.
I20260709 13:09:10.350697 24756 raft_consensus.cc:493] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.354655 24756 raft_consensus.cc:515] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:10.358274 24756 leader_election.cc:290] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:10.358355 24576 ts_tablet_manager.cc:1434] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.077s	user 0.008s	sys 0.000s
I20260709 13:09:10.360098 24382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bb19353ba334eac831ae2cce277298a" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:10.360858 24382 raft_consensus.cc:2468] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 0.
I20260709 13:09:10.361413 24576 tablet_bootstrap.cc:492] T 2039124a6d8848149440a70100af8755 P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:10.361836 24529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bb19353ba334eac831ae2cce277298a" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:10.362453 24529 raft_consensus.cc:2468] T 5bb19353ba334eac831ae2cce277298a P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 0.
I20260709 13:09:10.364053 24270 leader_election.cc:304] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381; no voters: 
I20260709 13:09:10.365298 24575 tablet_bootstrap.cc:492] T 3e71fd85d9424a08aed292991bccf5ee P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:10.365811 24575 ts_tablet_manager.cc:1403] T 3e71fd85d9424a08aed292991bccf5ee P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.196s	user 0.010s	sys 0.009s
I20260709 13:09:10.368009 24756 raft_consensus.cc:2804] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:10.368767 24575 raft_consensus.cc:359] T 3e71fd85d9424a08aed292991bccf5ee P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:10.369496 24575 raft_consensus.cc:385] T 3e71fd85d9424a08aed292991bccf5ee P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.370052 24577 tablet_bootstrap.cc:492] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:10.370576 24577 ts_tablet_manager.cc:1403] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.179s	user 0.028s	sys 0.035s
I20260709 13:09:10.372076 24306 raft_consensus.cc:3060] T 722754340deb4df18b716dc801c35c34 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.368743 24756 raft_consensus.cc:493] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:10.373405 24577 raft_consensus.cc:359] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:10.373965 24756 raft_consensus.cc:3060] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.374243 24577 raft_consensus.cc:385] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.374658 24577 raft_consensus.cc:740] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.375461 24577 consensus_queue.cc:260] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:10.380287 24205 catalog_manager.cc:5697] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "9c2a14143eb744cf83bfa5f46284a381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:10.384708 24577 ts_tablet_manager.cc:1434] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:09:10.388815 24579 tablet_bootstrap.cc:492] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:10.389416 24579 ts_tablet_manager.cc:1403] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.167s	user 0.030s	sys 0.028s
I20260709 13:09:10.392277 24756 raft_consensus.cc:515] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:10.393002 24579 raft_consensus.cc:359] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:10.393678 24579 raft_consensus.cc:385] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.393831 24454 raft_consensus.cc:3060] T 05840bb250764e02998cdddc290e5139 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.393980 24579 raft_consensus.cc:740] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.394335 24756 leader_election.cc:290] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Requested vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:10.394794 24579 consensus_queue.cc:260] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:10.369904 24575 raft_consensus.cc:740] T 3e71fd85d9424a08aed292991bccf5ee P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.395562 24772 consensus_queue.cc:1048] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:10.397552 24528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bb19353ba334eac831ae2cce277298a" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:10.397917 24579 ts_tablet_manager.cc:1434] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.004s
I20260709 13:09:10.398181 24528 raft_consensus.cc:3060] T 5bb19353ba334eac831ae2cce277298a P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.396592 24576 tablet_bootstrap.cc:654] T 2039124a6d8848149440a70100af8755 P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:10.399040 24579 tablet_bootstrap.cc:492] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:10.402064 24575 consensus_queue.cc:260] T 3e71fd85d9424a08aed292991bccf5ee P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:10.405192 24579 tablet_bootstrap.cc:654] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:10.409055 24575 ts_tablet_manager.cc:1434] T 3e71fd85d9424a08aed292991bccf5ee P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.043s	user 0.005s	sys 0.001s
I20260709 13:09:10.410147 24575 tablet_bootstrap.cc:492] T f4e53ea5cf694db697fa0e474da54423 P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:10.420302 24528 raft_consensus.cc:2468] T 5bb19353ba334eac831ae2cce277298a P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 1.
I20260709 13:09:10.422660 24380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bb19353ba334eac831ae2cce277298a" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:10.423700 24577 tablet_bootstrap.cc:492] T 40dd44ec1fb347f49c31a8b650ab1639 P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:10.427085 24380 raft_consensus.cc:3060] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.433137 24270 leader_election.cc:304] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381; no voters: 
I20260709 13:09:10.435093 24647 raft_consensus.cc:2804] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:10.437757 24647 raft_consensus.cc:697] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Becoming Leader. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Running, Role: LEADER
I20260709 13:09:10.440932 24647 consensus_queue.cc:237] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:10.479931 24577 tablet_bootstrap.cc:654] T 40dd44ec1fb347f49c31a8b650ab1639 P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:10.490355 24380 raft_consensus.cc:2468] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 1.
I20260709 13:09:10.485464 24649 raft_consensus.cc:493] T 3e71fd85d9424a08aed292991bccf5ee P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.504210 24649 raft_consensus.cc:515] T 3e71fd85d9424a08aed292991bccf5ee P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:10.493485 24630 consensus_queue.cc:1048] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:10.506595 24649 leader_election.cc:290] T 3e71fd85d9424a08aed292991bccf5ee P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:10.485994 24575 tablet_bootstrap.cc:654] T f4e53ea5cf694db697fa0e474da54423 P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:10.507804 24382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e71fd85d9424a08aed292991bccf5ee" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:10.508492 24382 raft_consensus.cc:2468] T 3e71fd85d9424a08aed292991bccf5ee P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 0.
I20260709 13:09:10.509675 24493 leader_election.cc:304] T 3e71fd85d9424a08aed292991bccf5ee P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:10.510612 24649 raft_consensus.cc:2804] T 3e71fd85d9424a08aed292991bccf5ee P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:10.511107 24649 raft_consensus.cc:493] T 3e71fd85d9424a08aed292991bccf5ee P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:10.511947 24649 raft_consensus.cc:3060] T 3e71fd85d9424a08aed292991bccf5ee P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.514479 24306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e71fd85d9424a08aed292991bccf5ee" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:10.496142 24579 tablet_bootstrap.cc:492] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:10.498095 24577 tablet_bootstrap.cc:492] T 40dd44ec1fb347f49c31a8b650ab1639 P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:10.499727 24205 catalog_manager.cc:5697] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "9c2a14143eb744cf83bfa5f46284a381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:10.502135 24576 tablet_bootstrap.cc:492] T 2039124a6d8848149440a70100af8755 P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
W20260709 13:09:10.516530 24494 leader_election.cc:343] T 3e71fd85d9424a08aed292991bccf5ee P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:10.517009 24576 ts_tablet_manager.cc:1403] T 2039124a6d8848149440a70100af8755 P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.156s	user 0.025s	sys 0.033s
I20260709 13:09:10.516561 24577 ts_tablet_manager.cc:1403] T 40dd44ec1fb347f49c31a8b650ab1639 P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.131s	user 0.012s	sys 0.006s
I20260709 13:09:10.519908 24576 raft_consensus.cc:359] T 2039124a6d8848149440a70100af8755 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:10.520612 24576 raft_consensus.cc:385] T 2039124a6d8848149440a70100af8755 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.520890 24576 raft_consensus.cc:740] T 2039124a6d8848149440a70100af8755 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.521656 24576 consensus_queue.cc:260] T 2039124a6d8848149440a70100af8755 P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:10.523775 24577 raft_consensus.cc:359] T 40dd44ec1fb347f49c31a8b650ab1639 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:10.524806 24577 raft_consensus.cc:385] T 40dd44ec1fb347f49c31a8b650ab1639 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.525089 24577 raft_consensus.cc:740] T 40dd44ec1fb347f49c31a8b650ab1639 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.525727 24577 consensus_queue.cc:260] T 40dd44ec1fb347f49c31a8b650ab1639 P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:10.526590 24576 ts_tablet_manager.cc:1434] T 2039124a6d8848149440a70100af8755 P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 13:09:10.527685 24576 tablet_bootstrap.cc:492] T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:10.528568 24577 ts_tablet_manager.cc:1434] T 40dd44ec1fb347f49c31a8b650ab1639 P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:09:10.529536 24577 tablet_bootstrap.cc:492] T a1b91f42722c4116950ce80bc02eccd0 P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:10.532930 24575 tablet_bootstrap.cc:492] T f4e53ea5cf694db697fa0e474da54423 P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:10.533411 24576 tablet_bootstrap.cc:654] T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:10.533586 24575 ts_tablet_manager.cc:1403] T f4e53ea5cf694db697fa0e474da54423 P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.124s	user 0.016s	sys 0.003s
I20260709 13:09:10.518921 24579 ts_tablet_manager.cc:1403] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.120s	user 0.021s	sys 0.015s
I20260709 13:09:10.537230 24575 raft_consensus.cc:359] T f4e53ea5cf694db697fa0e474da54423 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:10.538640 24575 raft_consensus.cc:385] T f4e53ea5cf694db697fa0e474da54423 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.539101 24575 raft_consensus.cc:740] T f4e53ea5cf694db697fa0e474da54423 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.543401 24575 consensus_queue.cc:260] T f4e53ea5cf694db697fa0e474da54423 P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:10.543653 24579 raft_consensus.cc:359] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:10.544875 24579 raft_consensus.cc:385] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.544935 24649 raft_consensus.cc:515] T 3e71fd85d9424a08aed292991bccf5ee P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:10.547258 24649 leader_election.cc:290] T 3e71fd85d9424a08aed292991bccf5ee P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Requested vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:10.535537 24577 tablet_bootstrap.cc:654] T a1b91f42722c4116950ce80bc02eccd0 P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:10.551503 24576 tablet_bootstrap.cc:492] T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:10.545454 24579 raft_consensus.cc:740] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.552131 24576 ts_tablet_manager.cc:1403] T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.004s
I20260709 13:09:10.552531 24380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e71fd85d9424a08aed292991bccf5ee" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:10.553172 24380 raft_consensus.cc:3060] T 3e71fd85d9424a08aed292991bccf5ee P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.553592 24575 ts_tablet_manager.cc:1434] T f4e53ea5cf694db697fa0e474da54423 P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 13:09:10.552753 24579 consensus_queue.cc:260] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:10.554941 24306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e71fd85d9424a08aed292991bccf5ee" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
W20260709 13:09:10.556324 24494 leader_election.cc:343] T 3e71fd85d9424a08aed292991bccf5ee P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:10.556816 24575 tablet_bootstrap.cc:492] T 72e12f99e2084675ab3f53ffb836e464 P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:10.555577 24576 raft_consensus.cc:359] T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:10.559372 24576 raft_consensus.cc:385] T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.559403 24579 ts_tablet_manager.cc:1434] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.023s	user 0.003s	sys 0.005s
I20260709 13:09:10.559790 24576 raft_consensus.cc:740] T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.560703 24579 tablet_bootstrap.cc:492] T 3e71fd85d9424a08aed292991bccf5ee P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:10.560878 24576 consensus_queue.cc:260] T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:10.567504 24576 ts_tablet_manager.cc:1434] T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:09:10.568593 24576 tablet_bootstrap.cc:492] T 72e12f99e2084675ab3f53ffb836e464 P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:10.570510 24579 tablet_bootstrap.cc:654] T 3e71fd85d9424a08aed292991bccf5ee P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:10.575656 24575 tablet_bootstrap.cc:654] T 72e12f99e2084675ab3f53ffb836e464 P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:10.581444 24576 tablet_bootstrap.cc:654] T 72e12f99e2084675ab3f53ffb836e464 P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:10.593161 24792 raft_consensus.cc:493] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.594316 24792 raft_consensus.cc:515] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:10.591656 24380 raft_consensus.cc:2468] T 3e71fd85d9424a08aed292991bccf5ee P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 1.
I20260709 13:09:10.594420 24630 consensus_queue.cc:1048] T 2039124a6d8848149440a70100af8755 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:10.597594 24493 leader_election.cc:304] T 3e71fd85d9424a08aed292991bccf5ee P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:10.599843 24306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ac163496084a3e977344fa9228a3c9" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
W20260709 13:09:10.601905 24346 leader_election.cc:343] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:10.602869 24630 raft_consensus.cc:2804] T 3e71fd85d9424a08aed292991bccf5ee P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:10.603966 24630 raft_consensus.cc:697] T 3e71fd85d9424a08aed292991bccf5ee P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Becoming Leader. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Running, Role: LEADER
I20260709 13:09:10.604863 24758 raft_consensus.cc:493] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.605585 24758 raft_consensus.cc:515] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:10.605225 24630 consensus_queue.cc:237] T 3e71fd85d9424a08aed292991bccf5ee P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:10.609653 24680 raft_consensus.cc:493] T 2039124a6d8848149440a70100af8755 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.610203 24680 raft_consensus.cc:515] T 2039124a6d8848149440a70100af8755 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:10.612149 24680 leader_election.cc:290] T 2039124a6d8848149440a70100af8755 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:10.612078 24792 leader_election.cc:290] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:10.613152 24792 raft_consensus.cc:493] T 40dd44ec1fb347f49c31a8b650ab1639 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.613814 24792 raft_consensus.cc:515] T 40dd44ec1fb347f49c31a8b650ab1639 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:10.615530 24381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2039124a6d8848149440a70100af8755" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:10.618386 24528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40dd44ec1fb347f49c31a8b650ab1639" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:10.619707 24529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2039124a6d8848149440a70100af8755" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
W20260709 13:09:10.620734 24345 leader_election.cc:343] T 40dd44ec1fb347f49c31a8b650ab1639 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:10.622025 24647 raft_consensus.cc:493] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.622726 24647 raft_consensus.cc:515] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:10.624125 24306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40dd44ec1fb347f49c31a8b650ab1639" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
W20260709 13:09:10.626190 24346 leader_election.cc:343] T 40dd44ec1fb347f49c31a8b650ab1639 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:10.626894 24346 leader_election.cc:304] T 40dd44ec1fb347f49c31a8b650ab1639 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:10.628069 24744 raft_consensus.cc:2749] T 40dd44ec1fb347f49c31a8b650ab1639 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:10.628773 24456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6427fa143f574eaba2ee4c979e5485fc" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:10.631098 24530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6427fa143f574eaba2ee4c979e5485fc" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:10.632055 24530 raft_consensus.cc:2468] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 0.
I20260709 13:09:10.634557 24647 leader_election.cc:290] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:10.637290 24792 leader_election.cc:290] T 40dd44ec1fb347f49c31a8b650ab1639 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:10.638697 24575 tablet_bootstrap.cc:492] T 72e12f99e2084675ab3f53ffb836e464 P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:10.639264 24575 ts_tablet_manager.cc:1403] T 72e12f99e2084675ab3f53ffb836e464 P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.085s	user 0.010s	sys 0.001s
I20260709 13:09:10.641466 24579 tablet_bootstrap.cc:492] T 3e71fd85d9424a08aed292991bccf5ee P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:10.641933 24579 ts_tablet_manager.cc:1403] T 3e71fd85d9424a08aed292991bccf5ee P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.081s	user 0.011s	sys 0.004s
I20260709 13:09:10.641999 24575 raft_consensus.cc:359] T 72e12f99e2084675ab3f53ffb836e464 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:10.642740 24575 raft_consensus.cc:385] T 72e12f99e2084675ab3f53ffb836e464 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.642983 24575 raft_consensus.cc:740] T 72e12f99e2084675ab3f53ffb836e464 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.636837 24270 leader_election.cc:304] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381; no voters: 
W20260709 13:09:10.644326 24270 leader_election.cc:343] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:10.643782 24575 consensus_queue.cc:260] T 72e12f99e2084675ab3f53ffb836e464 P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:10.644776 24579 raft_consensus.cc:359] T 3e71fd85d9424a08aed292991bccf5ee P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:10.645426 24579 raft_consensus.cc:385] T 3e71fd85d9424a08aed292991bccf5ee P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.645598 24696 raft_consensus.cc:2804] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:10.645673 24579 raft_consensus.cc:740] T 3e71fd85d9424a08aed292991bccf5ee P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.646111 24696 raft_consensus.cc:493] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:10.646600 24696 raft_consensus.cc:3060] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.646663 24579 consensus_queue.cc:260] T 3e71fd85d9424a08aed292991bccf5ee P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:10.646312 24575 ts_tablet_manager.cc:1434] T 72e12f99e2084675ab3f53ffb836e464 P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:09:10.649127 24577 tablet_bootstrap.cc:492] T a1b91f42722c4116950ce80bc02eccd0 P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:10.649487 24575 tablet_bootstrap.cc:492] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:10.649596 24577 ts_tablet_manager.cc:1403] T a1b91f42722c4116950ce80bc02eccd0 P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.120s	user 0.021s	sys 0.018s
I20260709 13:09:10.650933 24576 tablet_bootstrap.cc:492] T 72e12f99e2084675ab3f53ffb836e464 P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:10.651422 24576 ts_tablet_manager.cc:1403] T 72e12f99e2084675ab3f53ffb836e464 P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.083s	user 0.035s	sys 0.026s
I20260709 13:09:10.652992 24577 raft_consensus.cc:359] T a1b91f42722c4116950ce80bc02eccd0 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:10.653914 24577 raft_consensus.cc:385] T a1b91f42722c4116950ce80bc02eccd0 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.654103 24758 leader_election.cc:290] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:10.654268 24577 raft_consensus.cc:740] T a1b91f42722c4116950ce80bc02eccd0 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.654435 24456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6427fa143f574eaba2ee4c979e5485fc" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:10.655108 24577 consensus_queue.cc:260] T a1b91f42722c4116950ce80bc02eccd0 P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:10.656853 24306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6427fa143f574eaba2ee4c979e5485fc" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:10.657985 24577 ts_tablet_manager.cc:1434] T a1b91f42722c4116950ce80bc02eccd0 P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
W20260709 13:09:10.661680 24493 leader_election.cc:343] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:10.662777 24575 tablet_bootstrap.cc:654] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:10.663342 24419 leader_election.cc:304] T 2039124a6d8848149440a70100af8755 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32088c942fbb48239e0eed952d4df449; no voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:10.664630 24577 tablet_bootstrap.cc:492] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:10.658766 24579 ts_tablet_manager.cc:1434] T 3e71fd85d9424a08aed292991bccf5ee P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:09:10.665210 24576 raft_consensus.cc:359] T 72e12f99e2084675ab3f53ffb836e464 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:10.665910 24576 raft_consensus.cc:385] T 72e12f99e2084675ab3f53ffb836e464 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.666189 24576 raft_consensus.cc:740] T 72e12f99e2084675ab3f53ffb836e464 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.666322 24579 tablet_bootstrap.cc:492] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:10.666872 24576 consensus_queue.cc:260] T 72e12f99e2084675ab3f53ffb836e464 P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:10.670614 24812 raft_consensus.cc:2749] T 2039124a6d8848149440a70100af8755 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:10.676044 24454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ac163496084a3e977344fa9228a3c9" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
W20260709 13:09:10.677858 24345 leader_election.cc:343] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:10.678309 24345 leader_election.cc:304] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:10.679029 24744 raft_consensus.cc:2749] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:10.681345 24579 tablet_bootstrap.cc:654] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:10.684111 24576 ts_tablet_manager.cc:1434] T 72e12f99e2084675ab3f53ffb836e464 P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.032s	user 0.003s	sys 0.003s
I20260709 13:09:10.685204 24576 tablet_bootstrap.cc:492] T 6427fa143f574eaba2ee4c979e5485fc P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:10.685811 24575 tablet_bootstrap.cc:492] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:10.686503 24575 ts_tablet_manager.cc:1403] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.037s	user 0.008s	sys 0.004s
I20260709 13:09:10.685864 24577 tablet_bootstrap.cc:654] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:10.690958 24576 tablet_bootstrap.cc:654] T 6427fa143f574eaba2ee4c979e5485fc P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:10.691926 24454 raft_consensus.cc:3060] T 2039124a6d8848149440a70100af8755 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.661906 24201 catalog_manager.cc:5697] T 3e71fd85d9424a08aed292991bccf5ee P 838cb9997b0f437989d3f4e3e9904390 reported cstate change: term changed from 0 to 1, leader changed from <none> to 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4). New cstate: current_term: 1 leader_uuid: "838cb9997b0f437989d3f4e3e9904390" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:10.698879 24696 raft_consensus.cc:515] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:10.705080 24306 raft_consensus.cc:2393] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 838cb9997b0f437989d3f4e3e9904390 in current term 1: Already voted for candidate 9c2a14143eb744cf83bfa5f46284a381 in this term.
I20260709 13:09:10.691999 24575 raft_consensus.cc:359] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:10.706929 24575 raft_consensus.cc:385] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.707290 24577 tablet_bootstrap.cc:492] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:10.707854 24577 ts_tablet_manager.cc:1403] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.043s	user 0.013s	sys 0.005s
I20260709 13:09:10.708649 24696 leader_election.cc:290] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Requested vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:10.707113 24494 leader_election.cc:304] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390; no voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:10.710553 24577 raft_consensus.cc:359] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:10.711050 24630 raft_consensus.cc:3060] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.711311 24577 raft_consensus.cc:385] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.711712 24577 raft_consensus.cc:740] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.707293 24575 raft_consensus.cc:740] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.712430 24758 consensus_queue.cc:1048] T 2039124a6d8848149440a70100af8755 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:10.712951 24576 tablet_bootstrap.cc:492] T 6427fa143f574eaba2ee4c979e5485fc P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:10.712836 24575 consensus_queue.cc:260] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:10.713447 24576 ts_tablet_manager.cc:1403] T 6427fa143f574eaba2ee4c979e5485fc P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.028s	user 0.007s	sys 0.012s
I20260709 13:09:10.715552 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6427fa143f574eaba2ee4c979e5485fc" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:10.716279 24530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6427fa143f574eaba2ee4c979e5485fc" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
W20260709 13:09:10.717139 24270 leader_election.cc:343] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:10.717892 24576 raft_consensus.cc:359] T 6427fa143f574eaba2ee4c979e5485fc P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:10.718544 24576 raft_consensus.cc:385] T 6427fa143f574eaba2ee4c979e5485fc P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.722400 24576 raft_consensus.cc:740] T 6427fa143f574eaba2ee4c979e5485fc P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.725857 24630 raft_consensus.cc:2749] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:10.724838 24576 consensus_queue.cc:260] T 6427fa143f574eaba2ee4c979e5485fc P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:10.726291 24575 ts_tablet_manager.cc:1434] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.000s
I20260709 13:09:10.727598 24575 tablet_bootstrap.cc:492] T 2deba793363c4afca7d75b2fc3814304 P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:10.728149 24576 ts_tablet_manager.cc:1434] T 6427fa143f574eaba2ee4c979e5485fc P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 13:09:10.729144 24576 tablet_bootstrap.cc:492] T 2deba793363c4afca7d75b2fc3814304 P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:10.741242 24579 tablet_bootstrap.cc:492] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:10.743436 24577 consensus_queue.cc:260] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:10.744397 24579 ts_tablet_manager.cc:1403] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.078s	user 0.015s	sys 0.004s
I20260709 13:09:10.747537 24576 tablet_bootstrap.cc:654] T 2deba793363c4afca7d75b2fc3814304 P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:10.750694 24579 raft_consensus.cc:359] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:10.756119 24579 raft_consensus.cc:385] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.756486 24579 raft_consensus.cc:740] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.757215 24579 consensus_queue.cc:260] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:10.763051 24579 ts_tablet_manager.cc:1434] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 13:09:10.755470 24575 tablet_bootstrap.cc:654] T 2deba793363c4afca7d75b2fc3814304 P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:10.754518 24577 ts_tablet_manager.cc:1434] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.046s	user 0.006s	sys 0.000s
I20260709 13:09:10.768757 24576 tablet_bootstrap.cc:492] T 2deba793363c4afca7d75b2fc3814304 P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:10.781505 24577 tablet_bootstrap.cc:492] T 40cf5b30d13f4c3e86a9aa657c20c650 P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:10.781770 24579 tablet_bootstrap.cc:492] T 2deba793363c4afca7d75b2fc3814304 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:10.781962 24576 ts_tablet_manager.cc:1403] T 2deba793363c4afca7d75b2fc3814304 P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.053s	user 0.009s	sys 0.015s
I20260709 13:09:10.785626 24696 consensus_queue.cc:1048] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:10.785688 24576 raft_consensus.cc:359] T 2deba793363c4afca7d75b2fc3814304 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:10.786720 24576 raft_consensus.cc:385] T 2deba793363c4afca7d75b2fc3814304 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.786989 24576 raft_consensus.cc:740] T 2deba793363c4afca7d75b2fc3814304 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.736927 24530 raft_consensus.cc:2468] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 1.
I20260709 13:09:10.789150 24577 tablet_bootstrap.cc:654] T 40cf5b30d13f4c3e86a9aa657c20c650 P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:10.790261 24270 leader_election.cc:304] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381; no voters: 32088c942fbb48239e0eed952d4df449
I20260709 13:09:10.790776 24576 consensus_queue.cc:260] T 2deba793363c4afca7d75b2fc3814304 P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:10.791396 24696 raft_consensus.cc:2804] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:10.791747 24579 tablet_bootstrap.cc:654] T 2deba793363c4afca7d75b2fc3814304 P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:10.791966 24696 raft_consensus.cc:697] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Becoming Leader. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Running, Role: LEADER
I20260709 13:09:10.792908 24696 consensus_queue.cc:237] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:10.795948 24575 tablet_bootstrap.cc:492] T 2deba793363c4afca7d75b2fc3814304 P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:10.796589 24575 ts_tablet_manager.cc:1403] T 2deba793363c4afca7d75b2fc3814304 P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.069s	user 0.014s	sys 0.000s
I20260709 13:09:10.797472 24576 ts_tablet_manager.cc:1434] T 2deba793363c4afca7d75b2fc3814304 P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:09:10.798763 24576 tablet_bootstrap.cc:492] T f4e53ea5cf694db697fa0e474da54423 P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:10.800297 24575 raft_consensus.cc:359] T 2deba793363c4afca7d75b2fc3814304 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:10.801257 24575 raft_consensus.cc:385] T 2deba793363c4afca7d75b2fc3814304 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.801600 24575 raft_consensus.cc:740] T 2deba793363c4afca7d75b2fc3814304 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.802776 24575 consensus_queue.cc:260] T 2deba793363c4afca7d75b2fc3814304 P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:10.808128 24575 ts_tablet_manager.cc:1434] T 2deba793363c4afca7d75b2fc3814304 P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:09:10.809116 24575 tablet_bootstrap.cc:492] T a1b91f42722c4116950ce80bc02eccd0 P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:10.813740 24577 tablet_bootstrap.cc:492] T 40cf5b30d13f4c3e86a9aa657c20c650 P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:10.814283 24577 ts_tablet_manager.cc:1403] T 40cf5b30d13f4c3e86a9aa657c20c650 P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.033s	user 0.013s	sys 0.001s
I20260709 13:09:10.817960 24577 raft_consensus.cc:359] T 40cf5b30d13f4c3e86a9aa657c20c650 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:10.818854 24577 raft_consensus.cc:385] T 40cf5b30d13f4c3e86a9aa657c20c650 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.819296 24576 tablet_bootstrap.cc:654] T f4e53ea5cf694db697fa0e474da54423 P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:10.825438 24575 tablet_bootstrap.cc:654] T a1b91f42722c4116950ce80bc02eccd0 P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:10.819181 24577 raft_consensus.cc:740] T 40cf5b30d13f4c3e86a9aa657c20c650 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.829177 24577 consensus_queue.cc:260] T 40cf5b30d13f4c3e86a9aa657c20c650 P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:10.831984 24579 tablet_bootstrap.cc:492] T 2deba793363c4afca7d75b2fc3814304 P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:10.832559 24579 ts_tablet_manager.cc:1403] T 2deba793363c4afca7d75b2fc3814304 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.051s	user 0.018s	sys 0.010s
I20260709 13:09:10.835717 24579 raft_consensus.cc:359] T 2deba793363c4afca7d75b2fc3814304 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:10.836432 24579 raft_consensus.cc:385] T 2deba793363c4afca7d75b2fc3814304 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.836756 24579 raft_consensus.cc:740] T 2deba793363c4afca7d75b2fc3814304 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.837493 24579 consensus_queue.cc:260] T 2deba793363c4afca7d75b2fc3814304 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:10.841624 24201 catalog_manager.cc:5697] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "9c2a14143eb744cf83bfa5f46284a381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:10.847219 24756 consensus_queue.cc:1048] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:10.854399 24577 ts_tablet_manager.cc:1434] T 40cf5b30d13f4c3e86a9aa657c20c650 P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.040s	user 0.007s	sys 0.002s
I20260709 13:09:10.855526 24576 tablet_bootstrap.cc:492] T f4e53ea5cf694db697fa0e474da54423 P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:10.855655 24577 tablet_bootstrap.cc:492] T 84af43749e474087b0a5857a95dfb31d P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:10.856048 24576 ts_tablet_manager.cc:1403] T f4e53ea5cf694db697fa0e474da54423 P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.058s	user 0.013s	sys 0.015s
I20260709 13:09:10.864313 24579 ts_tablet_manager.cc:1434] T 2deba793363c4afca7d75b2fc3814304 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.031s	user 0.008s	sys 0.000s
I20260709 13:09:10.876187 24644 raft_consensus.cc:493] T 6427fa143f574eaba2ee4c979e5485fc P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.877041 24644 raft_consensus.cc:515] T 6427fa143f574eaba2ee4c979e5485fc P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:10.866626 24680 raft_consensus.cc:493] T 2deba793363c4afca7d75b2fc3814304 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.877842 24579 tablet_bootstrap.cc:492] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:10.880546 24530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6427fa143f574eaba2ee4c979e5485fc" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:10.881491 24530 raft_consensus.cc:2393] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 32088c942fbb48239e0eed952d4df449 in current term 1: Already voted for candidate 9c2a14143eb744cf83bfa5f46284a381 in this term.
I20260709 13:09:10.870810 24792 raft_consensus.cc:493] T 40cf5b30d13f4c3e86a9aa657c20c650 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.882570 24792 raft_consensus.cc:515] T 40cf5b30d13f4c3e86a9aa657c20c650 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:10.878711 24680 raft_consensus.cc:515] T 2deba793363c4afca7d75b2fc3814304 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:10.886076 24680 leader_election.cc:290] T 2deba793363c4afca7d75b2fc3814304 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:10.886835 24306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2deba793363c4afca7d75b2fc3814304" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:10.887513 24575 tablet_bootstrap.cc:492] T a1b91f42722c4116950ce80bc02eccd0 P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:10.887522 24306 raft_consensus.cc:2468] T 2deba793363c4afca7d75b2fc3814304 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 0.
I20260709 13:09:10.888249 24575 ts_tablet_manager.cc:1403] T a1b91f42722c4116950ce80bc02eccd0 P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.079s	user 0.012s	sys 0.021s
I20260709 13:09:10.888322 24307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40cf5b30d13f4c3e86a9aa657c20c650" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:10.889098 24420 leader_election.cc:304] T 2deba793363c4afca7d75b2fc3814304 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381; no voters: 
I20260709 13:09:10.889839 24577 tablet_bootstrap.cc:654] T 84af43749e474087b0a5857a95dfb31d P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
W20260709 13:09:10.891139 24346 leader_election.cc:343] T 40cf5b30d13f4c3e86a9aa657c20c650 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:10.889999 24680 raft_consensus.cc:2804] T 2deba793363c4afca7d75b2fc3814304 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:10.892066 24680 raft_consensus.cc:493] T 2deba793363c4afca7d75b2fc3814304 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:10.892402 24680 raft_consensus.cc:3060] T 2deba793363c4afca7d75b2fc3814304 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.892513 24690 consensus_queue.cc:1048] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:10.894364 24792 leader_election.cc:290] T 40cf5b30d13f4c3e86a9aa657c20c650 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:10.895489 24644 leader_election.cc:290] T 6427fa143f574eaba2ee4c979e5485fc P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:10.895565 24528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40cf5b30d13f4c3e86a9aa657c20c650" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:10.879989 24696 raft_consensus.cc:493] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.896425 24530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2deba793363c4afca7d75b2fc3814304" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:10.897202 24530 raft_consensus.cc:2468] T 2deba793363c4afca7d75b2fc3814304 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 0.
I20260709 13:09:10.896864 24696 raft_consensus.cc:515] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:10.897921 24307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6427fa143f574eaba2ee4c979e5485fc" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:10.900411 24420 leader_election.cc:304] T 6427fa143f574eaba2ee4c979e5485fc P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32088c942fbb48239e0eed952d4df449; no voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:10.896435 24579 tablet_bootstrap.cc:654] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
W20260709 13:09:10.901849 24345 leader_election.cc:343] T 40cf5b30d13f4c3e86a9aa657c20c650 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:10.903298 24345 leader_election.cc:304] T 40cf5b30d13f4c3e86a9aa657c20c650 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:10.905783 24792 raft_consensus.cc:2749] T 40cf5b30d13f4c3e86a9aa657c20c650 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:10.905923 24381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ae3dc25b8484b6f99d8f41685229cb8" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:10.906724 24381 raft_consensus.cc:2468] T 6ae3dc25b8484b6f99d8f41685229cb8 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 0.
I20260709 13:09:10.906874 24644 raft_consensus.cc:3060] T 6427fa143f574eaba2ee4c979e5485fc P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.911827 24577 tablet_bootstrap.cc:492] T 84af43749e474087b0a5857a95dfb31d P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:10.912436 24577 ts_tablet_manager.cc:1403] T 84af43749e474087b0a5857a95dfb31d P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.057s	user 0.015s	sys 0.004s
I20260709 13:09:10.912616 24270 leader_election.cc:304] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:10.914464 24756 raft_consensus.cc:2804] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:10.914888 24756 raft_consensus.cc:493] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:10.915385 24756 raft_consensus.cc:3060] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.917436 24577 raft_consensus.cc:359] T 84af43749e474087b0a5857a95dfb31d P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:10.921103 24575 raft_consensus.cc:359] T a1b91f42722c4116950ce80bc02eccd0 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:10.922133 24575 raft_consensus.cc:385] T a1b91f42722c4116950ce80bc02eccd0 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.921778 24577 raft_consensus.cc:385] T 84af43749e474087b0a5857a95dfb31d P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:10.922747 24577 raft_consensus.cc:740] T 84af43749e474087b0a5857a95dfb31d P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.923971 24577 consensus_queue.cc:260] T 84af43749e474087b0a5857a95dfb31d P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:10.927718 24577 ts_tablet_manager.cc:1434] T 84af43749e474087b0a5857a95dfb31d P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:09:10.931057 24577 tablet_bootstrap.cc:492] T 624f99e3165f49babbfce35effc9d627 P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:10.936658 24575 raft_consensus.cc:740] T a1b91f42722c4116950ce80bc02eccd0 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:10.937788 24854 consensus_queue.cc:1048] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:10.937719 24575 consensus_queue.cc:260] T a1b91f42722c4116950ce80bc02eccd0 P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:10.939818 24575 ts_tablet_manager.cc:1434] T a1b91f42722c4116950ce80bc02eccd0 P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.050s	user 0.007s	sys 0.000s
I20260709 13:09:10.940929 24575 tablet_bootstrap.cc:492] T 836ed40e5bb8485cb93cb03bc2508265 P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:10.944152 24577 tablet_bootstrap.cc:654] T 624f99e3165f49babbfce35effc9d627 P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:10.952476 24575 tablet_bootstrap.cc:654] T 836ed40e5bb8485cb93cb03bc2508265 P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:10.959815 24630 raft_consensus.cc:493] T 72e12f99e2084675ab3f53ffb836e464 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.960623 24630 raft_consensus.cc:515] T 72e12f99e2084675ab3f53ffb836e464 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:10.964368 24630 leader_election.cc:290] T 72e12f99e2084675ab3f53ffb836e464 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:10.964727 24812 raft_consensus.cc:493] T 72e12f99e2084675ab3f53ffb836e464 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.965533 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e12f99e2084675ab3f53ffb836e464" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:10.965512 24812 raft_consensus.cc:515] T 72e12f99e2084675ab3f53ffb836e464 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:10.968606 24812 leader_election.cc:290] T 72e12f99e2084675ab3f53ffb836e464 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:10.969611 24380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e12f99e2084675ab3f53ffb836e464" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:10.970618 24380 raft_consensus.cc:2468] T 72e12f99e2084675ab3f53ffb836e464 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 0.
I20260709 13:09:10.973394 24529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e12f99e2084675ab3f53ffb836e464" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:10.974366 24529 raft_consensus.cc:2468] T 72e12f99e2084675ab3f53ffb836e464 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 0.
I20260709 13:09:10.976034 24419 leader_election.cc:304] T 72e12f99e2084675ab3f53ffb836e464 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 838cb9997b0f437989d3f4e3e9904390; no voters: 
I20260709 13:09:10.973436 24380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e12f99e2084675ab3f53ffb836e464" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:10.978084 24380 raft_consensus.cc:2468] T 72e12f99e2084675ab3f53ffb836e464 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 0.
I20260709 13:09:10.981479 24493 leader_election.cc:304] T 72e12f99e2084675ab3f53ffb836e464 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:10.982741 24630 raft_consensus.cc:2804] T 72e12f99e2084675ab3f53ffb836e464 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:10.984102 24630 raft_consensus.cc:493] T 72e12f99e2084675ab3f53ffb836e464 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:10.984587 24630 raft_consensus.cc:3060] T 72e12f99e2084675ab3f53ffb836e464 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:10.969251 24455 raft_consensus.cc:2468] T 72e12f99e2084675ab3f53ffb836e464 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 0.
I20260709 13:09:10.990494 24812 raft_consensus.cc:2804] T 72e12f99e2084675ab3f53ffb836e464 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:10.993747 24812 raft_consensus.cc:493] T 72e12f99e2084675ab3f53ffb836e464 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:10.994370 24812 raft_consensus.cc:3060] T 72e12f99e2084675ab3f53ffb836e464 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:11.000828 24792 raft_consensus.cc:493] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.001405 24792 raft_consensus.cc:515] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:11.002607 24696 leader_election.cc:290] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:11.004215 24792 leader_election.cc:290] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:11.005932 24307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f7b6652eba4485ca0632f9fefc2e326" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:11.008246 24576 raft_consensus.cc:359] T f4e53ea5cf694db697fa0e474da54423 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:11.009061 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f7b6652eba4485ca0632f9fefc2e326" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
W20260709 13:09:11.009608 24346 leader_election.cc:343] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:11.026795 24772 raft_consensus.cc:493] T f4e53ea5cf694db697fa0e474da54423 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.039130 24792 raft_consensus.cc:493] T a1b91f42722c4116950ce80bc02eccd0 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:11.041275 24253 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:11.045845 24644 raft_consensus.cc:2749] T 6427fa143f574eaba2ee4c979e5485fc P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:11.039489 24680 raft_consensus.cc:515] T 2deba793363c4afca7d75b2fc3814304 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:11.074925 24792 raft_consensus.cc:515] T a1b91f42722c4116950ce80bc02eccd0 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:11.077530 24680 leader_election.cc:290] T 2deba793363c4afca7d75b2fc3814304 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 election: Requested vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:11.077971 24792 leader_election.cc:290] T a1b91f42722c4116950ce80bc02eccd0 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:11.079034 24576 raft_consensus.cc:385] T f4e53ea5cf694db697fa0e474da54423 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:11.079470 24576 raft_consensus.cc:740] T f4e53ea5cf694db697fa0e474da54423 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:11.080502 24576 consensus_queue.cc:260] T f4e53ea5cf694db697fa0e474da54423 P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:11.085423 24530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2deba793363c4afca7d75b2fc3814304" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:11.086336 24530 raft_consensus.cc:3060] T 2deba793363c4afca7d75b2fc3814304 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:11.085295 24529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b91f42722c4116950ce80bc02eccd0" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:11.092689 24529 raft_consensus.cc:2468] T a1b91f42722c4116950ce80bc02eccd0 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 0.
I20260709 13:09:11.098984 24647 raft_consensus.cc:493] T 3e71fd85d9424a08aed292991bccf5ee P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.099627 24647 raft_consensus.cc:515] T 3e71fd85d9424a08aed292991bccf5ee P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:11.102452 24647 leader_election.cc:290] T 3e71fd85d9424a08aed292991bccf5ee P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:11.104211 24528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e71fd85d9424a08aed292991bccf5ee" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:11.057623 24860 raft_consensus.cc:493] T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:11.073256 24346 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:45351, user_credentials={real_user=slave}} blocked reactor thread for 63708.7us
I20260709 13:09:11.113888 24345 leader_election.cc:304] T a1b91f42722c4116950ce80bc02eccd0 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:11.045987 24579 tablet_bootstrap.cc:492] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:11.112344 24382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e71fd85d9424a08aed292991bccf5ee" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:11.115231 24579 ts_tablet_manager.cc:1403] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.238s	user 0.028s	sys 0.032s
I20260709 13:09:11.115433 24382 raft_consensus.cc:2393] T 3e71fd85d9424a08aed292991bccf5ee P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9c2a14143eb744cf83bfa5f46284a381 in current term 1: Already voted for candidate 838cb9997b0f437989d3f4e3e9904390 in this term.
I20260709 13:09:11.073652 24772 raft_consensus.cc:515] T f4e53ea5cf694db697fa0e474da54423 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:11.117856 24772 leader_election.cc:290] T f4e53ea5cf694db697fa0e474da54423 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:11.118130 24307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2deba793363c4afca7d75b2fc3814304" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:11.115882 24792 raft_consensus.cc:2804] T a1b91f42722c4116950ce80bc02eccd0 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:11.118842 24307 raft_consensus.cc:3060] T 2deba793363c4afca7d75b2fc3814304 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:11.119228 24792 raft_consensus.cc:493] T a1b91f42722c4116950ce80bc02eccd0 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:11.118534 24579 raft_consensus.cc:359] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:11.119706 24792 raft_consensus.cc:3060] T a1b91f42722c4116950ce80bc02eccd0 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:11.120213 24579 raft_consensus.cc:385] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:11.120606 24579 raft_consensus.cc:740] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:11.121560 24579 consensus_queue.cc:260] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:11.121920 24382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4e53ea5cf694db697fa0e474da54423" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:11.122345 24270 leader_election.cc:304] T 3e71fd85d9424a08aed292991bccf5ee P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381; no voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:11.122627 24382 raft_consensus.cc:2468] T f4e53ea5cf694db697fa0e474da54423 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 0.
I20260709 13:09:11.124048 24579 ts_tablet_manager.cc:1434] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:09:11.134514 24579 tablet_bootstrap.cc:492] T 836ed40e5bb8485cb93cb03bc2508265 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:11.129549 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ae3dc25b8484b6f99d8f41685229cb8" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:11.131744 24493 leader_election.cc:304] T f4e53ea5cf694db697fa0e474da54423 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:11.138008 24772 raft_consensus.cc:2804] T f4e53ea5cf694db697fa0e474da54423 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:11.138557 24306 raft_consensus.cc:3060] T 3e71fd85d9424a08aed292991bccf5ee P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:11.141466 24792 raft_consensus.cc:515] T a1b91f42722c4116950ce80bc02eccd0 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:11.138561 24772 raft_consensus.cc:493] T f4e53ea5cf694db697fa0e474da54423 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:11.063318 24756 raft_consensus.cc:515] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:11.145099 24772 raft_consensus.cc:3060] T f4e53ea5cf694db697fa0e474da54423 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:11.147720 24792 leader_election.cc:290] T a1b91f42722c4116950ce80bc02eccd0 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 election: Requested vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:11.150614 24382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ae3dc25b8484b6f99d8f41685229cb8" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:11.151307 24382 raft_consensus.cc:3060] T 6ae3dc25b8484b6f99d8f41685229cb8 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:11.152900 24696 raft_consensus.cc:2749] T 3e71fd85d9424a08aed292991bccf5ee P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:11.154300 24853 raft_consensus.cc:493] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.154807 24853 raft_consensus.cc:515] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:11.159327 24382 raft_consensus.cc:2468] T 6ae3dc25b8484b6f99d8f41685229cb8 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 1.
I20260709 13:09:11.160780 24270 leader_election.cc:304] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:11.161638 24853 leader_election.cc:290] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:11.222718 24812 raft_consensus.cc:515] T 72e12f99e2084675ab3f53ffb836e464 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:11.162153 24647 raft_consensus.cc:2804] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:11.169656 24382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ac163496084a3e977344fa9228a3c9" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:11.224160 24382 raft_consensus.cc:2468] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 0.
I20260709 13:09:11.171566 24454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ac163496084a3e977344fa9228a3c9" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:11.176783 24758 raft_consensus.cc:493] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.225538 24758 raft_consensus.cc:515] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
W20260709 13:09:11.226487 24270 leader_election.cc:343] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:11.226955 24530 raft_consensus.cc:2468] T 2deba793363c4afca7d75b2fc3814304 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 1.
I20260709 13:09:11.228107 24758 leader_election.cc:290] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:11.228031 24630 raft_consensus.cc:515] T 72e12f99e2084675ab3f53ffb836e464 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:11.231089 24579 tablet_bootstrap.cc:654] T 836ed40e5bb8485cb93cb03bc2508265 P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:11.232103 24306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40dd44ec1fb347f49c31a8b650ab1639" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:11.176936 24529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b91f42722c4116950ce80bc02eccd0" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:11.178189 24807 consensus_queue.cc:1048] T 3e71fd85d9424a08aed292991bccf5ee P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:11.179065 24852 raft_consensus.cc:493] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:11.182313 24345 leader_election.cc:343] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:11.233366 24345 leader_election.cc:304] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:11.184650 24456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4e53ea5cf694db697fa0e474da54423" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:11.234033 24649 consensus_queue.cc:1048] T 3e71fd85d9424a08aed292991bccf5ee P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:11.234308 24456 raft_consensus.cc:2468] T f4e53ea5cf694db697fa0e474da54423 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 0.
I20260709 13:09:11.192965 24696 consensus_queue.cc:1048] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:11.235859 24419 leader_election.cc:304] T 2deba793363c4afca7d75b2fc3814304 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 838cb9997b0f437989d3f4e3e9904390; no voters: 
I20260709 13:09:11.237064 24680 raft_consensus.cc:2804] T 2deba793363c4afca7d75b2fc3814304 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:11.237807 24530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e12f99e2084675ab3f53ffb836e464" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
W20260709 13:09:11.238646 24345 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:39679, user_credentials={real_user=slave}} blocked reactor thread for 56362us
I20260709 13:09:11.239149 24680 raft_consensus.cc:697] T 2deba793363c4afca7d75b2fc3814304 P 32088c942fbb48239e0eed952d4df449 [term 1 LEADER]: Becoming Leader. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Running, Role: LEADER
I20260709 13:09:11.239390 24529 raft_consensus.cc:3060] T a1b91f42722c4116950ce80bc02eccd0 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:11.240924 24380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e12f99e2084675ab3f53ffb836e464" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:11.240474 24680 consensus_queue.cc:237] T 2deba793363c4afca7d75b2fc3814304 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:11.241577 24380 raft_consensus.cc:3060] T 72e12f99e2084675ab3f53ffb836e464 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:11.244082 24647 raft_consensus.cc:697] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Becoming Leader. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Running, Role: LEADER
I20260709 13:09:11.245144 24792 raft_consensus.cc:2749] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:11.238670 24530 raft_consensus.cc:2393] T 72e12f99e2084675ab3f53ffb836e464 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 32088c942fbb48239e0eed952d4df449 in current term 1: Already voted for candidate 838cb9997b0f437989d3f4e3e9904390 in this term.
I20260709 13:09:11.250772 24647 consensus_queue.cc:237] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:11.253468 24270 leader_election.cc:304] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449
I20260709 13:09:11.255426 24854 raft_consensus.cc:2804] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:11.255934 24854 raft_consensus.cc:493] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:11.256506 24854 raft_consensus.cc:3060] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:11.256258 24382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e12f99e2084675ab3f53ffb836e464" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:11.257962 24381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40dd44ec1fb347f49c31a8b650ab1639" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:11.258692 24381 raft_consensus.cc:2468] T 40dd44ec1fb347f49c31a8b650ab1639 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 0.
I20260709 13:09:11.260375 24454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72e12f99e2084675ab3f53ffb836e464" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:11.261252 24454 raft_consensus.cc:2393] T 72e12f99e2084675ab3f53ffb836e464 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 838cb9997b0f437989d3f4e3e9904390 in current term 1: Already voted for candidate 32088c942fbb48239e0eed952d4df449 in this term.
I20260709 13:09:11.260653 24419 leader_election.cc:304] T 72e12f99e2084675ab3f53ffb836e464 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32088c942fbb48239e0eed952d4df449; no voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:11.262320 24861 raft_consensus.cc:2749] T 72e12f99e2084675ab3f53ffb836e464 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:09:11.265230 24854 raft_consensus.cc:515] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:11.251483 24575 tablet_bootstrap.cc:492] T 836ed40e5bb8485cb93cb03bc2508265 P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:11.270097 24575 ts_tablet_manager.cc:1403] T 836ed40e5bb8485cb93cb03bc2508265 P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.329s	user 0.048s	sys 0.085s
I20260709 13:09:11.270300 24852 raft_consensus.cc:515] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:11.272428 24852 leader_election.cc:290] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:11.272845 24575 raft_consensus.cc:359] T 836ed40e5bb8485cb93cb03bc2508265 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:11.273511 24306 raft_consensus.cc:2468] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 0.
I20260709 13:09:11.203481 24452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b91f42722c4116950ce80bc02eccd0" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:11.210826 24307 raft_consensus.cc:2468] T 2deba793363c4afca7d75b2fc3814304 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 1.
I20260709 13:09:11.199494 24453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b91f42722c4116950ce80bc02eccd0" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:11.276358 24494 leader_election.cc:304] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381; no voters: 
I20260709 13:09:11.277346 24649 raft_consensus.cc:2804] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:11.277726 24649 raft_consensus.cc:493] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:11.278070 24649 raft_consensus.cc:3060] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:11.083467 24576 ts_tablet_manager.cc:1434] T f4e53ea5cf694db697fa0e474da54423 P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.227s	user 0.006s	sys 0.000s
I20260709 13:09:11.279541 24576 tablet_bootstrap.cc:492] T 72ac163496084a3e977344fa9228a3c9 P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:11.284906 24382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40dd44ec1fb347f49c31a8b650ab1639" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:11.285629 24382 raft_consensus.cc:2468] T 40dd44ec1fb347f49c31a8b650ab1639 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 0.
I20260709 13:09:11.286058 24649 raft_consensus.cc:515] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:11.286765 24381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ac163496084a3e977344fa9228a3c9" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:11.287359 24381 raft_consensus.cc:3060] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:11.286286 24204 catalog_manager.cc:5697] T 2deba793363c4afca7d75b2fc3814304 P 32088c942fbb48239e0eed952d4df449 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32088c942fbb48239e0eed952d4df449 (127.14.103.3). New cstate: current_term: 1 leader_uuid: "32088c942fbb48239e0eed952d4df449" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:11.215003 24577 tablet_bootstrap.cc:492] T 624f99e3165f49babbfce35effc9d627 P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:11.216651 24860 raft_consensus.cc:515] T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:11.292171 24577 ts_tablet_manager.cc:1403] T 624f99e3165f49babbfce35effc9d627 P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.361s	user 0.036s	sys 0.095s
I20260709 13:09:11.293985 24860 leader_election.cc:290] T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:11.294516 24528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40dd44ec1fb347f49c31a8b650ab1639" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:11.295219 24455 raft_consensus.cc:2468] T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 0.
I20260709 13:09:11.295632 24528 raft_consensus.cc:2393] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9c2a14143eb744cf83bfa5f46284a381 in current term 1: Already voted for candidate 838cb9997b0f437989d3f4e3e9904390 in this term.
I20260709 13:09:11.295437 24577 raft_consensus.cc:359] T 624f99e3165f49babbfce35effc9d627 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:11.296172 24577 raft_consensus.cc:385] T 624f99e3165f49babbfce35effc9d627 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:11.296438 24577 raft_consensus.cc:740] T 624f99e3165f49babbfce35effc9d627 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:11.297124 24577 consensus_queue.cc:260] T 624f99e3165f49babbfce35effc9d627 P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:11.299152 24577 ts_tablet_manager.cc:1434] T 624f99e3165f49babbfce35effc9d627 P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.000s
I20260709 13:09:11.224931 24812 leader_election.cc:290] T 72e12f99e2084675ab3f53ffb836e464 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 election: Requested vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:11.300743 24382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40dd44ec1fb347f49c31a8b650ab1639" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:11.301371 24382 raft_consensus.cc:3060] T 40dd44ec1fb347f49c31a8b650ab1639 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:11.301417 24307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40dd44ec1fb347f49c31a8b650ab1639" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:11.301928 24649 leader_election.cc:290] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Requested vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:11.302126 24307 raft_consensus.cc:3060] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:11.303418 24529 raft_consensus.cc:2468] T a1b91f42722c4116950ce80bc02eccd0 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 1.
I20260709 13:09:11.305276 24345 leader_election.cc:304] T a1b91f42722c4116950ce80bc02eccd0 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:11.307884 24792 raft_consensus.cc:2804] T a1b91f42722c4116950ce80bc02eccd0 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:11.308391 24792 raft_consensus.cc:697] T a1b91f42722c4116950ce80bc02eccd0 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 LEADER]: Becoming Leader. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Running, Role: LEADER
I20260709 13:09:11.309198 24792 consensus_queue.cc:237] T a1b91f42722c4116950ce80bc02eccd0 P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:11.232735 24630 leader_election.cc:290] T 72e12f99e2084675ab3f53ffb836e464 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Requested vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:11.273726 24575 raft_consensus.cc:385] T 836ed40e5bb8485cb93cb03bc2508265 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:11.212888 24756 leader_election.cc:290] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Requested vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:11.288393 24453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ac163496084a3e977344fa9228a3c9" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:11.312280 24575 raft_consensus.cc:740] T 836ed40e5bb8485cb93cb03bc2508265 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:11.214565 24451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ae3dc25b8484b6f99d8f41685229cb8" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:11.312901 24451 raft_consensus.cc:3060] T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:11.267323 24854 leader_election.cc:290] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Requested vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:11.316090 24270 leader_election.cc:304] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:11.317049 24307 raft_consensus.cc:2468] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 1.
I20260709 13:09:11.317660 24380 raft_consensus.cc:2468] T 72e12f99e2084675ab3f53ffb836e464 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 1.
I20260709 13:09:11.318166 24494 leader_election.cc:304] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381; no voters: 
I20260709 13:09:11.316275 24577 tablet_bootstrap.cc:492] T c6af3489742e4539866e5a3b10761ca2 P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:11.315908 24852 consensus_queue.cc:1048] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:11.319839 24630 raft_consensus.cc:2804] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:11.320823 24630 raft_consensus.cc:697] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Becoming Leader. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Running, Role: LEADER
I20260709 13:09:11.321586 24304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ae3dc25b8484b6f99d8f41685229cb8" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:11.322077 24493 leader_election.cc:304] T 72e12f99e2084675ab3f53ffb836e464 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:11.323009 24862 raft_consensus.cc:2804] T 72e12f99e2084675ab3f53ffb836e464 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:11.323474 24862 raft_consensus.cc:697] T 72e12f99e2084675ab3f53ffb836e464 P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Becoming Leader. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Running, Role: LEADER
I20260709 13:09:11.324465 24862 consensus_queue.cc:237] T 72e12f99e2084675ab3f53ffb836e464 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:11.327127 24577 tablet_bootstrap.cc:654] T c6af3489742e4539866e5a3b10761ca2 P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:11.329432 24852 raft_consensus.cc:2764] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:09:11.321833 24630 consensus_queue.cc:237] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:11.320612 24575 consensus_queue.cc:260] T 836ed40e5bb8485cb93cb03bc2508265 P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:11.338456 24576 tablet_bootstrap.cc:654] T 72ac163496084a3e977344fa9228a3c9 P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:11.342149 24575 ts_tablet_manager.cc:1434] T 836ed40e5bb8485cb93cb03bc2508265 P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.072s	user 0.006s	sys 0.001s
W20260709 13:09:11.334875 24270 leader_election.cc:343] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:11.354223 24575 tablet_bootstrap.cc:492] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:11.354619 24205 catalog_manager.cc:5697] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "9c2a14143eb744cf83bfa5f46284a381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:11.352542 24609 raft_consensus.cc:493] T 84af43749e474087b0a5857a95dfb31d P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.356966 24609 raft_consensus.cc:515] T 84af43749e474087b0a5857a95dfb31d P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
W20260709 13:09:11.360796 24345 leader_election.cc:343] T a1b91f42722c4116950ce80bc02eccd0 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:11.319789 24381 raft_consensus.cc:2468] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 1.
W20260709 13:09:11.366494 24345 leader_election.cc:343] T a1b91f42722c4116950ce80bc02eccd0 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:11.359135 24609 leader_election.cc:290] T 84af43749e474087b0a5857a95dfb31d P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:11.377763 24270 leader_election.cc:304] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449
I20260709 13:09:11.388254 24647 raft_consensus.cc:2804] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:11.390866 24649 raft_consensus.cc:493] T 836ed40e5bb8485cb93cb03bc2508265 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.395292 24649 raft_consensus.cc:515] T 836ed40e5bb8485cb93cb03bc2508265 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:11.396842 24382 raft_consensus.cc:2468] T 40dd44ec1fb347f49c31a8b650ab1639 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 1.
I20260709 13:09:11.405014 24452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "836ed40e5bb8485cb93cb03bc2508265" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
W20260709 13:09:11.407171 24493 leader_election.cc:343] T 836ed40e5bb8485cb93cb03bc2508265 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:11.407514 24451 raft_consensus.cc:2468] T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 1.
I20260709 13:09:11.445257 24649 leader_election.cc:290] T 836ed40e5bb8485cb93cb03bc2508265 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:11.426697 24772 raft_consensus.cc:515] T f4e53ea5cf694db697fa0e474da54423 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:11.448302 24772 leader_election.cc:290] T f4e53ea5cf694db697fa0e474da54423 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Requested vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:11.451918 24382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4e53ea5cf694db697fa0e474da54423" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:11.452579 24382 raft_consensus.cc:3060] T f4e53ea5cf694db697fa0e474da54423 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:11.452677 24452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4e53ea5cf694db697fa0e474da54423" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:11.453636 24452 raft_consensus.cc:3060] T f4e53ea5cf694db697fa0e474da54423 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:11.434369 24380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ae3dc25b8484b6f99d8f41685229cb8" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:11.458365 24380 raft_consensus.cc:2393] T 6ae3dc25b8484b6f99d8f41685229cb8 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 32088c942fbb48239e0eed952d4df449 in current term 1: Already voted for candidate 9c2a14143eb744cf83bfa5f46284a381 in this term.
I20260709 13:09:11.458761 24579 tablet_bootstrap.cc:492] T 836ed40e5bb8485cb93cb03bc2508265 P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:11.459260 24579 ts_tablet_manager.cc:1403] T 836ed40e5bb8485cb93cb03bc2508265 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.325s	user 0.050s	sys 0.073s
I20260709 13:09:11.443470 24577 tablet_bootstrap.cc:492] T c6af3489742e4539866e5a3b10761ca2 P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:11.460821 24577 ts_tablet_manager.cc:1403] T c6af3489742e4539866e5a3b10761ca2 P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.145s	user 0.007s	sys 0.018s
I20260709 13:09:11.462145 24579 raft_consensus.cc:359] T 836ed40e5bb8485cb93cb03bc2508265 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:11.462877 24579 raft_consensus.cc:385] T 836ed40e5bb8485cb93cb03bc2508265 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:11.463178 24579 raft_consensus.cc:740] T 836ed40e5bb8485cb93cb03bc2508265 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:11.463855 24579 consensus_queue.cc:260] T 836ed40e5bb8485cb93cb03bc2508265 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:11.465463 24579 ts_tablet_manager.cc:1434] T 836ed40e5bb8485cb93cb03bc2508265 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20260709 13:09:11.466439 24579 tablet_bootstrap.cc:492] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:11.466292 24382 raft_consensus.cc:2468] T f4e53ea5cf694db697fa0e474da54423 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 1.
I20260709 13:09:11.467604 24493 leader_election.cc:304] T f4e53ea5cf694db697fa0e474da54423 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:11.468564 24772 raft_consensus.cc:2804] T f4e53ea5cf694db697fa0e474da54423 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:11.469276 24772 raft_consensus.cc:697] T f4e53ea5cf694db697fa0e474da54423 P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Becoming Leader. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Running, Role: LEADER
I20260709 13:09:11.462669 24453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84af43749e474087b0a5857a95dfb31d" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:11.470136 24772 consensus_queue.cc:237] T f4e53ea5cf694db697fa0e474da54423 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:11.473073 24575 tablet_bootstrap.cc:654] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:11.475322 24579 tablet_bootstrap.cc:654] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:11.463510 24577 raft_consensus.cc:359] T c6af3489742e4539866e5a3b10761ca2 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
W20260709 13:09:11.476028 24345 leader_election.cc:343] T 84af43749e474087b0a5857a95dfb31d P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:11.476403 24577 raft_consensus.cc:385] T c6af3489742e4539866e5a3b10761ca2 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:11.476737 24577 raft_consensus.cc:740] T c6af3489742e4539866e5a3b10761ca2 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:11.477887 24577 consensus_queue.cc:260] T c6af3489742e4539866e5a3b10761ca2 P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:11.467314 24419 leader_election.cc:304] T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32088c942fbb48239e0eed952d4df449; no voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:11.485561 24680 raft_consensus.cc:2749] T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:11.489614 24576 tablet_bootstrap.cc:492] T 72ac163496084a3e977344fa9228a3c9 P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:11.502532 24576 ts_tablet_manager.cc:1403] T 72ac163496084a3e977344fa9228a3c9 P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.223s	user 0.018s	sys 0.034s
I20260709 13:09:11.506096 24576 raft_consensus.cc:359] T 72ac163496084a3e977344fa9228a3c9 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:11.506459 24307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "836ed40e5bb8485cb93cb03bc2508265" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:11.507028 24576 raft_consensus.cc:385] T 72ac163496084a3e977344fa9228a3c9 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:11.507248 24307 raft_consensus.cc:2468] T 836ed40e5bb8485cb93cb03bc2508265 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 0.
I20260709 13:09:11.490016 24529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84af43749e474087b0a5857a95dfb31d" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:11.507468 24576 raft_consensus.cc:740] T 72ac163496084a3e977344fa9228a3c9 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:11.492132 24647 raft_consensus.cc:697] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Becoming Leader. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Running, Role: LEADER
I20260709 13:09:11.516471 24576 consensus_queue.cc:260] T 72ac163496084a3e977344fa9228a3c9 P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:11.481813 24452 raft_consensus.cc:2468] T f4e53ea5cf694db697fa0e474da54423 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 1.
I20260709 13:09:11.518616 24647 consensus_queue.cc:237] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:11.522905 24576 ts_tablet_manager.cc:1434] T 72ac163496084a3e977344fa9228a3c9 P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.000s
I20260709 13:09:11.491613 24577 ts_tablet_manager.cc:1434] T c6af3489742e4539866e5a3b10761ca2 P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.031s	user 0.002s	sys 0.005s
I20260709 13:09:11.526346 24494 leader_election.cc:304] T 836ed40e5bb8485cb93cb03bc2508265 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381; no voters: 32088c942fbb48239e0eed952d4df449
I20260709 13:09:11.530529 24577 tablet_bootstrap.cc:492] T 915d8224f8b64cdab90dff1689046548 P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:11.531819 24630 raft_consensus.cc:2804] T 836ed40e5bb8485cb93cb03bc2508265 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:11.532305 24630 raft_consensus.cc:493] T 836ed40e5bb8485cb93cb03bc2508265 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:11.533236 24630 raft_consensus.cc:3060] T 836ed40e5bb8485cb93cb03bc2508265 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:11.533010 24205 catalog_manager.cc:5697] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390 reported cstate change: term changed from 0 to 1, leader changed from <none> to 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4). New cstate: current_term: 1 leader_uuid: "838cb9997b0f437989d3f4e3e9904390" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } }
W20260709 13:09:11.538564 24345 leader_election.cc:343] T 84af43749e474087b0a5857a95dfb31d P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:11.539067 24345 leader_election.cc:304] T 84af43749e474087b0a5857a95dfb31d P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449, 838cb9997b0f437989d3f4e3e9904390
I20260709 13:09:11.542620 24630 raft_consensus.cc:515] T 836ed40e5bb8485cb93cb03bc2508265 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:11.531775 24576 tablet_bootstrap.cc:492] T 836ed40e5bb8485cb93cb03bc2508265 P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:11.553148 24577 tablet_bootstrap.cc:654] T 915d8224f8b64cdab90dff1689046548 P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:11.552109 24579 tablet_bootstrap.cc:492] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:11.556228 24579 ts_tablet_manager.cc:1403] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.090s	user 0.024s	sys 0.014s
I20260709 13:09:11.557942 24792 raft_consensus.cc:2749] T 84af43749e474087b0a5857a95dfb31d P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:11.560846 24205 catalog_manager.cc:5697] T 72e12f99e2084675ab3f53ffb836e464 P 838cb9997b0f437989d3f4e3e9904390 reported cstate change: term changed from 0 to 1, leader changed from <none> to 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4). New cstate: current_term: 1 leader_uuid: "838cb9997b0f437989d3f4e3e9904390" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:11.562117 24579 raft_consensus.cc:359] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:11.569478 24579 raft_consensus.cc:385] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:11.569799 24579 raft_consensus.cc:740] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:11.594529 24577 tablet_bootstrap.cc:492] T 915d8224f8b64cdab90dff1689046548 P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:11.550673 24204 catalog_manager.cc:5697] T a1b91f42722c4116950ce80bc02eccd0 P c40e7ba102224d24a22ccd1f68aaae45 reported cstate change: term changed from 0 to 1, leader changed from <none> to c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2). New cstate: current_term: 1 leader_uuid: "c40e7ba102224d24a22ccd1f68aaae45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:11.595006 24577 ts_tablet_manager.cc:1403] T 915d8224f8b64cdab90dff1689046548 P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.065s	user 0.019s	sys 0.012s
I20260709 13:09:11.570168 24453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "836ed40e5bb8485cb93cb03bc2508265" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:11.587297 24575 tablet_bootstrap.cc:492] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:11.598507 24575 ts_tablet_manager.cc:1403] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.248s	user 0.013s	sys 0.006s
I20260709 13:09:11.591686 24307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "836ed40e5bb8485cb93cb03bc2508265" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
W20260709 13:09:11.598985 24493 leader_election.cc:343] T 836ed40e5bb8485cb93cb03bc2508265 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:11.599382 24307 raft_consensus.cc:3060] T 836ed40e5bb8485cb93cb03bc2508265 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:11.570706 24579 consensus_queue.cc:260] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:11.601498 24575 raft_consensus.cc:359] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:11.601927 24579 ts_tablet_manager.cc:1434] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.045s	user 0.001s	sys 0.004s
I20260709 13:09:11.602180 24575 raft_consensus.cc:385] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:11.602489 24575 raft_consensus.cc:740] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:11.603000 24579 tablet_bootstrap.cc:492] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:11.603516 24575 consensus_queue.cc:260] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:11.605734 24575 ts_tablet_manager.cc:1434] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:09:11.606711 24575 tablet_bootstrap.cc:492] T 363cde7d4cb84353a74a18cbd9701fa7 P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:11.607836 24680 raft_consensus.cc:493] T 72ac163496084a3e977344fa9228a3c9 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.608325 24680 raft_consensus.cc:515] T 72ac163496084a3e977344fa9228a3c9 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:11.598052 24577 raft_consensus.cc:359] T 915d8224f8b64cdab90dff1689046548 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:11.610431 24609 raft_consensus.cc:493] T 624f99e3165f49babbfce35effc9d627 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.590332 24630 leader_election.cc:290] T 836ed40e5bb8485cb93cb03bc2508265 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Requested vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:11.609478 24579 tablet_bootstrap.cc:654] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:11.611020 24609 raft_consensus.cc:515] T 624f99e3165f49babbfce35effc9d627 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:11.611362 24680 leader_election.cc:290] T 72ac163496084a3e977344fa9228a3c9 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:11.613160 24609 leader_election.cc:290] T 624f99e3165f49babbfce35effc9d627 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:11.613297 24304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ac163496084a3e977344fa9228a3c9" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:11.593852 24576 tablet_bootstrap.cc:654] T 836ed40e5bb8485cb93cb03bc2508265 P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:11.615525 24453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "624f99e3165f49babbfce35effc9d627" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
W20260709 13:09:11.617512 24345 leader_election.cc:343] T 624f99e3165f49babbfce35effc9d627 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:11.620827 24379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ac163496084a3e977344fa9228a3c9" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:11.628670 24379 raft_consensus.cc:2393] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 32088c942fbb48239e0eed952d4df449 in current term 1: Already voted for candidate 9c2a14143eb744cf83bfa5f46284a381 in this term.
I20260709 13:09:11.624002 24575 tablet_bootstrap.cc:654] T 363cde7d4cb84353a74a18cbd9701fa7 P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:11.632300 24576 tablet_bootstrap.cc:492] T 836ed40e5bb8485cb93cb03bc2508265 P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:11.625320 24307 raft_consensus.cc:2468] T 836ed40e5bb8485cb93cb03bc2508265 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 1.
I20260709 13:09:11.632860 24576 ts_tablet_manager.cc:1403] T 836ed40e5bb8485cb93cb03bc2508265 P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.101s	user 0.011s	sys 0.003s
I20260709 13:09:11.610517 24577 raft_consensus.cc:385] T 915d8224f8b64cdab90dff1689046548 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:11.633255 24577 raft_consensus.cc:740] T 915d8224f8b64cdab90dff1689046548 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:11.633337 24419 leader_election.cc:304] T 72ac163496084a3e977344fa9228a3c9 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32088c942fbb48239e0eed952d4df449; no voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:11.634418 24203 catalog_manager.cc:5697] T f4e53ea5cf694db697fa0e474da54423 P 838cb9997b0f437989d3f4e3e9904390 reported cstate change: term changed from 0 to 1, leader changed from <none> to 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4). New cstate: current_term: 1 leader_uuid: "838cb9997b0f437989d3f4e3e9904390" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:11.634156 24577 consensus_queue.cc:260] T 915d8224f8b64cdab90dff1689046548 P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:11.635426 24494 leader_election.cc:304] T 836ed40e5bb8485cb93cb03bc2508265 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381; no voters: 32088c942fbb48239e0eed952d4df449
I20260709 13:09:11.636732 24861 raft_consensus.cc:3060] T 72ac163496084a3e977344fa9228a3c9 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:11.637329 24577 ts_tablet_manager.cc:1434] T 915d8224f8b64cdab90dff1689046548 P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.042s	user 0.003s	sys 0.003s
I20260709 13:09:11.637939 24579 tablet_bootstrap.cc:492] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:11.638373 24577 tablet_bootstrap.cc:492] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:11.638448 24579 ts_tablet_manager.cc:1403] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.036s	user 0.008s	sys 0.006s
I20260709 13:09:11.639647 24649 raft_consensus.cc:2804] T 836ed40e5bb8485cb93cb03bc2508265 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:11.640230 24649 raft_consensus.cc:697] T 836ed40e5bb8485cb93cb03bc2508265 P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Becoming Leader. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Running, Role: LEADER
I20260709 13:09:11.640475 24527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "624f99e3165f49babbfce35effc9d627" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
W20260709 13:09:11.642022 24345 leader_election.cc:343] T 624f99e3165f49babbfce35effc9d627 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:11.642464 24345 leader_election.cc:304] T 624f99e3165f49babbfce35effc9d627 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449, 838cb9997b0f437989d3f4e3e9904390
I20260709 13:09:11.643450 24609 raft_consensus.cc:2749] T 624f99e3165f49babbfce35effc9d627 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:11.645304 24579 raft_consensus.cc:359] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:11.647329 24579 raft_consensus.cc:385] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:11.645993 24861 raft_consensus.cc:2749] T 72ac163496084a3e977344fa9228a3c9 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:11.648502 24609 raft_consensus.cc:493] T c6af3489742e4539866e5a3b10761ca2 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.648999 24609 raft_consensus.cc:515] T c6af3489742e4539866e5a3b10761ca2 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:11.650483 24649 consensus_queue.cc:237] T 836ed40e5bb8485cb93cb03bc2508265 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:11.651140 24609 leader_election.cc:290] T c6af3489742e4539866e5a3b10761ca2 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:11.652678 24575 tablet_bootstrap.cc:492] T 363cde7d4cb84353a74a18cbd9701fa7 P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:11.652982 24529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6af3489742e4539866e5a3b10761ca2" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:11.646596 24576 raft_consensus.cc:359] T 836ed40e5bb8485cb93cb03bc2508265 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:11.654459 24576 raft_consensus.cc:385] T 836ed40e5bb8485cb93cb03bc2508265 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:09:11.654422 24345 leader_election.cc:343] T c6af3489742e4539866e5a3b10761ca2 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:11.654901 24576 raft_consensus.cc:740] T 836ed40e5bb8485cb93cb03bc2508265 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:11.655620 24576 consensus_queue.cc:260] T 836ed40e5bb8485cb93cb03bc2508265 P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:11.655989 24304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6af3489742e4539866e5a3b10761ca2" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:11.653206 24575 ts_tablet_manager.cc:1403] T 363cde7d4cb84353a74a18cbd9701fa7 P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.047s	user 0.003s	sys 0.007s
I20260709 13:09:11.656354 24577 tablet_bootstrap.cc:654] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
W20260709 13:09:11.657721 24346 leader_election.cc:343] T c6af3489742e4539866e5a3b10761ca2 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:11.658293 24346 leader_election.cc:304] T c6af3489742e4539866e5a3b10761ca2 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:11.659209 24575 raft_consensus.cc:359] T 363cde7d4cb84353a74a18cbd9701fa7 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:11.660027 24575 raft_consensus.cc:385] T 363cde7d4cb84353a74a18cbd9701fa7 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:11.660288 24575 raft_consensus.cc:740] T 363cde7d4cb84353a74a18cbd9701fa7 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:11.660322 24792 raft_consensus.cc:2749] T c6af3489742e4539866e5a3b10761ca2 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:11.647634 24579 raft_consensus.cc:740] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:11.663041 24579 consensus_queue.cc:260] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:11.665426 24579 ts_tablet_manager.cc:1434] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.002s
I20260709 13:09:11.660959 24575 consensus_queue.cc:260] T 363cde7d4cb84353a74a18cbd9701fa7 P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:11.666572 24579 tablet_bootstrap.cc:492] T 363cde7d4cb84353a74a18cbd9701fa7 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:11.666134 24205 catalog_manager.cc:5697] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "9c2a14143eb744cf83bfa5f46284a381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:11.668349 24575 ts_tablet_manager.cc:1434] T 363cde7d4cb84353a74a18cbd9701fa7 P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:09:11.669422 24575 tablet_bootstrap.cc:492] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:11.672127 24576 ts_tablet_manager.cc:1434] T 836ed40e5bb8485cb93cb03bc2508265 P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.000s
I20260709 13:09:11.682235 24609 raft_consensus.cc:493] T 915d8224f8b64cdab90dff1689046548 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.682703 24576 tablet_bootstrap.cc:492] T a1b91f42722c4116950ce80bc02eccd0 P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:11.682862 24609 raft_consensus.cc:515] T 915d8224f8b64cdab90dff1689046548 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:11.685277 24609 leader_election.cc:290] T 915d8224f8b64cdab90dff1689046548 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:11.686084 24579 tablet_bootstrap.cc:654] T 363cde7d4cb84353a74a18cbd9701fa7 P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:11.687345 24453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915d8224f8b64cdab90dff1689046548" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
W20260709 13:09:11.689240 24345 leader_election.cc:343] T 915d8224f8b64cdab90dff1689046548 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:11.691809 24307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915d8224f8b64cdab90dff1689046548" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
W20260709 13:09:11.693738 24346 leader_election.cc:343] T 915d8224f8b64cdab90dff1689046548 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:11.694319 24346 leader_election.cc:304] T 915d8224f8b64cdab90dff1689046548 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:11.695768 24577 tablet_bootstrap.cc:492] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:11.695801 24609 raft_consensus.cc:2749] T 915d8224f8b64cdab90dff1689046548 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:11.696370 24577 ts_tablet_manager.cc:1403] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.058s	user 0.013s	sys 0.005s
I20260709 13:09:11.698046 24576 tablet_bootstrap.cc:654] T a1b91f42722c4116950ce80bc02eccd0 P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:11.698690 24577 raft_consensus.cc:359] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:11.699259 24577 raft_consensus.cc:385] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:11.699486 24577 raft_consensus.cc:740] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:11.700119 24577 consensus_queue.cc:260] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:11.702322 24577 ts_tablet_manager.cc:1434] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:09:11.703838 24575 tablet_bootstrap.cc:654] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:11.708375 24577 tablet_bootstrap.cc:492] T a3e3b771ce2c4165aad02954773c58d7 P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:11.714294 24577 tablet_bootstrap.cc:654] T a3e3b771ce2c4165aad02954773c58d7 P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:11.727764 24576 tablet_bootstrap.cc:492] T a1b91f42722c4116950ce80bc02eccd0 P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:11.728351 24576 ts_tablet_manager.cc:1403] T a1b91f42722c4116950ce80bc02eccd0 P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.046s	user 0.005s	sys 0.006s
I20260709 13:09:11.729362 24577 tablet_bootstrap.cc:492] T a3e3b771ce2c4165aad02954773c58d7 P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:11.729104 24861 consensus_queue.cc:1048] T 2deba793363c4afca7d75b2fc3814304 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:11.729830 24577 ts_tablet_manager.cc:1403] T a3e3b771ce2c4165aad02954773c58d7 P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.001s
I20260709 13:09:11.731089 24576 raft_consensus.cc:359] T a1b91f42722c4116950ce80bc02eccd0 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:11.732029 24576 raft_consensus.cc:385] T a1b91f42722c4116950ce80bc02eccd0 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:11.732280 24576 raft_consensus.cc:740] T a1b91f42722c4116950ce80bc02eccd0 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:11.733009 24576 consensus_queue.cc:260] T a1b91f42722c4116950ce80bc02eccd0 P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:11.744885 24576 ts_tablet_manager.cc:1434] T a1b91f42722c4116950ce80bc02eccd0 P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.016s	user 0.001s	sys 0.004s
I20260709 13:09:11.745946 24576 tablet_bootstrap.cc:492] T 2f7b6652eba4485ca0632f9fefc2e326 P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:11.753625 24203 catalog_manager.cc:5697] T 836ed40e5bb8485cb93cb03bc2508265 P 838cb9997b0f437989d3f4e3e9904390 reported cstate change: term changed from 0 to 1, leader changed from <none> to 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4). New cstate: current_term: 1 leader_uuid: "838cb9997b0f437989d3f4e3e9904390" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:11.755219 24680 consensus_queue.cc:1048] T 2deba793363c4afca7d75b2fc3814304 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:09:11.768998 24577 raft_consensus.cc:359] T a3e3b771ce2c4165aad02954773c58d7 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:11.771327 24577 raft_consensus.cc:385] T a3e3b771ce2c4165aad02954773c58d7 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:11.771750 24577 raft_consensus.cc:740] T a3e3b771ce2c4165aad02954773c58d7 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:11.773524 24577 consensus_queue.cc:260] T a3e3b771ce2c4165aad02954773c58d7 P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:11.775744 24861 raft_consensus.cc:493] T 836ed40e5bb8485cb93cb03bc2508265 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.778534 24861 raft_consensus.cc:515] T 836ed40e5bb8485cb93cb03bc2508265 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:11.781311 24861 leader_election.cc:290] T 836ed40e5bb8485cb93cb03bc2508265 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:11.781858 24861 raft_consensus.cc:493] T a1b91f42722c4116950ce80bc02eccd0 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.782500 24527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "836ed40e5bb8485cb93cb03bc2508265" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:11.782359 24861 raft_consensus.cc:515] T a1b91f42722c4116950ce80bc02eccd0 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:11.788270 24379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b91f42722c4116950ce80bc02eccd0" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:11.789947 24575 tablet_bootstrap.cc:492] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:11.790445 24575 ts_tablet_manager.cc:1403] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.121s	user 0.022s	sys 0.016s
I20260709 13:09:11.792311 24529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b91f42722c4116950ce80bc02eccd0" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:11.792904 24577 ts_tablet_manager.cc:1434] T a3e3b771ce2c4165aad02954773c58d7 P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.063s	user 0.008s	sys 0.000s
I20260709 13:09:11.793442 24529 raft_consensus.cc:2393] T a1b91f42722c4116950ce80bc02eccd0 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 32088c942fbb48239e0eed952d4df449 in current term 1: Already voted for candidate c40e7ba102224d24a22ccd1f68aaae45 in this term.
I20260709 13:09:11.793938 24577 tablet_bootstrap.cc:492] T dd5ac0d1debc4695a406617ec06f07a0 P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:11.795439 24419 leader_election.cc:304] T a1b91f42722c4116950ce80bc02eccd0 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32088c942fbb48239e0eed952d4df449; no voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:11.797973 24861 leader_election.cc:290] T a1b91f42722c4116950ce80bc02eccd0 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:11.798029 24812 raft_consensus.cc:3060] T a1b91f42722c4116950ce80bc02eccd0 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:11.809902 24579 tablet_bootstrap.cc:492] T 363cde7d4cb84353a74a18cbd9701fa7 P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:11.810196 24609 raft_consensus.cc:493] T a3e3b771ce2c4165aad02954773c58d7 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.810492 24579 ts_tablet_manager.cc:1403] T 363cde7d4cb84353a74a18cbd9701fa7 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.144s	user 0.022s	sys 0.030s
I20260709 13:09:11.810689 24609 raft_consensus.cc:515] T a3e3b771ce2c4165aad02954773c58d7 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:11.809904 24812 raft_consensus.cc:2749] T a1b91f42722c4116950ce80bc02eccd0 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:11.816015 24453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e3b771ce2c4165aad02954773c58d7" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
W20260709 13:09:11.817602 24345 leader_election.cc:343] T a3e3b771ce2c4165aad02954773c58d7 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:11.819404 24609 leader_election.cc:290] T a3e3b771ce2c4165aad02954773c58d7 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:11.827032 24577 tablet_bootstrap.cc:654] T dd5ac0d1debc4695a406617ec06f07a0 P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:11.828032 24304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "836ed40e5bb8485cb93cb03bc2508265" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:11.832314 24304 raft_consensus.cc:2393] T 836ed40e5bb8485cb93cb03bc2508265 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 32088c942fbb48239e0eed952d4df449 in current term 1: Already voted for candidate 838cb9997b0f437989d3f4e3e9904390 in this term.
I20260709 13:09:11.835842 24875 raft_consensus.cc:493] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.836269 24420 leader_election.cc:304] T 836ed40e5bb8485cb93cb03bc2508265 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32088c942fbb48239e0eed952d4df449; no voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:11.836347 24875 raft_consensus.cc:515] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:11.838398 24896 raft_consensus.cc:3060] T 836ed40e5bb8485cb93cb03bc2508265 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:11.839231 24875 leader_election.cc:290] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:11.841192 24380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40cf5b30d13f4c3e86a9aa657c20c650" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:11.841881 24380 raft_consensus.cc:2468] T 40cf5b30d13f4c3e86a9aa657c20c650 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 0.
I20260709 13:09:11.821868 24575 raft_consensus.cc:359] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:11.847532 24792 consensus_queue.cc:1048] T a1b91f42722c4116950ce80bc02eccd0 P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:11.829165 24576 tablet_bootstrap.cc:654] T 2f7b6652eba4485ca0632f9fefc2e326 P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:11.823395 24529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e3b771ce2c4165aad02954773c58d7" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:11.821748 24647 consensus_queue.cc:1048] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:11.849905 24756 consensus_queue.cc:1048] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:11.819489 24579 raft_consensus.cc:359] T 363cde7d4cb84353a74a18cbd9701fa7 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:11.851148 24579 raft_consensus.cc:385] T 363cde7d4cb84353a74a18cbd9701fa7 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:11.852603 24876 raft_consensus.cc:493] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.853108 24876 raft_consensus.cc:515] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:11.851477 24579 raft_consensus.cc:740] T 363cde7d4cb84353a74a18cbd9701fa7 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:11.854697 24579 consensus_queue.cc:260] T 363cde7d4cb84353a74a18cbd9701fa7 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:11.855530 24876 leader_election.cc:290] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:11.856281 24579 ts_tablet_manager.cc:1434] T 363cde7d4cb84353a74a18cbd9701fa7 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.045s	user 0.007s	sys 0.000s
I20260709 13:09:11.857365 24579 tablet_bootstrap.cc:492] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
W20260709 13:09:11.858989 24345 leader_election.cc:343] T a3e3b771ce2c4165aad02954773c58d7 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:11.858022 24772 consensus_queue.cc:1048] T 72e12f99e2084675ab3f53ffb836e464 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:11.859570 24345 leader_election.cc:304] T a3e3b771ce2c4165aad02954773c58d7 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449, 838cb9997b0f437989d3f4e3e9904390
I20260709 13:09:11.862702 24772 consensus_queue.cc:1048] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:11.866052 24270 leader_election.cc:304] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:11.869802 24876 raft_consensus.cc:2804] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:11.870247 24876 raft_consensus.cc:493] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:11.880242 24876 raft_consensus.cc:3060] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:11.870898 24579 tablet_bootstrap.cc:654] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:11.871889 24792 raft_consensus.cc:2749] T a3e3b771ce2c4165aad02954773c58d7 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:11.883914 24609 consensus_queue.cc:1048] T a1b91f42722c4116950ce80bc02eccd0 P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:11.877068 24380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f7b6652eba4485ca0632f9fefc2e326" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:11.885386 24380 raft_consensus.cc:2468] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 0.
I20260709 13:09:11.847958 24575 raft_consensus.cc:385] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:11.890275 24575 raft_consensus.cc:740] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:11.896981 24630 raft_consensus.cc:493] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.897742 24630 raft_consensus.cc:515] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:11.905103 24527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40cf5b30d13f4c3e86a9aa657c20c650" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:11.906117 24527 raft_consensus.cc:2468] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 0.
I20260709 13:09:11.891197 24575 consensus_queue.cc:260] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:11.922494 24307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40cf5b30d13f4c3e86a9aa657c20c650" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:11.924583 24630 leader_election.cc:290] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:11.927013 24579 tablet_bootstrap.cc:492] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:11.927966 24579 ts_tablet_manager.cc:1403] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.071s	user 0.012s	sys 0.000s
I20260709 13:09:11.929687 24452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f7b6652eba4485ca0632f9fefc2e326" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:11.930783 24579 raft_consensus.cc:359] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:11.932708 24896 raft_consensus.cc:2749] T 836ed40e5bb8485cb93cb03bc2508265 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:11.933105 24576 tablet_bootstrap.cc:492] T 2f7b6652eba4485ca0632f9fefc2e326 P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:11.935628 24576 ts_tablet_manager.cc:1403] T 2f7b6652eba4485ca0632f9fefc2e326 P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.190s	user 0.025s	sys 0.001s
I20260709 13:09:11.937155 24270 leader_election.cc:304] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:11.931479 24579 raft_consensus.cc:385] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:11.938992 24647 raft_consensus.cc:2804] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:11.939734 24647 raft_consensus.cc:493] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:11.940286 24647 raft_consensus.cc:3060] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:11.939406 24579 raft_consensus.cc:740] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:11.945119 24579 consensus_queue.cc:260] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:11.947125 24382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40cf5b30d13f4c3e86a9aa657c20c650" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:11.948026 24382 raft_consensus.cc:2468] T 40cf5b30d13f4c3e86a9aa657c20c650 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 0.
I20260709 13:09:11.941403 24772 consensus_queue.cc:1048] T 72e12f99e2084675ab3f53ffb836e464 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:09:11.960669 24575 ts_tablet_manager.cc:1434] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.170s	user 0.009s	sys 0.000s
I20260709 13:09:11.967248 24575 tablet_bootstrap.cc:492] T 84af43749e474087b0a5857a95dfb31d P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:11.936233 24876 raft_consensus.cc:515] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:11.964689 24576 raft_consensus.cc:359] T 2f7b6652eba4485ca0632f9fefc2e326 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:11.966122 24579 ts_tablet_manager.cc:1434] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.000s
I20260709 13:09:11.970837 24576 raft_consensus.cc:385] T 2f7b6652eba4485ca0632f9fefc2e326 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:11.971139 24576 raft_consensus.cc:740] T 2f7b6652eba4485ca0632f9fefc2e326 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:11.971849 24576 consensus_queue.cc:260] T 2f7b6652eba4485ca0632f9fefc2e326 P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:11.939496 24630 raft_consensus.cc:493] T 363cde7d4cb84353a74a18cbd9701fa7 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.978258 24647 raft_consensus.cc:515] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:11.993121 24576 ts_tablet_manager.cc:1434] T 2f7b6652eba4485ca0632f9fefc2e326 P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.056s	user 0.007s	sys 0.000s
I20260709 13:09:11.995476 24579 tablet_bootstrap.cc:492] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
W20260709 13:09:11.987823 24270 leader_election.cc:343] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:12.004657 24647 leader_election.cc:290] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Requested vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:12.020404 24576 tablet_bootstrap.cc:492] T 363cde7d4cb84353a74a18cbd9701fa7 P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:12.025789 24647 consensus_queue.cc:1048] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:12.023140 24307 raft_consensus.cc:2393] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 838cb9997b0f437989d3f4e3e9904390 in current term 1: Already voted for candidate 9c2a14143eb744cf83bfa5f46284a381 in this term.
I20260709 13:09:11.993764 24630 raft_consensus.cc:515] T 363cde7d4cb84353a74a18cbd9701fa7 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:12.054436 24792 raft_consensus.cc:493] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.055053 24792 raft_consensus.cc:515] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:12.022619 24493 leader_election.cc:304] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
W20260709 13:09:12.060169 24493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:46177, user_credentials={real_user=slave}} blocked reactor thread for 47026.6us
I20260709 13:09:12.060267 24772 raft_consensus.cc:2804] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:12.060838 24772 raft_consensus.cc:493] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:12.061229 24772 raft_consensus.cc:3060] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:12.062220 24452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f7b6652eba4485ca0632f9fefc2e326" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:12.062866 24452 raft_consensus.cc:3060] T 2f7b6652eba4485ca0632f9fefc2e326 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:12.065732 24306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd607d60354e4864ae5499ee1c1a05c9" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:12.069370 24630 leader_election.cc:290] T 363cde7d4cb84353a74a18cbd9701fa7 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:12.070417 24772 raft_consensus.cc:515] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:12.074051 24575 tablet_bootstrap.cc:654] T 84af43749e474087b0a5857a95dfb31d P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:12.098069 24577 tablet_bootstrap.cc:492] T dd5ac0d1debc4695a406617ec06f07a0 P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:12.109935 24577 ts_tablet_manager.cc:1403] T dd5ac0d1debc4695a406617ec06f07a0 P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.316s	user 0.043s	sys 0.049s
I20260709 13:09:12.097862 24876 leader_election.cc:290] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Requested vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:12.121668 24876 raft_consensus.cc:493] T 363cde7d4cb84353a74a18cbd9701fa7 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.122150 24876 raft_consensus.cc:515] T 363cde7d4cb84353a74a18cbd9701fa7 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:12.076171 24376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f7b6652eba4485ca0632f9fefc2e326" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:12.086287 24305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40cf5b30d13f4c3e86a9aa657c20c650" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:12.124518 24376 raft_consensus.cc:3060] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:12.124706 24451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363cde7d4cb84353a74a18cbd9701fa7" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:12.125607 24876 leader_election.cc:290] T 363cde7d4cb84353a74a18cbd9701fa7 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
W20260709 13:09:12.126839 24270 leader_election.cc:343] T 363cde7d4cb84353a74a18cbd9701fa7 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:12.124915 24305 raft_consensus.cc:2393] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 838cb9997b0f437989d3f4e3e9904390 in current term 1: Already voted for candidate 9c2a14143eb744cf83bfa5f46284a381 in this term.
I20260709 13:09:12.085758 24306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363cde7d4cb84353a74a18cbd9701fa7" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:12.074898 24772 leader_election.cc:290] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Requested vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:12.140965 24792 leader_election.cc:290] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:12.141868 24306 raft_consensus.cc:2468] T 363cde7d4cb84353a74a18cbd9701fa7 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 0.
I20260709 13:09:12.150056 24494 leader_election.cc:304] T 363cde7d4cb84353a74a18cbd9701fa7 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381; no voters: 
I20260709 13:09:12.149928 24529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40cf5b30d13f4c3e86a9aa657c20c650" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:12.151022 24630 raft_consensus.cc:2804] T 363cde7d4cb84353a74a18cbd9701fa7 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:12.151397 24630 raft_consensus.cc:493] T 363cde7d4cb84353a74a18cbd9701fa7 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:12.151794 24630 raft_consensus.cc:3060] T 363cde7d4cb84353a74a18cbd9701fa7 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:12.149848 24451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363cde7d4cb84353a74a18cbd9701fa7" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:12.153915 24579 tablet_bootstrap.cc:654] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:12.147070 24577 raft_consensus.cc:359] T dd5ac0d1debc4695a406617ec06f07a0 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:12.154551 24577 raft_consensus.cc:385] T dd5ac0d1debc4695a406617ec06f07a0 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:12.154979 24577 raft_consensus.cc:740] T dd5ac0d1debc4695a406617ec06f07a0 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:12.151466 24529 raft_consensus.cc:2393] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9c2a14143eb744cf83bfa5f46284a381 in current term 1: Already voted for candidate 838cb9997b0f437989d3f4e3e9904390 in this term.
I20260709 13:09:12.162427 24576 tablet_bootstrap.cc:654] T 363cde7d4cb84353a74a18cbd9701fa7 P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:12.166607 24527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363cde7d4cb84353a74a18cbd9701fa7" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:12.167627 24577 consensus_queue.cc:260] T dd5ac0d1debc4695a406617ec06f07a0 P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:12.115023 24873 consensus_queue.cc:1048] T f4e53ea5cf694db697fa0e474da54423 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.055s
I20260709 13:09:12.170857 24577 ts_tablet_manager.cc:1434] T dd5ac0d1debc4695a406617ec06f07a0 P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.061s	user 0.008s	sys 0.000s
I20260709 13:09:12.177482 24452 raft_consensus.cc:2468] T 2f7b6652eba4485ca0632f9fefc2e326 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 1.
I20260709 13:09:12.179955 24577 tablet_bootstrap.cc:492] T 142bbcbf57124963866a3967c8adebed P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:12.182226 24649 consensus_queue.cc:1048] T 836ed40e5bb8485cb93cb03bc2508265 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:09:12.184180 24493 leader_election.cc:343] T 363cde7d4cb84353a74a18cbd9701fa7 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:12.185843 24873 consensus_queue.cc:1048] T 836ed40e5bb8485cb93cb03bc2508265 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:09:12.203131 24575 tablet_bootstrap.cc:492] T 84af43749e474087b0a5857a95dfb31d P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:12.203719 24575 ts_tablet_manager.cc:1403] T 84af43749e474087b0a5857a95dfb31d P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.237s	user 0.015s	sys 0.006s
I20260709 13:09:12.205684 24270 leader_election.cc:304] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381; no voters: 
I20260709 13:09:12.211890 24876 raft_consensus.cc:2804] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:12.216708 24876 raft_consensus.cc:697] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Becoming Leader. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Running, Role: LEADER
I20260709 13:09:12.219206 24379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40cf5b30d13f4c3e86a9aa657c20c650" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:12.212431 24911 consensus_queue.cc:1048] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20260709 13:09:12.233028 24649 consensus_queue.cc:1048] T f4e53ea5cf694db697fa0e474da54423 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:12.233306 24876 consensus_queue.cc:237] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
W20260709 13:09:12.225546 24346 leader_election.cc:343] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:12.252599 24577 tablet_bootstrap.cc:654] T 142bbcbf57124963866a3967c8adebed P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:12.254916 24630 raft_consensus.cc:515] T 363cde7d4cb84353a74a18cbd9701fa7 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:12.238909 24452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd607d60354e4864ae5499ee1c1a05c9" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:12.263963 24575 raft_consensus.cc:359] T 84af43749e474087b0a5857a95dfb31d P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:12.264678 24575 raft_consensus.cc:385] T 84af43749e474087b0a5857a95dfb31d P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:12.244943 24379 raft_consensus.cc:3060] T 40cf5b30d13f4c3e86a9aa657c20c650 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:12.264957 24575 raft_consensus.cc:740] T 84af43749e474087b0a5857a95dfb31d P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:12.276942 24575 consensus_queue.cc:260] T 84af43749e474087b0a5857a95dfb31d P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
W20260709 13:09:12.277532 24493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:46177, user_credentials={real_user=slave}} blocked reactor thread for 51342.6us
I20260709 13:09:12.278855 24575 ts_tablet_manager.cc:1434] T 84af43749e474087b0a5857a95dfb31d P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.075s	user 0.006s	sys 0.000s
I20260709 13:09:12.280951 24579 tablet_bootstrap.cc:492] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:12.281467 24579 ts_tablet_manager.cc:1403] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.286s	user 0.013s	sys 0.002s
I20260709 13:09:12.281985 24575 tablet_bootstrap.cc:492] T c6af3489742e4539866e5a3b10761ca2 P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:12.283154 24756 raft_consensus.cc:493] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.284323 24756 raft_consensus.cc:515] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:12.284402 24579 raft_consensus.cc:359] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:12.285627 24579 raft_consensus.cc:385] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:12.285924 24579 raft_consensus.cc:740] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:12.286649 24579 consensus_queue.cc:260] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:12.240731 24376 raft_consensus.cc:2468] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 1.
W20260709 13:09:12.269932 24346 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:45351, user_credentials={real_user=slave}} blocked reactor thread for 48030.6us
I20260709 13:09:12.306010 24579 ts_tablet_manager.cc:1434] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.002s
I20260709 13:09:12.309530 24576 tablet_bootstrap.cc:492] T 363cde7d4cb84353a74a18cbd9701fa7 P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:12.310314 24576 ts_tablet_manager.cc:1403] T 363cde7d4cb84353a74a18cbd9701fa7 P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.308s	user 0.018s	sys 0.004s
I20260709 13:09:12.300503 24921 raft_consensus.cc:493] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:12.332095 24916 log.cc:927] Time spent T 72e12f99e2084675ab3f53ffb836e464 P c40e7ba102224d24a22ccd1f68aaae45: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:09:12.314762 24527 raft_consensus.cc:2393] T 363cde7d4cb84353a74a18cbd9701fa7 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9c2a14143eb744cf83bfa5f46284a381 in current term 1: Already voted for candidate 838cb9997b0f437989d3f4e3e9904390 in this term.
I20260709 13:09:12.334074 24579 tablet_bootstrap.cc:492] T 915d8224f8b64cdab90dff1689046548 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:12.339259 24630 leader_election.cc:290] T 363cde7d4cb84353a74a18cbd9701fa7 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Requested vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:12.325176 24756 leader_election.cc:290] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:12.346670 24577 tablet_bootstrap.cc:492] T 142bbcbf57124963866a3967c8adebed P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:12.347325 24577 ts_tablet_manager.cc:1403] T 142bbcbf57124963866a3967c8adebed P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.168s	user 0.012s	sys 0.012s
I20260709 13:09:12.352162 24579 tablet_bootstrap.cc:654] T 915d8224f8b64cdab90dff1689046548 P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:12.350832 24576 raft_consensus.cc:359] T 363cde7d4cb84353a74a18cbd9701fa7 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:12.356570 24576 raft_consensus.cc:385] T 363cde7d4cb84353a74a18cbd9701fa7 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:12.356855 24576 raft_consensus.cc:740] T 363cde7d4cb84353a74a18cbd9701fa7 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:12.357383 24576 consensus_queue.cc:260] T 363cde7d4cb84353a74a18cbd9701fa7 P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:12.380331 24577 raft_consensus.cc:359] T 142bbcbf57124963866a3967c8adebed P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:12.381371 24577 raft_consensus.cc:385] T 142bbcbf57124963866a3967c8adebed P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:12.382797 24454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363cde7d4cb84353a74a18cbd9701fa7" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:12.336128 24921 raft_consensus.cc:515] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:12.383572 24454 raft_consensus.cc:3060] T 363cde7d4cb84353a74a18cbd9701fa7 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:12.385126 24921 leader_election.cc:290] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:12.390287 24305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88feffc0859413482b0daf562d2f86a" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:12.381699 24577 raft_consensus.cc:740] T 142bbcbf57124963866a3967c8adebed P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:12.394827 24577 consensus_queue.cc:260] T 142bbcbf57124963866a3967c8adebed P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:12.400768 24304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363cde7d4cb84353a74a18cbd9701fa7" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:12.414016 24577 ts_tablet_manager.cc:1434] T 142bbcbf57124963866a3967c8adebed P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.066s	user 0.006s	sys 0.002s
I20260709 13:09:12.410848 24304 raft_consensus.cc:3060] T 363cde7d4cb84353a74a18cbd9701fa7 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:09:12.422540 24494 leader_election.cc:343] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:12.434594 24454 raft_consensus.cc:2468] T 363cde7d4cb84353a74a18cbd9701fa7 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 1.
I20260709 13:09:12.440631 24453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88feffc0859413482b0daf562d2f86a" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
W20260709 13:09:12.443755 24345 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:39679, user_credentials={real_user=slave}} blocked reactor thread for 229757us
I20260709 13:09:12.445199 24376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40cf5b30d13f4c3e86a9aa657c20c650" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:12.418519 24205 catalog_manager.cc:5697] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "9c2a14143eb744cf83bfa5f46284a381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:12.455842 24575 tablet_bootstrap.cc:654] T c6af3489742e4539866e5a3b10761ca2 P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:12.475245 24576 ts_tablet_manager.cc:1434] T 363cde7d4cb84353a74a18cbd9701fa7 P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.165s	user 0.016s	sys 0.004s
I20260709 13:09:12.477809 24579 tablet_bootstrap.cc:492] T 915d8224f8b64cdab90dff1689046548 P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:12.478381 24579 ts_tablet_manager.cc:1403] T 915d8224f8b64cdab90dff1689046548 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.150s	user 0.020s	sys 0.017s
I20260709 13:09:12.481500 24304 raft_consensus.cc:2468] T 363cde7d4cb84353a74a18cbd9701fa7 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 1.
I20260709 13:09:12.483003 24382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142bbcbf57124963866a3967c8adebed" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:12.485580 24270 leader_election.cc:304] T 363cde7d4cb84353a74a18cbd9701fa7 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381; no voters: 32088c942fbb48239e0eed952d4df449, 838cb9997b0f437989d3f4e3e9904390
I20260709 13:09:12.496451 24772 raft_consensus.cc:493] T 84af43749e474087b0a5857a95dfb31d P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.496459 24528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142bbcbf57124963866a3967c8adebed" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:12.492646 24579 raft_consensus.cc:359] T 915d8224f8b64cdab90dff1689046548 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
W20260709 13:09:12.498382 24493 leader_election.cc:343] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:12.505883 24579 raft_consensus.cc:385] T 915d8224f8b64cdab90dff1689046548 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:12.502051 24577 tablet_bootstrap.cc:492] T 3cf2c3be9fd244d4b9d4703d424d3c5d P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:12.507264 24579 raft_consensus.cc:740] T 915d8224f8b64cdab90dff1689046548 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:12.508368 24576 tablet_bootstrap.cc:492] T 915d8224f8b64cdab90dff1689046548 P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:12.507994 24579 consensus_queue.cc:260] T 915d8224f8b64cdab90dff1689046548 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:12.509473 24379 raft_consensus.cc:2468] T 40cf5b30d13f4c3e86a9aa657c20c650 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 1.
I20260709 13:09:12.505436 24876 raft_consensus.cc:2749] T 363cde7d4cb84353a74a18cbd9701fa7 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:12.512281 24772 raft_consensus.cc:515] T 84af43749e474087b0a5857a95dfb31d P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:12.514328 24772 leader_election.cc:290] T 84af43749e474087b0a5857a95dfb31d P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:12.505056 24382 raft_consensus.cc:2468] T 142bbcbf57124963866a3967c8adebed P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 0.
I20260709 13:09:12.520354 24875 raft_consensus.cc:493] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.520925 24875 raft_consensus.cc:515] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:12.523030 24875 leader_election.cc:290] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:12.523574 24493 leader_election.cc:304] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390; no voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:12.527297 24921 raft_consensus.cc:2749] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:12.527326 24493 leader_election.cc:304] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390; no voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:12.530025 24921 raft_consensus.cc:2749] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:09:12.530750 24270 leader_election.cc:343] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:12.531199 24493 leader_election.cc:304] T 363cde7d4cb84353a74a18cbd9701fa7 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 838cb9997b0f437989d3f4e3e9904390; no voters: 
I20260709 13:09:12.533704 24454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd5ac0d1debc4695a406617ec06f07a0" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:12.534759 24579 ts_tablet_manager.cc:1434] T 915d8224f8b64cdab90dff1689046548 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.056s	user 0.007s	sys 0.000s
I20260709 13:09:12.535387 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84af43749e474087b0a5857a95dfb31d" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
W20260709 13:09:12.539028 24253 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:12.539494 24579 tablet_bootstrap.cc:492] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:12.540693 24921 raft_consensus.cc:2804] T 363cde7d4cb84353a74a18cbd9701fa7 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:12.541345 24921 raft_consensus.cc:697] T 363cde7d4cb84353a74a18cbd9701fa7 P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Becoming Leader. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Running, Role: LEADER
I20260709 13:09:12.542346 24921 consensus_queue.cc:237] T 363cde7d4cb84353a74a18cbd9701fa7 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
W20260709 13:09:12.547665 24270 leader_election.cc:343] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:12.558876 24382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd5ac0d1debc4695a406617ec06f07a0" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:12.559758 24382 raft_consensus.cc:2468] T dd5ac0d1debc4695a406617ec06f07a0 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 0.
W20260709 13:09:12.565797 24493 leader_election.cc:343] T 84af43749e474087b0a5857a95dfb31d P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:12.569192 24576 tablet_bootstrap.cc:654] T 915d8224f8b64cdab90dff1689046548 P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
W20260709 13:09:12.566633 24345 leader_election.cc:343] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:12.569908 24345 leader_election.cc:304] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:12.573453 24575 tablet_bootstrap.cc:492] T c6af3489742e4539866e5a3b10761ca2 P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:12.573969 24575 ts_tablet_manager.cc:1403] T c6af3489742e4539866e5a3b10761ca2 P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.292s	user 0.013s	sys 0.003s
I20260709 13:09:12.571928 24909 raft_consensus.cc:2749] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:12.578100 24575 raft_consensus.cc:359] T c6af3489742e4539866e5a3b10761ca2 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:12.578886 24575 raft_consensus.cc:385] T c6af3489742e4539866e5a3b10761ca2 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:12.579720 24575 raft_consensus.cc:740] T c6af3489742e4539866e5a3b10761ca2 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:12.580448 24575 consensus_queue.cc:260] T c6af3489742e4539866e5a3b10761ca2 P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:12.582063 24575 ts_tablet_manager.cc:1434] T c6af3489742e4539866e5a3b10761ca2 P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:09:12.584625 24649 consensus_queue.cc:1048] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:12.585557 24577 tablet_bootstrap.cc:654] T 3cf2c3be9fd244d4b9d4703d424d3c5d P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:12.586388 24576 tablet_bootstrap.cc:492] T 915d8224f8b64cdab90dff1689046548 P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:12.586283 24270 leader_election.cc:304] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 838cb9997b0f437989d3f4e3e9904390
I20260709 13:09:12.586911 24576 ts_tablet_manager.cc:1403] T 915d8224f8b64cdab90dff1689046548 P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.079s	user 0.013s	sys 0.000s
I20260709 13:09:12.587114 24875 raft_consensus.cc:2804] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:12.587623 24875 raft_consensus.cc:697] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Becoming Leader. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Running, Role: LEADER
I20260709 13:09:12.588441 24875 consensus_queue.cc:237] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:12.589648 24576 raft_consensus.cc:359] T 915d8224f8b64cdab90dff1689046548 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:12.590327 24576 raft_consensus.cc:385] T 915d8224f8b64cdab90dff1689046548 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:12.590585 24576 raft_consensus.cc:740] T 915d8224f8b64cdab90dff1689046548 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:12.591218 24576 consensus_queue.cc:260] T 915d8224f8b64cdab90dff1689046548 P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:12.592419 24575 tablet_bootstrap.cc:492] T 9c88d903166c4812a88653e389ab5776 P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:12.593367 24576 ts_tablet_manager.cc:1434] T 915d8224f8b64cdab90dff1689046548 P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:09:12.593469 24270 leader_election.cc:304] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 838cb9997b0f437989d3f4e3e9904390
I20260709 13:09:12.596318 24756 raft_consensus.cc:2804] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:12.596697 24756 raft_consensus.cc:493] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:12.597036 24756 raft_consensus.cc:3060] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:12.603906 24576 tablet_bootstrap.cc:492] T b88feffc0859413482b0daf562d2f86a P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:12.604010 24579 tablet_bootstrap.cc:654] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:12.615011 24270 leader_election.cc:304] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449
I20260709 13:09:12.617022 24690 raft_consensus.cc:2804] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:12.617520 24690 raft_consensus.cc:493] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:12.617951 24690 raft_consensus.cc:3060] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:12.625133 24575 tablet_bootstrap.cc:654] T 9c88d903166c4812a88653e389ab5776 P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:12.636220 24576 tablet_bootstrap.cc:654] T b88feffc0859413482b0daf562d2f86a P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:12.623974 24376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84af43749e474087b0a5857a95dfb31d" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:12.646943 24376 raft_consensus.cc:2468] T 84af43749e474087b0a5857a95dfb31d P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 0.
I20260709 13:09:12.648897 24493 leader_election.cc:304] T 84af43749e474087b0a5857a95dfb31d P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449
I20260709 13:09:12.649895 24921 raft_consensus.cc:2804] T 84af43749e474087b0a5857a95dfb31d P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:12.650291 24921 raft_consensus.cc:493] T 84af43749e474087b0a5857a95dfb31d P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:12.650629 24921 raft_consensus.cc:3060] T 84af43749e474087b0a5857a95dfb31d P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:12.611549 24756 raft_consensus.cc:515] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:12.657538 24647 consensus_queue.cc:1048] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:09:12.673898 24577 tablet_bootstrap.cc:492] T 3cf2c3be9fd244d4b9d4703d424d3c5d P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:12.674546 24577 ts_tablet_manager.cc:1403] T 3cf2c3be9fd244d4b9d4703d424d3c5d P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.221s	user 0.009s	sys 0.022s
I20260709 13:09:12.678380 24577 raft_consensus.cc:359] T 3cf2c3be9fd244d4b9d4703d424d3c5d P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:12.679615 24577 raft_consensus.cc:385] T 3cf2c3be9fd244d4b9d4703d424d3c5d P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:12.680115 24577 raft_consensus.cc:740] T 3cf2c3be9fd244d4b9d4703d424d3c5d P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:12.688771 24577 consensus_queue.cc:260] T 3cf2c3be9fd244d4b9d4703d424d3c5d P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:12.684499 24921 raft_consensus.cc:515] T 84af43749e474087b0a5857a95dfb31d P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:12.693555 24577 ts_tablet_manager.cc:1434] T 3cf2c3be9fd244d4b9d4703d424d3c5d P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.001s
I20260709 13:09:12.697322 24579 tablet_bootstrap.cc:492] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:12.698042 24579 ts_tablet_manager.cc:1403] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.162s	user 0.014s	sys 0.000s
I20260709 13:09:12.704449 24577 tablet_bootstrap.cc:492] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:12.722880 24577 tablet_bootstrap.cc:654] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:12.730536 24921 leader_election.cc:290] T 84af43749e474087b0a5857a95dfb31d P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Requested vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:12.731189 24374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84af43749e474087b0a5857a95dfb31d" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:12.733459 24374 raft_consensus.cc:3060] T 84af43749e474087b0a5857a95dfb31d P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:12.733566 24376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142bbcbf57124963866a3967c8adebed" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:12.733570 24451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84af43749e474087b0a5857a95dfb31d" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:12.734498 24376 raft_consensus.cc:3060] T 142bbcbf57124963866a3967c8adebed P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:12.729585 24528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142bbcbf57124963866a3967c8adebed" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
W20260709 13:09:12.735970 24493 leader_election.cc:343] T 84af43749e474087b0a5857a95dfb31d P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:12.737150 24270 leader_election.cc:343] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:12.739498 24756 leader_election.cc:290] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Requested vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:12.742133 24374 raft_consensus.cc:2468] T 84af43749e474087b0a5857a95dfb31d P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 1.
I20260709 13:09:12.730931 24690 raft_consensus.cc:515] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:12.750229 24493 leader_election.cc:304] T 84af43749e474087b0a5857a95dfb31d P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449
I20260709 13:09:12.752166 24921 raft_consensus.cc:2804] T 84af43749e474087b0a5857a95dfb31d P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:12.754793 24579 raft_consensus.cc:359] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:12.755878 24579 raft_consensus.cc:385] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:12.756233 24579 raft_consensus.cc:740] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:12.757076 24579 consensus_queue.cc:260] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:12.758842 24577 tablet_bootstrap.cc:492] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:12.759106 24579 ts_tablet_manager.cc:1434] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.061s	user 0.006s	sys 0.000s
I20260709 13:09:12.759362 24577 ts_tablet_manager.cc:1403] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.055s	user 0.014s	sys 0.002s
I20260709 13:09:12.760411 24579 tablet_bootstrap.cc:492] T c6af3489742e4539866e5a3b10761ca2 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:12.767262 24579 tablet_bootstrap.cc:654] T c6af3489742e4539866e5a3b10761ca2 P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:12.772377 24577 raft_consensus.cc:359] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:12.773538 24577 raft_consensus.cc:385] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:12.773970 24577 raft_consensus.cc:740] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:12.775339 24575 tablet_bootstrap.cc:492] T 9c88d903166c4812a88653e389ab5776 P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:12.775913 24575 ts_tablet_manager.cc:1403] T 9c88d903166c4812a88653e389ab5776 P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.184s	user 0.022s	sys 0.021s
I20260709 13:09:12.775216 24577 consensus_queue.cc:260] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:12.779340 24577 ts_tablet_manager.cc:1434] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.003s
I20260709 13:09:12.781785 24577 tablet_bootstrap.cc:492] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:12.782774 24575 raft_consensus.cc:359] T 9c88d903166c4812a88653e389ab5776 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:12.783782 24575 raft_consensus.cc:385] T 9c88d903166c4812a88653e389ab5776 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:12.784124 24575 raft_consensus.cc:740] T 9c88d903166c4812a88653e389ab5776 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:12.785020 24575 consensus_queue.cc:260] T 9c88d903166c4812a88653e389ab5776 P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:12.768668 24921 raft_consensus.cc:697] T 84af43749e474087b0a5857a95dfb31d P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Becoming Leader. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Running, Role: LEADER
I20260709 13:09:12.792811 24575 ts_tablet_manager.cc:1434] T 9c88d903166c4812a88653e389ab5776 P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.003s
I20260709 13:09:12.793943 24575 tablet_bootstrap.cc:492] T 142bbcbf57124963866a3967c8adebed P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:12.791934 24921 consensus_queue.cc:237] T 84af43749e474087b0a5857a95dfb31d P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:12.792850 24376 raft_consensus.cc:2468] T 142bbcbf57124963866a3967c8adebed P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 1.
I20260709 13:09:12.800390 24576 tablet_bootstrap.cc:492] T b88feffc0859413482b0daf562d2f86a P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:12.800927 24576 ts_tablet_manager.cc:1403] T b88feffc0859413482b0daf562d2f86a P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.198s	user 0.015s	sys 0.003s
I20260709 13:09:12.801369 24690 leader_election.cc:290] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Requested vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:12.805590 24270 leader_election.cc:304] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 838cb9997b0f437989d3f4e3e9904390
I20260709 13:09:12.805987 24690 consensus_queue.cc:1048] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:12.806849 24941 raft_consensus.cc:2804] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:12.807407 24941 raft_consensus.cc:697] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Becoming Leader. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Running, Role: LEADER
I20260709 13:09:12.808321 24941 consensus_queue.cc:237] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:12.811393 24577 tablet_bootstrap.cc:654] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:12.812551 24454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd5ac0d1debc4695a406617ec06f07a0" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:12.814229 24647 raft_consensus.cc:493] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.814720 24647 raft_consensus.cc:515] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:12.816624 24647 leader_election.cc:290] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:12.817967 24376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd5ac0d1debc4695a406617ec06f07a0" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:12.818642 24376 raft_consensus.cc:3060] T dd5ac0d1debc4695a406617ec06f07a0 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:09:12.834180 24270 leader_election.cc:343] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:12.803218 24579 tablet_bootstrap.cc:492] T c6af3489742e4539866e5a3b10761ca2 P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:12.841715 24579 ts_tablet_manager.cc:1403] T c6af3489742e4539866e5a3b10761ca2 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.082s	user 0.018s	sys 0.007s
I20260709 13:09:12.835083 24528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88feffc0859413482b0daf562d2f86a" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:12.803644 24576 raft_consensus.cc:359] T b88feffc0859413482b0daf562d2f86a P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:12.844074 24576 raft_consensus.cc:385] T b88feffc0859413482b0daf562d2f86a P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:12.844125 24528 raft_consensus.cc:2468] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 0.
I20260709 13:09:12.844527 24576 raft_consensus.cc:740] T b88feffc0859413482b0daf562d2f86a P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:12.845655 24576 consensus_queue.cc:260] T b88feffc0859413482b0daf562d2f86a P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:12.847471 24579 raft_consensus.cc:359] T c6af3489742e4539866e5a3b10761ca2 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:12.848174 24579 raft_consensus.cc:385] T c6af3489742e4539866e5a3b10761ca2 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:12.848421 24579 raft_consensus.cc:740] T c6af3489742e4539866e5a3b10761ca2 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:12.848831 24576 ts_tablet_manager.cc:1434] T b88feffc0859413482b0daf562d2f86a P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.048s	user 0.008s	sys 0.000s
I20260709 13:09:12.850111 24576 tablet_bootstrap.cc:492] T a3e3b771ce2c4165aad02954773c58d7 P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:12.853763 24575 tablet_bootstrap.cc:654] T 142bbcbf57124963866a3967c8adebed P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:12.858913 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88feffc0859413482b0daf562d2f86a" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:12.859781 24455 raft_consensus.cc:2468] T b88feffc0859413482b0daf562d2f86a P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 0.
I20260709 13:09:12.846168 24270 leader_election.cc:304] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381; no voters: 
I20260709 13:09:12.863979 24875 raft_consensus.cc:2804] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:12.864408 24875 raft_consensus.cc:493] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:12.864768 24875 raft_consensus.cc:3060] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:12.874007 24921 raft_consensus.cc:493] T c6af3489742e4539866e5a3b10761ca2 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.874583 24921 raft_consensus.cc:515] T c6af3489742e4539866e5a3b10761ca2 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:12.877290 24376 raft_consensus.cc:2468] T dd5ac0d1debc4695a406617ec06f07a0 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 1.
I20260709 13:09:12.877475 24921 leader_election.cc:290] T c6af3489742e4539866e5a3b10761ca2 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:12.877707 24304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6af3489742e4539866e5a3b10761ca2" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:12.879765 24377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6af3489742e4539866e5a3b10761ca2" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:12.880671 24377 raft_consensus.cc:2468] T c6af3489742e4539866e5a3b10761ca2 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 0.
I20260709 13:09:12.880923 24875 raft_consensus.cc:515] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:12.881820 24493 leader_election.cc:304] T c6af3489742e4539866e5a3b10761ca2 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:12.882947 24921 raft_consensus.cc:2804] T c6af3489742e4539866e5a3b10761ca2 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:12.883339 24921 raft_consensus.cc:493] T c6af3489742e4539866e5a3b10761ca2 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:12.883663 24921 raft_consensus.cc:3060] T c6af3489742e4539866e5a3b10761ca2 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:12.885399 24875 leader_election.cc:290] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Requested vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:12.891387 24921 raft_consensus.cc:515] T c6af3489742e4539866e5a3b10761ca2 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:12.894608 24305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6af3489742e4539866e5a3b10761ca2" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:12.849097 24579 consensus_queue.cc:260] T c6af3489742e4539866e5a3b10761ca2 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:12.895345 24202 catalog_manager.cc:5697] T 363cde7d4cb84353a74a18cbd9701fa7 P 838cb9997b0f437989d3f4e3e9904390 reported cstate change: term changed from 0 to 1, leader changed from <none> to 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4). New cstate: current_term: 1 leader_uuid: "838cb9997b0f437989d3f4e3e9904390" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:12.897305 24202 catalog_manager.cc:5697] T 84af43749e474087b0a5857a95dfb31d P 838cb9997b0f437989d3f4e3e9904390 reported cstate change: term changed from 0 to 1, leader changed from <none> to 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4). New cstate: current_term: 1 leader_uuid: "838cb9997b0f437989d3f4e3e9904390" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:12.898202 24579 ts_tablet_manager.cc:1434] T c6af3489742e4539866e5a3b10761ca2 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.056s	user 0.005s	sys 0.000s
I20260709 13:09:12.898347 24374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6af3489742e4539866e5a3b10761ca2" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:12.899204 24374 raft_consensus.cc:3060] T c6af3489742e4539866e5a3b10761ca2 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:12.899478 24579 tablet_bootstrap.cc:492] T edd24afe4d7847e3980c6e586b9dac5e P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:12.899871 24304 raft_consensus.cc:2468] T c6af3489742e4539866e5a3b10761ca2 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 0.
I20260709 13:09:12.905028 24921 leader_election.cc:290] T c6af3489742e4539866e5a3b10761ca2 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Requested vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:12.906463 24528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88feffc0859413482b0daf562d2f86a" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:12.907485 24528 raft_consensus.cc:3060] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:12.911304 24374 raft_consensus.cc:2468] T c6af3489742e4539866e5a3b10761ca2 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 1.
I20260709 13:09:12.913220 24493 leader_election.cc:304] T c6af3489742e4539866e5a3b10761ca2 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:12.914535 24921 raft_consensus.cc:2804] T c6af3489742e4539866e5a3b10761ca2 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:12.915608 24680 raft_consensus.cc:493] T 915d8224f8b64cdab90dff1689046548 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.916539 24680 raft_consensus.cc:515] T 915d8224f8b64cdab90dff1689046548 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:12.916689 24921 raft_consensus.cc:697] T c6af3489742e4539866e5a3b10761ca2 P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Becoming Leader. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Running, Role: LEADER
I20260709 13:09:12.919481 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88feffc0859413482b0daf562d2f86a" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:12.921559 24455 raft_consensus.cc:3060] T b88feffc0859413482b0daf562d2f86a P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:12.929579 24304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915d8224f8b64cdab90dff1689046548" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:12.939810 24576 tablet_bootstrap.cc:654] T a3e3b771ce2c4165aad02954773c58d7 P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:12.984400 24304 raft_consensus.cc:2468] T 915d8224f8b64cdab90dff1689046548 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 0.
I20260709 13:09:12.986438 24420 leader_election.cc:304] T 915d8224f8b64cdab90dff1689046548 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381; no voters: 
I20260709 13:09:12.987520 24928 raft_consensus.cc:2804] T 915d8224f8b64cdab90dff1689046548 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:12.988013 24928 raft_consensus.cc:493] T 915d8224f8b64cdab90dff1689046548 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:12.988389 24928 raft_consensus.cc:3060] T 915d8224f8b64cdab90dff1689046548 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:13.000166 24270 leader_election.cc:304] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449
I20260709 13:09:13.002445 24941 raft_consensus.cc:2804] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:13.003024 24941 raft_consensus.cc:697] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Becoming Leader. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Running, Role: LEADER
I20260709 13:09:13.004333 24941 consensus_queue.cc:237] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:13.022641 24928 raft_consensus.cc:515] T 915d8224f8b64cdab90dff1689046548 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:13.046752 24609 raft_consensus.cc:493] T 3cf2c3be9fd244d4b9d4703d424d3c5d P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:13.070812 24609 raft_consensus.cc:515] T 3cf2c3be9fd244d4b9d4703d424d3c5d P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:13.073225 24609 leader_election.cc:290] T 3cf2c3be9fd244d4b9d4703d424d3c5d P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:13.074810 24530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf2c3be9fd244d4b9d4703d424d3c5d" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:13.076723 24304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915d8224f8b64cdab90dff1689046548" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:13.077423 24304 raft_consensus.cc:3060] T 915d8224f8b64cdab90dff1689046548 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:13.051945 24579 tablet_bootstrap.cc:654] T edd24afe4d7847e3980c6e586b9dac5e P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:13.054217 24921 consensus_queue.cc:237] T c6af3489742e4539866e5a3b10761ca2 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:13.087810 24576 tablet_bootstrap.cc:492] T a3e3b771ce2c4165aad02954773c58d7 P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:13.088326 24576 ts_tablet_manager.cc:1403] T a3e3b771ce2c4165aad02954773c58d7 P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.238s	user 0.016s	sys 0.004s
I20260709 13:09:13.056740 24680 leader_election.cc:290] T 915d8224f8b64cdab90dff1689046548 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:13.057945 24575 tablet_bootstrap.cc:492] T 142bbcbf57124963866a3967c8adebed P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:13.058374 24577 tablet_bootstrap.cc:492] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
W20260709 13:09:13.059981 24270 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.2:46177, user_credentials={real_user=slave}} blocked reactor thread for 59913.8us
I20260709 13:09:13.089681 24577 ts_tablet_manager.cc:1403] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.308s	user 0.036s	sys 0.015s
I20260709 13:09:13.089300 24203 catalog_manager.cc:5697] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "9c2a14143eb744cf83bfa5f46284a381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:13.092175 24203 catalog_manager.cc:5697] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "9c2a14143eb744cf83bfa5f46284a381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:13.092751 24577 raft_consensus.cc:359] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:13.093519 24577 raft_consensus.cc:385] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:13.093808 24577 raft_consensus.cc:740] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:13.094600 24577 consensus_queue.cc:260] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
W20260709 13:09:13.097843 24345 leader_election.cc:343] T 3cf2c3be9fd244d4b9d4703d424d3c5d P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:13.102036 24609 raft_consensus.cc:493] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:13.102592 24609 raft_consensus.cc:515] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:13.104298 24577 ts_tablet_manager.cc:1434] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.001s
I20260709 13:09:13.105415 24577 tablet_bootstrap.cc:492] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45: Bootstrap starting.
I20260709 13:09:13.106076 24305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f795acd16fd472b8c9360fb14e7e5d7" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
W20260709 13:09:13.107735 24346 leader_election.cc:343] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:13.108388 24609 leader_election.cc:290] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:13.110327 24928 leader_election.cc:290] T 915d8224f8b64cdab90dff1689046548 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 election: Requested vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:13.110839 24576 raft_consensus.cc:359] T a3e3b771ce2c4165aad02954773c58d7 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:13.064554 24455 raft_consensus.cc:2468] T b88feffc0859413482b0daf562d2f86a P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 1.
I20260709 13:09:13.117657 24451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf2c3be9fd244d4b9d4703d424d3c5d" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:13.069005 24379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915d8224f8b64cdab90dff1689046548" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:13.125178 24379 raft_consensus.cc:2468] T 915d8224f8b64cdab90dff1689046548 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 0.
I20260709 13:09:13.118783 24530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f795acd16fd472b8c9360fb14e7e5d7" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:13.119742 24630 raft_consensus.cc:493] T 9c88d903166c4812a88653e389ab5776 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:13.128116 24630 raft_consensus.cc:515] T 9c88d903166c4812a88653e389ab5776 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:13.130153 24379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915d8224f8b64cdab90dff1689046548" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:13.130759 24379 raft_consensus.cc:3060] T 915d8224f8b64cdab90dff1689046548 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:13.132165 24630 leader_election.cc:290] T 9c88d903166c4812a88653e389ab5776 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
W20260709 13:09:13.132357 24345 leader_election.cc:343] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:13.132890 24345 leader_election.cc:304] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:13.133564 24609 raft_consensus.cc:2749] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:13.138284 24305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c88d903166c4812a88653e389ab5776" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:13.139580 24579 tablet_bootstrap.cc:492] T edd24afe4d7847e3980c6e586b9dac5e P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:13.140122 24579 ts_tablet_manager.cc:1403] T edd24afe4d7847e3980c6e586b9dac5e P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.241s	user 0.009s	sys 0.003s
W20260709 13:09:13.141294 24494 leader_election.cc:343] T 9c88d903166c4812a88653e389ab5776 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:13.144157 24579 raft_consensus.cc:359] T edd24afe4d7847e3980c6e586b9dac5e P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:13.144825 24579 raft_consensus.cc:385] T edd24afe4d7847e3980c6e586b9dac5e P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:13.145092 24579 raft_consensus.cc:740] T edd24afe4d7847e3980c6e586b9dac5e P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
W20260709 13:09:13.145136 24345 leader_election.cc:343] T 3cf2c3be9fd244d4b9d4703d424d3c5d P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:13.145602 24345 leader_election.cc:304] T 3cf2c3be9fd244d4b9d4703d424d3c5d P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449, 838cb9997b0f437989d3f4e3e9904390
I20260709 13:09:13.145766 24579 consensus_queue.cc:260] T edd24afe4d7847e3980c6e586b9dac5e P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:13.146278 24609 raft_consensus.cc:2749] T 3cf2c3be9fd244d4b9d4703d424d3c5d P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:13.147688 24451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c88d903166c4812a88653e389ab5776" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:13.148380 24579 ts_tablet_manager.cc:1434] T edd24afe4d7847e3980c6e586b9dac5e P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:09:13.070019 24528 raft_consensus.cc:2468] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 1.
I20260709 13:09:13.089710 24575 ts_tablet_manager.cc:1403] T 142bbcbf57124963866a3967c8adebed P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.296s	user 0.012s	sys 0.002s
I20260709 13:09:13.149515 24579 tablet_bootstrap.cc:492] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:13.151633 24575 raft_consensus.cc:359] T 142bbcbf57124963866a3967c8adebed P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:13.152415 24575 raft_consensus.cc:385] T 142bbcbf57124963866a3967c8adebed P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:13.152726 24575 raft_consensus.cc:740] T 142bbcbf57124963866a3967c8adebed P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:13.153429 24575 consensus_queue.cc:260] T 142bbcbf57124963866a3967c8adebed P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:13.155612 24647 consensus_queue.cc:1048] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:13.156172 24575 ts_tablet_manager.cc:1434] T 142bbcbf57124963866a3967c8adebed P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.005s
I20260709 13:09:13.156500 24579 tablet_bootstrap.cc:654] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:13.157210 24575 tablet_bootstrap.cc:492] T a3e3b771ce2c4165aad02954773c58d7 P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:13.153077 24204 catalog_manager.cc:5697] T c6af3489742e4539866e5a3b10761ca2 P 838cb9997b0f437989d3f4e3e9904390 reported cstate change: term changed from 0 to 1, leader changed from <none> to 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4). New cstate: current_term: 1 leader_uuid: "838cb9997b0f437989d3f4e3e9904390" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:13.111560 24576 raft_consensus.cc:385] T a3e3b771ce2c4165aad02954773c58d7 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:13.164063 24576 raft_consensus.cc:740] T a3e3b771ce2c4165aad02954773c58d7 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
W20260709 13:09:13.167802 24493 leader_election.cc:343] T 9c88d903166c4812a88653e389ab5776 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:13.168349 24493 leader_election.cc:304] T 9c88d903166c4812a88653e389ab5776 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390; no voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:13.171125 24630 raft_consensus.cc:2749] T 9c88d903166c4812a88653e389ab5776 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:13.164108 24304 raft_consensus.cc:2468] T 915d8224f8b64cdab90dff1689046548 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 1.
I20260709 13:09:13.172693 24630 consensus_queue.cc:1048] T 363cde7d4cb84353a74a18cbd9701fa7 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:13.178191 24630 consensus_queue.cc:1048] T 363cde7d4cb84353a74a18cbd9701fa7 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:13.180907 24270 leader_election.cc:304] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381; no voters: 
I20260709 13:09:13.187831 24647 raft_consensus.cc:2804] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:13.189009 24647 raft_consensus.cc:697] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Becoming Leader. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Running, Role: LEADER
I20260709 13:09:13.144533 24379 raft_consensus.cc:2468] T 915d8224f8b64cdab90dff1689046548 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 1.
I20260709 13:09:13.187831 24576 consensus_queue.cc:260] T a3e3b771ce2c4165aad02954773c58d7 P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:13.190122 24647 consensus_queue.cc:237] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:13.224637 24576 ts_tablet_manager.cc:1434] T a3e3b771ce2c4165aad02954773c58d7 P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.136s	user 0.008s	sys 0.000s
I20260709 13:09:13.226210 24577 tablet_bootstrap.cc:654] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:13.238723 24579 tablet_bootstrap.cc:492] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:13.239338 24579 ts_tablet_manager.cc:1403] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.090s	user 0.033s	sys 0.023s
I20260709 13:09:13.242246 24579 raft_consensus.cc:359] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:13.242897 24579 raft_consensus.cc:385] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:13.243162 24579 raft_consensus.cc:740] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:13.243837 24579 consensus_queue.cc:260] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:13.245920 24575 tablet_bootstrap.cc:654] T a3e3b771ce2c4165aad02954773c58d7 P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:13.246187 24579 ts_tablet_manager.cc:1434] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:09:13.247247 24579 tablet_bootstrap.cc:492] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:13.249776 24530 raft_consensus.cc:3060] T 142bbcbf57124963866a3967c8adebed P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:13.261299 24875 consensus_queue.cc:1048] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:09:13.263108 24420 leader_election.cc:304] T 915d8224f8b64cdab90dff1689046548 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381; no voters: 
I20260709 13:09:13.271659 24609 raft_consensus.cc:493] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:13.275338 24579 tablet_bootstrap.cc:654] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:13.270076 24928 raft_consensus.cc:2804] T 915d8224f8b64cdab90dff1689046548 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:13.274677 24609 raft_consensus.cc:515] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:13.278553 24928 raft_consensus.cc:697] T 915d8224f8b64cdab90dff1689046548 P 32088c942fbb48239e0eed952d4df449 [term 1 LEADER]: Becoming Leader. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Running, Role: LEADER
I20260709 13:09:13.279431 24928 consensus_queue.cc:237] T 915d8224f8b64cdab90dff1689046548 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:13.280251 24609 leader_election.cc:290] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:13.281229 24205 catalog_manager.cc:5697] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "9c2a14143eb744cf83bfa5f46284a381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:13.282917 24941 raft_consensus.cc:493] T edd24afe4d7847e3980c6e586b9dac5e P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:13.286314 24941 raft_consensus.cc:515] T edd24afe4d7847e3980c6e586b9dac5e P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:13.285351 24630 consensus_queue.cc:1048] T 84af43749e474087b0a5857a95dfb31d P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:13.288544 24941 leader_election.cc:290] T edd24afe4d7847e3980c6e586b9dac5e P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:13.295740 24576 tablet_bootstrap.cc:492] T dd5ac0d1debc4695a406617ec06f07a0 P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:13.311002 24577 tablet_bootstrap.cc:492] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45: No bootstrap required, opened a new log
I20260709 13:09:13.311779 24577 ts_tablet_manager.cc:1403] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45: Time spent bootstrapping tablet: real 0.207s	user 0.017s	sys 0.000s
I20260709 13:09:13.314849 24577 raft_consensus.cc:359] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:13.322897 24577 raft_consensus.cc:385] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:13.323210 24577 raft_consensus.cc:740] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Initialized, Role: FOLLOWER
I20260709 13:09:13.315225 24690 consensus_queue.cc:1048] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 13:09:13.332075 24304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e219ba07753f49a5abe454b27cd1494e" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:13.323946 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edd24afe4d7847e3980c6e586b9dac5e" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
W20260709 13:09:13.357549 24270 leader_election.cc:343] T edd24afe4d7847e3980c6e586b9dac5e P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:13.358135 24346 leader_election.cc:343] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:13.361119 24529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edd24afe4d7847e3980c6e586b9dac5e" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:13.366516 24454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e219ba07753f49a5abe454b27cd1494e" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:13.354382 24203 catalog_manager.cc:5697] T 915d8224f8b64cdab90dff1689046548 P 32088c942fbb48239e0eed952d4df449 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32088c942fbb48239e0eed952d4df449 (127.14.103.3). New cstate: current_term: 1 leader_uuid: "32088c942fbb48239e0eed952d4df449" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:13.323895 24577 consensus_queue.cc:260] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
W20260709 13:09:13.382548 24270 leader_election.cc:343] T edd24afe4d7847e3980c6e586b9dac5e P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:13.386040 24270 leader_election.cc:304] T edd24afe4d7847e3980c6e586b9dac5e P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381; no voters: 32088c942fbb48239e0eed952d4df449, 838cb9997b0f437989d3f4e3e9904390
I20260709 13:09:13.386375 24576 tablet_bootstrap.cc:654] T dd5ac0d1debc4695a406617ec06f07a0 P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:13.392925 24204 catalog_manager.cc:5697] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "9c2a14143eb744cf83bfa5f46284a381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:13.410036 24965 raft_consensus.cc:2749] T edd24afe4d7847e3980c6e586b9dac5e P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:13.438457 24577 ts_tablet_manager.cc:1434] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45: Time spent starting tablet: real 0.126s	user 0.007s	sys 0.000s
W20260709 13:09:13.442786 24493 consensus_peers.cc:597] T 84af43749e474087b0a5857a95dfb31d P 838cb9997b0f437989d3f4e3e9904390 -> Peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Couldn't send request to peer 32088c942fbb48239e0eed952d4df449. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:13.485066 24270 consensus_peers.cc:597] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381 -> Peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Couldn't send request to peer 32088c942fbb48239e0eed952d4df449. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:13.495437 24575 tablet_bootstrap.cc:492] T a3e3b771ce2c4165aad02954773c58d7 P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:13.496194 24575 ts_tablet_manager.cc:1403] T a3e3b771ce2c4165aad02954773c58d7 P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.339s	user 0.030s	sys 0.027s
I20260709 13:09:13.497308 24579 tablet_bootstrap.cc:492] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:13.497890 24579 ts_tablet_manager.cc:1403] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.251s	user 0.035s	sys 0.050s
W20260709 13:09:13.512902 24345 leader_election.cc:343] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:13.513522 24345 leader_election.cc:304] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:13.511445 24609 raft_consensus.cc:493] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:13.518072 24609 raft_consensus.cc:515] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:13.520294 24609 leader_election.cc:290] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:13.513803 24909 raft_consensus.cc:493] T 624f99e3165f49babbfce35effc9d627 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:13.521195 24909 raft_consensus.cc:515] T 624f99e3165f49babbfce35effc9d627 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:13.521654 24576 tablet_bootstrap.cc:492] T dd5ac0d1debc4695a406617ec06f07a0 P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:13.522166 24576 ts_tablet_manager.cc:1403] T dd5ac0d1debc4695a406617ec06f07a0 P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.227s	user 0.018s	sys 0.004s
I20260709 13:09:13.522593 24304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b8d15473a964a0c8b8c283c4552706e" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:13.523262 24909 leader_election.cc:290] T 624f99e3165f49babbfce35effc9d627 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
W20260709 13:09:13.524083 24346 leader_election.cc:343] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:13.520748 24609 raft_consensus.cc:2749] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:13.525146 24576 raft_consensus.cc:359] T dd5ac0d1debc4695a406617ec06f07a0 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:13.525828 24576 raft_consensus.cc:385] T dd5ac0d1debc4695a406617ec06f07a0 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:13.526113 24576 raft_consensus.cc:740] T dd5ac0d1debc4695a406617ec06f07a0 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:13.526239 24579 raft_consensus.cc:359] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:13.527076 24579 raft_consensus.cc:385] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:13.526729 24576 consensus_queue.cc:260] T dd5ac0d1debc4695a406617ec06f07a0 P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:13.527447 24579 raft_consensus.cc:740] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:13.528376 24579 consensus_queue.cc:260] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:13.529352 24576 ts_tablet_manager.cc:1434] T dd5ac0d1debc4695a406617ec06f07a0 P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.002s
I20260709 13:09:13.540349 24576 tablet_bootstrap.cc:492] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:13.541606 24579 ts_tablet_manager.cc:1434] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.043s	user 0.005s	sys 0.001s
I20260709 13:09:13.542829 24579 tablet_bootstrap.cc:492] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:13.546118 24576 tablet_bootstrap.cc:654] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:13.549083 24875 raft_consensus.cc:493] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:13.549737 24875 raft_consensus.cc:515] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:13.554299 24875 leader_election.cc:290] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:13.555842 24575 raft_consensus.cc:359] T a3e3b771ce2c4165aad02954773c58d7 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:13.556710 24575 raft_consensus.cc:385] T a3e3b771ce2c4165aad02954773c58d7 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:13.556941 24575 raft_consensus.cc:740] T a3e3b771ce2c4165aad02954773c58d7 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:13.559310 24454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd607d60354e4864ae5499ee1c1a05c9" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
W20260709 13:09:13.561080 24345 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:39679, user_credentials={real_user=slave}} blocked reactor thread for 45903.5us
I20260709 13:09:13.561574 24965 raft_consensus.cc:493] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:13.562049 24965 raft_consensus.cc:515] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
W20260709 13:09:13.563261 24270 leader_election.cc:343] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:13.563565 24575 consensus_queue.cc:260] T a3e3b771ce2c4165aad02954773c58d7 P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:13.565316 24965 leader_election.cc:290] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:13.566990 24454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c88d903166c4812a88653e389ab5776" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:13.569125 24527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c88d903166c4812a88653e389ab5776" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
W20260709 13:09:13.569314 24270 leader_election.cc:343] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:13.569825 24527 raft_consensus.cc:2468] T 9c88d903166c4812a88653e389ab5776 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 0.
I20260709 13:09:13.572971 24575 ts_tablet_manager.cc:1434] T a3e3b771ce2c4165aad02954773c58d7 P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.076s	user 0.006s	sys 0.000s
I20260709 13:09:13.573644 24379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd607d60354e4864ae5499ee1c1a05c9" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:13.574005 24575 tablet_bootstrap.cc:492] T 0f795acd16fd472b8c9360fb14e7e5d7 P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:13.574437 24379 raft_consensus.cc:2468] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 0.
I20260709 13:09:13.575412 24270 leader_election.cc:304] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381; no voters: 32088c942fbb48239e0eed952d4df449
I20260709 13:09:13.576587 24965 raft_consensus.cc:2804] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:13.576958 24965 raft_consensus.cc:493] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:13.577353 24965 raft_consensus.cc:3060] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:13.582157 24527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b8d15473a964a0c8b8c283c4552706e" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:13.594861 24579 tablet_bootstrap.cc:654] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:13.595048 24928 raft_consensus.cc:493] T a3e3b771ce2c4165aad02954773c58d7 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:13.595803 24928 raft_consensus.cc:515] T a3e3b771ce2c4165aad02954773c58d7 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:13.599814 24376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e3b771ce2c4165aad02954773c58d7" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:13.600677 24376 raft_consensus.cc:2468] T a3e3b771ce2c4165aad02954773c58d7 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 0.
I20260709 13:09:13.602291 24419 leader_election.cc:304] T a3e3b771ce2c4165aad02954773c58d7 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:13.603492 24966 raft_consensus.cc:2804] T a3e3b771ce2c4165aad02954773c58d7 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:13.604009 24966 raft_consensus.cc:493] T a3e3b771ce2c4165aad02954773c58d7 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:13.604420 24966 raft_consensus.cc:3060] T a3e3b771ce2c4165aad02954773c58d7 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:13.606837 24575 tablet_bootstrap.cc:654] T 0f795acd16fd472b8c9360fb14e7e5d7 P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:13.611107 24454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "624f99e3165f49babbfce35effc9d627" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:13.613418 24527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "624f99e3165f49babbfce35effc9d627" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:13.617192 24928 leader_election.cc:290] T a3e3b771ce2c4165aad02954773c58d7 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:13.617666 24527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e3b771ce2c4165aad02954773c58d7" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:13.618347 24527 raft_consensus.cc:2468] T a3e3b771ce2c4165aad02954773c58d7 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 0.
I20260709 13:09:13.622406 24575 tablet_bootstrap.cc:492] T 0f795acd16fd472b8c9360fb14e7e5d7 P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:13.623097 24575 ts_tablet_manager.cc:1403] T 0f795acd16fd472b8c9360fb14e7e5d7 P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.049s	user 0.009s	sys 0.001s
I20260709 13:09:13.623261 24966 raft_consensus.cc:515] T a3e3b771ce2c4165aad02954773c58d7 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:13.625823 24647 consensus_queue.cc:1048] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:13.626462 24270 leader_election.cc:304] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449
I20260709 13:09:13.627116 24647 consensus_queue.cc:1048] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:13.628052 24382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e3b771ce2c4165aad02954773c58d7" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:13.628182 24963 raft_consensus.cc:2804] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:13.628707 24963 raft_consensus.cc:493] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:13.629087 24963 raft_consensus.cc:3060] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:13.628707 24382 raft_consensus.cc:3060] T a3e3b771ce2c4165aad02954773c58d7 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:13.631413 24965 raft_consensus.cc:515] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:13.640673 24965 leader_election.cc:290] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Requested vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
W20260709 13:09:13.638835 24345 leader_election.cc:343] T 624f99e3165f49babbfce35effc9d627 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:13.641628 24530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e3b771ce2c4165aad02954773c58d7" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:13.632246 24576 tablet_bootstrap.cc:492] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:13.642328 24530 raft_consensus.cc:3060] T a3e3b771ce2c4165aad02954773c58d7 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:13.642475 24576 ts_tablet_manager.cc:1403] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.102s	user 0.010s	sys 0.032s
I20260709 13:09:13.644306 24579 tablet_bootstrap.cc:492] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:13.644796 24579 ts_tablet_manager.cc:1403] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.102s	user 0.013s	sys 0.000s
I20260709 13:09:13.642055 24966 leader_election.cc:290] T a3e3b771ce2c4165aad02954773c58d7 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 election: Requested vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:13.645445 24576 raft_consensus.cc:359] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:13.654529 24454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c88d903166c4812a88653e389ab5776" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:13.649657 24530 raft_consensus.cc:2468] T a3e3b771ce2c4165aad02954773c58d7 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 1.
I20260709 13:09:13.658262 24419 leader_election.cc:304] T a3e3b771ce2c4165aad02954773c58d7 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 838cb9997b0f437989d3f4e3e9904390; no voters: 
I20260709 13:09:13.651484 24528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c88d903166c4812a88653e389ab5776" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:13.647615 24579 raft_consensus.cc:359] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:13.660454 24928 raft_consensus.cc:2804] T a3e3b771ce2c4165aad02954773c58d7 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:13.660735 24579 raft_consensus.cc:385] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:13.660692 24528 raft_consensus.cc:3060] T 9c88d903166c4812a88653e389ab5776 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:13.661128 24928 raft_consensus.cc:697] T a3e3b771ce2c4165aad02954773c58d7 P 32088c942fbb48239e0eed952d4df449 [term 1 LEADER]: Becoming Leader. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Running, Role: LEADER
I20260709 13:09:13.661263 24579 raft_consensus.cc:740] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
W20260709 13:09:13.656710 24345 leader_election.cc:343] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:13.662256 24928 consensus_queue.cc:237] T a3e3b771ce2c4165aad02954773c58d7 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:13.662472 24579 consensus_queue.cc:260] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:13.663127 24345 leader_election.cc:304] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:13.654479 24576 raft_consensus.cc:385] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:13.663950 24576 raft_consensus.cc:740] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:13.664714 24576 consensus_queue.cc:260] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:13.666436 24576 ts_tablet_manager.cc:1434] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.024s	user 0.003s	sys 0.001s
I20260709 13:09:13.655252 24963 raft_consensus.cc:515] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:13.667451 24576 tablet_bootstrap.cc:492] T 84af43749e474087b0a5857a95dfb31d P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:13.668406 24963 leader_election.cc:290] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Requested vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:13.669986 24528 raft_consensus.cc:2468] T 9c88d903166c4812a88653e389ab5776 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 1.
I20260709 13:09:13.670504 24451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd607d60354e4864ae5499ee1c1a05c9" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:13.651408 24382 raft_consensus.cc:2468] T a3e3b771ce2c4165aad02954773c58d7 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 1.
I20260709 13:09:13.668813 24454 raft_consensus.cc:3060] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:13.674963 24976 raft_consensus.cc:2749] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:13.675859 24575 raft_consensus.cc:359] T 0f795acd16fd472b8c9360fb14e7e5d7 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:13.680018 24575 raft_consensus.cc:385] T 0f795acd16fd472b8c9360fb14e7e5d7 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:13.680428 24575 raft_consensus.cc:740] T 0f795acd16fd472b8c9360fb14e7e5d7 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
W20260709 13:09:13.677718 24345 leader_election.cc:343] T 624f99e3165f49babbfce35effc9d627 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:13.681023 24454 raft_consensus.cc:2468] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 1.
I20260709 13:09:13.681138 24345 leader_election.cc:304] T 624f99e3165f49babbfce35effc9d627 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449, 838cb9997b0f437989d3f4e3e9904390
I20260709 13:09:13.682073 24976 raft_consensus.cc:2749] T 624f99e3165f49babbfce35effc9d627 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:09:13.684234 24270 leader_election.cc:343] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:13.684990 24270 leader_election.cc:304] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381; no voters: 
I20260709 13:09:13.686304 24941 raft_consensus.cc:2804] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:13.686759 24941 raft_consensus.cc:697] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Becoming Leader. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Running, Role: LEADER
I20260709 13:09:13.687456 24579 ts_tablet_manager.cc:1434] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.042s	user 0.006s	sys 0.000s
I20260709 13:09:13.687537 24941 consensus_queue.cc:237] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:13.690789 24576 tablet_bootstrap.cc:654] T 84af43749e474087b0a5857a95dfb31d P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:13.697288 24921 consensus_queue.cc:1048] T c6af3489742e4539866e5a3b10761ca2 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:13.703742 24377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd607d60354e4864ae5499ee1c1a05c9" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:13.704470 24377 raft_consensus.cc:3060] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:13.709154 24875 consensus_queue.cc:1048] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:13.697450 24575 consensus_queue.cc:260] T 0f795acd16fd472b8c9360fb14e7e5d7 P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:13.712344 24304 raft_consensus.cc:3060] T c6af3489742e4539866e5a3b10761ca2 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:13.714912 24377 raft_consensus.cc:2468] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 1.
I20260709 13:09:13.723160 24928 consensus_queue.cc:1048] T 915d8224f8b64cdab90dff1689046548 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:13.726437 24941 raft_consensus.cc:493] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:13.727008 24941 raft_consensus.cc:515] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:13.729604 24941 leader_election.cc:290] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:13.730020 24579 tablet_bootstrap.cc:492] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:13.730456 24203 catalog_manager.cc:5697] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "9c2a14143eb744cf83bfa5f46284a381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:13.733158 24527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f795acd16fd472b8c9360fb14e7e5d7" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:13.748172 24527 raft_consensus.cc:2468] T 0f795acd16fd472b8c9360fb14e7e5d7 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 0.
I20260709 13:09:13.751103 24205 catalog_manager.cc:5697] T a3e3b771ce2c4165aad02954773c58d7 P 32088c942fbb48239e0eed952d4df449 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32088c942fbb48239e0eed952d4df449 (127.14.103.3). New cstate: current_term: 1 leader_uuid: "32088c942fbb48239e0eed952d4df449" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:13.753983 24270 leader_election.cc:304] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381; no voters: 
I20260709 13:09:13.755401 24965 raft_consensus.cc:2804] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:13.756359 24965 raft_consensus.cc:493] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:13.757189 24965 raft_consensus.cc:3060] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:13.765178 24579 tablet_bootstrap.cc:654] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:13.774854 24576 tablet_bootstrap.cc:492] T 84af43749e474087b0a5857a95dfb31d P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:13.775343 24576 ts_tablet_manager.cc:1403] T 84af43749e474087b0a5857a95dfb31d P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.108s	user 0.009s	sys 0.004s
I20260709 13:09:13.783752 24941 consensus_queue.cc:1048] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:09:13.790764 24270 leader_election.cc:304] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449
I20260709 13:09:13.792870 24575 ts_tablet_manager.cc:1434] T 0f795acd16fd472b8c9360fb14e7e5d7 P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.169s	user 0.006s	sys 0.003s
I20260709 13:09:13.807915 24875 raft_consensus.cc:2804] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:13.796401 24649 consensus_queue.cc:1048] T c6af3489742e4539866e5a3b10761ca2 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 13:09:13.795131 24376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f795acd16fd472b8c9360fb14e7e5d7" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:13.813170 24376 raft_consensus.cc:2468] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 0.
I20260709 13:09:13.819659 24928 consensus_queue.cc:1048] T 915d8224f8b64cdab90dff1689046548 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:13.838900 24575 tablet_bootstrap.cc:492] T 624f99e3165f49babbfce35effc9d627 P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:13.851204 24575 tablet_bootstrap.cc:654] T 624f99e3165f49babbfce35effc9d627 P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:13.787099 24579 tablet_bootstrap.cc:492] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:13.854714 24579 ts_tablet_manager.cc:1403] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.138s	user 0.017s	sys 0.007s
I20260709 13:09:13.861873 24579 raft_consensus.cc:359] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:13.862560 24579 raft_consensus.cc:385] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:13.862848 24579 raft_consensus.cc:740] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:13.842222 24965 raft_consensus.cc:515] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:13.868294 24576 raft_consensus.cc:359] T 84af43749e474087b0a5857a95dfb31d P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:13.869220 24576 raft_consensus.cc:385] T 84af43749e474087b0a5857a95dfb31d P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:13.869596 24576 raft_consensus.cc:740] T 84af43749e474087b0a5857a95dfb31d P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
W20260709 13:09:13.869798 24270 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:46177, user_credentials={real_user=slave}} blocked reactor thread for 67801.5us
I20260709 13:09:13.870831 24576 consensus_queue.cc:260] T 84af43749e474087b0a5857a95dfb31d P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:13.873489 24527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f795acd16fd472b8c9360fb14e7e5d7" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:13.874101 24527 raft_consensus.cc:3060] T 0f795acd16fd472b8c9360fb14e7e5d7 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:13.877195 24375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f795acd16fd472b8c9360fb14e7e5d7" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:13.878944 24576 ts_tablet_manager.cc:1434] T 84af43749e474087b0a5857a95dfb31d P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.103s	user 0.002s	sys 0.006s
I20260709 13:09:13.880131 24375 raft_consensus.cc:3060] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:13.869027 24965 leader_election.cc:290] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Requested vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:13.881882 24579 consensus_queue.cc:260] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
W20260709 13:09:13.886348 24253 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:13.887354 24576 tablet_bootstrap.cc:492] T 624f99e3165f49babbfce35effc9d627 P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:13.888578 24928 raft_consensus.cc:493] T dd5ac0d1debc4695a406617ec06f07a0 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:13.889096 24928 raft_consensus.cc:515] T dd5ac0d1debc4695a406617ec06f07a0 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:13.891268 24928 leader_election.cc:290] T dd5ac0d1debc4695a406617ec06f07a0 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:13.894255 24576 tablet_bootstrap.cc:654] T 624f99e3165f49babbfce35effc9d627 P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:13.894641 24376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd5ac0d1debc4695a406617ec06f07a0" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:13.912688 24307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd5ac0d1debc4695a406617ec06f07a0" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:13.917618 24966 raft_consensus.cc:493] T 84af43749e474087b0a5857a95dfb31d P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:13.917837 24420 leader_election.cc:304] T dd5ac0d1debc4695a406617ec06f07a0 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32088c942fbb48239e0eed952d4df449; no voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:13.918206 24966 raft_consensus.cc:515] T 84af43749e474087b0a5857a95dfb31d P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:13.919848 24680 raft_consensus.cc:2749] T dd5ac0d1debc4695a406617ec06f07a0 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:13.925500 24966 leader_election.cc:290] T 84af43749e474087b0a5857a95dfb31d P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:13.927011 24530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84af43749e474087b0a5857a95dfb31d" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:13.931085 24379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84af43749e474087b0a5857a95dfb31d" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:13.933811 24419 leader_election.cc:304] T 84af43749e474087b0a5857a95dfb31d P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32088c942fbb48239e0eed952d4df449; no voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:13.934912 24966 raft_consensus.cc:2749] T 84af43749e474087b0a5857a95dfb31d P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:13.901626 24875 raft_consensus.cc:697] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Becoming Leader. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Running, Role: LEADER
I20260709 13:09:13.939275 24875 consensus_queue.cc:237] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:13.972815 24375 raft_consensus.cc:2468] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 1.
W20260709 13:09:13.977813 24270 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:39679, user_credentials={real_user=slave}} blocked reactor thread for 78853.4us
I20260709 13:09:13.995925 24579 ts_tablet_manager.cc:1434] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.141s	user 0.007s	sys 0.001s
I20260709 13:09:13.998250 24527 raft_consensus.cc:2468] T 0f795acd16fd472b8c9360fb14e7e5d7 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 1.
I20260709 13:09:14.009725 24576 tablet_bootstrap.cc:492] T 624f99e3165f49babbfce35effc9d627 P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:14.010212 24576 ts_tablet_manager.cc:1403] T 624f99e3165f49babbfce35effc9d627 P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.123s	user 0.033s	sys 0.040s
I20260709 13:09:14.011399 24270 leader_election.cc:304] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:14.012509 24647 raft_consensus.cc:2804] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:14.014555 24647 raft_consensus.cc:697] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Becoming Leader. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Running, Role: LEADER
I20260709 13:09:14.015362 24647 consensus_queue.cc:237] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:14.012517 24579 tablet_bootstrap.cc:492] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:14.032671 24575 tablet_bootstrap.cc:492] T 624f99e3165f49babbfce35effc9d627 P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:14.033217 24575 ts_tablet_manager.cc:1403] T 624f99e3165f49babbfce35effc9d627 P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.195s	user 0.019s	sys 0.019s
I20260709 13:09:14.035924 24575 raft_consensus.cc:359] T 624f99e3165f49babbfce35effc9d627 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:14.036576 24575 raft_consensus.cc:385] T 624f99e3165f49babbfce35effc9d627 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.036844 24575 raft_consensus.cc:740] T 624f99e3165f49babbfce35effc9d627 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.037474 24575 consensus_queue.cc:260] T 624f99e3165f49babbfce35effc9d627 P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:14.039376 24576 raft_consensus.cc:359] T 624f99e3165f49babbfce35effc9d627 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:14.040062 24576 raft_consensus.cc:385] T 624f99e3165f49babbfce35effc9d627 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.040323 24576 raft_consensus.cc:740] T 624f99e3165f49babbfce35effc9d627 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.040939 24576 consensus_queue.cc:260] T 624f99e3165f49babbfce35effc9d627 P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:14.047848 24205 catalog_manager.cc:5697] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "9c2a14143eb744cf83bfa5f46284a381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:14.055112 24579 tablet_bootstrap.cc:654] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:14.049535 24576 ts_tablet_manager.cc:1434] T 624f99e3165f49babbfce35effc9d627 P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.000s
I20260709 13:09:14.056810 24576 tablet_bootstrap.cc:492] T edd24afe4d7847e3980c6e586b9dac5e P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:14.062871 24576 tablet_bootstrap.cc:654] T edd24afe4d7847e3980c6e586b9dac5e P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:14.051035 24575 ts_tablet_manager.cc:1434] T 624f99e3165f49babbfce35effc9d627 P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.001s
I20260709 13:09:14.064559 24575 tablet_bootstrap.cc:492] T edd24afe4d7847e3980c6e586b9dac5e P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:14.074512 24454 raft_consensus.cc:3060] T dd5ac0d1debc4695a406617ec06f07a0 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:14.075731 24575 tablet_bootstrap.cc:654] T edd24afe4d7847e3980c6e586b9dac5e P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:14.082005 24928 consensus_queue.cc:1048] T a3e3b771ce2c4165aad02954773c58d7 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:14.096678 24579 tablet_bootstrap.cc:492] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
I20260709 13:09:14.097205 24579 ts_tablet_manager.cc:1403] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.085s	user 0.011s	sys 0.000s
I20260709 13:09:14.109900 24579 raft_consensus.cc:359] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:14.110633 24579 raft_consensus.cc:385] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.110924 24579 raft_consensus.cc:740] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.111590 24928 consensus_queue.cc:1048] T a3e3b771ce2c4165aad02954773c58d7 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:14.113873 24579 consensus_queue.cc:260] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:14.116194 24579 ts_tablet_manager.cc:1434] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 13:09:14.121579 24579 tablet_bootstrap.cc:492] T 86262701775a456dacd47fcdc1a5d13b P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:14.132513 24579 tablet_bootstrap.cc:654] T 86262701775a456dacd47fcdc1a5d13b P 9c2a14143eb744cf83bfa5f46284a381: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:14.154834 24921 raft_consensus.cc:493] T 624f99e3165f49babbfce35effc9d627 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:14.155442 24921 raft_consensus.cc:515] T 624f99e3165f49babbfce35effc9d627 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:14.158993 24921 leader_election.cc:290] T 624f99e3165f49babbfce35effc9d627 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:14.161084 24647 consensus_queue.cc:1048] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:14.163170 24647 raft_consensus.cc:493] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:14.163758 24647 raft_consensus.cc:515] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:14.166687 24647 leader_election.cc:290] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:14.168691 24528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b8d15473a964a0c8b8c283c4552706e" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
W20260709 13:09:14.170322 24270 leader_election.cc:343] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:14.171079 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "624f99e3165f49babbfce35effc9d627" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:14.172048 24455 raft_consensus.cc:2468] T 624f99e3165f49babbfce35effc9d627 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 0.
I20260709 13:09:14.179641 24647 consensus_queue.cc:1048] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:14.188217 24493 leader_election.cc:304] T 624f99e3165f49babbfce35effc9d627 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 838cb9997b0f437989d3f4e3e9904390; no voters: 
I20260709 13:09:14.189055 24630 raft_consensus.cc:2804] T 624f99e3165f49babbfce35effc9d627 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:14.191546 24630 raft_consensus.cc:493] T 624f99e3165f49babbfce35effc9d627 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:14.192070 24630 raft_consensus.cc:3060] T 624f99e3165f49babbfce35effc9d627 P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:14.216419 24377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b8d15473a964a0c8b8c283c4552706e" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:14.217427 24377 raft_consensus.cc:2468] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 0.
I20260709 13:09:14.239722 24203 catalog_manager.cc:5697] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "9c2a14143eb744cf83bfa5f46284a381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:14.252172 24270 leader_election.cc:304] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 838cb9997b0f437989d3f4e3e9904390
I20260709 13:09:14.253356 24647 raft_consensus.cc:2804] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:14.253747 24647 raft_consensus.cc:493] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:14.254114 24647 raft_consensus.cc:3060] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:14.260329 24377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "624f99e3165f49babbfce35effc9d627" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:14.261121 24377 raft_consensus.cc:2468] T 624f99e3165f49babbfce35effc9d627 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 0.
I20260709 13:09:14.265655 24647 raft_consensus.cc:515] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:14.269475 24647 leader_election.cc:290] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Requested vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:14.288115 24451 raft_consensus.cc:3060] T 84af43749e474087b0a5857a95dfb31d P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:14.304422 24529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b8d15473a964a0c8b8c283c4552706e" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:14.314872 24630 raft_consensus.cc:515] T 624f99e3165f49babbfce35effc9d627 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:14.337514 24576 tablet_bootstrap.cc:492] T edd24afe4d7847e3980c6e586b9dac5e P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:14.319334 24579 tablet_bootstrap.cc:492] T 86262701775a456dacd47fcdc1a5d13b P 9c2a14143eb744cf83bfa5f46284a381: No bootstrap required, opened a new log
W20260709 13:09:14.321007 24270 leader_election.cc:343] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:14.331875 24379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b8d15473a964a0c8b8c283c4552706e" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:14.338078 24576 ts_tablet_manager.cc:1403] T edd24afe4d7847e3980c6e586b9dac5e P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.282s	user 0.045s	sys 0.060s
I20260709 13:09:14.338311 24579 ts_tablet_manager.cc:1403] T 86262701775a456dacd47fcdc1a5d13b P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.221s	user 0.032s	sys 0.024s
I20260709 13:09:14.338614 24379 raft_consensus.cc:3060] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:14.341089 24576 raft_consensus.cc:359] T edd24afe4d7847e3980c6e586b9dac5e P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:14.341774 24576 raft_consensus.cc:385] T edd24afe4d7847e3980c6e586b9dac5e P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.342015 24576 raft_consensus.cc:740] T edd24afe4d7847e3980c6e586b9dac5e P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.342664 24576 consensus_queue.cc:260] T edd24afe4d7847e3980c6e586b9dac5e P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:14.343708 24375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "624f99e3165f49babbfce35effc9d627" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:14.344326 24375 raft_consensus.cc:3060] T 624f99e3165f49babbfce35effc9d627 P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:14.344291 24454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "624f99e3165f49babbfce35effc9d627" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:14.345093 24454 raft_consensus.cc:3060] T 624f99e3165f49babbfce35effc9d627 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:14.344717 24576 ts_tablet_manager.cc:1434] T edd24afe4d7847e3980c6e586b9dac5e P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:09:14.349818 24576 tablet_bootstrap.cc:492] T bd607d60354e4864ae5499ee1c1a05c9 P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:14.352819 24375 raft_consensus.cc:2468] T 624f99e3165f49babbfce35effc9d627 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 1.
I20260709 13:09:14.354501 24493 leader_election.cc:304] T 624f99e3165f49babbfce35effc9d627 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:14.355715 24772 raft_consensus.cc:2804] T 624f99e3165f49babbfce35effc9d627 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:14.356204 24772 raft_consensus.cc:697] T 624f99e3165f49babbfce35effc9d627 P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Becoming Leader. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Running, Role: LEADER
I20260709 13:09:14.356501 24680 raft_consensus.cc:493] T edd24afe4d7847e3980c6e586b9dac5e P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:14.357062 24680 raft_consensus.cc:515] T edd24afe4d7847e3980c6e586b9dac5e P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:14.357149 24772 consensus_queue.cc:237] T 624f99e3165f49babbfce35effc9d627 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:14.357875 24576 tablet_bootstrap.cc:654] T bd607d60354e4864ae5499ee1c1a05c9 P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:14.359279 24680 leader_election.cc:290] T edd24afe4d7847e3980c6e586b9dac5e P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:14.360992 24304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edd24afe4d7847e3980c6e586b9dac5e" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:14.361630 24304 raft_consensus.cc:2468] T edd24afe4d7847e3980c6e586b9dac5e P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 0.
I20260709 13:09:14.362952 24420 leader_election.cc:304] T edd24afe4d7847e3980c6e586b9dac5e P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381; no voters: 
I20260709 13:09:14.363859 24529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edd24afe4d7847e3980c6e586b9dac5e" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:14.364837 24966 raft_consensus.cc:2804] T edd24afe4d7847e3980c6e586b9dac5e P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:14.365273 24966 raft_consensus.cc:493] T edd24afe4d7847e3980c6e586b9dac5e P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:14.365605 24966 raft_consensus.cc:3060] T edd24afe4d7847e3980c6e586b9dac5e P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:09:14.365273 24419 leader_election.cc:343] T edd24afe4d7847e3980c6e586b9dac5e P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:14.369657 24576 tablet_bootstrap.cc:492] T bd607d60354e4864ae5499ee1c1a05c9 P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:14.370265 24576 ts_tablet_manager.cc:1403] T bd607d60354e4864ae5499ee1c1a05c9 P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.001s
I20260709 13:09:14.373914 24576 raft_consensus.cc:359] T bd607d60354e4864ae5499ee1c1a05c9 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:14.374819 24576 raft_consensus.cc:385] T bd607d60354e4864ae5499ee1c1a05c9 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.375160 24576 raft_consensus.cc:740] T bd607d60354e4864ae5499ee1c1a05c9 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.376529 24576 consensus_queue.cc:260] T bd607d60354e4864ae5499ee1c1a05c9 P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:14.379415 24576 ts_tablet_manager.cc:1434] T bd607d60354e4864ae5499ee1c1a05c9 P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:09:14.380259 24379 raft_consensus.cc:2468] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 1.
I20260709 13:09:14.380813 24576 tablet_bootstrap.cc:492] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:14.381332 24454 raft_consensus.cc:2468] T 624f99e3165f49babbfce35effc9d627 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 1.
W20260709 13:09:14.383054 24270 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.4:33251, user_credentials={real_user=slave}} blocked reactor thread for 62062us
I20260709 13:09:14.391341 24203 catalog_manager.cc:5697] T 624f99e3165f49babbfce35effc9d627 P 838cb9997b0f437989d3f4e3e9904390 reported cstate change: term changed from 0 to 1, leader changed from <none> to 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4). New cstate: current_term: 1 leader_uuid: "838cb9997b0f437989d3f4e3e9904390" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:14.379441 24966 raft_consensus.cc:515] T edd24afe4d7847e3980c6e586b9dac5e P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:14.399045 24647 raft_consensus.cc:493] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:14.399580 24647 raft_consensus.cc:515] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:14.399840 24529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edd24afe4d7847e3980c6e586b9dac5e" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:14.401379 24647 leader_election.cc:290] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
W20260709 13:09:14.401449 24419 leader_election.cc:343] T edd24afe4d7847e3980c6e586b9dac5e P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:14.399016 24575 tablet_bootstrap.cc:492] T edd24afe4d7847e3980c6e586b9dac5e P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:14.402302 24575 ts_tablet_manager.cc:1403] T edd24afe4d7847e3980c6e586b9dac5e P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.338s	user 0.057s	sys 0.103s
I20260709 13:09:14.388985 24630 leader_election.cc:290] T 624f99e3165f49babbfce35effc9d627 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Requested vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:14.388016 24579 raft_consensus.cc:359] T 86262701775a456dacd47fcdc1a5d13b P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:14.404026 24579 raft_consensus.cc:385] T 86262701775a456dacd47fcdc1a5d13b P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.404379 24579 raft_consensus.cc:740] T 86262701775a456dacd47fcdc1a5d13b P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.405149 24575 raft_consensus.cc:359] T edd24afe4d7847e3980c6e586b9dac5e P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:14.405220 24579 consensus_queue.cc:260] T 86262701775a456dacd47fcdc1a5d13b P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:14.406002 24575 raft_consensus.cc:385] T edd24afe4d7847e3980c6e586b9dac5e P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.406338 24575 raft_consensus.cc:740] T edd24afe4d7847e3980c6e586b9dac5e P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.405172 24576 tablet_bootstrap.cc:654] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:14.406982 24575 consensus_queue.cc:260] T edd24afe4d7847e3980c6e586b9dac5e P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:14.409159 24575 ts_tablet_manager.cc:1434] T edd24afe4d7847e3980c6e586b9dac5e P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:09:14.409966 24579 ts_tablet_manager.cc:1434] T 86262701775a456dacd47fcdc1a5d13b P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.071s	user 0.005s	sys 0.001s
I20260709 13:09:14.415946 24966 leader_election.cc:290] T edd24afe4d7847e3980c6e586b9dac5e P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 election: Requested vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:14.418965 24576 tablet_bootstrap.cc:492] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:14.417629 24270 leader_election.cc:304] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 838cb9997b0f437989d3f4e3e9904390
I20260709 13:09:14.417544 24306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edd24afe4d7847e3980c6e586b9dac5e" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:14.420214 24306 raft_consensus.cc:3060] T edd24afe4d7847e3980c6e586b9dac5e P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:14.428171 24306 raft_consensus.cc:2468] T edd24afe4d7847e3980c6e586b9dac5e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 1.
I20260709 13:09:14.419602 24576 ts_tablet_manager.cc:1403] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.039s	user 0.018s	sys 0.002s
I20260709 13:09:14.429490 24420 leader_election.cc:304] T edd24afe4d7847e3980c6e586b9dac5e P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381; no voters: 838cb9997b0f437989d3f4e3e9904390
I20260709 13:09:14.410734 24575 tablet_bootstrap.cc:492] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:14.431435 24576 raft_consensus.cc:359] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:14.432130 24576 raft_consensus.cc:385] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.432402 24576 raft_consensus.cc:740] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.433218 24576 consensus_queue.cc:260] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:14.434101 24966 raft_consensus.cc:2804] T edd24afe4d7847e3980c6e586b9dac5e P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:14.434863 24994 raft_consensus.cc:2804] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:14.435909 24966 raft_consensus.cc:697] T edd24afe4d7847e3980c6e586b9dac5e P 32088c942fbb48239e0eed952d4df449 [term 1 LEADER]: Becoming Leader. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Running, Role: LEADER
I20260709 13:09:14.435961 24994 raft_consensus.cc:697] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Becoming Leader. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Running, Role: LEADER
I20260709 13:09:14.436810 24994 consensus_queue.cc:237] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:14.435377 24576 ts_tablet_manager.cc:1434] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:09:14.440358 24575 tablet_bootstrap.cc:654] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:14.440177 24647 raft_consensus.cc:493] T 86262701775a456dacd47fcdc1a5d13b P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:14.441018 24647 raft_consensus.cc:515] T 86262701775a456dacd47fcdc1a5d13b P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:14.442826 24647 leader_election.cc:290] T 86262701775a456dacd47fcdc1a5d13b P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:14.447149 24451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e219ba07753f49a5abe454b27cd1494e" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:14.448627 24379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e219ba07753f49a5abe454b27cd1494e" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:14.449286 24379 raft_consensus.cc:2468] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 0.
I20260709 13:09:14.451066 24875 consensus_queue.cc:1048] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:14.452538 24576 tablet_bootstrap.cc:492] T 86262701775a456dacd47fcdc1a5d13b P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:14.457562 24454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86262701775a456dacd47fcdc1a5d13b" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:14.457746 24529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86262701775a456dacd47fcdc1a5d13b" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:14.436829 24966 consensus_queue.cc:237] T edd24afe4d7847e3980c6e586b9dac5e P 32088c942fbb48239e0eed952d4df449 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:14.463249 24680 raft_consensus.cc:493] T bd607d60354e4864ae5499ee1c1a05c9 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:14.463817 24680 raft_consensus.cc:515] T bd607d60354e4864ae5499ee1c1a05c9 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:14.466485 24680 leader_election.cc:290] T bd607d60354e4864ae5499ee1c1a05c9 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:14.466665 24379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd607d60354e4864ae5499ee1c1a05c9" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:14.467499 24379 raft_consensus.cc:2393] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 32088c942fbb48239e0eed952d4df449 in current term 1: Already voted for candidate 9c2a14143eb744cf83bfa5f46284a381 in this term.
I20260709 13:09:14.468020 24306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd607d60354e4864ae5499ee1c1a05c9" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:14.471904 24575 tablet_bootstrap.cc:492] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:14.471808 24420 leader_election.cc:304] T bd607d60354e4864ae5499ee1c1a05c9 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32088c942fbb48239e0eed952d4df449; no voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:14.472455 24575 ts_tablet_manager.cc:1403] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.062s	user 0.018s	sys 0.011s
I20260709 13:09:14.473228 24928 raft_consensus.cc:3060] T bd607d60354e4864ae5499ee1c1a05c9 P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:09:14.475138 24270 leader_election.cc:343] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:14.475318 24575 raft_consensus.cc:359] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:14.476022 24575 raft_consensus.cc:385] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.476302 24575 raft_consensus.cc:740] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.477299 24576 tablet_bootstrap.cc:654] T 86262701775a456dacd47fcdc1a5d13b P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:14.476954 24575 consensus_queue.cc:260] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
W20260709 13:09:14.477650 24270 leader_election.cc:343] T 86262701775a456dacd47fcdc1a5d13b P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:14.478827 24270 leader_election.cc:304] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449
I20260709 13:09:14.481236 24202 catalog_manager.cc:5697] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "9c2a14143eb744cf83bfa5f46284a381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:14.483858 24994 raft_consensus.cc:2804] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:14.484284 24994 raft_consensus.cc:493] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:14.487053 24575 ts_tablet_manager.cc:1434] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:09:14.484633 24994 raft_consensus.cc:3060] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:14.493134 24575 tablet_bootstrap.cc:492] T 86262701775a456dacd47fcdc1a5d13b P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
W20260709 13:09:14.502024 24270 leader_election.cc:343] T 86262701775a456dacd47fcdc1a5d13b P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:14.502759 24270 leader_election.cc:304] T 86262701775a456dacd47fcdc1a5d13b P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381; no voters: 32088c942fbb48239e0eed952d4df449, 838cb9997b0f437989d3f4e3e9904390
I20260709 13:09:14.508754 24647 raft_consensus.cc:2749] T 86262701775a456dacd47fcdc1a5d13b P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:09:14.539863 24420 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:45351, user_credentials={real_user=slave}} blocked reactor thread for 66979.1us
I20260709 13:09:14.542415 24928 raft_consensus.cc:2749] T bd607d60354e4864ae5499ee1c1a05c9 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:14.561081 24202 catalog_manager.cc:5697] T edd24afe4d7847e3980c6e586b9dac5e P 32088c942fbb48239e0eed952d4df449 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32088c942fbb48239e0eed952d4df449 (127.14.103.3). New cstate: current_term: 1 leader_uuid: "32088c942fbb48239e0eed952d4df449" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:14.587786 24690 consensus_queue.cc:1048] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:14.588013 24994 raft_consensus.cc:515] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:14.590099 24994 leader_election.cc:290] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Requested vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:14.593241 24647 consensus_queue.cc:1048] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:14.595506 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e219ba07753f49a5abe454b27cd1494e" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:14.598512 24576 tablet_bootstrap.cc:492] T 86262701775a456dacd47fcdc1a5d13b P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:14.600771 24576 ts_tablet_manager.cc:1403] T 86262701775a456dacd47fcdc1a5d13b P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.148s	user 0.018s	sys 0.015s
I20260709 13:09:14.602741 24575 tablet_bootstrap.cc:654] T 86262701775a456dacd47fcdc1a5d13b P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:14.604592 24377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e219ba07753f49a5abe454b27cd1494e" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:14.605248 24377 raft_consensus.cc:3060] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:14.605350 24921 raft_consensus.cc:493] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:14.605948 24921 raft_consensus.cc:515] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
W20260709 13:09:14.606436 24270 leader_election.cc:343] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:14.616739 24921 leader_election.cc:290] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:14.618841 25019 raft_consensus.cc:493] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:14.619442 25019 raft_consensus.cc:515] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:14.621894 25019 leader_election.cc:290] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:14.624871 24451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf2c3be9fd244d4b9d4703d424d3c5d" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:14.625082 24529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf2c3be9fd244d4b9d4703d424d3c5d" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:14.625583 24451 raft_consensus.cc:2468] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 0.
I20260709 13:09:14.625718 24529 raft_consensus.cc:2468] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 0.
I20260709 13:09:14.627609 24493 leader_election.cc:304] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 838cb9997b0f437989d3f4e3e9904390; no voters: 
I20260709 13:09:14.627225 24419 leader_election.cc:304] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 838cb9997b0f437989d3f4e3e9904390; no voters: 
I20260709 13:09:14.632797 24576 raft_consensus.cc:359] T 86262701775a456dacd47fcdc1a5d13b P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:14.639540 24381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf2c3be9fd244d4b9d4703d424d3c5d" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:14.640039 24576 raft_consensus.cc:385] T 86262701775a456dacd47fcdc1a5d13b P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.640290 24381 raft_consensus.cc:2468] T 3cf2c3be9fd244d4b9d4703d424d3c5d P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 0.
I20260709 13:09:14.640473 24576 raft_consensus.cc:740] T 86262701775a456dacd47fcdc1a5d13b P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.641466 24576 consensus_queue.cc:260] T 86262701775a456dacd47fcdc1a5d13b P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:14.643991 25021 raft_consensus.cc:2804] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:14.644433 25021 raft_consensus.cc:493] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:14.644758 25021 raft_consensus.cc:3060] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:14.648825 24928 raft_consensus.cc:2804] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:14.651445 24576 ts_tablet_manager.cc:1434] T 86262701775a456dacd47fcdc1a5d13b P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.049s	user 0.006s	sys 0.001s
I20260709 13:09:14.649272 24928 raft_consensus.cc:493] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:14.626847 24921 raft_consensus.cc:493] T edd24afe4d7847e3980c6e586b9dac5e P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:14.660554 24928 raft_consensus.cc:3060] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:14.668363 24921 raft_consensus.cc:515] T edd24afe4d7847e3980c6e586b9dac5e P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:14.672204 24382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf2c3be9fd244d4b9d4703d424d3c5d" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:14.673110 24382 raft_consensus.cc:2468] T 3cf2c3be9fd244d4b9d4703d424d3c5d P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 0.
I20260709 13:09:14.673141 24576 tablet_bootstrap.cc:492] T e219ba07753f49a5abe454b27cd1494e P 32088c942fbb48239e0eed952d4df449: Bootstrap starting.
I20260709 13:09:14.679100 24994 consensus_queue.cc:1048] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:14.682178 24451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edd24afe4d7847e3980c6e586b9dac5e" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:14.695896 24921 leader_election.cc:290] T edd24afe4d7847e3980c6e586b9dac5e P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:14.697875 24306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edd24afe4d7847e3980c6e586b9dac5e" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:14.700657 24306 raft_consensus.cc:2393] T edd24afe4d7847e3980c6e586b9dac5e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 838cb9997b0f437989d3f4e3e9904390 in current term 1: Already voted for candidate 32088c942fbb48239e0eed952d4df449 in this term.
I20260709 13:09:14.701362 24377 raft_consensus.cc:2468] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 1.
I20260709 13:09:14.702889 24494 leader_election.cc:304] T edd24afe4d7847e3980c6e586b9dac5e P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390; no voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:14.704496 24772 raft_consensus.cc:3060] T edd24afe4d7847e3980c6e586b9dac5e P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:14.710073 24270 leader_election.cc:304] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449
I20260709 13:09:14.715183 24647 raft_consensus.cc:2804] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:14.715651 24647 raft_consensus.cc:697] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Becoming Leader. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Running, Role: LEADER
I20260709 13:09:14.716388 24647 consensus_queue.cc:237] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:14.718225 24928 raft_consensus.cc:515] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:14.719558 24576 tablet_bootstrap.cc:654] T e219ba07753f49a5abe454b27cd1494e P 32088c942fbb48239e0eed952d4df449: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:14.720422 25021 raft_consensus.cc:515] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
W20260709 13:09:14.747334 24419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:46177, user_credentials={real_user=slave}} blocked reactor thread for 48458.4us
I20260709 13:09:14.763624 24928 leader_election.cc:290] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 election: Requested vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:14.781724 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf2c3be9fd244d4b9d4703d424d3c5d" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:14.781841 24528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf2c3be9fd244d4b9d4703d424d3c5d" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:14.784360 24528 raft_consensus.cc:2393] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 32088c942fbb48239e0eed952d4df449 in current term 1: Already voted for candidate 838cb9997b0f437989d3f4e3e9904390 in this term.
I20260709 13:09:14.785169 24379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf2c3be9fd244d4b9d4703d424d3c5d" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:14.785847 24379 raft_consensus.cc:3060] T 3cf2c3be9fd244d4b9d4703d424d3c5d P c40e7ba102224d24a22ccd1f68aaae45 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:14.780372 25021 leader_election.cc:290] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Requested vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:14.791088 24455 raft_consensus.cc:2393] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 838cb9997b0f437989d3f4e3e9904390 in current term 1: Already voted for candidate 32088c942fbb48239e0eed952d4df449 in this term.
I20260709 13:09:14.801932 24203 catalog_manager.cc:5697] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "9c2a14143eb744cf83bfa5f46284a381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:14.803131 24376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf2c3be9fd244d4b9d4703d424d3c5d" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:14.808882 24575 tablet_bootstrap.cc:492] T 86262701775a456dacd47fcdc1a5d13b P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:14.809432 24575 ts_tablet_manager.cc:1403] T 86262701775a456dacd47fcdc1a5d13b P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.317s	user 0.031s	sys 0.054s
I20260709 13:09:14.809921 24772 raft_consensus.cc:2749] T edd24afe4d7847e3980c6e586b9dac5e P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:14.817162 24575 raft_consensus.cc:359] T 86262701775a456dacd47fcdc1a5d13b P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:14.818341 24419 leader_election.cc:304] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32088c942fbb48239e0eed952d4df449; no voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:14.819307 24575 raft_consensus.cc:385] T 86262701775a456dacd47fcdc1a5d13b P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.819860 24575 raft_consensus.cc:740] T 86262701775a456dacd47fcdc1a5d13b P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.825141 24575 consensus_queue.cc:260] T 86262701775a456dacd47fcdc1a5d13b P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:14.829362 24575 ts_tablet_manager.cc:1434] T 86262701775a456dacd47fcdc1a5d13b P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.001s
W20260709 13:09:14.830286 24270 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:33251, user_credentials={real_user=slave}} blocked reactor thread for 98894.4us
I20260709 13:09:14.831869 24630 raft_consensus.cc:493] T 86262701775a456dacd47fcdc1a5d13b P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:14.834017 24630 raft_consensus.cc:515] T 86262701775a456dacd47fcdc1a5d13b P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:14.830233 24966 raft_consensus.cc:2749] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:09:14.857396 24575 tablet_bootstrap.cc:492] T 5b8d15473a964a0c8b8c283c4552706e P 838cb9997b0f437989d3f4e3e9904390: Bootstrap starting.
I20260709 13:09:14.870435 24575 tablet_bootstrap.cc:654] T 5b8d15473a964a0c8b8c283c4552706e P 838cb9997b0f437989d3f4e3e9904390: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:14.863200 24630 leader_election.cc:290] T 86262701775a456dacd47fcdc1a5d13b P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:14.855764 24379 raft_consensus.cc:2468] T 3cf2c3be9fd244d4b9d4703d424d3c5d P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 1.
I20260709 13:09:14.863536 24306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86262701775a456dacd47fcdc1a5d13b" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:14.872215 24306 raft_consensus.cc:2468] T 86262701775a456dacd47fcdc1a5d13b P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 0.
I20260709 13:09:14.873379 24494 leader_election.cc:304] T 86262701775a456dacd47fcdc1a5d13b P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381; no voters: 
I20260709 13:09:14.874320 25021 raft_consensus.cc:2804] T 86262701775a456dacd47fcdc1a5d13b P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:14.874686 25021 raft_consensus.cc:493] T 86262701775a456dacd47fcdc1a5d13b P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:14.864185 24454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86262701775a456dacd47fcdc1a5d13b" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:14.875072 25021 raft_consensus.cc:3060] T 86262701775a456dacd47fcdc1a5d13b P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:14.876847 24576 tablet_bootstrap.cc:492] T e219ba07753f49a5abe454b27cd1494e P 32088c942fbb48239e0eed952d4df449: No bootstrap required, opened a new log
I20260709 13:09:14.877738 24576 ts_tablet_manager.cc:1403] T e219ba07753f49a5abe454b27cd1494e P 32088c942fbb48239e0eed952d4df449: Time spent bootstrapping tablet: real 0.205s	user 0.019s	sys 0.013s
I20260709 13:09:14.879768 24493 leader_election.cc:304] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449
I20260709 13:09:14.875322 24454 raft_consensus.cc:2468] T 86262701775a456dacd47fcdc1a5d13b P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 0.
I20260709 13:09:14.880622 24576 raft_consensus.cc:359] T e219ba07753f49a5abe454b27cd1494e P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:14.881366 24576 raft_consensus.cc:385] T e219ba07753f49a5abe454b27cd1494e P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.881680 24576 raft_consensus.cc:740] T e219ba07753f49a5abe454b27cd1494e P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.882054 24921 raft_consensus.cc:2804] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:14.882596 24921 raft_consensus.cc:697] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Becoming Leader. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Running, Role: LEADER
I20260709 13:09:14.882470 24576 consensus_queue.cc:260] T e219ba07753f49a5abe454b27cd1494e P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:14.883471 24921 consensus_queue.cc:237] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } }
I20260709 13:09:14.893273 24576 ts_tablet_manager.cc:1434] T e219ba07753f49a5abe454b27cd1494e P 32088c942fbb48239e0eed952d4df449: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:09:14.904143 25021 raft_consensus.cc:515] T 86262701775a456dacd47fcdc1a5d13b P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:14.906829 24454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86262701775a456dacd47fcdc1a5d13b" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:14.907384 24454 raft_consensus.cc:3060] T 86262701775a456dacd47fcdc1a5d13b P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:14.911804 24575 tablet_bootstrap.cc:492] T 5b8d15473a964a0c8b8c283c4552706e P 838cb9997b0f437989d3f4e3e9904390: No bootstrap required, opened a new log
I20260709 13:09:14.912389 24575 ts_tablet_manager.cc:1403] T 5b8d15473a964a0c8b8c283c4552706e P 838cb9997b0f437989d3f4e3e9904390: Time spent bootstrapping tablet: real 0.057s	user 0.013s	sys 0.000s
I20260709 13:09:14.915231 24575 raft_consensus.cc:359] T 5b8d15473a964a0c8b8c283c4552706e P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:14.919035 24575 raft_consensus.cc:385] T 5b8d15473a964a0c8b8c283c4552706e P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:14.919349 24575 raft_consensus.cc:740] T 5b8d15473a964a0c8b8c283c4552706e P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Initialized, Role: FOLLOWER
I20260709 13:09:14.920120 24575 consensus_queue.cc:260] T 5b8d15473a964a0c8b8c283c4552706e P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:14.922144 24575 ts_tablet_manager.cc:1434] T 5b8d15473a964a0c8b8c283c4552706e P 838cb9997b0f437989d3f4e3e9904390: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:09:14.922849 24454 raft_consensus.cc:2468] T 86262701775a456dacd47fcdc1a5d13b P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 1.
I20260709 13:09:14.928783 25021 leader_election.cc:290] T 86262701775a456dacd47fcdc1a5d13b P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Requested vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:14.929256 24306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86262701775a456dacd47fcdc1a5d13b" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:14.929936 24306 raft_consensus.cc:3060] T 86262701775a456dacd47fcdc1a5d13b P 9c2a14143eb744cf83bfa5f46284a381 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:14.929370 24205 catalog_manager.cc:5697] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 838cb9997b0f437989d3f4e3e9904390 reported cstate change: term changed from 0 to 1, leader changed from <none> to 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4). New cstate: current_term: 1 leader_uuid: "838cb9997b0f437989d3f4e3e9904390" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:14.936242 24921 consensus_queue.cc:1048] T 624f99e3165f49babbfce35effc9d627 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:14.938423 24493 leader_election.cc:304] T 86262701775a456dacd47fcdc1a5d13b P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 838cb9997b0f437989d3f4e3e9904390; no voters: 
I20260709 13:09:14.944193 24921 raft_consensus.cc:2804] T 86262701775a456dacd47fcdc1a5d13b P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:14.944711 24921 raft_consensus.cc:697] T 86262701775a456dacd47fcdc1a5d13b P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Becoming Leader. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Running, Role: LEADER
I20260709 13:09:14.945600 24921 consensus_queue.cc:237] T 86262701775a456dacd47fcdc1a5d13b P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:14.964206 24772 consensus_queue.cc:1048] T 624f99e3165f49babbfce35effc9d627 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:14.988584 24530 raft_consensus.cc:3060] T 5b8d15473a964a0c8b8c283c4552706e P 838cb9997b0f437989d3f4e3e9904390 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:14.990761 24306 raft_consensus.cc:2468] T 86262701775a456dacd47fcdc1a5d13b P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 1.
W20260709 13:09:14.997241 24253 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:15.018101 24647 consensus_queue.cc:1048] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:15.020654 24680 consensus_queue.cc:1048] T edd24afe4d7847e3980c6e586b9dac5e P 32088c942fbb48239e0eed952d4df449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:15.020777 24204 catalog_manager.cc:5697] T 86262701775a456dacd47fcdc1a5d13b P 838cb9997b0f437989d3f4e3e9904390 reported cstate change: term changed from 0 to 1, leader changed from <none> to 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4). New cstate: current_term: 1 leader_uuid: "838cb9997b0f437989d3f4e3e9904390" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:15.023532 24680 consensus_queue.cc:1048] T edd24afe4d7847e3980c6e586b9dac5e P 32088c942fbb48239e0eed952d4df449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:15.038281 25033 consensus_queue.cc:1048] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:15.133241 24451 raft_consensus.cc:3060] T e219ba07753f49a5abe454b27cd1494e P 32088c942fbb48239e0eed952d4df449 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:15.142987 24875 consensus_queue.cc:1048] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:15.160176 25032 consensus_queue.cc:1048] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:15.331161 24921 consensus_queue.cc:1048] T 86262701775a456dacd47fcdc1a5d13b P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:15.343430 24649 consensus_queue.cc:1048] T 86262701775a456dacd47fcdc1a5d13b P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:15.385304 24921 consensus_queue.cc:1048] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:15.401542 24630 consensus_queue.cc:1048] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:15.899323 14748 tablet_server.cc:179] TabletServer@127.14.103.1:0 shutting down...
W20260709 13:09:15.907982 24494 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 1 similar messages]
W20260709 13:09:15.922142 24494 consensus_peers.cc:597] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:15.950630 24346 consensus_peers.cc:597] T 54fa63b278394f569a9ecffed4dec233 P c40e7ba102224d24a22ccd1f68aaae45 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:15.974390 14748 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:09:15.975234 14748 tablet_replica.cc:333] T 86262701775a456dacd47fcdc1a5d13b P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:15.976013 14748 raft_consensus.cc:2243] T 86262701775a456dacd47fcdc1a5d13b P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:15.976487 14748 raft_consensus.cc:2272] T 86262701775a456dacd47fcdc1a5d13b P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:15.979521 14748 tablet_replica.cc:333] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:15.980252 14748 raft_consensus.cc:2243] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:15.981187 14748 raft_consensus.cc:2272] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:15.983596 14748 tablet_replica.cc:333] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:15.984457 14748 raft_consensus.cc:2243] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:15.985417 14748 raft_consensus.cc:2272] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:15.988003 14748 tablet_replica.cc:333] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:15.988893 14748 raft_consensus.cc:2243] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:15.989838 14748 raft_consensus.cc:2272] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:15.992691 14748 tablet_replica.cc:333] T edd24afe4d7847e3980c6e586b9dac5e P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:15.993563 14748 raft_consensus.cc:2243] T edd24afe4d7847e3980c6e586b9dac5e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:15.994095 14748 raft_consensus.cc:2272] T edd24afe4d7847e3980c6e586b9dac5e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:15.995373 24346 consensus_peers.cc:597] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:15.998375 14748 tablet_replica.cc:333] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:15.999745 14748 raft_consensus.cc:2243] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:16.000691 14748 raft_consensus.cc:2272] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.003018 14748 tablet_replica.cc:333] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:16.003865 14748 raft_consensus.cc:2243] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:16.004840 14748 raft_consensus.cc:2272] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:16.006247 24494 consensus_peers.cc:597] T c6af3489742e4539866e5a3b10761ca2 P 838cb9997b0f437989d3f4e3e9904390 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:16.013516 14748 tablet_replica.cc:333] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:16.014541 14748 raft_consensus.cc:2243] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:16.015539 14748 raft_consensus.cc:2272] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.018059 14748 tablet_replica.cc:333] T 363cde7d4cb84353a74a18cbd9701fa7 P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:16.020681 14748 raft_consensus.cc:2243] T 363cde7d4cb84353a74a18cbd9701fa7 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:16.021217 14748 raft_consensus.cc:2272] T 363cde7d4cb84353a74a18cbd9701fa7 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:16.022302 24346 consensus_peers.cc:597] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:16.023244 14748 tablet_replica.cc:333] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:16.024089 14748 raft_consensus.cc:2243] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:16.024945 14748 raft_consensus.cc:2272] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.026657 14748 tablet_replica.cc:333] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:16.027271 14748 raft_consensus.cc:2243] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:16.028074 14748 raft_consensus.cc:2272] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.029942 14748 tablet_replica.cc:333] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:16.030692 14748 raft_consensus.cc:2243] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:16.031463 14748 raft_consensus.cc:2272] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.034112 14748 tablet_replica.cc:333] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:16.034957 14748 raft_consensus.cc:2243] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:16.035864 14748 raft_consensus.cc:2272] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.038724 14748 tablet_replica.cc:333] T 722754340deb4df18b716dc801c35c34 P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:16.039297 14748 raft_consensus.cc:2243] T 722754340deb4df18b716dc801c35c34 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:16.039736 14748 raft_consensus.cc:2272] T 722754340deb4df18b716dc801c35c34 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.041549 14748 tablet_replica.cc:333] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:16.042122 14748 raft_consensus.cc:2243] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:16.042924 14748 raft_consensus.cc:2272] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.045275 14748 tablet_replica.cc:333] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:16.045957 14748 raft_consensus.cc:2243] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:16.046370 14748 raft_consensus.cc:2272] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:16.047811 24494 consensus_peers.cc:597] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:16.048475 14748 tablet_replica.cc:333] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:16.049074 14748 raft_consensus.cc:2243] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:16.049526 14748 raft_consensus.cc:2272] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.051472 14748 tablet_replica.cc:333] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:16.052256 14748 raft_consensus.cc:2243] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:16.053119 14748 raft_consensus.cc:2272] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.055524 14748 tablet_replica.cc:333] T b147866f4e5f4596b0b3989418cc4b0a P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:16.056341 14748 raft_consensus.cc:2243] T b147866f4e5f4596b0b3989418cc4b0a P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:16.056816 14748 raft_consensus.cc:2272] T b147866f4e5f4596b0b3989418cc4b0a P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.059226 14748 tablet_replica.cc:333] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:16.059988 14748 raft_consensus.cc:2243] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:09:16.060209 24494 consensus_peers.cc:597] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:16.060786 14748 raft_consensus.cc:2272] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.062841 14748 tablet_replica.cc:333] T 836ed40e5bb8485cb93cb03bc2508265 P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:16.063566 14748 raft_consensus.cc:2243] T 836ed40e5bb8485cb93cb03bc2508265 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:16.064042 14748 raft_consensus.cc:2272] T 836ed40e5bb8485cb93cb03bc2508265 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.065817 14748 tablet_replica.cc:333] T 2deba793363c4afca7d75b2fc3814304 P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:16.066527 14748 raft_consensus.cc:2243] T 2deba793363c4afca7d75b2fc3814304 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:16.066932 14748 raft_consensus.cc:2272] T 2deba793363c4afca7d75b2fc3814304 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.068974 14748 tablet_replica.cc:333] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:16.069751 14748 raft_consensus.cc:2243] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:16.070606 14748 raft_consensus.cc:2272] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.073748 14748 tablet_replica.cc:333] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:16.074631 14748 raft_consensus.cc:2243] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:16.075250 14748 raft_consensus.cc:2272] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.078043 14748 tablet_replica.cc:333] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:16.078739 14748 raft_consensus.cc:2243] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:16.079535 14748 raft_consensus.cc:2272] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.081573 14748 tablet_replica.cc:333] T 54fa63b278394f569a9ecffed4dec233 P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:16.082369 14748 raft_consensus.cc:2243] T 54fa63b278394f569a9ecffed4dec233 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:16.082859 14748 raft_consensus.cc:2272] T 54fa63b278394f569a9ecffed4dec233 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.085011 14748 tablet_replica.cc:333] T e4dd51cda55249b48d7bfa39d5feb398 P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:16.085878 14748 raft_consensus.cc:2243] T e4dd51cda55249b48d7bfa39d5feb398 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:16.086323 14748 raft_consensus.cc:2272] T e4dd51cda55249b48d7bfa39d5feb398 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.088353 14748 tablet_replica.cc:333] T c6af3489742e4539866e5a3b10761ca2 P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:16.088840 14748 raft_consensus.cc:2243] T c6af3489742e4539866e5a3b10761ca2 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:16.089188 14748 raft_consensus.cc:2272] T c6af3489742e4539866e5a3b10761ca2 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.091029 14748 tablet_replica.cc:333] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:16.091807 14748 raft_consensus.cc:2243] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:16.092757 14748 raft_consensus.cc:2272] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.095117 14748 tablet_replica.cc:333] T 9b7cd923257143648cfe6a296e18b642 P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
W20260709 13:09:16.095413 24420 consensus_peers.cc:597] T 915d8224f8b64cdab90dff1689046548 P 32088c942fbb48239e0eed952d4df449 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:16.095918 14748 raft_consensus.cc:2243] T 9b7cd923257143648cfe6a296e18b642 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:16.096524 14748 raft_consensus.cc:2272] T 9b7cd923257143648cfe6a296e18b642 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.098578 14748 tablet_replica.cc:333] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:16.099164 14748 raft_consensus.cc:2243] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:16.099980 14748 raft_consensus.cc:2272] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.101807 14748 tablet_replica.cc:333] T 915d8224f8b64cdab90dff1689046548 P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:16.102384 14748 raft_consensus.cc:2243] T 915d8224f8b64cdab90dff1689046548 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:16.102789 14748 raft_consensus.cc:2272] T 915d8224f8b64cdab90dff1689046548 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.104578 14748 tablet_replica.cc:333] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:16.105135 14748 raft_consensus.cc:2243] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:16.105933 14748 raft_consensus.cc:2272] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.107904 14748 tablet_replica.cc:333] T 3e71fd85d9424a08aed292991bccf5ee P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:16.108497 14748 raft_consensus.cc:2243] T 3e71fd85d9424a08aed292991bccf5ee P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:16.108932 14748 raft_consensus.cc:2272] T 3e71fd85d9424a08aed292991bccf5ee P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.111122 14748 tablet_replica.cc:333] T 958ccd87b65945468e9247c40ca27c33 P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:16.112018 14748 raft_consensus.cc:2243] T 958ccd87b65945468e9247c40ca27c33 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:16.112536 14748 raft_consensus.cc:2272] T 958ccd87b65945468e9247c40ca27c33 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.115275 14748 tablet_replica.cc:333] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:16.116065 14748 raft_consensus.cc:2243] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:16.116858 14748 raft_consensus.cc:2272] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.118953 14748 tablet_replica.cc:333] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:16.119513 14748 raft_consensus.cc:2243] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:16.120294 14748 raft_consensus.cc:2272] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.122157 14748 tablet_replica.cc:333] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:16.122684 14748 raft_consensus.cc:2243] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:16.123394 14748 raft_consensus.cc:2272] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.125870 14748 tablet_replica.cc:333] T 861acffd0d804c8fbd1a63771d7937c7 P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:16.126518 14748 raft_consensus.cc:2243] T 861acffd0d804c8fbd1a63771d7937c7 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:16.127048 14748 raft_consensus.cc:2272] T 861acffd0d804c8fbd1a63771d7937c7 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.129731 14748 tablet_replica.cc:333] T 4af7fc369d124452beb5fd07cf8592ec P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:16.130285 14748 raft_consensus.cc:2243] T 4af7fc369d124452beb5fd07cf8592ec P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:16.130750 14748 raft_consensus.cc:2272] T 4af7fc369d124452beb5fd07cf8592ec P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.132907 14748 tablet_replica.cc:333] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:16.133440 14748 raft_consensus.cc:2243] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:16.133839 14748 raft_consensus.cc:2272] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.135599 14748 tablet_replica.cc:333] T 93b6310306174b4ca68b467ea7f5d23e P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:16.136179 14748 raft_consensus.cc:2243] T 93b6310306174b4ca68b467ea7f5d23e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:16.136576 14748 raft_consensus.cc:2272] T 93b6310306174b4ca68b467ea7f5d23e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.138492 14748 tablet_replica.cc:333] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:16.138996 14748 raft_consensus.cc:2243] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:16.139778 14748 raft_consensus.cc:2272] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.141598 14748 tablet_replica.cc:333] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:16.142176 14748 raft_consensus.cc:2243] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:16.143117 14748 raft_consensus.cc:2272] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.145711 14748 tablet_replica.cc:333] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381: stopping tablet replica
I20260709 13:09:16.146342 14748 raft_consensus.cc:2243] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:09:16.146694 24420 consensus_peers.cc:597] T edd24afe4d7847e3980c6e586b9dac5e P 32088c942fbb48239e0eed952d4df449 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:16.147326 14748 raft_consensus.cc:2272] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:16.157792 24494 consensus_peers.cc:597] T 86262701775a456dacd47fcdc1a5d13b P 838cb9997b0f437989d3f4e3e9904390 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:16.175582 24420 consensus_peers.cc:597] T 958ccd87b65945468e9247c40ca27c33 P 32088c942fbb48239e0eed952d4df449 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:16.196038 24494 consensus_peers.cc:597] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:16.213387 14748 tablet_server.cc:196] TabletServer@127.14.103.1:0 shutdown complete.
W20260709 13:09:16.214720 24420 consensus_peers.cc:597] T e4dd51cda55249b48d7bfa39d5feb398 P 32088c942fbb48239e0eed952d4df449 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:45351: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:16.236630 24346 consensus_peers.cc:597] T 4af7fc369d124452beb5fd07cf8592ec P c40e7ba102224d24a22ccd1f68aaae45 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:45351: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:16.256843 24420 consensus_peers.cc:597] T 9b7cd923257143648cfe6a296e18b642 P 32088c942fbb48239e0eed952d4df449 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:45351: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:16.272941 14748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:16.281255 25060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:16.282845 25061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:16.285755 25063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:16.286584 14748 server_base.cc:1061] running on GCE node
I20260709 13:09:16.290901 14748 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:16.291225 14748 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:16.291507 14748 hybrid_clock.cc:648] HybridClock initialized: now 1783602556291483 us; error 0 us; skew 500 ppm
I20260709 13:09:16.292236 14748 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:16.295356 14748 webserver.cc:533] Webserver started at http://127.14.103.1:39811/ using document root <none> and password file <none>
I20260709 13:09:16.295897 14748 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:16.296103 14748 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:09:16.299533 14748 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.001s	sys 0.002s
I20260709 13:09:16.308379 25069 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
W20260709 13:09:16.309087 24494 consensus_peers.cc:597] T 3e71fd85d9424a08aed292991bccf5ee P 838cb9997b0f437989d3f4e3e9904390 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:45351: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:16.309885 14748 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 13:09:16.310341 14748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602247531521-14748-0/minicluster-data/ts-0-root
uuid: "9c2a14143eb744cf83bfa5f46284a381"
format_stamp: "Formatted at 2026-07-09 13:09:04 on dist-test-slave-fpp7"
I20260709 13:09:16.311053 14748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602247531521-14748-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602247531521-14748-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602247531521-14748-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:16.331346 14748 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:16.332892 14748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:16.334353 14748 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:09:16.338900 25076 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
W20260709 13:09:16.366994 24494 consensus_peers.cc:597] T 363cde7d4cb84353a74a18cbd9701fa7 P 838cb9997b0f437989d3f4e3e9904390 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:45351: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:16.408584 24346 consensus_peers.cc:597] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:45351: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:16.444233 24494 consensus_peers.cc:597] T 836ed40e5bb8485cb93cb03bc2508265 P 838cb9997b0f437989d3f4e3e9904390 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:45351: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:16.444577 24420 consensus_peers.cc:597] T 93b6310306174b4ca68b467ea7f5d23e P 32088c942fbb48239e0eed952d4df449 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:45351: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:16.512514 24420 consensus_peers.cc:597] T 2deba793363c4afca7d75b2fc3814304 P 32088c942fbb48239e0eed952d4df449 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:45351: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:16.581540 14748 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 13:09:16.581943 14748 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.244s	user 0.009s	sys 0.000s
I20260709 13:09:16.582283 14748 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 13:09:16.587122 25076 tablet_bootstrap.cc:492] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:16.605424 25076 tablet_bootstrap.cc:492] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:16.606484 25076 tablet_bootstrap.cc:492] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:16.607199 25076 ts_tablet_manager.cc:1403] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.001s
I20260709 13:09:16.610066 25076 raft_consensus.cc:359] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:16.610837 25076 raft_consensus.cc:740] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:16.611819 25076 consensus_queue.cc:260] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:16.614892 25076 ts_tablet_manager.cc:1434] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.004s
I20260709 13:09:16.615721 25076 tablet_bootstrap.cc:492] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:16.630484 25076 tablet_bootstrap.cc:492] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:16.631217 25076 tablet_bootstrap.cc:492] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:16.631781 25076 ts_tablet_manager.cc:1403] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.016s	user 0.013s	sys 0.000s
I20260709 13:09:16.633479 25076 raft_consensus.cc:359] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:16.633952 25076 raft_consensus.cc:740] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:16.634460 25076 consensus_queue.cc:260] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:16.635830 25076 ts_tablet_manager.cc:1434] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:09:16.636409 25076 tablet_bootstrap.cc:492] T edd24afe4d7847e3980c6e586b9dac5e P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:16.651790 25076 tablet_bootstrap.cc:492] T edd24afe4d7847e3980c6e586b9dac5e P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:16.652704 25076 tablet_bootstrap.cc:492] T edd24afe4d7847e3980c6e586b9dac5e P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:16.653302 25076 ts_tablet_manager.cc:1403] T edd24afe4d7847e3980c6e586b9dac5e P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.017s	user 0.007s	sys 0.004s
I20260709 13:09:16.655537 25076 raft_consensus.cc:359] T edd24afe4d7847e3980c6e586b9dac5e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:16.656239 25076 raft_consensus.cc:740] T edd24afe4d7847e3980c6e586b9dac5e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:16.656858 25076 consensus_queue.cc:260] T edd24afe4d7847e3980c6e586b9dac5e P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:16.658432 25076 ts_tablet_manager.cc:1434] T edd24afe4d7847e3980c6e586b9dac5e P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:09:16.659056 25076 tablet_bootstrap.cc:492] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:16.670811 25076 tablet_bootstrap.cc:492] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:16.671770 25076 tablet_bootstrap.cc:492] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:16.672386 25076 ts_tablet_manager.cc:1403] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.013s	user 0.012s	sys 0.000s
I20260709 13:09:16.674747 25076 raft_consensus.cc:359] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:16.675418 25076 raft_consensus.cc:740] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:16.676103 25076 consensus_queue.cc:260] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:16.677539 25076 ts_tablet_manager.cc:1434] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:09:16.678171 25076 tablet_bootstrap.cc:492] T 54fa63b278394f569a9ecffed4dec233 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:16.689606 25076 tablet_bootstrap.cc:492] T 54fa63b278394f569a9ecffed4dec233 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:16.690342 25076 tablet_bootstrap.cc:492] T 54fa63b278394f569a9ecffed4dec233 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:16.690852 25076 ts_tablet_manager.cc:1403] T 54fa63b278394f569a9ecffed4dec233 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 13:09:16.692694 25076 raft_consensus.cc:359] T 54fa63b278394f569a9ecffed4dec233 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:16.693200 25076 raft_consensus.cc:740] T 54fa63b278394f569a9ecffed4dec233 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:16.693990 25076 consensus_queue.cc:260] T 54fa63b278394f569a9ecffed4dec233 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:16.695859 25076 ts_tablet_manager.cc:1434] T 54fa63b278394f569a9ecffed4dec233 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:09:16.696609 25076 tablet_bootstrap.cc:492] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:16.707857 25076 tablet_bootstrap.cc:492] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:16.708612 25076 tablet_bootstrap.cc:492] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:16.709125 25076 ts_tablet_manager.cc:1403] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.013s	user 0.007s	sys 0.004s
I20260709 13:09:16.710804 25076 raft_consensus.cc:359] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:16.711310 25076 raft_consensus.cc:740] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:16.711959 25076 consensus_queue.cc:260] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:16.713541 25076 ts_tablet_manager.cc:1434] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:09:16.714154 25076 tablet_bootstrap.cc:492] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:16.725920 25076 tablet_bootstrap.cc:492] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:16.726835 25076 tablet_bootstrap.cc:492] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:16.727380 25076 ts_tablet_manager.cc:1403] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 13:09:16.729691 25076 raft_consensus.cc:359] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:16.730351 25076 raft_consensus.cc:740] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:16.730968 25076 consensus_queue.cc:260] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:16.732924 25076 ts_tablet_manager.cc:1434] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:09:16.733795 25076 tablet_bootstrap.cc:492] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:16.785321 25076 tablet_bootstrap.cc:492] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:16.786980 25076 tablet_bootstrap.cc:492] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:16.787993 25076 ts_tablet_manager.cc:1403] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.054s	user 0.029s	sys 0.023s
I20260709 13:09:16.790441 25076 raft_consensus.cc:359] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:16.791152 25076 raft_consensus.cc:740] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:16.791949 25076 consensus_queue.cc:260] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:16.793886 25076 ts_tablet_manager.cc:1434] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.001s
I20260709 13:09:16.798516 25076 tablet_bootstrap.cc:492] T 3e71fd85d9424a08aed292991bccf5ee P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:16.813463 25076 tablet_bootstrap.cc:492] T 3e71fd85d9424a08aed292991bccf5ee P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:16.814182 25076 tablet_bootstrap.cc:492] T 3e71fd85d9424a08aed292991bccf5ee P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:16.814772 25076 ts_tablet_manager.cc:1403] T 3e71fd85d9424a08aed292991bccf5ee P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.016s	user 0.011s	sys 0.000s
I20260709 13:09:16.817008 25076 raft_consensus.cc:359] T 3e71fd85d9424a08aed292991bccf5ee P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:16.817662 25076 raft_consensus.cc:740] T 3e71fd85d9424a08aed292991bccf5ee P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:16.818210 25076 consensus_queue.cc:260] T 3e71fd85d9424a08aed292991bccf5ee P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:16.819579 25076 ts_tablet_manager.cc:1434] T 3e71fd85d9424a08aed292991bccf5ee P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:09:16.820267 25076 tablet_bootstrap.cc:492] T 836ed40e5bb8485cb93cb03bc2508265 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:16.821341 14748 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 13:09:16.821828 14748 ts_tablet_manager.cc:595] Time spent register tablets: real 0.240s	user 0.216s	sys 0.007s
I20260709 13:09:16.832697 25076 tablet_bootstrap.cc:492] T 836ed40e5bb8485cb93cb03bc2508265 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:16.833937 25076 tablet_bootstrap.cc:492] T 836ed40e5bb8485cb93cb03bc2508265 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:16.834589 25076 ts_tablet_manager.cc:1403] T 836ed40e5bb8485cb93cb03bc2508265 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.014s	user 0.011s	sys 0.000s
I20260709 13:09:16.836843 25076 raft_consensus.cc:359] T 836ed40e5bb8485cb93cb03bc2508265 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:16.837491 25076 raft_consensus.cc:740] T 836ed40e5bb8485cb93cb03bc2508265 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:16.838246 25076 consensus_queue.cc:260] T 836ed40e5bb8485cb93cb03bc2508265 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:16.840202 25076 ts_tablet_manager.cc:1434] T 836ed40e5bb8485cb93cb03bc2508265 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:09:16.840966 25076 tablet_bootstrap.cc:492] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:16.867208 25076 tablet_bootstrap.cc:492] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:16.868422 25076 tablet_bootstrap.cc:492] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:16.869189 25076 ts_tablet_manager.cc:1403] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.028s	user 0.026s	sys 0.000s
I20260709 13:09:16.871940 25076 raft_consensus.cc:359] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:16.872640 25076 raft_consensus.cc:740] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:16.873382 25076 consensus_queue.cc:260] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:16.875723 25076 ts_tablet_manager.cc:1434] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:09:16.877436 25076 tablet_bootstrap.cc:492] T 93b6310306174b4ca68b467ea7f5d23e P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:16.899154 25076 tablet_bootstrap.cc:492] T 93b6310306174b4ca68b467ea7f5d23e P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:16.900046 25076 tablet_bootstrap.cc:492] T 93b6310306174b4ca68b467ea7f5d23e P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:16.900625 25076 ts_tablet_manager.cc:1403] T 93b6310306174b4ca68b467ea7f5d23e P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.023s	user 0.011s	sys 0.011s
I20260709 13:09:16.902978 25076 raft_consensus.cc:359] T 93b6310306174b4ca68b467ea7f5d23e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:16.903491 25076 raft_consensus.cc:740] T 93b6310306174b4ca68b467ea7f5d23e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:16.904116 25076 consensus_queue.cc:260] T 93b6310306174b4ca68b467ea7f5d23e P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:16.906070 25076 ts_tablet_manager.cc:1434] T 93b6310306174b4ca68b467ea7f5d23e P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.005s	user 0.001s	sys 0.004s
I20260709 13:09:16.906877 25076 tablet_bootstrap.cc:492] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:16.924505 25076 tablet_bootstrap.cc:492] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:16.925278 25076 tablet_bootstrap.cc:492] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:16.925820 25076 ts_tablet_manager.cc:1403] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.004s
I20260709 13:09:16.928143 14748 rpc_server.cc:307] RPC server started. Bound to: 127.14.103.1:45351
I20260709 13:09:16.927843 25076 raft_consensus.cc:359] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:16.928289 25149 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.103.1:45351 every 8 connection(s)
I20260709 13:09:16.928732 25076 raft_consensus.cc:740] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:16.929430 25076 consensus_queue.cc:260] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:16.931022 25076 ts_tablet_manager.cc:1434] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:09:16.931747 25076 tablet_bootstrap.cc:492] T 722754340deb4df18b716dc801c35c34 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:16.948998 25076 tablet_bootstrap.cc:492] T 722754340deb4df18b716dc801c35c34 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:16.949858 25076 tablet_bootstrap.cc:492] T 722754340deb4df18b716dc801c35c34 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:16.950624 25076 ts_tablet_manager.cc:1403] T 722754340deb4df18b716dc801c35c34 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.000s
I20260709 13:09:16.953217 25076 raft_consensus.cc:359] T 722754340deb4df18b716dc801c35c34 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:16.953648 25076 raft_consensus.cc:740] T 722754340deb4df18b716dc801c35c34 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:16.954236 25076 consensus_queue.cc:260] T 722754340deb4df18b716dc801c35c34 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:16.956044 25076 ts_tablet_manager.cc:1434] T 722754340deb4df18b716dc801c35c34 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:09:16.956782 25076 tablet_bootstrap.cc:492] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:16.962960 25150 heartbeater.cc:344] Connected to a master server at 127.14.103.62:41611
I20260709 13:09:16.963528 25150 heartbeater.cc:461] Registering TS with master...
I20260709 13:09:16.964311 25150 heartbeater.cc:507] Master 127.14.103.62:41611 requested a full tablet report, sending...
I20260709 13:09:16.969575 25076 tablet_bootstrap.cc:492] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:16.970335 25076 tablet_bootstrap.cc:492] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:16.970883 25076 ts_tablet_manager.cc:1403] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.014s	user 0.011s	sys 0.000s
I20260709 13:09:16.972692 25076 raft_consensus.cc:359] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:16.973166 25076 raft_consensus.cc:740] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:16.973701 25076 consensus_queue.cc:260] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:16.975163 25076 ts_tablet_manager.cc:1434] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:09:16.975833 25076 tablet_bootstrap.cc:492] T 915d8224f8b64cdab90dff1689046548 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:16.977348 24204 ts_manager.cc:194] Re-registered known tserver with Master: 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:16.985713 25076 tablet_bootstrap.cc:492] T 915d8224f8b64cdab90dff1689046548 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:16.986436 25076 tablet_bootstrap.cc:492] T 915d8224f8b64cdab90dff1689046548 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:16.986927 25076 ts_tablet_manager.cc:1403] T 915d8224f8b64cdab90dff1689046548 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:09:16.988986 25076 raft_consensus.cc:359] T 915d8224f8b64cdab90dff1689046548 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:16.989512 25076 raft_consensus.cc:740] T 915d8224f8b64cdab90dff1689046548 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:16.990036 25076 consensus_queue.cc:260] T 915d8224f8b64cdab90dff1689046548 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:16.991531 25076 ts_tablet_manager.cc:1434] T 915d8224f8b64cdab90dff1689046548 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:09:16.992169 25076 tablet_bootstrap.cc:492] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:17.003641 25076 tablet_bootstrap.cc:492] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:17.004650 25076 tablet_bootstrap.cc:492] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:17.005290 25076 ts_tablet_manager.cc:1403] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.013s	user 0.004s	sys 0.007s
I20260709 13:09:17.008013 25076 raft_consensus.cc:359] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:17.009362 25076 raft_consensus.cc:740] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:17.009945 25076 consensus_queue.cc:260] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:17.013094 24204 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57330
I20260709 13:09:17.014878 25076 ts_tablet_manager.cc:1434] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:09:17.015734 25076 tablet_bootstrap.cc:492] T 9b7cd923257143648cfe6a296e18b642 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:17.017546 24649 raft_consensus.cc:493] T 5b8d15473a964a0c8b8c283c4552706e P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.018069 24649 raft_consensus.cc:515] T 5b8d15473a964a0c8b8c283c4552706e P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:17.018399 25150 heartbeater.cc:499] Master 127.14.103.62:41611 was elected leader, sending a full tablet report...
I20260709 13:09:17.019984 24649 leader_election.cc:290] T 5b8d15473a964a0c8b8c283c4552706e P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:17.021718 24377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b8d15473a964a0c8b8c283c4552706e" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:17.027544 25122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b8d15473a964a0c8b8c283c4552706e" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
W20260709 13:09:17.036397 24494 leader_election.cc:343] T 5b8d15473a964a0c8b8c283c4552706e P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:17.036875 24494 leader_election.cc:304] T 5b8d15473a964a0c8b8c283c4552706e P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390; no voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:17.038555 25021 raft_consensus.cc:2749] T 5b8d15473a964a0c8b8c283c4552706e P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:17.040419 25076 tablet_bootstrap.cc:492] T 9b7cd923257143648cfe6a296e18b642 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:17.041425 25076 tablet_bootstrap.cc:492] T 9b7cd923257143648cfe6a296e18b642 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:17.042120 25076 ts_tablet_manager.cc:1403] T 9b7cd923257143648cfe6a296e18b642 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.027s	user 0.017s	sys 0.008s
I20260709 13:09:17.044373 25076 raft_consensus.cc:359] T 9b7cd923257143648cfe6a296e18b642 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:17.045034 25076 raft_consensus.cc:740] T 9b7cd923257143648cfe6a296e18b642 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:17.045674 25076 consensus_queue.cc:260] T 9b7cd923257143648cfe6a296e18b642 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:17.046322 25054 raft_consensus.cc:493] T b88feffc0859413482b0daf562d2f86a P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.046861 25054 raft_consensus.cc:515] T b88feffc0859413482b0daf562d2f86a P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:17.047597 25076 ts_tablet_manager.cc:1434] T 9b7cd923257143648cfe6a296e18b642 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.000s
I20260709 13:09:17.048488 25076 tablet_bootstrap.cc:492] T 861acffd0d804c8fbd1a63771d7937c7 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:17.048910 25054 leader_election.cc:290] T b88feffc0859413482b0daf562d2f86a P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:17.054797 24530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88feffc0859413482b0daf562d2f86a" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:17.082037 25123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88feffc0859413482b0daf562d2f86a" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
W20260709 13:09:17.092640 24420 leader_election.cc:343] T b88feffc0859413482b0daf562d2f86a P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:17.093077 24420 leader_election.cc:304] T b88feffc0859413482b0daf562d2f86a P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32088c942fbb48239e0eed952d4df449; no voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:17.093343 25052 raft_consensus.cc:493] T dd5ac0d1debc4695a406617ec06f07a0 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.093842 25052 raft_consensus.cc:515] T dd5ac0d1debc4695a406617ec06f07a0 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:17.095395 25054 raft_consensus.cc:2749] T b88feffc0859413482b0daf562d2f86a P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:17.098706 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd5ac0d1debc4695a406617ec06f07a0" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:17.101547 25054 raft_consensus.cc:493] T dd5ac0d1debc4695a406617ec06f07a0 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.102210 25054 raft_consensus.cc:515] T dd5ac0d1debc4695a406617ec06f07a0 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:17.096891 25052 leader_election.cc:290] T dd5ac0d1debc4695a406617ec06f07a0 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:17.111953 25078 raft_consensus.cc:493] T 2f7b6652eba4485ca0632f9fefc2e326 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.112577 25078 raft_consensus.cc:515] T 2f7b6652eba4485ca0632f9fefc2e326 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:17.114730 25078 leader_election.cc:290] T 2f7b6652eba4485ca0632f9fefc2e326 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:17.116043 25078 raft_consensus.cc:493] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.116494 25078 raft_consensus.cc:515] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:17.120527 24377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f7b6652eba4485ca0632f9fefc2e326" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:17.121945 25123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f7b6652eba4485ca0632f9fefc2e326" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
W20260709 13:09:17.124010 24420 leader_election.cc:343] T 2f7b6652eba4485ca0632f9fefc2e326 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:17.124396 24420 leader_election.cc:304] T 2f7b6652eba4485ca0632f9fefc2e326 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32088c942fbb48239e0eed952d4df449; no voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:17.125396 25124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd5ac0d1debc4695a406617ec06f07a0" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
W20260709 13:09:17.129199 24346 leader_election.cc:343] T dd5ac0d1debc4695a406617ec06f07a0 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:17.129602 24346 leader_election.cc:304] T dd5ac0d1debc4695a406617ec06f07a0 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:17.130924 25076 tablet_bootstrap.cc:492] T 861acffd0d804c8fbd1a63771d7937c7 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:17.130946 24530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c88d903166c4812a88653e389ab5776" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:17.131486 25124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c88d903166c4812a88653e389ab5776" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:17.131968 25076 tablet_bootstrap.cc:492] T 861acffd0d804c8fbd1a63771d7937c7 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:17.132221 25124 raft_consensus.cc:2468] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 1.
I20260709 13:09:17.133399 25054 leader_election.cc:290] T dd5ac0d1debc4695a406617ec06f07a0 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:17.133587 25077 raft_consensus.cc:2749] T dd5ac0d1debc4695a406617ec06f07a0 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:17.134644 24377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd5ac0d1debc4695a406617ec06f07a0" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:17.135164 25054 raft_consensus.cc:2749] T 2f7b6652eba4485ca0632f9fefc2e326 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:17.135267 24377 raft_consensus.cc:2468] T dd5ac0d1debc4695a406617ec06f07a0 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 1.
I20260709 13:09:17.136241 24420 leader_election.cc:304] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381; no voters: 838cb9997b0f437989d3f4e3e9904390
I20260709 13:09:17.137748 25076 ts_tablet_manager.cc:1403] T 861acffd0d804c8fbd1a63771d7937c7 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.089s	user 0.014s	sys 0.009s
I20260709 13:09:17.140024 24419 leader_election.cc:304] T dd5ac0d1debc4695a406617ec06f07a0 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:17.140297 25079 raft_consensus.cc:2804] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:17.140725 25162 raft_consensus.cc:2804] T dd5ac0d1debc4695a406617ec06f07a0 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:17.141140 25162 raft_consensus.cc:493] T dd5ac0d1debc4695a406617ec06f07a0 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.141475 25162 raft_consensus.cc:3060] T dd5ac0d1debc4695a406617ec06f07a0 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:17.141363 25123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd5ac0d1debc4695a406617ec06f07a0" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:17.149467 25054 raft_consensus.cc:493] T e219ba07753f49a5abe454b27cd1494e P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.145004 25052 raft_consensus.cc:493] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
W20260709 13:09:17.143040 24420 leader_election.cc:343] T dd5ac0d1debc4695a406617ec06f07a0 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:17.143085 25078 leader_election.cc:290] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:17.153725 25054 raft_consensus.cc:515] T e219ba07753f49a5abe454b27cd1494e P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:17.153729 25052 raft_consensus.cc:515] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:17.156759 25054 leader_election.cc:290] T e219ba07753f49a5abe454b27cd1494e P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:17.157586 25123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd607d60354e4864ae5499ee1c1a05c9" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:17.157765 25124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e219ba07753f49a5abe454b27cd1494e" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:17.158057 24377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e219ba07753f49a5abe454b27cd1494e" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:17.158288 25123 raft_consensus.cc:2468] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 1.
W20260709 13:09:17.159246 24420 leader_election.cc:343] T e219ba07753f49a5abe454b27cd1494e P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:17.159772 24419 leader_election.cc:304] T e219ba07753f49a5abe454b27cd1494e P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32088c942fbb48239e0eed952d4df449; no voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:17.159807 24346 leader_election.cc:304] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:17.140725 25079 raft_consensus.cc:493] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.161687 24979 raft_consensus.cc:2804] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:17.162015 25054 raft_consensus.cc:2749] T e219ba07753f49a5abe454b27cd1494e P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:17.169333 24979 raft_consensus.cc:493] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.164904 25161 raft_consensus.cc:493] T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.170169 24979 raft_consensus.cc:3060] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:17.170317 25161 raft_consensus.cc:515] T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:17.173584 24377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ae3dc25b8484b6f99d8f41685229cb8" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:17.165366 25078 raft_consensus.cc:493] T c60b42ac6d7e48bfae725dc113758659 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.165948 25052 leader_election.cc:290] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:17.142670 25076 raft_consensus.cc:359] T 861acffd0d804c8fbd1a63771d7937c7 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:17.175894 25076 raft_consensus.cc:740] T 861acffd0d804c8fbd1a63771d7937c7 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:17.175729 25078 raft_consensus.cc:515] T c60b42ac6d7e48bfae725dc113758659 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:17.176205 25123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ae3dc25b8484b6f99d8f41685229cb8" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:17.176803 25021 raft_consensus.cc:493] T 53b7256c3dd84f5b952d1f9d47691f4e P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.176611 25076 consensus_queue.cc:260] T 861acffd0d804c8fbd1a63771d7937c7 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:17.176921 25123 raft_consensus.cc:2468] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 1.
I20260709 13:09:17.182294 25021 raft_consensus.cc:515] T 53b7256c3dd84f5b952d1f9d47691f4e P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:17.183642 25076 ts_tablet_manager.cc:1434] T 861acffd0d804c8fbd1a63771d7937c7 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.044s	user 0.004s	sys 0.001s
I20260709 13:09:17.184468 24420 leader_election.cc:304] T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381; no voters: c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:17.178339 24530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c60b42ac6d7e48bfae725dc113758659" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:17.177785 25161 leader_election.cc:290] T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:17.180033 25124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c60b42ac6d7e48bfae725dc113758659" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:17.189389 25054 raft_consensus.cc:2804] T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:17.189772 25054 raft_consensus.cc:493] T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.180845 25162 raft_consensus.cc:515] T dd5ac0d1debc4695a406617ec06f07a0 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:17.190130 25054 raft_consensus.cc:3060] T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:17.185251 25123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b7256c3dd84f5b952d1f9d47691f4e" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:17.191923 25162 leader_election.cc:290] T dd5ac0d1debc4695a406617ec06f07a0 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 election: Requested vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:17.185500 25076 tablet_bootstrap.cc:492] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:17.184625 25021 leader_election.cc:290] T 53b7256c3dd84f5b952d1f9d47691f4e P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:17.168897 25079 raft_consensus.cc:3060] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:17.186581 24377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b7256c3dd84f5b952d1f9d47691f4e" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:17.178669 25078 leader_election.cc:290] T c60b42ac6d7e48bfae725dc113758659 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:17.194588 24649 raft_consensus.cc:493] T 5bb19353ba334eac831ae2cce277298a P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
W20260709 13:09:17.198627 24494 leader_election.cc:343] T 53b7256c3dd84f5b952d1f9d47691f4e P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:17.194502 24379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd5ac0d1debc4695a406617ec06f07a0" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:17.205931 24649 raft_consensus.cc:515] T 5bb19353ba334eac831ae2cce277298a P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:17.206373 24379 raft_consensus.cc:3060] T dd5ac0d1debc4695a406617ec06f07a0 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:09:17.207011 24420 leader_election.cc:343] T c60b42ac6d7e48bfae725dc113758659 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:17.207319 24420 leader_election.cc:304] T c60b42ac6d7e48bfae725dc113758659 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32088c942fbb48239e0eed952d4df449; no voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:17.203296 25123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd5ac0d1debc4695a406617ec06f07a0" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:17.209477 24649 leader_election.cc:290] T 5bb19353ba334eac831ae2cce277298a P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:17.200294 25052 raft_consensus.cc:493] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.210323 25052 raft_consensus.cc:515] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:17.210994 25123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bb19353ba334eac831ae2cce277298a" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:17.212857 24530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f795acd16fd472b8c9360fb14e7e5d7" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:17.213410 25079 raft_consensus.cc:515] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:17.214779 25052 leader_election.cc:290] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:17.215699 24379 raft_consensus.cc:2468] T dd5ac0d1debc4695a406617ec06f07a0 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 2.
I20260709 13:09:17.215853 25079 leader_election.cc:290] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 election: Requested vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:17.216936 25124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f795acd16fd472b8c9360fb14e7e5d7" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:17.217728 25124 raft_consensus.cc:2468] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 1.
I20260709 13:09:17.217898 24530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c88d903166c4812a88653e389ab5776" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:17.218027 25052 raft_consensus.cc:493] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.218204 24419 leader_election.cc:304] T dd5ac0d1debc4695a406617ec06f07a0 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:17.218940 24346 leader_election.cc:304] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 838cb9997b0f437989d3f4e3e9904390
I20260709 13:09:17.218567 25052 raft_consensus.cc:515] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
W20260709 13:09:17.219898 24494 leader_election.cc:343] T 5bb19353ba334eac831ae2cce277298a P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:17.220070 25077 raft_consensus.cc:2804] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:17.220484 25077 raft_consensus.cc:493] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.220835 25077 raft_consensus.cc:3060] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:17.221874 25052 leader_election.cc:290] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:17.222882 25052 raft_consensus.cc:493] T 8775bc34e96d48ae8f5598affd26efdc P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.222944 24530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b8d15473a964a0c8b8c283c4552706e" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:17.222393 25079 raft_consensus.cc:2804] T dd5ac0d1debc4695a406617ec06f07a0 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:17.206059 24493 leader_election.cc:304] T 53b7256c3dd84f5b952d1f9d47691f4e P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390; no voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:17.223718 24530 raft_consensus.cc:2468] T 5b8d15473a964a0c8b8c283c4552706e P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 1.
I20260709 13:09:17.230792 24649 raft_consensus.cc:2749] T 53b7256c3dd84f5b952d1f9d47691f4e P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:17.232532 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd607d60354e4864ae5499ee1c1a05c9" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:17.229328 25052 raft_consensus.cc:515] T 8775bc34e96d48ae8f5598affd26efdc P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:17.236876 24377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bb19353ba334eac831ae2cce277298a" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:17.232769 24345 leader_election.cc:304] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:17.238394 25123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b8d15473a964a0c8b8c283c4552706e" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
W20260709 13:09:17.230511 24420 leader_election.cc:343] T dd5ac0d1debc4695a406617ec06f07a0 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:17.238941 25021 raft_consensus.cc:493] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.239431 25021 raft_consensus.cc:515] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:17.240514 25123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c88d903166c4812a88653e389ab5776" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:17.241087 25123 raft_consensus.cc:3060] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:17.242152 25021 leader_election.cc:290] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
W20260709 13:09:17.239910 24346 leader_election.cc:343] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:17.245332 25124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6427fa143f574eaba2ee4c979e5485fc" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:17.245966 25124 raft_consensus.cc:2468] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 1.
I20260709 13:09:17.248013 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6427fa143f574eaba2ee4c979e5485fc" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:17.249454 25123 raft_consensus.cc:2468] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 2.
I20260709 13:09:17.229992 25161 raft_consensus.cc:2749] T c60b42ac6d7e48bfae725dc113758659 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:17.248926 24494 leader_election.cc:304] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381; no voters: 
I20260709 13:09:17.250523 24420 leader_election.cc:304] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381; no voters: 838cb9997b0f437989d3f4e3e9904390
I20260709 13:09:17.250912 25052 leader_election.cc:290] T 8775bc34e96d48ae8f5598affd26efdc P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:17.251505 25052 raft_consensus.cc:493] T 142bbcbf57124963866a3967c8adebed P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.251657 25161 raft_consensus.cc:2804] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:17.252353 25161 raft_consensus.cc:697] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449 [term 2 LEADER]: Becoming Leader. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Running, Role: LEADER
I20260709 13:09:17.252556 25021 raft_consensus.cc:2804] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:17.253077 25021 raft_consensus.cc:493] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.252032 25052 raft_consensus.cc:515] T 142bbcbf57124963866a3967c8adebed P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:17.255316 24493 leader_election.cc:304] T 5bb19353ba334eac831ae2cce277298a P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390; no voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:17.255153 25021 raft_consensus.cc:3060] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:17.257225 25052 leader_election.cc:290] T 142bbcbf57124963866a3967c8adebed P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:17.257881 25166 raft_consensus.cc:2804] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:17.258293 25166 raft_consensus.cc:493] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.258307 25124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142bbcbf57124963866a3967c8adebed" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:17.258684 25166 raft_consensus.cc:3060] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:17.259049 25124 raft_consensus.cc:2468] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 1.
I20260709 13:09:17.260279 24346 leader_election.cc:304] T 142bbcbf57124963866a3967c8adebed P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:17.260963 24530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142bbcbf57124963866a3967c8adebed" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:17.253284 25161 consensus_queue.cc:237] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:17.256275 24649 raft_consensus.cc:2749] T 5bb19353ba334eac831ae2cce277298a P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:17.262439 25165 raft_consensus.cc:493] T 40cf5b30d13f4c3e86a9aa657c20c650 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.262967 25165 raft_consensus.cc:515] T 40cf5b30d13f4c3e86a9aa657c20c650 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:17.249486 25077 raft_consensus.cc:515] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:17.266150 25077 leader_election.cc:290] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 election: Requested vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:17.266376 25052 raft_consensus.cc:2804] T 142bbcbf57124963866a3967c8adebed P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:17.266258 25166 raft_consensus.cc:515] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:17.266930 25052 raft_consensus.cc:493] T 142bbcbf57124963866a3967c8adebed P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.256246 25123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8775bc34e96d48ae8f5598affd26efdc" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:17.230856 25079 raft_consensus.cc:697] T dd5ac0d1debc4695a406617ec06f07a0 P 32088c942fbb48239e0eed952d4df449 [term 2 LEADER]: Becoming Leader. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Running, Role: LEADER
I20260709 13:09:17.230165 25078 raft_consensus.cc:493] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.202358 24979 raft_consensus.cc:515] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:17.267829 24649 raft_consensus.cc:493] T 0f795acd16fd472b8c9360fb14e7e5d7 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.269908 24529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b8d15473a964a0c8b8c283c4552706e" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:17.270449 24529 raft_consensus.cc:3060] T 5b8d15473a964a0c8b8c283c4552706e P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:17.270200 24649 raft_consensus.cc:515] T 0f795acd16fd472b8c9360fb14e7e5d7 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:17.271124 24979 leader_election.cc:290] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 election: Requested vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:17.272058 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8775bc34e96d48ae8f5598affd26efdc" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:17.272198 24649 leader_election.cc:290] T 0f795acd16fd472b8c9360fb14e7e5d7 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:17.267474 25052 raft_consensus.cc:3060] T 142bbcbf57124963866a3967c8adebed P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:17.273573 25165 leader_election.cc:290] T 40cf5b30d13f4c3e86a9aa657c20c650 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:17.253224 25054 raft_consensus.cc:515] T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:17.275614 25054 leader_election.cc:290] T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 election: Requested vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:17.276377 25124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ae3dc25b8484b6f99d8f41685229cb8" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:17.277001 25124 raft_consensus.cc:3060] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:17.277194 25122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f795acd16fd472b8c9360fb14e7e5d7" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:17.277379 24377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ae3dc25b8484b6f99d8f41685229cb8" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:17.277834 25122 raft_consensus.cc:2468] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 1.
I20260709 13:09:17.279085 24494 leader_election.cc:304] T 0f795acd16fd472b8c9360fb14e7e5d7 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381; no voters: 
I20260709 13:09:17.268997 24530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f795acd16fd472b8c9360fb14e7e5d7" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:17.279824 24530 raft_consensus.cc:3060] T 0f795acd16fd472b8c9360fb14e7e5d7 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:17.280651 24528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40cf5b30d13f4c3e86a9aa657c20c650" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:17.281544 24529 raft_consensus.cc:2468] T 5b8d15473a964a0c8b8c283c4552706e P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 2.
I20260709 13:09:17.283330 24377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f795acd16fd472b8c9360fb14e7e5d7" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:17.291147 24377 raft_consensus.cc:2393] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 838cb9997b0f437989d3f4e3e9904390 in current term 2: Already voted for candidate c40e7ba102224d24a22ccd1f68aaae45 in this term.
I20260709 13:09:17.269501 25078 raft_consensus.cc:515] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:17.268350 25166 leader_election.cc:290] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 election: Requested vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:17.296321 24451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd607d60354e4864ae5499ee1c1a05c9" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:17.294605 24345 leader_election.cc:304] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:17.269151 25079 consensus_queue.cc:237] T dd5ac0d1debc4695a406617ec06f07a0 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:17.285653 25124 raft_consensus.cc:2468] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 2.
I20260709 13:09:17.286187 24530 raft_consensus.cc:2468] T 0f795acd16fd472b8c9360fb14e7e5d7 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 2.
I20260709 13:09:17.298683 24649 raft_consensus.cc:2764] T 0f795acd16fd472b8c9360fb14e7e5d7 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:09:17.289461 25052 raft_consensus.cc:515] T 142bbcbf57124963866a3967c8adebed P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:17.299922 25166 raft_consensus.cc:2804] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:17.300470 25166 raft_consensus.cc:697] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45 [term 2 LEADER]: Becoming Leader. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Running, Role: LEADER
I20260709 13:09:17.300701 25052 leader_election.cc:290] T 142bbcbf57124963866a3967c8adebed P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 election: Requested vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:17.301334 25166 consensus_queue.cc:237] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:17.301734 24420 leader_election.cc:304] T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381; no voters: c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:17.303552 24530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142bbcbf57124963866a3967c8adebed" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:17.303785 25054 raft_consensus.cc:2804] T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:17.304658 25054 raft_consensus.cc:697] T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449 [term 2 LEADER]: Becoming Leader. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Running, Role: LEADER
I20260709 13:09:17.305987 25054 consensus_queue.cc:237] T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:17.307861 25078 leader_election.cc:290] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:17.308070 24377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90d9e64f97544895b9b97dfdc5ceb359" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:17.310438 25123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90d9e64f97544895b9b97dfdc5ceb359" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:17.311095 25123 raft_consensus.cc:2468] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 1.
I20260709 13:09:17.311375 25079 raft_consensus.cc:493] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.312033 25079 raft_consensus.cc:515] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
W20260709 13:09:17.313900 24346 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:45351, user_credentials={real_user=slave}} blocked reactor thread for 53700.5us
I20260709 13:09:17.307240 25076 tablet_bootstrap.cc:492] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:17.316654 25076 tablet_bootstrap.cc:492] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:17.316907 25124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b8d15473a964a0c8b8c283c4552706e" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:17.317315 25076 ts_tablet_manager.cc:1403] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.132s	user 0.021s	sys 0.000s
I20260709 13:09:17.317761 25121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40cf5b30d13f4c3e86a9aa657c20c650" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:17.319031 25120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142bbcbf57124963866a3967c8adebed" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:17.319185 25124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8775bc34e96d48ae8f5598affd26efdc" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:17.319713 25120 raft_consensus.cc:3060] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:17.319998 25076 raft_consensus.cc:359] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:17.320675 25076 raft_consensus.cc:740] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:17.321375 25076 consensus_queue.cc:260] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:17.307885 24345 leader_election.cc:304] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:17.324429 25052 raft_consensus.cc:2804] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:17.324882 25052 raft_consensus.cc:697] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 LEADER]: Becoming Leader. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Running, Role: LEADER
I20260709 13:09:17.324959 25122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd607d60354e4864ae5499ee1c1a05c9" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
W20260709 13:09:17.325062 24346 leader_election.cc:343] T 8775bc34e96d48ae8f5598affd26efdc P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:17.325742 25052 consensus_queue.cc:237] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:17.323504 25076 ts_tablet_manager.cc:1434] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:09:17.327441 25076 tablet_bootstrap.cc:492] T 958ccd87b65945468e9247c40ca27c33 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:17.327934 24420 leader_election.cc:304] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381; no voters: c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:17.297539 25021 raft_consensus.cc:515] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:17.326202 24346 leader_election.cc:304] T 8775bc34e96d48ae8f5598affd26efdc P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381
W20260709 13:09:17.329759 24420 leader_election.cc:343] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:17.331583 24377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8775bc34e96d48ae8f5598affd26efdc" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
W20260709 13:09:17.333367 24346 leader_election.cc:343] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:17.332191 24202 catalog_manager.cc:5697] T dd5ac0d1debc4695a406617ec06f07a0 P 32088c942fbb48239e0eed952d4df449 reported cstate change: term changed from 1 to 2, leader changed from 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1) to 32088c942fbb48239e0eed952d4df449 (127.14.103.3). New cstate: current_term: 2 leader_uuid: "32088c942fbb48239e0eed952d4df449" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: HEALTHY } } }
W20260709 13:09:17.334632 24346 leader_election.cc:343] T 40cf5b30d13f4c3e86a9aa657c20c650 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:17.335258 24346 leader_election.cc:304] T 40cf5b30d13f4c3e86a9aa657c20c650 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:17.333856 25079 leader_election.cc:290] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:17.340333 25167 raft_consensus.cc:2749] T 8775bc34e96d48ae8f5598affd26efdc P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:17.342093 24377 raft_consensus.cc:2468] T 8775bc34e96d48ae8f5598affd26efdc P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 1.
I20260709 13:09:17.344386 25077 raft_consensus.cc:2749] T 40cf5b30d13f4c3e86a9aa657c20c650 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:17.333321 25161 raft_consensus.cc:2804] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:17.344995 25161 raft_consensus.cc:493] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.345356 25161 raft_consensus.cc:3060] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:17.335577 25122 raft_consensus.cc:3060] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:17.356720 25077 raft_consensus.cc:493] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.362798 25077 raft_consensus.cc:515] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:17.367449 24419 leader_election.cc:304] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, c40e7ba102224d24a22ccd1f68aaae45; no voters: 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:17.368991 24630 raft_consensus.cc:493] T 9711acbe7d044647bf774ee849c8a032 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.369637 24630 raft_consensus.cc:515] T 9711acbe7d044647bf774ee849c8a032 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:17.357787 25123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f795acd16fd472b8c9360fb14e7e5d7" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:17.367937 25079 raft_consensus.cc:493] T 373185ca8ab04fdb91c7947ec0668808 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.370819 25123 raft_consensus.cc:3060] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:17.371169 25079 raft_consensus.cc:515] T 373185ca8ab04fdb91c7947ec0668808 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:17.371608 24630 leader_election.cc:290] T 9711acbe7d044647bf774ee849c8a032 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:17.374583 25166 raft_consensus.cc:493] T 6ae3dc25b8484b6f99d8f41685229cb8 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.374972 24379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9711acbe7d044647bf774ee849c8a032" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:17.375135 25166 raft_consensus.cc:515] T 6ae3dc25b8484b6f99d8f41685229cb8 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:17.372701 25162 raft_consensus.cc:2804] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:17.375634 25079 leader_election.cc:290] T 373185ca8ab04fdb91c7947ec0668808 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:17.380685 24630 raft_consensus.cc:493] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.376521 25162 raft_consensus.cc:493] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.381400 25077 leader_election.cc:290] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:17.381253 24630 raft_consensus.cc:515] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:17.381808 25162 raft_consensus.cc:3060] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:17.383111 25121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b7256c3dd84f5b952d1f9d47691f4e" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:17.383805 25121 raft_consensus.cc:2468] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 1.
I20260709 13:09:17.386904 24530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b7256c3dd84f5b952d1f9d47691f4e" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:17.387465 24530 raft_consensus.cc:2468] T 53b7256c3dd84f5b952d1f9d47691f4e P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 1.
I20260709 13:09:17.388980 24630 leader_election.cc:290] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:17.389647 24346 leader_election.cc:304] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:17.388897 24451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ae3dc25b8484b6f99d8f41685229cb8" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:17.392309 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88feffc0859413482b0daf562d2f86a" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:17.393092 24455 raft_consensus.cc:2468] T b88feffc0859413482b0daf562d2f86a P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 1.
I20260709 13:09:17.396890 24493 leader_election.cc:304] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 838cb9997b0f437989d3f4e3e9904390; no voters: 
I20260709 13:09:17.399410 24630 raft_consensus.cc:2804] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:17.400126 24630 raft_consensus.cc:493] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.400602 24630 raft_consensus.cc:3060] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:17.400624 24204 catalog_manager.cc:5697] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45 reported cstate change: term changed from 1 to 2, leader changed from 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1) to c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2). New cstate: current_term: 2 leader_uuid: "c40e7ba102224d24a22ccd1f68aaae45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:17.400192 25161 raft_consensus.cc:515] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:17.402114 24377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "373185ca8ab04fdb91c7947ec0668808" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:17.378816 25119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ae3dc25b8484b6f99d8f41685229cb8" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:17.377640 25166 leader_election.cc:290] T 6ae3dc25b8484b6f99d8f41685229cb8 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:17.396567 25077 raft_consensus.cc:2804] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:17.405306 24649 raft_consensus.cc:493] T 72d2665a42a94a82b1ceec249960fdf6 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.405807 24649 raft_consensus.cc:515] T 72d2665a42a94a82b1ceec249960fdf6 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:17.406678 25077 raft_consensus.cc:493] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.407009 25021 leader_election.cc:290] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 election: Requested vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:17.407135 25077 raft_consensus.cc:3060] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:17.409626 24451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d2665a42a94a82b1ceec249960fdf6" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:17.411514 24649 leader_election.cc:290] T 72d2665a42a94a82b1ceec249960fdf6 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:17.412022 25119 raft_consensus.cc:2393] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c40e7ba102224d24a22ccd1f68aaae45 in current term 2: Already voted for candidate 32088c942fbb48239e0eed952d4df449 in this term.
I20260709 13:09:17.411968 24649 raft_consensus.cc:493] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.412803 24649 raft_consensus.cc:515] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:17.415246 24346 leader_election.cc:304] T 6ae3dc25b8484b6f99d8f41685229cb8 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:17.417780 24649 leader_election.cc:290] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:17.415539 24377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40cf5b30d13f4c3e86a9aa657c20c650" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:17.421656 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6427fa143f574eaba2ee4c979e5485fc" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:17.421990 24377 raft_consensus.cc:2468] T 40cf5b30d13f4c3e86a9aa657c20c650 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 1.
I20260709 13:09:17.425366 24649 raft_consensus.cc:493] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.425999 24649 raft_consensus.cc:515] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:17.423739 24493 leader_election.cc:304] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:17.433808 25161 leader_election.cc:290] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 election: Requested vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:17.434526 25121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9711acbe7d044647bf774ee849c8a032" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:17.435487 25077 raft_consensus.cc:515] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:17.435976 25167 raft_consensus.cc:493] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.423964 25166 raft_consensus.cc:3060] T 6ae3dc25b8484b6f99d8f41685229cb8 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:17.435736 24376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90d9e64f97544895b9b97dfdc5ceb359" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:17.436539 25165 raft_consensus.cc:493] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.436491 25167 raft_consensus.cc:515] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:17.437376 25165 raft_consensus.cc:515] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:17.438939 25167 leader_election.cc:290] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:17.440018 25124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6427fa143f574eaba2ee4c979e5485fc" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:17.439441 25165 leader_election.cc:290] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:17.440992 25117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d2665a42a94a82b1ceec249960fdf6" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:17.441334 25124 raft_consensus.cc:3060] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:17.442034 25123 raft_consensus.cc:2468] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 2.
I20260709 13:09:17.442461 25117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3b35ea0300f4b9e9afa80cf1dc9ba0f" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:17.443338 25123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "373185ca8ab04fdb91c7947ec0668808" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:17.443717 25116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40cf5b30d13f4c3e86a9aa657c20c650" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:17.444242 25161 raft_consensus.cc:493] T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.444823 25021 raft_consensus.cc:2804] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:17.444743 25161 raft_consensus.cc:515] T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:17.445255 25021 raft_consensus.cc:493] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.445654 25021 raft_consensus.cc:3060] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:09:17.446420 24346 leader_election.cc:343] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:17.447199 25117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90d9e64f97544895b9b97dfdc5ceb359" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:17.447813 25117 raft_consensus.cc:3060] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:17.449395 25122 raft_consensus.cc:2468] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 2.
I20260709 13:09:17.449757 24377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f32a46373be4a29b6ac72efa35494b4" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:17.450850 24346 leader_election.cc:304] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449
I20260709 13:09:17.451078 25079 raft_consensus.cc:493] T bd607d60354e4864ae5499ee1c1a05c9 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.451565 25079 raft_consensus.cc:515] T bd607d60354e4864ae5499ee1c1a05c9 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:17.440275 25121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88feffc0859413482b0daf562d2f86a" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:17.451994 25119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ac163496084a3e977344fa9228a3c9" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:17.452821 25119 raft_consensus.cc:2468] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 1.
W20260709 13:09:17.453860 24494 leader_election.cc:343] T 9711acbe7d044647bf774ee849c8a032 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:17.454219 25167 raft_consensus.cc:493] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.454324 24494 leader_election.cc:304] T 9711acbe7d044647bf774ee849c8a032 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390; no voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:17.454784 25167 raft_consensus.cc:515] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:17.447227 25161 leader_election.cc:290] T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:17.454948 24346 leader_election.cc:304] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:17.457592 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ac163496084a3e977344fa9228a3c9" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
W20260709 13:09:17.458163 24494 leader_election.cc:343] T 72d2665a42a94a82b1ceec249960fdf6 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:17.458699 24494 leader_election.cc:304] T 72d2665a42a94a82b1ceec249960fdf6 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390; no voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:17.457868 24451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3b35ea0300f4b9e9afa80cf1dc9ba0f" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:17.470340 24649 leader_election.cc:290] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:17.471788 24649 raft_consensus.cc:2749] T 9711acbe7d044647bf774ee849c8a032 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:17.400889 25120 raft_consensus.cc:2468] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 2.
I20260709 13:09:17.451942 25165 raft_consensus.cc:2804] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:17.476038 24649 raft_consensus.cc:493] T 9c88d903166c4812a88653e389ab5776 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.476691 24649 raft_consensus.cc:515] T 9c88d903166c4812a88653e389ab5776 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:17.464254 25079 leader_election.cc:290] T bd607d60354e4864ae5499ee1c1a05c9 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:17.481863 24649 leader_election.cc:290] T 9c88d903166c4812a88653e389ab5776 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:17.466842 24979 raft_consensus.cc:2804] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:17.464586 25124 raft_consensus.cc:2468] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 2.
I20260709 13:09:17.473515 25077 leader_election.cc:290] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 election: Requested vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
W20260709 13:09:17.482093 24420 leader_election.cc:343] T 373185ca8ab04fdb91c7947ec0668808 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:17.499720 24420 leader_election.cc:304] T 373185ca8ab04fdb91c7947ec0668808 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32088c942fbb48239e0eed952d4df449; no voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:17.495275 24649 raft_consensus.cc:2749] T 72d2665a42a94a82b1ceec249960fdf6 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:17.494508 24630 raft_consensus.cc:515] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:17.433837 25162 raft_consensus.cc:515] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:17.478996 25161 raft_consensus.cc:493] T 6427fa143f574eaba2ee4c979e5485fc P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.494560 25078 raft_consensus.cc:493] T 72ac163496084a3e977344fa9228a3c9 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.494990 24979 raft_consensus.cc:493] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.541447 24630 leader_election.cc:290] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 election: Requested vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:17.539825 24451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c60b42ac6d7e48bfae725dc113758659" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:17.534746 24649 raft_consensus.cc:493] T 142bbcbf57124963866a3967c8adebed P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.494309 25165 raft_consensus.cc:697] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 LEADER]: Becoming Leader. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Running, Role: LEADER
I20260709 13:09:17.497957 25079 raft_consensus.cc:493] T 72d2665a42a94a82b1ceec249960fdf6 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.549258 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c88d903166c4812a88653e389ab5776" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:17.555214 24649 raft_consensus.cc:515] T 142bbcbf57124963866a3967c8adebed P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:17.537083 24377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd607d60354e4864ae5499ee1c1a05c9" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:17.552779 24451 raft_consensus.cc:2468] T c60b42ac6d7e48bfae725dc113758659 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 1.
I20260709 13:09:17.562680 24528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9711acbe7d044647bf774ee849c8a032" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:17.561528 24530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b7256c3dd84f5b952d1f9d47691f4e" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:17.616259 24530 raft_consensus.cc:3060] T 53b7256c3dd84f5b952d1f9d47691f4e P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:17.561528 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88feffc0859413482b0daf562d2f86a" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:17.616928 24455 raft_consensus.cc:3060] T b88feffc0859413482b0daf562d2f86a P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:17.617750 24346 leader_election.cc:304] T 142bbcbf57124963866a3967c8adebed P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 838cb9997b0f437989d3f4e3e9904390
I20260709 13:09:17.552512 25077 raft_consensus.cc:493] T 90d9e64f97544895b9b97dfdc5ceb359 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.615788 24528 raft_consensus.cc:2468] T 9711acbe7d044647bf774ee849c8a032 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 1.
I20260709 13:09:17.621675 24493 leader_election.cc:304] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 838cb9997b0f437989d3f4e3e9904390; no voters: 
I20260709 13:09:17.552570 24979 raft_consensus.cc:3060] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:17.641438 24630 raft_consensus.cc:2804] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:17.641868 24630 raft_consensus.cc:493] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.642179 24630 raft_consensus.cc:3060] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:17.541692 25161 raft_consensus.cc:515] T 6427fa143f574eaba2ee4c979e5485fc P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:17.619974 25124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f32a46373be4a29b6ac72efa35494b4" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:17.559021 25167 leader_election.cc:290] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:17.557348 25120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b7256c3dd84f5b952d1f9d47691f4e" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:17.650167 24455 raft_consensus.cc:2468] T b88feffc0859413482b0daf562d2f86a P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 2.
I20260709 13:09:17.554320 25078 raft_consensus.cc:515] T 72ac163496084a3e977344fa9228a3c9 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:17.634855 25054 raft_consensus.cc:493] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.556341 25079 raft_consensus.cc:515] T 72d2665a42a94a82b1ceec249960fdf6 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:17.660161 24649 leader_election.cc:290] T 142bbcbf57124963866a3967c8adebed P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:17.635859 25077 raft_consensus.cc:515] T 90d9e64f97544895b9b97dfdc5ceb359 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:17.635074 25052 raft_consensus.cc:493] T 4f32a46373be4a29b6ac72efa35494b4 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.672566 24630 raft_consensus.cc:515] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:17.685755 25076 tablet_bootstrap.cc:492] T 958ccd87b65945468e9247c40ca27c33 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:09:17.664261 25172 log.cc:927] Time spent T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:09:17.687397 25182 raft_consensus.cc:493] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
W20260709 13:09:17.664942 25174 log.cc:927] Time spent T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:09:17.689868 25054 raft_consensus.cc:515] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:17.672294 25078 leader_election.cc:290] T 72ac163496084a3e977344fa9228a3c9 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:17.667330 25165 consensus_queue.cc:237] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
W20260709 13:09:17.667143 25178 log.cc:927] Time spent T 00000000000000000000000000000000 P a20e5c2ad87d4af086eb0cf554c1d829: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 13:09:17.671826 25124 raft_consensus.cc:2468] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 1.
I20260709 13:09:17.671416 25161 leader_election.cc:290] T 6427fa143f574eaba2ee4c979e5485fc P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:17.673187 25164 raft_consensus.cc:2749] T 373185ca8ab04fdb91c7947ec0668808 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:17.710688 25076 tablet_bootstrap.cc:492] T 958ccd87b65945468e9247c40ca27c33 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:17.668325 25162 leader_election.cc:290] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 election: Requested vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
W20260709 13:09:17.714519 24346 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:45351, user_credentials={real_user=slave}} blocked reactor thread for 97030.3us
W20260709 13:09:17.712083 24494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:45351, user_credentials={real_user=slave}} blocked reactor thread for 256387us
I20260709 13:09:17.684990 25119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd607d60354e4864ae5499ee1c1a05c9" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:17.704937 25180 raft_consensus.cc:493] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.722379 25076 ts_tablet_manager.cc:1403] T 958ccd87b65945468e9247c40ca27c33 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.395s	user 0.023s	sys 0.009s
I20260709 13:09:17.671633 25117 raft_consensus.cc:2468] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 2.
I20260709 13:09:17.704849 25167 raft_consensus.cc:493] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.715378 24530 raft_consensus.cc:2468] T 53b7256c3dd84f5b952d1f9d47691f4e P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 2.
I20260709 13:09:17.687234 25120 raft_consensus.cc:3060] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:17.708268 25079 leader_election.cc:290] T 72d2665a42a94a82b1ceec249960fdf6 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:17.729269 24451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c60b42ac6d7e48bfae725dc113758659" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:17.736866 24451 raft_consensus.cc:3060] T c60b42ac6d7e48bfae725dc113758659 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:17.729621 24630 leader_election.cc:290] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 election: Requested vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:17.721635 25077 leader_election.cc:290] T 90d9e64f97544895b9b97dfdc5ceb359 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:17.739385 25077 raft_consensus.cc:2804] T 142bbcbf57124963866a3967c8adebed P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:17.732592 24493 leader_election.cc:304] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 838cb9997b0f437989d3f4e3e9904390; no voters: 
I20260709 13:09:17.739845 25077 raft_consensus.cc:697] T 142bbcbf57124963866a3967c8adebed P c40e7ba102224d24a22ccd1f68aaae45 [term 2 LEADER]: Becoming Leader. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Running, Role: LEADER
I20260709 13:09:17.740661 25077 consensus_queue.cc:237] T 142bbcbf57124963866a3967c8adebed P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:17.745003 24451 raft_consensus.cc:2468] T c60b42ac6d7e48bfae725dc113758659 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 2.
I20260709 13:09:17.704628 25052 raft_consensus.cc:515] T 4f32a46373be4a29b6ac72efa35494b4 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:17.707577 25182 raft_consensus.cc:515] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:17.753082 24630 raft_consensus.cc:2804] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:17.772768 24630 raft_consensus.cc:697] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [term 2 LEADER]: Becoming Leader. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Running, Role: LEADER
I20260709 13:09:17.737712 25180 raft_consensus.cc:515] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:17.773603 24630 consensus_queue.cc:237] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:17.753476 24345 leader_election.cc:304] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:17.746579 25166 raft_consensus.cc:2749] T 6ae3dc25b8484b6f99d8f41685229cb8 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:17.746354 25175 raft_consensus.cc:493] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.781016 25119 raft_consensus.cc:2393] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 32088c942fbb48239e0eed952d4df449 in current term 2: Already voted for candidate c40e7ba102224d24a22ccd1f68aaae45 in this term.
I20260709 13:09:17.797644 24528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6427fa143f574eaba2ee4c979e5485fc" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:17.798471 24528 raft_consensus.cc:2393] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 32088c942fbb48239e0eed952d4df449 in current term 2: Already voted for candidate 838cb9997b0f437989d3f4e3e9904390 in this term.
I20260709 13:09:17.801644 24528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d2665a42a94a82b1ceec249960fdf6" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:17.802205 24528 raft_consensus.cc:2468] T 72d2665a42a94a82b1ceec249960fdf6 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 1.
I20260709 13:09:17.747015 25167 raft_consensus.cc:515] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
W20260709 13:09:17.803148 24494 leader_election.cc:343] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:17.806110 24494 leader_election.cc:343] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:17.807183 24494 leader_election.cc:304] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381; no voters: 32088c942fbb48239e0eed952d4df449
I20260709 13:09:17.811142 24419 leader_election.cc:304] T 72d2665a42a94a82b1ceec249960fdf6 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 838cb9997b0f437989d3f4e3e9904390; no voters: 
I20260709 13:09:17.791822 25186 raft_consensus.cc:493] T 42bcfb18b77e41afa60057ccfd718fa9 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.814354 24381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142bbcbf57124963866a3967c8adebed" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:17.797735 25117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9711acbe7d044647bf774ee849c8a032" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:17.816727 24649 raft_consensus.cc:2804] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:17.815395 25166 raft_consensus.cc:2749] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:17.818976 24649 raft_consensus.cc:697] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390 [term 2 LEADER]: Becoming Leader. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Running, Role: LEADER
I20260709 13:09:17.799721 25124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c60b42ac6d7e48bfae725dc113758659" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:17.802877 25054 leader_election.cc:290] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:17.821147 24528 raft_consensus.cc:3060] T 9c88d903166c4812a88653e389ab5776 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:17.823318 24649 consensus_queue.cc:237] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:17.798142 25175 raft_consensus.cc:515] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
W20260709 13:09:17.835034 24420 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:45351, user_credentials={real_user=slave}} blocked reactor thread for 352982us
I20260709 13:09:17.847270 24528 raft_consensus.cc:1275] T 9c88d903166c4812a88653e389ab5776 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Refusing update from remote peer 32088c942fbb48239e0eed952d4df449: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:17.816742 25191 raft_consensus.cc:493] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.817315 25164 raft_consensus.cc:2804] T 72d2665a42a94a82b1ceec249960fdf6 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:09:17.869124 24346 leader_election.cc:343] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:17.821931 25122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c88d903166c4812a88653e389ab5776" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
W20260709 13:09:17.882303 24493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:39679, user_credentials={real_user=slave}} blocked reactor thread for 130429us
I20260709 13:09:17.881032 25076 raft_consensus.cc:359] T 958ccd87b65945468e9247c40ca27c33 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:17.883025 24493 leader_election.cc:304] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 838cb9997b0f437989d3f4e3e9904390; no voters: 
I20260709 13:09:17.883659 25076 raft_consensus.cc:740] T 958ccd87b65945468e9247c40ca27c33 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:17.868515 25164 raft_consensus.cc:493] T 72d2665a42a94a82b1ceec249960fdf6 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:17.820393 25186 raft_consensus.cc:515] T 42bcfb18b77e41afa60057ccfd718fa9 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:17.869333 25021 raft_consensus.cc:515] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:17.896586 24979 raft_consensus.cc:515] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:17.898595 25079 consensus_queue.cc:1048] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:17.899371 25175 leader_election.cc:290] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:17.893644 25117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88feffc0859413482b0daf562d2f86a" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:17.904305 25167 leader_election.cc:290] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:17.900264 25124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "373185ca8ab04fdb91c7947ec0668808" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:17.891544 25164 raft_consensus.cc:3060] T 72d2665a42a94a82b1ceec249960fdf6 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:17.870363 25191 raft_consensus.cc:515] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:17.877307 25119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90d9e64f97544895b9b97dfdc5ceb359" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:17.895781 25122 raft_consensus.cc:2393] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 838cb9997b0f437989d3f4e3e9904390 in current term 2: Already voted for candidate 32088c942fbb48239e0eed952d4df449 in this term.
I20260709 13:09:17.891830 25181 raft_consensus.cc:2804] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:09:17.908540 24345 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:39679, user_credentials={real_user=slave}} blocked reactor thread for 155126us
I20260709 13:09:17.913021 25021 leader_election.cc:290] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 election: Requested vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:17.913525 25124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f32a46373be4a29b6ac72efa35494b4" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:17.918460 24381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8775bc34e96d48ae8f5598affd26efdc" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:17.919217 24381 raft_consensus.cc:3060] T 8775bc34e96d48ae8f5598affd26efdc P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:17.892465 25076 consensus_queue.cc:260] T 958ccd87b65945468e9247c40ca27c33 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:17.925347 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90d9e64f97544895b9b97dfdc5ceb359" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:17.926498 24455 raft_consensus.cc:2393] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c40e7ba102224d24a22ccd1f68aaae45 in current term 2: Already voted for candidate 32088c942fbb48239e0eed952d4df449 in this term.
I20260709 13:09:17.932042 25119 raft_consensus.cc:2393] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c40e7ba102224d24a22ccd1f68aaae45 in current term 2: Already voted for candidate 32088c942fbb48239e0eed952d4df449 in this term.
I20260709 13:09:17.932653 25115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "142bbcbf57124963866a3967c8adebed" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:17.933605 25115 raft_consensus.cc:2393] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 838cb9997b0f437989d3f4e3e9904390 in current term 2: Already voted for candidate c40e7ba102224d24a22ccd1f68aaae45 in this term.
I20260709 13:09:17.934768 25116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e219ba07753f49a5abe454b27cd1494e" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:17.935177 25191 leader_election.cc:290] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:17.899411 25052 leader_election.cc:290] T 4f32a46373be4a29b6ac72efa35494b4 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:17.907058 25180 leader_election.cc:290] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:17.937736 25186 leader_election.cc:290] T 42bcfb18b77e41afa60057ccfd718fa9 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:17.938812 25181 raft_consensus.cc:697] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390 [term 2 LEADER]: Becoming Leader. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Running, Role: LEADER
I20260709 13:09:17.941663 24528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bb19353ba334eac831ae2cce277298a" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:17.941871 24530 raft_consensus.cc:1275] T 5b8d15473a964a0c8b8c283c4552706e P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Refusing update from remote peer c40e7ba102224d24a22ccd1f68aaae45: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:17.942575 24528 raft_consensus.cc:2468] T 5bb19353ba334eac831ae2cce277298a P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 1.
I20260709 13:09:17.916424 24979 leader_election.cc:290] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 election: Requested vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:17.955950 25076 ts_tablet_manager.cc:1434] T 958ccd87b65945468e9247c40ca27c33 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.228s	user 0.004s	sys 0.003s
I20260709 13:09:17.956658 24381 raft_consensus.cc:2468] T 8775bc34e96d48ae8f5598affd26efdc P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 2.
I20260709 13:09:17.957240 24454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e219ba07753f49a5abe454b27cd1494e" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:17.957897 24454 raft_consensus.cc:2468] T e219ba07753f49a5abe454b27cd1494e P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 1.
I20260709 13:09:17.959035 25076 tablet_bootstrap.cc:492] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:17.962378 24454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "373185ca8ab04fdb91c7947ec0668808" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:17.963198 24454 raft_consensus.cc:2468] T 373185ca8ab04fdb91c7947ec0668808 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 1.
I20260709 13:09:17.959424 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f32a46373be4a29b6ac72efa35494b4" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:17.966147 24455 raft_consensus.cc:2468] T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 1.
I20260709 13:09:17.956097 25124 raft_consensus.cc:2468] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 1.
W20260709 13:09:17.970532 24419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:33251, user_credentials={real_user=slave}} blocked reactor thread for 102646us
I20260709 13:09:17.968477 25122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8775bc34e96d48ae8f5598affd26efdc" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:17.977211 24452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ac163496084a3e977344fa9228a3c9" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:17.977927 24452 raft_consensus.cc:3060] T 72ac163496084a3e977344fa9228a3c9 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:17.985208 24381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ac163496084a3e977344fa9228a3c9" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:17.985744 24377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3b35ea0300f4b9e9afa80cf1dc9ba0f" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:17.986124 24381 raft_consensus.cc:2393] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 32088c942fbb48239e0eed952d4df449 in current term 2: Already voted for candidate c40e7ba102224d24a22ccd1f68aaae45 in this term.
I20260709 13:09:17.986415 24377 raft_consensus.cc:2468] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 1.
I20260709 13:09:17.986544 25123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bb19353ba334eac831ae2cce277298a" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:17.997279 25120 raft_consensus.cc:2468] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 2.
I20260709 13:09:17.995498 24452 raft_consensus.cc:2468] T 72ac163496084a3e977344fa9228a3c9 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 2.
I20260709 13:09:18.001214 25164 raft_consensus.cc:515] T 72d2665a42a94a82b1ceec249960fdf6 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:17.933634 25182 leader_election.cc:290] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:18.002466 24419 leader_election.cc:304] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:17.968748 25124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ac163496084a3e977344fa9228a3c9" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:18.022011 24420 leader_election.cc:304] T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381; no voters: c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:18.024919 24345 leader_election.cc:304] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:17.951018 25181 consensus_queue.cc:237] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:18.029479 24345 leader_election.cc:304] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:18.032017 25203 raft_consensus.cc:2804] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:18.035164 25167 raft_consensus.cc:2804] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:18.035689 25203 raft_consensus.cc:493] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:18.036119 25203 raft_consensus.cc:3060] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:18.038199 25164 leader_election.cc:290] T 72d2665a42a94a82b1ceec249960fdf6 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 election: Requested vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:18.040300 25116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42bcfb18b77e41afa60057ccfd718fa9" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:18.040342 25167 raft_consensus.cc:697] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45 [term 2 LEADER]: Becoming Leader. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Running, Role: LEADER
I20260709 13:09:18.041177 24345 leader_election.cc:304] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:18.041975 25116 raft_consensus.cc:1275] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Refusing update from remote peer 32088c942fbb48239e0eed952d4df449: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:18.041688 25167 consensus_queue.cc:237] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:18.042397 24979 raft_consensus.cc:2804] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:18.042842 24979 raft_consensus.cc:493] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:18.043187 24979 raft_consensus.cc:3060] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:18.046547 24420 leader_election.cc:304] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381; no voters: c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:18.046929 24419 leader_election.cc:304] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:18.048635 24379 raft_consensus.cc:1275] T 6ae3dc25b8484b6f99d8f41685229cb8 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Refusing update from remote peer 32088c942fbb48239e0eed952d4df449: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:18.012025 25115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c60b42ac6d7e48bfae725dc113758659" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:18.050177 25190 raft_consensus.cc:2804] T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:18.052004 25190 raft_consensus.cc:493] T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:18.052541 25190 raft_consensus.cc:3060] T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:18.034281 25124 raft_consensus.cc:2468] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 1.
I20260709 13:09:18.055702 25117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ac163496084a3e977344fa9228a3c9" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:18.056398 25117 raft_consensus.cc:3060] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:18.056754 24381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40cf5b30d13f4c3e86a9aa657c20c650" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:18.057413 24381 raft_consensus.cc:3060] T 40cf5b30d13f4c3e86a9aa657c20c650 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:18.058184 25182 consensus_queue.cc:1048] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:18.050704 24420 leader_election.cc:304] T bd607d60354e4864ae5499ee1c1a05c9 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32088c942fbb48239e0eed952d4df449; no voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:18.061265 24379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42bcfb18b77e41afa60057ccfd718fa9" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:18.063616 24528 raft_consensus.cc:1275] T 0f795acd16fd472b8c9360fb14e7e5d7 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Refusing update from remote peer c40e7ba102224d24a22ccd1f68aaae45: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:18.063954 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f7b6652eba4485ca0632f9fefc2e326" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449" is_pre_election: true
I20260709 13:09:18.064702 24455 raft_consensus.cc:2468] T 2f7b6652eba4485ca0632f9fefc2e326 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 1.
W20260709 13:09:18.067921 24420 leader_election.cc:343] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:18.078147 24381 raft_consensus.cc:2468] T 40cf5b30d13f4c3e86a9aa657c20c650 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 2.
I20260709 13:09:18.079037 24345 leader_election.cc:304] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:18.080035 24345 leader_election.cc:304] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:18.081151 24530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42bcfb18b77e41afa60057ccfd718fa9" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390" is_pre_election: true
I20260709 13:09:18.081758 24530 raft_consensus.cc:2468] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 1.
I20260709 13:09:18.062423 24379 raft_consensus.cc:2468] T 42bcfb18b77e41afa60057ccfd718fa9 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 1.
I20260709 13:09:18.050333 25164 raft_consensus.cc:2804] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:18.084517 24345 leader_election.cc:304] T 4f32a46373be4a29b6ac72efa35494b4 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:18.090665 25120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6427fa143f574eaba2ee4c979e5485fc" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:18.091574 25120 raft_consensus.cc:2393] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 32088c942fbb48239e0eed952d4df449 in current term 2: Already voted for candidate 838cb9997b0f437989d3f4e3e9904390 in this term.
W20260709 13:09:18.092280 24494 leader_election.cc:343] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:18.092646 25123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d2665a42a94a82b1ceec249960fdf6" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:18.093626 25122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3b35ea0300f4b9e9afa80cf1dc9ba0f" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:18.094795 24528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d2665a42a94a82b1ceec249960fdf6" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
W20260709 13:09:18.095072 24494 leader_election.cc:343] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:18.095396 24528 raft_consensus.cc:3060] T 72d2665a42a94a82b1ceec249960fdf6 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:18.095888 25205 raft_consensus.cc:3060] T bd607d60354e4864ae5499ee1c1a05c9 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:18.097033 24494 leader_election.cc:304] T 142bbcbf57124963866a3967c8adebed P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390; no voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:18.084954 25180 raft_consensus.cc:2804] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:18.097877 24649 raft_consensus.cc:3060] T 142bbcbf57124963866a3967c8adebed P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:18.083132 25186 raft_consensus.cc:2804] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:18.098551 25186 raft_consensus.cc:493] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:18.098940 25186 raft_consensus.cc:3060] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:18.103722 25116 raft_consensus.cc:1275] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Refusing update from remote peer c40e7ba102224d24a22ccd1f68aaae45: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:18.104282 25114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f7b6652eba4485ca0632f9fefc2e326" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:18.083362 25209 raft_consensus.cc:2804] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:18.097801 25180 raft_consensus.cc:493] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:18.108013 25185 raft_consensus.cc:2804] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:18.146775 25185 raft_consensus.cc:493] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:18.147222 25185 raft_consensus.cc:3060] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:18.108580 25209 raft_consensus.cc:697] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449 [term 2 LEADER]: Becoming Leader. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Running, Role: LEADER
I20260709 13:09:18.152835 25209 consensus_queue.cc:237] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:18.152832 25076 tablet_bootstrap.cc:492] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:18.122627 25164 raft_consensus.cc:697] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449 [term 2 LEADER]: Becoming Leader. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Running, Role: LEADER
I20260709 13:09:18.127523 25190 raft_consensus.cc:515] T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:18.156946 25164 consensus_queue.cc:237] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:18.158731 25190 leader_election.cc:290] T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 election: Requested vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:18.128651 24205 catalog_manager.cc:5697] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 reported cstate change: term changed from 1 to 2, leader changed from 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1) to 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4). New cstate: current_term: 2 leader_uuid: "838cb9997b0f437989d3f4e3e9904390" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:18.124636 24979 raft_consensus.cc:515] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:18.155727 25076 tablet_bootstrap.cc:492] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:18.162671 24979 leader_election.cc:290] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 election: Requested vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:18.125375 25121 raft_consensus.cc:1275] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Refusing update from remote peer 32088c942fbb48239e0eed952d4df449: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:18.163251 25076 ts_tablet_manager.cc:1403] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.207s	user 0.015s	sys 0.004s
I20260709 13:09:18.129180 25078 consensus_queue.cc:1048] T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 13:09:18.160459 24377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f32a46373be4a29b6ac72efa35494b4" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:18.166150 25076 raft_consensus.cc:359] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:18.166891 25076 raft_consensus.cc:740] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:18.166612 25185 raft_consensus.cc:515] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:18.107707 24649 raft_consensus.cc:2749] T 142bbcbf57124963866a3967c8adebed P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:18.144460 24420 leader_election.cc:304] T 72ac163496084a3e977344fa9228a3c9 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381; no voters: c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:18.118809 25117 raft_consensus.cc:2468] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 2.
I20260709 13:09:18.119288 24345 leader_election.cc:304] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
W20260709 13:09:18.169631 24494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:45351, user_credentials={real_user=slave}} blocked reactor thread for 72701.3us
W20260709 13:09:18.109334 24346 leader_election.cc:343] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:18.170343 24979 raft_consensus.cc:2804] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:18.170405 24420 leader_election.cc:304] T 6427fa143f574eaba2ee4c979e5485fc P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32088c942fbb48239e0eed952d4df449; no voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:18.139212 25186 raft_consensus.cc:515] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:18.171195 24979 raft_consensus.cc:697] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 LEADER]: Becoming Leader. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Running, Role: LEADER
I20260709 13:09:18.121507 24376 raft_consensus.cc:1275] T dd5ac0d1debc4695a406617ec06f07a0 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Refusing update from remote peer 32088c942fbb48239e0eed952d4df449: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:18.172124 24979 consensus_queue.cc:237] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:18.136849 25114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42bcfb18b77e41afa60057ccfd718fa9" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:18.114918 24528 raft_consensus.cc:2468] T 72d2665a42a94a82b1ceec249960fdf6 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 2.
I20260709 13:09:18.119227 25195 raft_consensus.cc:2804] T 4f32a46373be4a29b6ac72efa35494b4 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:18.120170 25180 raft_consensus.cc:3060] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:09:18.169492 24345 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:39679, user_credentials={real_user=slave}} blocked reactor thread for 50288.5us
I20260709 13:09:18.174361 25195 raft_consensus.cc:493] T 4f32a46373be4a29b6ac72efa35494b4 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:18.174906 24345 leader_election.cc:304] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
W20260709 13:09:18.175266 24420 leader_election.cc:343] T 72d2665a42a94a82b1ceec249960fdf6 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:18.175473 25190 raft_consensus.cc:3060] T 6427fa143f574eaba2ee4c979e5485fc P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:18.122640 25205 raft_consensus.cc:2749] T bd607d60354e4864ae5499ee1c1a05c9 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:18.122936 25123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d2665a42a94a82b1ceec249960fdf6" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:18.118597 25203 raft_consensus.cc:515] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:18.177645 25167 raft_consensus.cc:2804] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:18.177850 25114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f32a46373be4a29b6ac72efa35494b4" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:18.178090 25167 raft_consensus.cc:493] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:18.178572 25167 raft_consensus.cc:3060] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:18.178592 25203 leader_election.cc:290] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 election: Requested vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:18.174976 25195 raft_consensus.cc:3060] T 4f32a46373be4a29b6ac72efa35494b4 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:18.181643 24529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bb19353ba334eac831ae2cce277298a" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:18.182260 24529 raft_consensus.cc:3060] T 5bb19353ba334eac831ae2cce277298a P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:09:18.170665 24346 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:45351, user_credentials={real_user=slave}} blocked reactor thread for 61362.2us
I20260709 13:09:18.184919 24346 leader_election.cc:304] T 90d9e64f97544895b9b97dfdc5ceb359 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:18.173188 25186 leader_election.cc:290] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 election: Requested vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
W20260709 13:09:18.186193 24346 leader_election.cc:343] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:18.187009 25186 raft_consensus.cc:3060] T 90d9e64f97544895b9b97dfdc5ceb359 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:18.174455 25078 raft_consensus.cc:2764] T 72ac163496084a3e977344fa9228a3c9 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:09:18.187899 25205 consensus_queue.cc:1048] T dd5ac0d1debc4695a406617ec06f07a0 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:09:18.188407 24346 leader_election.cc:343] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:18.190992 24529 raft_consensus.cc:2468] T 5bb19353ba334eac831ae2cce277298a P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 2.
I20260709 13:09:18.192581 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e219ba07753f49a5abe454b27cd1494e" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:18.193451 24455 raft_consensus.cc:3060] T e219ba07753f49a5abe454b27cd1494e P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:18.145406 25122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40cf5b30d13f4c3e86a9aa657c20c650" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:18.137818 25204 consensus_queue.cc:1048] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.060s
I20260709 13:09:18.197191 24203 catalog_manager.cc:5697] T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449 reported cstate change: term changed from 1 to 2, leader changed from 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1) to 32088c942fbb48239e0eed952d4df449 (127.14.103.3). New cstate: current_term: 2 leader_uuid: "32088c942fbb48239e0eed952d4df449" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:18.210004 24455 raft_consensus.cc:2468] T e219ba07753f49a5abe454b27cd1494e P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 2.
I20260709 13:09:18.216228 24494 leader_election.cc:304] T 9c88d903166c4812a88653e389ab5776 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390; no voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:18.217792 25186 raft_consensus.cc:2749] T 90d9e64f97544895b9b97dfdc5ceb359 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:09:18.218567 24494 leader_election.cc:343] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:18.192615 24493 leader_election.cc:304] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
W20260709 13:09:18.176343 24420 leader_election.cc:343] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:18.220860 24420 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.1:45351, user_credentials={real_user=slave}} blocked reactor thread for 44544.5us
I20260709 13:09:18.220932 24493 leader_election.cc:304] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:18.222100 25193 raft_consensus.cc:2804] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:09:18.222474 24420 consensus_peers.cc:597] T dd5ac0d1debc4695a406617ec06f07a0 P 32088c942fbb48239e0eed952d4df449 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:18.192688 24346 leader_election.cc:343] T 42bcfb18b77e41afa60057ccfd718fa9 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:18.189147 25114 raft_consensus.cc:3060] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:18.200542 25165 consensus_queue.cc:1048] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:18.222523 25193 raft_consensus.cc:493] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
W20260709 13:09:18.223762 24346 consensus_peers.cc:597] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:18.223995 25193 raft_consensus.cc:3060] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:09:18.224931 24346 leader_election.cc:343] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:18.202302 25167 raft_consensus.cc:515] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:18.167814 25076 consensus_queue.cc:260] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:18.228564 25117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e219ba07753f49a5abe454b27cd1494e" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:18.229000 25123 raft_consensus.cc:3060] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:18.229231 24345 leader_election.cc:304] T 42bcfb18b77e41afa60057ccfd718fa9 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:18.220861 25181 raft_consensus.cc:2804] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:18.230160 25167 leader_election.cc:290] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 election: Requested vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:18.230666 25203 raft_consensus.cc:2804] T 42bcfb18b77e41afa60057ccfd718fa9 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:18.224856 25165 consensus_queue.cc:1048] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 13:09:18.231158 25203 raft_consensus.cc:493] T 42bcfb18b77e41afa60057ccfd718fa9 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:18.231575 25203 raft_consensus.cc:3060] T 42bcfb18b77e41afa60057ccfd718fa9 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:18.234762 25181 raft_consensus.cc:697] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [term 2 LEADER]: Becoming Leader. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Running, Role: LEADER
I20260709 13:09:18.235812 25181 consensus_queue.cc:237] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:18.236091 25122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bb19353ba334eac831ae2cce277298a" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:18.237056 24345 leader_election.cc:304] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:18.237936 25076 ts_tablet_manager.cc:1434] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.074s	user 0.008s	sys 0.000s
I20260709 13:09:18.206750 25195 raft_consensus.cc:515] T 4f32a46373be4a29b6ac72efa35494b4 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:18.239434 24377 raft_consensus.cc:2393] T 4f32a46373be4a29b6ac72efa35494b4 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 32088c942fbb48239e0eed952d4df449 in current term 2: Already voted for candidate c40e7ba102224d24a22ccd1f68aaae45 in this term.
I20260709 13:09:18.240507 25195 leader_election.cc:290] T 4f32a46373be4a29b6ac72efa35494b4 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 election: Requested vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:18.208096 25180 raft_consensus.cc:515] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:18.242601 25122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9711acbe7d044647bf774ee849c8a032" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:18.242765 25180 leader_election.cc:290] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 election: Requested vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:18.210150 24203 catalog_manager.cc:5697] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449 reported cstate change: term changed from 1 to 2, leader changed from 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1) to 32088c942fbb48239e0eed952d4df449 (127.14.103.3). New cstate: current_term: 2 leader_uuid: "32088c942fbb48239e0eed952d4df449" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:18.133980 24202 catalog_manager.cc:5697] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45 reported cstate change: term changed from 1 to 2, leader changed from 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1) to c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2). New cstate: current_term: 2 leader_uuid: "c40e7ba102224d24a22ccd1f68aaae45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:18.219877 24649 raft_consensus.cc:2749] T 9c88d903166c4812a88653e389ab5776 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:18.238214 25205 consensus_queue.cc:1048] T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:18.246490 25182 raft_consensus.cc:2804] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:18.244899 24202 catalog_manager.cc:5697] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45 reported cstate change: term changed from 1 to 2, leader changed from 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1) to c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2). New cstate: current_term: 2 leader_uuid: "c40e7ba102224d24a22ccd1f68aaae45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:18.247032 25182 raft_consensus.cc:697] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45 [term 2 LEADER]: Becoming Leader. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Running, Role: LEADER
I20260709 13:09:18.247944 25182 consensus_queue.cc:237] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:18.238922 25076 tablet_bootstrap.cc:492] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:18.253634 24529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9711acbe7d044647bf774ee849c8a032" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:18.254417 24529 raft_consensus.cc:3060] T 9711acbe7d044647bf774ee849c8a032 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:18.247897 24202 catalog_manager.cc:5697] T 142bbcbf57124963866a3967c8adebed P c40e7ba102224d24a22ccd1f68aaae45 reported cstate change: term changed from 1 to 2, leader changed from 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1) to c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2). New cstate: current_term: 2 leader_uuid: "c40e7ba102224d24a22ccd1f68aaae45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:18.258704 25123 raft_consensus.cc:2468] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 2.
I20260709 13:09:18.281407 24345 leader_election.cc:304] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:18.281430 24452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f32a46373be4a29b6ac72efa35494b4" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:18.282505 24452 raft_consensus.cc:2393] T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate c40e7ba102224d24a22ccd1f68aaae45 in current term 2: Already voted for candidate 32088c942fbb48239e0eed952d4df449 in this term.
I20260709 13:09:18.282871 24451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "373185ca8ab04fdb91c7947ec0668808" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:18.283195 25195 raft_consensus.cc:2804] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:18.283528 24451 raft_consensus.cc:3060] T 373185ca8ab04fdb91c7947ec0668808 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:18.242312 25203 raft_consensus.cc:515] T 42bcfb18b77e41afa60057ccfd718fa9 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:18.288017 25203 leader_election.cc:290] T 42bcfb18b77e41afa60057ccfd718fa9 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 election: Requested vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:18.291762 25195 raft_consensus.cc:697] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45 [term 2 LEADER]: Becoming Leader. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Running, Role: LEADER
I20260709 13:09:18.292023 25185 leader_election.cc:290] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 election: Requested vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:18.293562 24377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3b35ea0300f4b9e9afa80cf1dc9ba0f" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:18.294291 24377 raft_consensus.cc:3060] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:18.215157 25190 raft_consensus.cc:2749] T 6427fa143f574eaba2ee4c979e5485fc P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:18.294312 25195 consensus_queue.cc:237] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:18.301005 24528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42bcfb18b77e41afa60057ccfd718fa9" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:18.302882 24451 raft_consensus.cc:2468] T 373185ca8ab04fdb91c7947ec0668808 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 2.
I20260709 13:09:18.303917 24529 raft_consensus.cc:2468] T 9711acbe7d044647bf774ee849c8a032 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 2.
I20260709 13:09:18.314010 24527 raft_consensus.cc:1275] T 142bbcbf57124963866a3967c8adebed P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Refusing update from remote peer c40e7ba102224d24a22ccd1f68aaae45: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:18.307425 25119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3b35ea0300f4b9e9afa80cf1dc9ba0f" candidate_uuid: "32088c942fbb48239e0eed952d4df449" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
W20260709 13:09:18.316720 24420 leader_election.cc:343] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:18.273197 25193 raft_consensus.cc:515] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:18.292119 24455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f7b6652eba4485ca0632f9fefc2e326" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:18.317937 24455 raft_consensus.cc:3060] T 2f7b6652eba4485ca0632f9fefc2e326 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:09:18.318859 24494 leader_election.cc:343] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:18.320065 24494 leader_election.cc:343] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:18.316838 24345 leader_election.cc:304] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:18.322515 24345 leader_election.cc:304] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:18.269893 25122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f7b6652eba4485ca0632f9fefc2e326" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:18.310341 25114 raft_consensus.cc:2468] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 2.
W20260709 13:09:18.321658 24346 leader_election.cc:343] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:18.333736 24346 leader_election.cc:343] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:18.334731 24346 leader_election.cc:343] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:18.332916 24419 leader_election.cc:304] T 72d2665a42a94a82b1ceec249960fdf6 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 838cb9997b0f437989d3f4e3e9904390; no voters: 
I20260709 13:09:18.328114 25167 raft_consensus.cc:2804] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:18.348137 24377 raft_consensus.cc:2468] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32088c942fbb48239e0eed952d4df449 in term 2.
I20260709 13:09:18.348320 25167 raft_consensus.cc:697] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 LEADER]: Becoming Leader. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Running, Role: LEADER
I20260709 13:09:18.349114 25119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f32a46373be4a29b6ac72efa35494b4" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:18.349227 25167 consensus_queue.cc:237] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:18.350018 25119 raft_consensus.cc:2393] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate c40e7ba102224d24a22ccd1f68aaae45 in current term 2: Already voted for candidate 32088c942fbb48239e0eed952d4df449 in this term.
I20260709 13:09:18.350917 25119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42bcfb18b77e41afa60057ccfd718fa9" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:18.351089 25120 raft_consensus.cc:1275] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Refusing update from remote peer c40e7ba102224d24a22ccd1f68aaae45: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:18.352008 25124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "373185ca8ab04fdb91c7947ec0668808" candidate_uuid: "c40e7ba102224d24a22ccd1f68aaae45" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:18.354139 24528 raft_consensus.cc:2393] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate c40e7ba102224d24a22ccd1f68aaae45 in current term 2: Already voted for candidate 838cb9997b0f437989d3f4e3e9904390 in this term.
I20260709 13:09:18.357287 24346 leader_election.cc:304] T 4f32a46373be4a29b6ac72efa35494b4 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381
W20260709 13:09:18.358739 24346 leader_election.cc:343] T 42bcfb18b77e41afa60057ccfd718fa9 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:18.327915 25203 raft_consensus.cc:2804] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:09:18.360967 24346 leader_election.cc:343] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:18.361577 25165 consensus_queue.cc:1048] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:18.363863 25183 raft_consensus.cc:2804] T 72d2665a42a94a82b1ceec249960fdf6 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:18.364617 25183 raft_consensus.cc:697] T 72d2665a42a94a82b1ceec249960fdf6 P 32088c942fbb48239e0eed952d4df449 [term 2 LEADER]: Becoming Leader. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Running, Role: LEADER
I20260709 13:09:18.367647 24345 leader_election.cc:304] T 42bcfb18b77e41afa60057ccfd718fa9 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c40e7ba102224d24a22ccd1f68aaae45; no voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:18.354220 25115 raft_consensus.cc:1275] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Refusing update from remote peer c40e7ba102224d24a22ccd1f68aaae45: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:18.332204 25180 consensus_queue.cc:1048] T 142bbcbf57124963866a3967c8adebed P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
W20260709 13:09:18.375134 24346 leader_election.cc:343] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:18.380214 24420 leader_election.cc:304] T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381; no voters: c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:18.383853 24451 raft_consensus.cc:1275] T 6427fa143f574eaba2ee4c979e5485fc P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Refusing update from remote peer 838cb9997b0f437989d3f4e3e9904390: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:18.372611 25052 raft_consensus.cc:2749] T 42bcfb18b77e41afa60057ccfd718fa9 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:09:18.385484 24455 raft_consensus.cc:2468] T 2f7b6652eba4485ca0632f9fefc2e326 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c40e7ba102224d24a22ccd1f68aaae45 in term 2.
I20260709 13:09:18.385716 24649 consensus_queue.cc:1048] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:09:18.391099 24345 leader_election.cc:304] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, c40e7ba102224d24a22ccd1f68aaae45; no voters: 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:18.392091 25167 raft_consensus.cc:2804] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:18.392601 25167 raft_consensus.cc:697] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 LEADER]: Becoming Leader. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Running, Role: LEADER
I20260709 13:09:18.376144 25203 raft_consensus.cc:697] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 LEADER]: Becoming Leader. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Running, Role: LEADER
I20260709 13:09:18.388888 25204 raft_consensus.cc:2804] T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:09:18.410845 25226 log.cc:927] Time spent T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:09:18.412609 25197 raft_consensus.cc:2749] T 4f32a46373be4a29b6ac72efa35494b4 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:09:18.422576 25204 raft_consensus.cc:697] T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449 [term 2 LEADER]: Becoming Leader. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Running, Role: LEADER
I20260709 13:09:18.423566 25204 consensus_queue.cc:237] T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:18.424407 25203 consensus_queue.cc:237] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:18.432849 25193 leader_election.cc:290] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 election: Requested vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:18.440398 24451 raft_consensus.cc:1275] T b88feffc0859413482b0daf562d2f86a P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Refusing update from remote peer 838cb9997b0f437989d3f4e3e9904390: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:18.464627 25167 consensus_queue.cc:237] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:18.473716 24649 consensus_queue.cc:1048] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:18.396306 25183 consensus_queue.cc:237] T 72d2665a42a94a82b1ceec249960fdf6 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
W20260709 13:09:18.470292 25234 log.cc:927] Time spent T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45: Append to log took a long time: real 0.065s	user 0.000s	sys 0.008s
I20260709 13:09:18.487460 24419 leader_election.cc:304] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 32088c942fbb48239e0eed952d4df449 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, c40e7ba102224d24a22ccd1f68aaae45; no voters: 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:18.420724 25197 consensus_queue.cc:1048] T 142bbcbf57124963866a3967c8adebed P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.065s
I20260709 13:09:18.504123 25185 raft_consensus.cc:2804] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:18.485822 24377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42bcfb18b77e41afa60057ccfd718fa9" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:18.519207 24377 raft_consensus.cc:2393] T 42bcfb18b77e41afa60057ccfd718fa9 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 838cb9997b0f437989d3f4e3e9904390 in current term 2: Already voted for candidate c40e7ba102224d24a22ccd1f68aaae45 in this term.
I20260709 13:09:18.525466 25185 raft_consensus.cc:697] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 32088c942fbb48239e0eed952d4df449 [term 2 LEADER]: Becoming Leader. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Running, Role: LEADER
I20260709 13:09:18.532688 25185 consensus_queue.cc:237] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 32088c942fbb48239e0eed952d4df449 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:18.539801 25119 raft_consensus.cc:1275] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Refusing update from remote peer 838cb9997b0f437989d3f4e3e9904390: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:18.540907 24452 raft_consensus.cc:1275] T bd607d60354e4864ae5499ee1c1a05c9 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Refusing update from remote peer c40e7ba102224d24a22ccd1f68aaae45: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:09:18.542558 25178 log.cc:927] Time spent T 00000000000000000000000000000000 P a20e5c2ad87d4af086eb0cf554c1d829: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:09:18.540011 25122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42bcfb18b77e41afa60057ccfd718fa9" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:18.557546 25052 consensus_queue.cc:1048] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:09:18.613987 25122 raft_consensus.cc:1275] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Refusing update from remote peer c40e7ba102224d24a22ccd1f68aaae45: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:09:18.616317 24494 consensus_peers.cc:597] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:18.617228 24494 consensus_peers.cc:597] T c6af3489742e4539866e5a3b10761ca2 P 838cb9997b0f437989d3f4e3e9904390 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:18.622920 25206 consensus_queue.cc:1048] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:09:18.623783 25236 log.cc:927] Time spent T 9c88d903166c4812a88653e389ab5776 P 838cb9997b0f437989d3f4e3e9904390: Append to log took a long time: real 0.135s	user 0.002s	sys 0.000s
I20260709 13:09:18.626379 25076 tablet_bootstrap.cc:492] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:18.627331 25076 tablet_bootstrap.cc:492] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
W20260709 13:09:18.627367 24494 leader_election.cc:343] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:18.627964 24494 leader_election.cc:304] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390; no voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:18.629153 25206 raft_consensus.cc:2749] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:09:18.632441 25076 ts_tablet_manager.cc:1403] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.394s	user 0.024s	sys 0.005s
I20260709 13:09:18.639653 24453 raft_consensus.cc:1275] T c60b42ac6d7e48bfae725dc113758659 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Refusing update from remote peer 838cb9997b0f437989d3f4e3e9904390: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:09:18.646575 25239 log.cc:927] Time spent T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449: Append to log took a long time: real 0.158s	user 0.000s	sys 0.003s
I20260709 13:09:18.653187 25076 raft_consensus.cc:359] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:18.653954 25076 raft_consensus.cc:740] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:18.648095 25167 consensus_queue.cc:1048] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:09:18.673715 24530 raft_consensus.cc:1275] T 53b7256c3dd84f5b952d1f9d47691f4e P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Refusing update from remote peer c40e7ba102224d24a22ccd1f68aaae45: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:18.674675 25206 consensus_queue.cc:1048] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:18.677982 25197 consensus_queue.cc:1048] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:09:18.688923 24346 consensus_peers.cc:597] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:18.694454 24377 raft_consensus.cc:1275] T 90d9e64f97544895b9b97dfdc5ceb359 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Refusing update from remote peer 32088c942fbb48239e0eed952d4df449: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:18.696017 24381 raft_consensus.cc:1275] T 8775bc34e96d48ae8f5598affd26efdc P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Refusing update from remote peer 32088c942fbb48239e0eed952d4df449: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:18.701918 25190 consensus_queue.cc:1048] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:18.686720 25076 consensus_queue.cc:260] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:18.705089 25190 consensus_queue.cc:1048] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:18.712815 25076 ts_tablet_manager.cc:1434] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.080s	user 0.007s	sys 0.000s
I20260709 13:09:18.714005 25076 tablet_bootstrap.cc:492] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
W20260709 13:09:18.722837 24345 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:33251, user_credentials={real_user=slave}} blocked reactor thread for 47068.8us
I20260709 13:09:18.723872 24205 catalog_manager.cc:5697] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45 reported cstate change: term changed from 1 to 2, leader changed from 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1) to c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2). New cstate: current_term: 2 leader_uuid: "c40e7ba102224d24a22ccd1f68aaae45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:18.731047 24452 raft_consensus.cc:1275] T 72ac163496084a3e977344fa9228a3c9 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Refusing update from remote peer c40e7ba102224d24a22ccd1f68aaae45: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:18.741963 24205 catalog_manager.cc:5697] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45 reported cstate change: term changed from 1 to 2, leader changed from 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1) to c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2). New cstate: current_term: 2 leader_uuid: "c40e7ba102224d24a22ccd1f68aaae45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:18.743966 25197 consensus_queue.cc:1048] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:18.747926 25114 raft_consensus.cc:1275] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Refusing update from remote peer 32088c942fbb48239e0eed952d4df449: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:18.757308 25076 tablet_bootstrap.cc:492] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:18.760244 25076 tablet_bootstrap.cc:492] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:18.760954 25076 ts_tablet_manager.cc:1403] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.047s	user 0.015s	sys 0.006s
I20260709 13:09:18.764333 25076 raft_consensus.cc:359] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:18.765012 25076 raft_consensus.cc:740] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:18.765969 25076 consensus_queue.cc:260] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:18.768621 25076 ts_tablet_manager.cc:1434] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260709 13:09:18.776989 25076 tablet_bootstrap.cc:492] T c6af3489742e4539866e5a3b10761ca2 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:18.781148 25079 consensus_queue.cc:1048] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:18.768525 25115 raft_consensus.cc:1275] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Refusing update from remote peer c40e7ba102224d24a22ccd1f68aaae45: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:09:18.785638 24420 consensus_peers.cc:597] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:18.794360 24494 consensus_peers.cc:597] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:18.805866 24346 consensus_peers.cc:597] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:18.822937 25117 raft_consensus.cc:3060] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:18.829120 25167 consensus_queue.cc:1048] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:09:18.832199 24494 consensus_peers.cc:597] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:18.830974 24205 catalog_manager.cc:5697] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45 reported cstate change: term changed from 1 to 2, leader changed from 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1) to c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2). New cstate: current_term: 2 leader_uuid: "c40e7ba102224d24a22ccd1f68aaae45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:18.832587 24381 raft_consensus.cc:1275] T 40cf5b30d13f4c3e86a9aa657c20c650 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Refusing update from remote peer 838cb9997b0f437989d3f4e3e9904390: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:18.879693 24526 raft_consensus.cc:1275] T 5bb19353ba334eac831ae2cce277298a P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Refusing update from remote peer c40e7ba102224d24a22ccd1f68aaae45: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:18.880654 25114 raft_consensus.cc:1275] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Refusing update from remote peer 32088c942fbb48239e0eed952d4df449: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:18.882534 25187 consensus_queue.cc:1048] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:18.882927 25185 consensus_queue.cc:1048] T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:18.899434 24452 raft_consensus.cc:1275] T 373185ca8ab04fdb91c7947ec0668808 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Refusing update from remote peer c40e7ba102224d24a22ccd1f68aaae45: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:09:18.905438 24420 consensus_peers.cc:597] T e4dd51cda55249b48d7bfa39d5feb398 P 32088c942fbb48239e0eed952d4df449 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:18.906147 24530 raft_consensus.cc:1275] T 9711acbe7d044647bf774ee849c8a032 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Refusing update from remote peer c40e7ba102224d24a22ccd1f68aaae45: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:18.910977 24377 raft_consensus.cc:1275] T 4f32a46373be4a29b6ac72efa35494b4 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Refusing update from remote peer 32088c942fbb48239e0eed952d4df449: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:18.914618 25167 consensus_queue.cc:1048] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:09:18.915264 24494 consensus_peers.cc:597] T 86262701775a456dacd47fcdc1a5d13b P 838cb9997b0f437989d3f4e3e9904390 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:18.917778 25183 consensus_queue.cc:1048] T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:18.843369 24205 catalog_manager.cc:5697] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45 reported cstate change: term changed from 1 to 2, leader changed from 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1) to c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2). New cstate: current_term: 2 leader_uuid: "c40e7ba102224d24a22ccd1f68aaae45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } }
W20260709 13:09:18.943625 24420 consensus_peers.cc:597] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 32088c942fbb48239e0eed952d4df449 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:18.946317 25167 consensus_queue.cc:1048] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:18.969239 25181 consensus_queue.cc:1048] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:09:18.999519 24346 consensus_peers.cc:597] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:19.000433 24346 consensus_peers.cc:597] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:19.001461 24346 consensus_peers.cc:597] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:18.934969 24205 catalog_manager.cc:5697] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45 reported cstate change: term changed from 1 to 2, leader changed from 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1) to c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2). New cstate: current_term: 2 leader_uuid: "c40e7ba102224d24a22ccd1f68aaae45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:19.005528 24205 catalog_manager.cc:5697] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45 reported cstate change: term changed from 1 to 2, leader changed from 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1) to c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2). New cstate: current_term: 2 leader_uuid: "c40e7ba102224d24a22ccd1f68aaae45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:19.020968 24205 catalog_manager.cc:5697] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45 reported cstate change: term changed from 1 to 2, leader changed from 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1) to c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2). New cstate: current_term: 2 leader_uuid: "c40e7ba102224d24a22ccd1f68aaae45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:19.032231 24452 raft_consensus.cc:1275] T e219ba07753f49a5abe454b27cd1494e P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Refusing update from remote peer c40e7ba102224d24a22ccd1f68aaae45: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:19.045416 25076 tablet_bootstrap.cc:492] T c6af3489742e4539866e5a3b10761ca2 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:19.046290 25052 consensus_queue.cc:1048] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 13:09:19.060531 25076 tablet_bootstrap.cc:492] T c6af3489742e4539866e5a3b10761ca2 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:19.061293 25076 ts_tablet_manager.cc:1403] T c6af3489742e4539866e5a3b10761ca2 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.285s	user 0.068s	sys 0.142s
I20260709 13:09:19.063860 25076 raft_consensus.cc:359] T c6af3489742e4539866e5a3b10761ca2 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:19.064466 25076 raft_consensus.cc:740] T c6af3489742e4539866e5a3b10761ca2 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:19.065091 25076 consensus_queue.cc:260] T c6af3489742e4539866e5a3b10761ca2 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
W20260709 13:09:19.118535 24346 consensus_peers.cc:597] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:19.122346 24420 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:45351, user_credentials={real_user=slave}} blocked reactor thread for 56699.3us
I20260709 13:09:19.125042 25076 ts_tablet_manager.cc:1434] T c6af3489742e4539866e5a3b10761ca2 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.063s	user 0.006s	sys 0.001s
I20260709 13:09:19.128315 25076 tablet_bootstrap.cc:492] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
W20260709 13:09:19.141044 24346 consensus_peers.cc:597] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:19.158954 24455 raft_consensus.cc:1275] T 2f7b6652eba4485ca0632f9fefc2e326 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Refusing update from remote peer c40e7ba102224d24a22ccd1f68aaae45: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:19.189788 25052 consensus_queue.cc:1048] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:19.201575 25115 raft_consensus.cc:1275] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Refusing update from remote peer 32088c942fbb48239e0eed952d4df449: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:09:19.203575 24494 consensus_peers.cc:597] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:19.223688 24381 raft_consensus.cc:1275] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Refusing update from remote peer 32088c942fbb48239e0eed952d4df449: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:19.230815 25183 consensus_queue.cc:1048] T 72d2665a42a94a82b1ceec249960fdf6 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:19.231777 24529 raft_consensus.cc:1275] T 72d2665a42a94a82b1ceec249960fdf6 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Refusing update from remote peer 32088c942fbb48239e0eed952d4df449: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:09:19.241267 24346 consensus_peers.cc:597] T 4af7fc369d124452beb5fd07cf8592ec P c40e7ba102224d24a22ccd1f68aaae45 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:19.256367 25161 consensus_queue.cc:1048] T 72d2665a42a94a82b1ceec249960fdf6 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:09:19.286059 25164 consensus_queue.cc:1048] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 32088c942fbb48239e0eed952d4df449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:19.413285 25116 raft_consensus.cc:3060] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:09:19.424677 25276 log.cc:927] Time spent T 2f7b6652eba4485ca0632f9fefc2e326 P 32088c942fbb48239e0eed952d4df449: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
W20260709 13:09:19.441243 24493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:39679, user_credentials={real_user=slave}} blocked reactor thread for 81749.9us
W20260709 13:09:19.447634 25278 log.cc:927] Time spent T a3b35ea0300f4b9e9afa80cf1dc9ba0f P c40e7ba102224d24a22ccd1f68aaae45: Append to log took a long time: real 0.055s	user 0.000s	sys 0.002s
I20260709 13:09:19.478341 25076 tablet_bootstrap.cc:492] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:19.479610 25076 tablet_bootstrap.cc:492] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:19.482139 25076 ts_tablet_manager.cc:1403] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.355s	user 0.027s	sys 0.028s
I20260709 13:09:19.484148 24204 catalog_manager.cc:5697] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449 reported cstate change: term changed from 1 to 2, leader changed from 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1) to 32088c942fbb48239e0eed952d4df449 (127.14.103.3). New cstate: current_term: 2 leader_uuid: "32088c942fbb48239e0eed952d4df449" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:19.485801 25076 raft_consensus.cc:359] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:19.486554 25076 raft_consensus.cc:740] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:19.487141 24204 catalog_manager.cc:5697] T 6427fa143f574eaba2ee4c979e5485fc P 32088c942fbb48239e0eed952d4df449 reported cstate change: term changed from 1 to 2, leader changed from 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1) to 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4). New cstate: current_term: 2 leader_uuid: "838cb9997b0f437989d3f4e3e9904390" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } }
I20260709 13:09:19.489730 24204 catalog_manager.cc:5697] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 32088c942fbb48239e0eed952d4df449 reported cstate change: term changed from 1 to 2, leader changed from 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1) to 32088c942fbb48239e0eed952d4df449 (127.14.103.3). New cstate: current_term: 2 leader_uuid: "32088c942fbb48239e0eed952d4df449" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:19.492549 24204 catalog_manager.cc:5697] T 72d2665a42a94a82b1ceec249960fdf6 P 32088c942fbb48239e0eed952d4df449 reported cstate change: term changed from 1 to 2, leader changed from 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1) to 32088c942fbb48239e0eed952d4df449 (127.14.103.3). New cstate: current_term: 2 leader_uuid: "32088c942fbb48239e0eed952d4df449" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } }
W20260709 13:09:19.496037 24494 consensus_peers.cc:597] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:19.487440 25076 consensus_queue.cc:260] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:19.510782 24204 catalog_manager.cc:5697] T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449 reported cstate change: term changed from 1 to 2, leader changed from 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1) to 32088c942fbb48239e0eed952d4df449 (127.14.103.3). New cstate: current_term: 2 leader_uuid: "32088c942fbb48239e0eed952d4df449" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:19.527604 25076 ts_tablet_manager.cc:1434] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.045s	user 0.006s	sys 0.003s
I20260709 13:09:19.600515 24204 catalog_manager.cc:5697] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449 reported cstate change: term changed from 1 to 2, leader changed from 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1) to 32088c942fbb48239e0eed952d4df449 (127.14.103.3). New cstate: current_term: 2 leader_uuid: "32088c942fbb48239e0eed952d4df449" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:19.789584 24204 catalog_manager.cc:5697] T c60b42ac6d7e48bfae725dc113758659 P 32088c942fbb48239e0eed952d4df449 reported cstate change: term changed from 1 to 2, leader changed from 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1) to 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4). New cstate: current_term: 2 leader_uuid: "838cb9997b0f437989d3f4e3e9904390" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } }
I20260709 13:09:19.802592 25076 tablet_bootstrap.cc:492] T 86262701775a456dacd47fcdc1a5d13b P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
W20260709 13:09:19.932806 24494 consensus_peers.cc:597] T 363cde7d4cb84353a74a18cbd9701fa7 P 838cb9997b0f437989d3f4e3e9904390 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:20.124943 24346 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:45351, user_credentials={real_user=slave}} blocked reactor thread for 201379us
W20260709 13:09:20.129781 24419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.4:33251, user_credentials={real_user=slave}} blocked reactor thread for 436286us
W20260709 13:09:20.137805 24493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:46177, user_credentials={real_user=slave}} blocked reactor thread for 191070us
I20260709 13:09:20.155085 25076 tablet_bootstrap.cc:492] T 86262701775a456dacd47fcdc1a5d13b P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:20.156524 25076 tablet_bootstrap.cc:492] T 86262701775a456dacd47fcdc1a5d13b P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:20.157392 25076 ts_tablet_manager.cc:1403] T 86262701775a456dacd47fcdc1a5d13b P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.607s	user 0.079s	sys 0.088s
I20260709 13:09:20.195417 25179 raft_consensus.cc:493] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:20.196169 25179 raft_consensus.cc:515] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:20.201363 25076 raft_consensus.cc:359] T 86262701775a456dacd47fcdc1a5d13b P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:20.202103 25076 raft_consensus.cc:740] T 86262701775a456dacd47fcdc1a5d13b P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:20.202778 25076 consensus_queue.cc:260] T 86262701775a456dacd47fcdc1a5d13b P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:20.225744 25076 ts_tablet_manager.cc:1434] T 86262701775a456dacd47fcdc1a5d13b P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.068s	user 0.004s	sys 0.002s
W20260709 13:09:20.247495 24420 consensus_peers.cc:597] T 2deba793363c4afca7d75b2fc3814304 P 32088c942fbb48239e0eed952d4df449 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:20.252827 25119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42bcfb18b77e41afa60057ccfd718fa9" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381" is_pre_election: true
I20260709 13:09:20.264839 25076 tablet_bootstrap.cc:492] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:20.270044 25179 leader_election.cc:290] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:20.271440 24379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42bcfb18b77e41afa60057ccfd718fa9" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45" is_pre_election: true
I20260709 13:09:20.272284 24379 raft_consensus.cc:2468] T 42bcfb18b77e41afa60057ccfd718fa9 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 2.
W20260709 13:09:20.277043 24494 leader_election.cc:343] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:20.278623 24493 leader_election.cc:304] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:20.279840 25282 raft_consensus.cc:2804] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:20.280278 25282 raft_consensus.cc:493] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:20.280773 25282 raft_consensus.cc:3060] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:20.296559 25282 raft_consensus.cc:515] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:20.298725 25282 leader_election.cc:290] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 3 election: Requested vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:20.303205 25116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42bcfb18b77e41afa60057ccfd718fa9" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
I20260709 13:09:20.310111 24379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42bcfb18b77e41afa60057ccfd718fa9" candidate_uuid: "838cb9997b0f437989d3f4e3e9904390" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:20.310801 24379 raft_consensus.cc:3060] T 42bcfb18b77e41afa60057ccfd718fa9 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:09:20.316383 24494 leader_election.cc:343] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 3 election: Tablet error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:20.320348 24379 raft_consensus.cc:2468] T 42bcfb18b77e41afa60057ccfd718fa9 P c40e7ba102224d24a22ccd1f68aaae45 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 838cb9997b0f437989d3f4e3e9904390 in term 3.
I20260709 13:09:20.330813 24493 leader_election.cc:304] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, c40e7ba102224d24a22ccd1f68aaae45; no voters: 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:20.332451 25179 raft_consensus.cc:2804] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:20.332959 25179 raft_consensus.cc:697] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [term 3 LEADER]: Becoming Leader. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Running, Role: LEADER
I20260709 13:09:20.334117 25179 consensus_queue.cc:237] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:20.351109 24202 catalog_manager.cc:5697] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 reported cstate change: term changed from 1 to 2, leader changed from 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1) to 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4). New cstate: current_term: 2 leader_uuid: "838cb9997b0f437989d3f4e3e9904390" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:20.355708 24204 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34412 (request call id 110) took 1554 ms. Trace:
I20260709 13:09:20.360817 25076 tablet_bootstrap.cc:492] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:20.361869 25076 tablet_bootstrap.cc:492] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:20.362520 25076 ts_tablet_manager.cc:1403] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.124s	user 0.018s	sys 0.004s
I20260709 13:09:20.356220 24204 rpcz_store.cc:276] 0709 13:09:18.801339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:18.801491 (+   152us) service_pool.cc:224] Handling call
0709 13:09:20.355600 (+1554109us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:20.140671 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:20.152366 (+ 11695us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:20.152406 (+    40us) write_op.cc:435] Acquired schema lock
0709 13:09:20.152420 (+    14us) tablet.cc:662] Decoding operations
0709 13:09:20.153295 (+   875us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:20.153351 (+    56us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:20.153378 (+    27us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:09:20.153981 (+   603us) tablet.cc:701] Row locks acquired
0709 13:09:20.153996 (+    15us) write_op.cc:260] PREPARE: finished
0709 13:09:20.154156 (+   160us) write_op.cc:270] Start()
0709 13:09:20.154264 (+   108us) write_op.cc:276] Timestamp: P: 1783602560154104 usec, L: 0
0709 13:09:20.154280 (+    16us) op_driver.cc:348] REPLICATION: starting
0709 13:09:20.154937 (+   657us) log.cc:844] Serialized 2844 byte log entry
0709 13:09:20.246374 (+ 91437us) op_driver.cc:464] REPLICATION: finished
0709 13:09:20.308378 (+ 62004us) write_op.cc:301] APPLY: starting
0709 13:09:20.308457 (+    79us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:20.308660 (+   203us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:20.308780 (+   120us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:20.309578 (+   798us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:20.309772 (+   194us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 13:09:20.309815 (+    43us) write_op.cc:312] APPLY: finished
0709 13:09:20.310077 (+   262us) log.cc:844] Serialized 58 byte log entry
0709 13:09:20.310246 (+   169us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:20.332696 (+ 22450us) write_op.cc:454] Released schema lock
0709 13:09:20.332743 (+    47us) write_op.cc:341] FINISH: Updating metrics
0709 13:09:20.333113 (+   370us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:20.333148 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":523,"reported_tablets":13,"child_traces":[["op",{"apply.queue_time_us":61757,"apply.run_cpu_time_us":2407,"apply.run_wall_time_us":24699,"num_ops":7,"prepare.queue_time_us":292665,"prepare.run_cpu_time_us":3676,"prepare.run_wall_time_us":35817,"replication_time_us":91922,"thread_start_us":262561,"threads_started":2}]]}
I20260709 13:09:20.369962 25076 raft_consensus.cc:359] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:20.370661 25076 raft_consensus.cc:740] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:20.371340 25076 consensus_queue.cc:260] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:20.375727 25076 ts_tablet_manager.cc:1434] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.003s
I20260709 13:09:20.376591 25076 tablet_bootstrap.cc:492] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:20.386173 24202 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34424 (request call id 115) took 1703 ms. Trace:
I20260709 13:09:20.407821 24203 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57330 (request call id 15) took 1627 ms. Trace:
I20260709 13:09:20.408305 24203 rpcz_store.cc:276] 0709 13:09:18.780713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:18.780898 (+   185us) service_pool.cc:224] Handling call
0709 13:09:20.407619 (+1626721us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":5}
I20260709 13:09:20.419183 25076 tablet_bootstrap.cc:492] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:20.420583 25076 tablet_bootstrap.cc:492] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:20.421474 25076 ts_tablet_manager.cc:1403] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.045s	user 0.014s	sys 0.011s
I20260709 13:09:20.424959 25076 raft_consensus.cc:359] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:20.425832 25076 raft_consensus.cc:740] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:20.426493 25076 consensus_queue.cc:260] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:20.386610 24202 rpcz_store.cc:276] 0709 13:09:18.682728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:18.682902 (+   174us) service_pool.cc:224] Handling call
0709 13:09:18.700019 (+ 17117us) spinlock_profiling.cc:243] Waited 2.31 ms on lock 0x7b5400151dd4. stack: 00007f73e7289343 00007f73e7289240 00007f73e6c181d4 00007f73f0956304 00007f73f0956299 00007f73ea312090 00007f73ea3b0f3a 00007f73f01671cc 00007f73f026537e 00007f73ee7262a4 00007f73ee726232 00007f73ee726161 00007f73ee7260dc 00007f73ee725392 00007f73e86e348c 00007f73e86e28c6
0709 13:09:20.386095 (+1686076us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:20.354343 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:20.355091 (+   748us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:20.355132 (+    41us) write_op.cc:435] Acquired schema lock
0709 13:09:20.355154 (+    22us) tablet.cc:662] Decoding operations
0709 13:09:20.355547 (+   393us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:20.355611 (+    64us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:20.355643 (+    32us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:09:20.355856 (+   213us) tablet.cc:701] Row locks acquired
0709 13:09:20.355870 (+    14us) write_op.cc:260] PREPARE: finished
0709 13:09:20.356012 (+   142us) write_op.cc:270] Start()
0709 13:09:20.356127 (+   115us) write_op.cc:276] Timestamp: P: 1783602560355968 usec, L: 0
0709 13:09:20.356146 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:09:20.356816 (+   670us) log.cc:844] Serialized 537 byte log entry
0709 13:09:20.358721 (+  1905us) op_driver.cc:464] REPLICATION: finished
0709 13:09:20.359087 (+   366us) write_op.cc:301] APPLY: starting
0709 13:09:20.359159 (+    72us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:20.359251 (+    92us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:20.359338 (+    87us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:20.359576 (+   238us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:20.359735 (+   159us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:09:20.359779 (+    44us) write_op.cc:312] APPLY: finished
0709 13:09:20.359944 (+   165us) log.cc:844] Serialized 22 byte log entry
0709 13:09:20.360077 (+   133us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:20.360153 (+    76us) write_op.cc:454] Released schema lock
0709 13:09:20.360189 (+    36us) write_op.cc:341] FINISH: Updating metrics
0709 13:09:20.360554 (+   365us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:20.360591 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":25951,"reported_tablets":11,"spinlock_wait_cycles":5316352,"child_traces":[["op",{"apply.queue_time_us":107,"apply.run_cpu_time_us":1397,"apply.run_wall_time_us":1397,"num_ops":1,"prepare.queue_time_us":202,"prepare.run_cpu_time_us":2817,"prepare.run_wall_time_us":2817,"replication_time_us":2447}]]}
I20260709 13:09:20.433274 25076 ts_tablet_manager.cc:1434] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.001s
I20260709 13:09:20.434319 25076 tablet_bootstrap.cc:492] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:20.452592 25076 tablet_bootstrap.cc:492] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:20.453526 25076 tablet_bootstrap.cc:492] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:20.454053 25076 ts_tablet_manager.cc:1403] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.004s
I20260709 13:09:20.456727 25076 raft_consensus.cc:359] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:20.457270 25076 raft_consensus.cc:740] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:20.457816 25076 consensus_queue.cc:260] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:20.459707 25076 ts_tablet_manager.cc:1434] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:09:20.460414 25076 tablet_bootstrap.cc:492] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:20.482631 25076 tablet_bootstrap.cc:492] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:20.483623 25076 tablet_bootstrap.cc:492] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:20.484333 25076 ts_tablet_manager.cc:1403] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.024s	user 0.015s	sys 0.005s
I20260709 13:09:20.486879 25076 raft_consensus.cc:359] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:20.486402 24201 catalog_manager.cc:5697] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 reported cstate change: term changed from 1 to 3, leader changed from 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1) to 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4). New cstate: current_term: 3 leader_uuid: "838cb9997b0f437989d3f4e3e9904390" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:20.487560 25076 raft_consensus.cc:740] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:20.488261 25076 consensus_queue.cc:260] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:20.490101 25076 ts_tablet_manager.cc:1434] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:09:20.491379 25076 tablet_bootstrap.cc:492] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:20.508441 25076 tablet_bootstrap.cc:492] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:20.509641 25076 tablet_bootstrap.cc:492] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:20.510360 25076 ts_tablet_manager.cc:1403] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.019s	user 0.016s	sys 0.000s
I20260709 13:09:20.512807 25076 raft_consensus.cc:359] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:20.513417 25076 raft_consensus.cc:740] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:20.513870 25076 consensus_queue.cc:260] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:20.515957 25076 ts_tablet_manager.cc:1434] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:09:20.516789 25076 tablet_bootstrap.cc:492] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:20.564832 25076 tablet_bootstrap.cc:492] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:20.566033 25076 tablet_bootstrap.cc:492] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:20.566684 25076 ts_tablet_manager.cc:1403] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.050s	user 0.018s	sys 0.005s
I20260709 13:09:20.570115 25076 raft_consensus.cc:359] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:20.570804 25076 raft_consensus.cc:740] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:20.571516 25076 consensus_queue.cc:260] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:20.573097 25076 ts_tablet_manager.cc:1434] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.001s
I20260709 13:09:20.573923 25076 tablet_bootstrap.cc:492] T e4dd51cda55249b48d7bfa39d5feb398 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:20.589442 25076 tablet_bootstrap.cc:492] T e4dd51cda55249b48d7bfa39d5feb398 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:20.590319 25076 tablet_bootstrap.cc:492] T e4dd51cda55249b48d7bfa39d5feb398 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:20.590869 25076 ts_tablet_manager.cc:1403] T e4dd51cda55249b48d7bfa39d5feb398 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.000s
I20260709 13:09:20.592909 25076 raft_consensus.cc:359] T e4dd51cda55249b48d7bfa39d5feb398 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:20.593415 25076 raft_consensus.cc:740] T e4dd51cda55249b48d7bfa39d5feb398 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:20.593912 25076 consensus_queue.cc:260] T e4dd51cda55249b48d7bfa39d5feb398 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:20.595980 25076 ts_tablet_manager.cc:1434] T e4dd51cda55249b48d7bfa39d5feb398 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:09:20.596818 25076 tablet_bootstrap.cc:492] T 4af7fc369d124452beb5fd07cf8592ec P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:20.608062 25076 tablet_bootstrap.cc:492] T 4af7fc369d124452beb5fd07cf8592ec P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:20.608919 25076 tablet_bootstrap.cc:492] T 4af7fc369d124452beb5fd07cf8592ec P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:20.609508 25076 ts_tablet_manager.cc:1403] T 4af7fc369d124452beb5fd07cf8592ec P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.013s	user 0.011s	sys 0.000s
I20260709 13:09:20.611410 25076 raft_consensus.cc:359] T 4af7fc369d124452beb5fd07cf8592ec P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:20.612118 25076 raft_consensus.cc:740] T 4af7fc369d124452beb5fd07cf8592ec P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:20.612762 25076 consensus_queue.cc:260] T 4af7fc369d124452beb5fd07cf8592ec P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:20.613173 25119 raft_consensus.cc:3060] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:20.614480 25076 ts_tablet_manager.cc:1434] T 4af7fc369d124452beb5fd07cf8592ec P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:09:20.616406 25076 tablet_bootstrap.cc:492] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:20.618248 25117 raft_consensus.cc:3060] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:20.641444 25076 tablet_bootstrap.cc:492] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:20.642411 25076 tablet_bootstrap.cc:492] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:20.643050 25076 ts_tablet_manager.cc:1403] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.027s	user 0.013s	sys 0.000s
I20260709 13:09:20.645648 25076 raft_consensus.cc:359] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:20.646221 25076 raft_consensus.cc:740] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:20.646811 25076 consensus_queue.cc:260] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:20.648882 25076 ts_tablet_manager.cc:1434] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:09:20.649529 25076 tablet_bootstrap.cc:492] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:20.671525 25076 tablet_bootstrap.cc:492] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:20.672477 25076 tablet_bootstrap.cc:492] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:20.673156 25076 ts_tablet_manager.cc:1403] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.007s
I20260709 13:09:20.675374 25076 raft_consensus.cc:359] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:20.675904 25076 raft_consensus.cc:740] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:20.676371 25076 consensus_queue.cc:260] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:20.678156 25076 ts_tablet_manager.cc:1434] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:09:20.679243 25076 tablet_bootstrap.cc:492] T 363cde7d4cb84353a74a18cbd9701fa7 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:20.693221 25076 tablet_bootstrap.cc:492] T 363cde7d4cb84353a74a18cbd9701fa7 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:20.694123 25076 tablet_bootstrap.cc:492] T 363cde7d4cb84353a74a18cbd9701fa7 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:20.694676 25076 ts_tablet_manager.cc:1403] T 363cde7d4cb84353a74a18cbd9701fa7 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.016s	user 0.004s	sys 0.008s
I20260709 13:09:20.696592 25076 raft_consensus.cc:359] T 363cde7d4cb84353a74a18cbd9701fa7 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:20.697103 25076 raft_consensus.cc:740] T 363cde7d4cb84353a74a18cbd9701fa7 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:20.697647 25076 consensus_queue.cc:260] T 363cde7d4cb84353a74a18cbd9701fa7 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:20.699393 25076 ts_tablet_manager.cc:1434] T 363cde7d4cb84353a74a18cbd9701fa7 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:09:20.700488 25076 tablet_bootstrap.cc:492] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:20.717365 25076 tablet_bootstrap.cc:492] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:20.718281 25076 tablet_bootstrap.cc:492] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:20.718917 25076 ts_tablet_manager.cc:1403] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.004s
I20260709 13:09:20.721134 25076 raft_consensus.cc:359] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:20.721624 25076 raft_consensus.cc:740] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:20.722249 25076 consensus_queue.cc:260] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:20.724191 25076 ts_tablet_manager.cc:1434] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:09:20.725076 25076 tablet_bootstrap.cc:492] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:20.749750 25076 tablet_bootstrap.cc:492] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:20.750723 25076 tablet_bootstrap.cc:492] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:20.751371 25076 ts_tablet_manager.cc:1403] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.027s	user 0.010s	sys 0.011s
I20260709 13:09:20.754340 25076 raft_consensus.cc:359] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:20.755002 25076 raft_consensus.cc:740] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:20.755728 25076 consensus_queue.cc:260] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:20.758010 25076 ts_tablet_manager.cc:1434] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:09:20.758783 25076 tablet_bootstrap.cc:492] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:20.758973 25119 raft_consensus.cc:3060] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:20.778535 25076 tablet_bootstrap.cc:492] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:20.779587 25076 tablet_bootstrap.cc:492] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:20.780285 25076 ts_tablet_manager.cc:1403] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.004s
I20260709 13:09:20.782246 25076 raft_consensus.cc:359] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:20.782779 25076 raft_consensus.cc:740] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:20.783427 25076 consensus_queue.cc:260] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:20.785418 25076 ts_tablet_manager.cc:1434] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:09:20.786460 25076 tablet_bootstrap.cc:492] T b147866f4e5f4596b0b3989418cc4b0a P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:20.800482 25076 tablet_bootstrap.cc:492] T b147866f4e5f4596b0b3989418cc4b0a P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:20.801378 25076 tablet_bootstrap.cc:492] T b147866f4e5f4596b0b3989418cc4b0a P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:20.802022 25076 ts_tablet_manager.cc:1403] T b147866f4e5f4596b0b3989418cc4b0a P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.016s	user 0.013s	sys 0.000s
I20260709 13:09:20.804359 25076 raft_consensus.cc:359] T b147866f4e5f4596b0b3989418cc4b0a P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:20.804863 25076 raft_consensus.cc:740] T b147866f4e5f4596b0b3989418cc4b0a P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:20.805476 25076 consensus_queue.cc:260] T b147866f4e5f4596b0b3989418cc4b0a P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:20.807540 25076 ts_tablet_manager.cc:1434] T b147866f4e5f4596b0b3989418cc4b0a P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:09:20.808642 25076 tablet_bootstrap.cc:492] T 2deba793363c4afca7d75b2fc3814304 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:20.823758 25076 tablet_bootstrap.cc:492] T 2deba793363c4afca7d75b2fc3814304 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:20.824667 25076 tablet_bootstrap.cc:492] T 2deba793363c4afca7d75b2fc3814304 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:20.825323 25076 ts_tablet_manager.cc:1403] T 2deba793363c4afca7d75b2fc3814304 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.017s	user 0.014s	sys 0.000s
I20260709 13:09:20.827876 25076 raft_consensus.cc:359] T 2deba793363c4afca7d75b2fc3814304 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:20.828522 25076 raft_consensus.cc:740] T 2deba793363c4afca7d75b2fc3814304 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:20.829173 25076 consensus_queue.cc:260] T 2deba793363c4afca7d75b2fc3814304 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:20.830621 25076 ts_tablet_manager.cc:1434] T 2deba793363c4afca7d75b2fc3814304 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:09:20.831421 25076 tablet_bootstrap.cc:492] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:20.845834 25076 tablet_bootstrap.cc:492] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:20.845925 25117 raft_consensus.cc:3060] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:09:20.846822 25076 tablet_bootstrap.cc:492] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:20.847505 25076 ts_tablet_manager.cc:1403] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.000s
I20260709 13:09:20.850085 25076 raft_consensus.cc:359] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:20.850867 25076 raft_consensus.cc:740] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:20.851516 25076 consensus_queue.cc:260] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:20.853331 25117 raft_consensus.cc:1275] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [term 3 FOLLOWER]: Refusing update from remote peer 838cb9997b0f437989d3f4e3e9904390: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:09:20.853600 25076 ts_tablet_manager.cc:1434] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:09:20.854470 25076 tablet_bootstrap.cc:492] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap starting.
I20260709 13:09:20.854825 25207 consensus_queue.cc:1048] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:20.879259 24382 raft_consensus.cc:1275] T 42bcfb18b77e41afa60057ccfd718fa9 P c40e7ba102224d24a22ccd1f68aaae45 [term 3 FOLLOWER]: Refusing update from remote peer 838cb9997b0f437989d3f4e3e9904390: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:09:20.880841 25179 consensus_queue.cc:1048] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:20.902061 25076 tablet_bootstrap.cc:492] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:20.903033 25076 tablet_bootstrap.cc:492] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381: Bootstrap complete.
I20260709 13:09:20.903684 25076 ts_tablet_manager.cc:1403] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381: Time spent bootstrapping tablet: real 0.049s	user 0.024s	sys 0.020s
I20260709 13:09:20.906148 25076 raft_consensus.cc:359] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:20.906798 25076 raft_consensus.cc:740] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Initialized, Role: FOLLOWER
I20260709 13:09:20.907519 25076 consensus_queue.cc:260] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:20.910014 25076 ts_tablet_manager.cc:1434] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:09:20.914969 25117 raft_consensus.cc:3060] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:20.976689 25117 raft_consensus.cc:3060] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:20.992827 25117 raft_consensus.cc:3060] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:21.039043 25117 raft_consensus.cc:3060] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:21.092126 25117 raft_consensus.cc:3060] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:21.207958 25117 raft_consensus.cc:3060] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:21.250444 25117 raft_consensus.cc:3060] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:26.948410 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:26.998085 24382 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5eaa917e65d9403dbb241f84ee96ba9d"
dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
mode: GRACEFUL
new_leader_uuid: "9c2a14143eb744cf83bfa5f46284a381"
 from {username='slave'} at 127.0.0.1:53088
I20260709 13:09:26.998683 24382 raft_consensus.cc:606] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45 [term 1 LEADER]: Received request to transfer leadership to TS 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:27.009449 24452 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8775bc34e96d48ae8f5598affd26efdc"
dest_uuid: "32088c942fbb48239e0eed952d4df449"
mode: GRACEFUL
new_leader_uuid: "9c2a14143eb744cf83bfa5f46284a381"
 from {username='slave'} at 127.0.0.1:38570
I20260709 13:09:27.010085 24452 raft_consensus.cc:606] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449 [term 2 LEADER]: Received request to transfer leadership to TS 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:27.012916 24452 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "90d9e64f97544895b9b97dfdc5ceb359"
dest_uuid: "32088c942fbb48239e0eed952d4df449"
mode: GRACEFUL
new_leader_uuid: "9c2a14143eb744cf83bfa5f46284a381"
 from {username='slave'} at 127.0.0.1:38570
I20260709 13:09:27.013439 24452 raft_consensus.cc:606] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449 [term 2 LEADER]: Received request to transfer leadership to TS 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:27.015987 24452 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "915d8224f8b64cdab90dff1689046548"
dest_uuid: "32088c942fbb48239e0eed952d4df449"
mode: GRACEFUL
new_leader_uuid: "9c2a14143eb744cf83bfa5f46284a381"
 from {username='slave'} at 127.0.0.1:38570
I20260709 13:09:27.016577 24452 raft_consensus.cc:606] T 915d8224f8b64cdab90dff1689046548 P 32088c942fbb48239e0eed952d4df449 [term 1 LEADER]: Received request to transfer leadership to TS 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:27.018945 24452 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9c88d903166c4812a88653e389ab5776"
dest_uuid: "32088c942fbb48239e0eed952d4df449"
mode: GRACEFUL
new_leader_uuid: "9c2a14143eb744cf83bfa5f46284a381"
 from {username='slave'} at 127.0.0.1:38570
I20260709 13:09:27.019542 24452 raft_consensus.cc:606] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449 [term 2 LEADER]: Received request to transfer leadership to TS 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:27.033104 24526 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b88feffc0859413482b0daf562d2f86a"
dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
mode: GRACEFUL
new_leader_uuid: "9c2a14143eb744cf83bfa5f46284a381"
 from {username='slave'} at 127.0.0.1:33304
I20260709 13:09:27.033742 24526 raft_consensus.cc:606] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [term 2 LEADER]: Received request to transfer leadership to TS 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:27.035022 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:09:27.036752 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:09:27.133949 25340 raft_consensus.cc:993] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45: : Instructing follower 9c2a14143eb744cf83bfa5f46284a381 to start an election
I20260709 13:09:27.134522 24979 raft_consensus.cc:1081] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45 [term 1 LEADER]: Signalling peer 9c2a14143eb744cf83bfa5f46284a381 to start an election
I20260709 13:09:27.135921 25119 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5eaa917e65d9403dbb241f84ee96ba9d"
dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
 from {username='slave'} at 127.0.0.1:49168
I20260709 13:09:27.136423 25119 raft_consensus.cc:493] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:27.136719 25119 raft_consensus.cc:3060] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:27.142275 25119 raft_consensus.cc:515] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:27.144102 25119 leader_election.cc:290] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 2 election: Requested vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:27.191951 25341 raft_consensus.cc:993] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449: : Instructing follower 9c2a14143eb744cf83bfa5f46284a381 to start an election
I20260709 13:09:27.192438 25341 raft_consensus.cc:1081] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449 [term 2 LEADER]: Signalling peer 9c2a14143eb744cf83bfa5f46284a381 to start an election
I20260709 13:09:27.204248 25119 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "90d9e64f97544895b9b97dfdc5ceb359"
dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
 from {username='slave'} at 127.0.0.1:49156
I20260709 13:09:27.204795 25119 raft_consensus.cc:493] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:27.205137 25119 raft_consensus.cc:3060] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:27.209888 24382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eaa917e65d9403dbb241f84ee96ba9d" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:27.210558 24382 raft_consensus.cc:3055] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:27.210834 24382 raft_consensus.cc:740] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Running, Role: LEADER
I20260709 13:09:27.210825 24526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5eaa917e65d9403dbb241f84ee96ba9d" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:27.211709 24526 raft_consensus.cc:3060] T 5eaa917e65d9403dbb241f84ee96ba9d P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:27.211982 24382 consensus_queue.cc:260] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:27.212095 25119 raft_consensus.cc:515] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:27.214561 24382 raft_consensus.cc:3060] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:27.218568 25119 leader_election.cc:290] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 3 election: Requested vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:27.220394 24379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90d9e64f97544895b9b97dfdc5ceb359" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:27.220665 24526 raft_consensus.cc:2468] T 5eaa917e65d9403dbb241f84ee96ba9d P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 2.
I20260709 13:09:27.220975 24379 raft_consensus.cc:3060] T 90d9e64f97544895b9b97dfdc5ceb359 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:27.222667 25070 leader_election.cc:304] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381; no voters: 
I20260709 13:09:27.227641 24379 raft_consensus.cc:2468] T 90d9e64f97544895b9b97dfdc5ceb359 P c40e7ba102224d24a22ccd1f68aaae45 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 3.
I20260709 13:09:27.230005 25352 raft_consensus.cc:2804] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:27.231127 24382 raft_consensus.cc:2468] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 2.
I20260709 13:09:27.229889 25070 leader_election.cc:304] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:27.238955 25354 raft_consensus.cc:2804] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:27.239784 25354 raft_consensus.cc:697] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [term 3 LEADER]: Becoming Leader. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Running, Role: LEADER
I20260709 13:09:27.240726 24379 raft_consensus.cc:1240] T 90d9e64f97544895b9b97dfdc5ceb359 P c40e7ba102224d24a22ccd1f68aaae45 [term 3 FOLLOWER]: Rejecting Update request from peer 32088c942fbb48239e0eed952d4df449 for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:27.243961 25354 consensus_queue.cc:237] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:27.247201 25352 raft_consensus.cc:697] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381 [term 2 LEADER]: Becoming Leader. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Running, Role: LEADER
I20260709 13:09:27.248328 24451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90d9e64f97544895b9b97dfdc5ceb359" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:27.248874 25352 consensus_queue.cc:237] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:27.247622 25349 consensus_queue.cc:1059] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:09:27.257382 24451 raft_consensus.cc:3055] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:27.257701 24451 raft_consensus.cc:740] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Running, Role: LEADER
I20260709 13:09:27.258329 24451 consensus_queue.cc:260] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:27.259305 24451 raft_consensus.cc:3060] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:27.266040 24451 raft_consensus.cc:2468] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 3.
I20260709 13:09:27.265698 24200 catalog_manager.cc:5697] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381 reported cstate change: term changed from 1 to 2, leader changed from c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2) to 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "9c2a14143eb744cf83bfa5f46284a381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } }
W20260709 13:09:27.266546 25341 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 13:09:27.293705 24200 catalog_manager.cc:5697] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 reported cstate change: term changed from 2 to 3, leader changed from 32088c942fbb48239e0eed952d4df449 (127.14.103.3) to 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1). New cstate: current_term: 3 leader_uuid: "9c2a14143eb744cf83bfa5f46284a381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:27.315135 25179 raft_consensus.cc:993] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390: : Instructing follower 9c2a14143eb744cf83bfa5f46284a381 to start an election
I20260709 13:09:27.315843 25348 raft_consensus.cc:1081] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [term 2 LEADER]: Signalling peer 9c2a14143eb744cf83bfa5f46284a381 to start an election
I20260709 13:09:27.317396 25117 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b88feffc0859413482b0daf562d2f86a"
dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
 from {username='slave'} at 127.0.0.1:49148
I20260709 13:09:27.317847 25117 raft_consensus.cc:493] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:27.318115 25117 raft_consensus.cc:3060] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:27.322484 25117 raft_consensus.cc:515] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:27.323967 25117 leader_election.cc:290] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 3 election: Requested vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:27.324837 24451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88feffc0859413482b0daf562d2f86a" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:27.325412 24451 raft_consensus.cc:3060] T b88feffc0859413482b0daf562d2f86a P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:27.326073 24526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88feffc0859413482b0daf562d2f86a" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:27.326633 24526 raft_consensus.cc:3055] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:27.326962 24526 raft_consensus.cc:740] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Running, Role: LEADER
I20260709 13:09:27.327754 24526 consensus_queue.cc:260] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:27.329208 24526 raft_consensus.cc:3060] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:27.337684 24451 raft_consensus.cc:2468] T b88feffc0859413482b0daf562d2f86a P 32088c942fbb48239e0eed952d4df449 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 3.
I20260709 13:09:27.338805 25070 leader_election.cc:304] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381; no voters: 
I20260709 13:09:27.339658 25354 raft_consensus.cc:2804] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:27.340176 25354 raft_consensus.cc:697] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 [term 3 LEADER]: Becoming Leader. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Running, Role: LEADER
I20260709 13:09:27.341027 25354 consensus_queue.cc:237] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:27.341362 24526 raft_consensus.cc:2468] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 3.
I20260709 13:09:27.351229 24200 catalog_manager.cc:5697] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 reported cstate change: term changed from 2 to 3, leader changed from 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4) to 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1). New cstate: current_term: 3 leader_uuid: "9c2a14143eb744cf83bfa5f46284a381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:27.390694 25349 raft_consensus.cc:993] T 915d8224f8b64cdab90dff1689046548 P 32088c942fbb48239e0eed952d4df449: : Instructing follower 9c2a14143eb744cf83bfa5f46284a381 to start an election
I20260709 13:09:27.391239 25341 raft_consensus.cc:1081] T 915d8224f8b64cdab90dff1689046548 P 32088c942fbb48239e0eed952d4df449 [term 1 LEADER]: Signalling peer 9c2a14143eb744cf83bfa5f46284a381 to start an election
I20260709 13:09:27.391918 25287 raft_consensus.cc:993] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449: : Instructing follower 9c2a14143eb744cf83bfa5f46284a381 to start an election
I20260709 13:09:27.392410 25341 raft_consensus.cc:1081] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449 [term 2 LEADER]: Signalling peer 9c2a14143eb744cf83bfa5f46284a381 to start an election
I20260709 13:09:27.392689 25119 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "915d8224f8b64cdab90dff1689046548"
dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
 from {username='slave'} at 127.0.0.1:49156
I20260709 13:09:27.393236 25119 raft_consensus.cc:493] T 915d8224f8b64cdab90dff1689046548 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:27.393537 25119 raft_consensus.cc:3060] T 915d8224f8b64cdab90dff1689046548 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:27.393900 25117 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8775bc34e96d48ae8f5598affd26efdc"
dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
 from {username='slave'} at 127.0.0.1:49156
I20260709 13:09:27.394369 25117 raft_consensus.cc:493] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:27.394640 25117 raft_consensus.cc:3060] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:27.398634 25119 raft_consensus.cc:515] T 915d8224f8b64cdab90dff1689046548 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:27.398900 25117 raft_consensus.cc:515] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:27.400369 25119 leader_election.cc:290] T 915d8224f8b64cdab90dff1689046548 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 2 election: Requested vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:27.401142 24379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915d8224f8b64cdab90dff1689046548" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:27.400439 25117 leader_election.cc:290] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 3 election: Requested vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:27.401443 24451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8775bc34e96d48ae8f5598affd26efdc" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:27.401732 24379 raft_consensus.cc:3060] T 915d8224f8b64cdab90dff1689046548 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:27.401829 24453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915d8224f8b64cdab90dff1689046548" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:27.402100 24451 raft_consensus.cc:3055] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:27.402527 24453 raft_consensus.cc:3055] T 915d8224f8b64cdab90dff1689046548 P 32088c942fbb48239e0eed952d4df449 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:27.402872 24453 raft_consensus.cc:740] T 915d8224f8b64cdab90dff1689046548 P 32088c942fbb48239e0eed952d4df449 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Running, Role: LEADER
I20260709 13:09:27.402546 24451 raft_consensus.cc:740] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Running, Role: LEADER
I20260709 13:09:27.403745 24382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8775bc34e96d48ae8f5598affd26efdc" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:27.404316 24382 raft_consensus.cc:3060] T 8775bc34e96d48ae8f5598affd26efdc P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:27.403746 24453 consensus_queue.cc:260] T 915d8224f8b64cdab90dff1689046548 P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:27.403898 24451 consensus_queue.cc:260] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:27.405200 24453 raft_consensus.cc:3060] T 915d8224f8b64cdab90dff1689046548 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:27.405227 24451 raft_consensus.cc:3060] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:27.423338 24382 raft_consensus.cc:2468] T 8775bc34e96d48ae8f5598affd26efdc P c40e7ba102224d24a22ccd1f68aaae45 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 3.
I20260709 13:09:27.424592 25070 leader_election.cc:304] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:27.425261 24379 raft_consensus.cc:2468] T 915d8224f8b64cdab90dff1689046548 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 2.
I20260709 13:09:27.425508 25354 raft_consensus.cc:2804] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:27.426043 25354 raft_consensus.cc:697] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 [term 3 LEADER]: Becoming Leader. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Running, Role: LEADER
I20260709 13:09:27.426834 25354 consensus_queue.cc:237] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:27.427925 25070 leader_election.cc:304] T 915d8224f8b64cdab90dff1689046548 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:27.429061 25352 raft_consensus.cc:2804] T 915d8224f8b64cdab90dff1689046548 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:27.429551 25352 raft_consensus.cc:697] T 915d8224f8b64cdab90dff1689046548 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 LEADER]: Becoming Leader. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Running, Role: LEADER
I20260709 13:09:27.430361 25352 consensus_queue.cc:237] T 915d8224f8b64cdab90dff1689046548 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:27.427983 24451 raft_consensus.cc:2468] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 3.
I20260709 13:09:27.437116 24453 raft_consensus.cc:2468] T 915d8224f8b64cdab90dff1689046548 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 2.
I20260709 13:09:27.443904 24200 catalog_manager.cc:5697] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 reported cstate change: term changed from 2 to 3, leader changed from 32088c942fbb48239e0eed952d4df449 (127.14.103.3) to 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1). New cstate: current_term: 3 leader_uuid: "9c2a14143eb744cf83bfa5f46284a381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:27.445044 24200 catalog_manager.cc:5697] T 915d8224f8b64cdab90dff1689046548 P 9c2a14143eb744cf83bfa5f46284a381 reported cstate change: term changed from 1 to 2, leader changed from 32088c942fbb48239e0eed952d4df449 (127.14.103.3) to 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "9c2a14143eb744cf83bfa5f46284a381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:27.452762 25287 raft_consensus.cc:993] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449: : Instructing follower 9c2a14143eb744cf83bfa5f46284a381 to start an election
I20260709 13:09:27.453214 25341 raft_consensus.cc:1081] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449 [term 2 LEADER]: Signalling peer 9c2a14143eb744cf83bfa5f46284a381 to start an election
I20260709 13:09:27.454629 25119 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9c88d903166c4812a88653e389ab5776"
dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
 from {username='slave'} at 127.0.0.1:49156
I20260709 13:09:27.455122 25119 raft_consensus.cc:493] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:27.455452 25119 raft_consensus.cc:3060] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:27.461236 25119 raft_consensus.cc:515] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:27.462739 25119 leader_election.cc:290] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 3 election: Requested vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:27.463979 24453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c88d903166c4812a88653e389ab5776" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:27.464110 24526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c88d903166c4812a88653e389ab5776" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:27.464514 24453 raft_consensus.cc:3055] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:27.464782 24526 raft_consensus.cc:3060] T 9c88d903166c4812a88653e389ab5776 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:27.464933 24453 raft_consensus.cc:740] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 32088c942fbb48239e0eed952d4df449, State: Running, Role: LEADER
I20260709 13:09:27.465760 24453 consensus_queue.cc:260] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:27.467007 24453 raft_consensus.cc:3060] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:27.470772 24526 raft_consensus.cc:2468] T 9c88d903166c4812a88653e389ab5776 P 838cb9997b0f437989d3f4e3e9904390 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 3.
I20260709 13:09:27.472288 25070 leader_election.cc:304] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381; no voters: 
I20260709 13:09:27.473130 25352 raft_consensus.cc:2804] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:27.473582 25352 raft_consensus.cc:697] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [term 3 LEADER]: Becoming Leader. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Running, Role: LEADER
I20260709 13:09:27.474368 25352 consensus_queue.cc:237] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:27.475157 24453 raft_consensus.cc:2468] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 3.
I20260709 13:09:27.484651 24200 catalog_manager.cc:5697] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 reported cstate change: term changed from 2 to 3, leader changed from 32088c942fbb48239e0eed952d4df449 (127.14.103.3) to 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1). New cstate: current_term: 3 leader_uuid: "9c2a14143eb744cf83bfa5f46284a381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:27.646358 24526 raft_consensus.cc:1275] T 5eaa917e65d9403dbb241f84ee96ba9d P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Refusing update from remote peer 9c2a14143eb744cf83bfa5f46284a381: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:27.647578 25352 consensus_queue.cc:1048] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:27.659087 24379 raft_consensus.cc:1275] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Refusing update from remote peer 9c2a14143eb744cf83bfa5f46284a381: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:27.660524 25354 consensus_queue.cc:1048] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:27.739981 24375 raft_consensus.cc:1275] T 90d9e64f97544895b9b97dfdc5ceb359 P c40e7ba102224d24a22ccd1f68aaae45 [term 3 FOLLOWER]: Refusing update from remote peer 9c2a14143eb744cf83bfa5f46284a381: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:27.741379 25354 consensus_queue.cc:1048] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:27.754374 24453 raft_consensus.cc:1275] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449 [term 3 FOLLOWER]: Refusing update from remote peer 9c2a14143eb744cf83bfa5f46284a381: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:27.756057 25354 consensus_queue.cc:1048] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:27.820379 24453 raft_consensus.cc:1275] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449 [term 3 FOLLOWER]: Refusing update from remote peer 9c2a14143eb744cf83bfa5f46284a381: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:27.821702 25353 consensus_queue.cc:1048] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:27.831068 24375 raft_consensus.cc:1275] T 8775bc34e96d48ae8f5598affd26efdc P c40e7ba102224d24a22ccd1f68aaae45 [term 3 FOLLOWER]: Refusing update from remote peer 9c2a14143eb744cf83bfa5f46284a381: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:27.834254 25352 consensus_queue.cc:1048] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:27.843133 24524 raft_consensus.cc:1275] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [term 3 FOLLOWER]: Refusing update from remote peer 9c2a14143eb744cf83bfa5f46284a381: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:27.851794 25354 consensus_queue.cc:1048] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:27.872089 24451 raft_consensus.cc:1275] T b88feffc0859413482b0daf562d2f86a P 32088c942fbb48239e0eed952d4df449 [term 3 FOLLOWER]: Refusing update from remote peer 9c2a14143eb744cf83bfa5f46284a381: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:27.886435 25354 consensus_queue.cc:1048] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:27.971877 24524 raft_consensus.cc:1275] T 9c88d903166c4812a88653e389ab5776 P 838cb9997b0f437989d3f4e3e9904390 [term 3 FOLLOWER]: Refusing update from remote peer 9c2a14143eb744cf83bfa5f46284a381: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:27.973233 25354 consensus_queue.cc:1048] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:27.977371 24451 raft_consensus.cc:1275] T 915d8224f8b64cdab90dff1689046548 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Refusing update from remote peer 9c2a14143eb744cf83bfa5f46284a381: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:27.978924 25354 consensus_queue.cc:1048] T 915d8224f8b64cdab90dff1689046548 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:27.983417 24453 raft_consensus.cc:1275] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449 [term 3 FOLLOWER]: Refusing update from remote peer 9c2a14143eb744cf83bfa5f46284a381: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:27.985183 25354 consensus_queue.cc:1048] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:27.994788 24375 raft_consensus.cc:1275] T 915d8224f8b64cdab90dff1689046548 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Refusing update from remote peer 9c2a14143eb744cf83bfa5f46284a381: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:27.999948 25353 consensus_queue.cc:1048] T 915d8224f8b64cdab90dff1689046548 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:28.037786 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:28.074707 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:28.110280 24524 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1adea9b113494bed8c9d57fdbee6e22b"
dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
mode: GRACEFUL
new_leader_uuid: "9c2a14143eb744cf83bfa5f46284a381"
 from {username='slave'} at 127.0.0.1:33304
I20260709 13:09:28.110805 24524 raft_consensus.cc:606] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Received request to transfer leadership to TS 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:28.113404 24524 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "40dd44ec1fb347f49c31a8b650ab1639"
dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
mode: GRACEFUL
new_leader_uuid: "9c2a14143eb744cf83bfa5f46284a381"
 from {username='slave'} at 127.0.0.1:33304
I20260709 13:09:28.113897 24524 raft_consensus.cc:606] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Received request to transfer leadership to TS 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:28.116539 24524 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "42bcfb18b77e41afa60057ccfd718fa9"
dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
mode: GRACEFUL
new_leader_uuid: "9c2a14143eb744cf83bfa5f46284a381"
 from {username='slave'} at 127.0.0.1:33304
I20260709 13:09:28.117105 24524 raft_consensus.cc:606] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [term 3 LEADER]: Received request to transfer leadership to TS 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:28.119352 24524 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "722754340deb4df18b716dc801c35c34"
dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
mode: GRACEFUL
new_leader_uuid: "9c2a14143eb744cf83bfa5f46284a381"
 from {username='slave'} at 127.0.0.1:33304
I20260709 13:09:28.119824 24524 raft_consensus.cc:606] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Received request to transfer leadership to TS 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:28.121860 24524 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "861acffd0d804c8fbd1a63771d7937c7"
dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
mode: GRACEFUL
new_leader_uuid: "9c2a14143eb744cf83bfa5f46284a381"
 from {username='slave'} at 127.0.0.1:33304
I20260709 13:09:28.122332 24524 raft_consensus.cc:606] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Received request to transfer leadership to TS 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:28.124423 24375 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b147866f4e5f4596b0b3989418cc4b0a"
dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
mode: GRACEFUL
new_leader_uuid: "9c2a14143eb744cf83bfa5f46284a381"
 from {username='slave'} at 127.0.0.1:53088
I20260709 13:09:28.125010 24375 raft_consensus.cc:606] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45 [term 1 LEADER]: Received request to transfer leadership to TS 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:28.126286 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:09:28.127485 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:09:28.155405 25179 raft_consensus.cc:993] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390: : Instructing follower 9c2a14143eb744cf83bfa5f46284a381 to start an election
I20260709 13:09:28.155994 25348 raft_consensus.cc:1081] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Signalling peer 9c2a14143eb744cf83bfa5f46284a381 to start an election
I20260709 13:09:28.157567 25117 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "40dd44ec1fb347f49c31a8b650ab1639"
dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
 from {username='slave'} at 127.0.0.1:49148
I20260709 13:09:28.158039 25117 raft_consensus.cc:493] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:28.158327 25117 raft_consensus.cc:3060] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:28.162789 25117 raft_consensus.cc:515] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:28.164794 25117 leader_election.cc:290] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 2 election: Requested vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:28.165478 24524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40dd44ec1fb347f49c31a8b650ab1639" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:28.165946 24382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40dd44ec1fb347f49c31a8b650ab1639" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:28.166276 24524 raft_consensus.cc:3055] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:28.166580 24382 raft_consensus.cc:3060] T 40dd44ec1fb347f49c31a8b650ab1639 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:28.166723 24524 raft_consensus.cc:740] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Running, Role: LEADER
I20260709 13:09:28.167814 24524 consensus_queue.cc:260] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:28.169214 24524 raft_consensus.cc:3060] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:28.171736 24382 raft_consensus.cc:2468] T 40dd44ec1fb347f49c31a8b650ab1639 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 2.
I20260709 13:09:28.172752 25070 leader_election.cc:304] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:28.173451 25354 raft_consensus.cc:2804] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:28.173925 25354 raft_consensus.cc:697] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 LEADER]: Becoming Leader. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Running, Role: LEADER
I20260709 13:09:28.174976 24524 raft_consensus.cc:2468] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 2.
I20260709 13:09:28.174738 25354 consensus_queue.cc:237] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:28.184006 24201 catalog_manager.cc:5697] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381 reported cstate change: term changed from 1 to 2, leader changed from 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4) to 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "9c2a14143eb744cf83bfa5f46284a381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:28.207540 25348 raft_consensus.cc:993] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390: : Instructing follower 9c2a14143eb744cf83bfa5f46284a381 to start an election
I20260709 13:09:28.208098 25179 raft_consensus.cc:1081] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [term 3 LEADER]: Signalling peer 9c2a14143eb744cf83bfa5f46284a381 to start an election
I20260709 13:09:28.209532 25119 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "42bcfb18b77e41afa60057ccfd718fa9"
dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
 from {username='slave'} at 127.0.0.1:49148
I20260709 13:09:28.210055 25119 raft_consensus.cc:493] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:28.210358 25119 raft_consensus.cc:3060] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:28.215801 25119 raft_consensus.cc:515] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:28.218415 25119 leader_election.cc:290] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 4 election: Requested vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:28.218631 24526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42bcfb18b77e41afa60057ccfd718fa9" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:28.219228 24526 raft_consensus.cc:3055] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:09:28.219506 24526 raft_consensus.cc:740] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Running, Role: LEADER
I20260709 13:09:28.220152 24526 consensus_queue.cc:260] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:28.221210 24526 raft_consensus.cc:3060] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:28.221529 24382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42bcfb18b77e41afa60057ccfd718fa9" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:28.222118 24382 raft_consensus.cc:3060] T 42bcfb18b77e41afa60057ccfd718fa9 P c40e7ba102224d24a22ccd1f68aaae45 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:28.227964 24526 raft_consensus.cc:2468] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 4.
I20260709 13:09:28.228281 24382 raft_consensus.cc:2468] T 42bcfb18b77e41afa60057ccfd718fa9 P c40e7ba102224d24a22ccd1f68aaae45 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 4.
I20260709 13:09:28.229327 25070 leader_election.cc:304] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381; no voters: 
I20260709 13:09:28.230099 25354 raft_consensus.cc:2804] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:09:28.230576 25354 raft_consensus.cc:697] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [term 4 LEADER]: Becoming Leader. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Running, Role: LEADER
I20260709 13:09:28.231237 25354 consensus_queue.cc:237] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:28.240525 24201 catalog_manager.cc:5697] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 reported cstate change: term changed from 3 to 4, leader changed from 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4) to 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1). New cstate: current_term: 4 leader_uuid: "9c2a14143eb744cf83bfa5f46284a381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:28.276680 25179 raft_consensus.cc:993] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390: : Instructing follower 9c2a14143eb744cf83bfa5f46284a381 to start an election
I20260709 13:09:28.277173 25179 raft_consensus.cc:1081] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Signalling peer 9c2a14143eb744cf83bfa5f46284a381 to start an election
I20260709 13:09:28.278627 25119 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "861acffd0d804c8fbd1a63771d7937c7"
dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
 from {username='slave'} at 127.0.0.1:49148
I20260709 13:09:28.279213 25119 raft_consensus.cc:493] T 861acffd0d804c8fbd1a63771d7937c7 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:28.279603 25119 raft_consensus.cc:3060] T 861acffd0d804c8fbd1a63771d7937c7 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:28.306181 25119 raft_consensus.cc:515] T 861acffd0d804c8fbd1a63771d7937c7 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:28.308087 25119 leader_election.cc:290] T 861acffd0d804c8fbd1a63771d7937c7 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 2 election: Requested vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:28.310353 25340 raft_consensus.cc:993] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45: : Instructing follower 9c2a14143eb744cf83bfa5f46284a381 to start an election
I20260709 13:09:28.313859 25361 raft_consensus.cc:1081] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45 [term 1 LEADER]: Signalling peer 9c2a14143eb744cf83bfa5f46284a381 to start an election
I20260709 13:09:28.322000 24453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861acffd0d804c8fbd1a63771d7937c7" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:28.322710 24453 raft_consensus.cc:3060] T 861acffd0d804c8fbd1a63771d7937c7 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:28.323804 25119 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b147866f4e5f4596b0b3989418cc4b0a"
dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
 from {username='slave'} at 127.0.0.1:49168
I20260709 13:09:28.324379 25119 raft_consensus.cc:493] T b147866f4e5f4596b0b3989418cc4b0a P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:28.324769 25119 raft_consensus.cc:3060] T b147866f4e5f4596b0b3989418cc4b0a P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:28.339294 25119 raft_consensus.cc:515] T b147866f4e5f4596b0b3989418cc4b0a P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:28.340720 24453 raft_consensus.cc:2468] T 861acffd0d804c8fbd1a63771d7937c7 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 2.
I20260709 13:09:28.341122 25119 leader_election.cc:290] T b147866f4e5f4596b0b3989418cc4b0a P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 2 election: Requested vote from peers c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177), 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679)
I20260709 13:09:28.341970 24526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861acffd0d804c8fbd1a63771d7937c7" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:28.342612 24526 raft_consensus.cc:3055] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:28.342914 24526 raft_consensus.cc:740] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Running, Role: LEADER
I20260709 13:09:28.343369 25070 leader_election.cc:304] T 861acffd0d804c8fbd1a63771d7937c7 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381; no voters: 
I20260709 13:09:28.344261 25353 raft_consensus.cc:2804] T 861acffd0d804c8fbd1a63771d7937c7 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:28.344736 25353 raft_consensus.cc:697] T 861acffd0d804c8fbd1a63771d7937c7 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 LEADER]: Becoming Leader. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Running, Role: LEADER
I20260709 13:09:28.345153 24526 consensus_queue.cc:260] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:28.345546 25353 consensus_queue.cc:237] T 861acffd0d804c8fbd1a63771d7937c7 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:28.346277 24382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b147866f4e5f4596b0b3989418cc4b0a" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:28.346446 24526 raft_consensus.cc:3060] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:28.346858 24382 raft_consensus.cc:3055] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:28.347162 24382 raft_consensus.cc:740] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Running, Role: LEADER
I20260709 13:09:28.348033 24382 consensus_queue.cc:260] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:28.349213 24382 raft_consensus.cc:3060] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:28.352181 24453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b147866f4e5f4596b0b3989418cc4b0a" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:28.358158 24382 raft_consensus.cc:2468] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 2.
I20260709 13:09:28.358741 24453 raft_consensus.cc:3060] T b147866f4e5f4596b0b3989418cc4b0a P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:28.359472 24526 raft_consensus.cc:2468] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 2.
I20260709 13:09:28.363409 25070 leader_election.cc:304] T b147866f4e5f4596b0b3989418cc4b0a P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:28.364255 25353 raft_consensus.cc:2804] T b147866f4e5f4596b0b3989418cc4b0a P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:28.364742 25353 raft_consensus.cc:697] T b147866f4e5f4596b0b3989418cc4b0a P 9c2a14143eb744cf83bfa5f46284a381 [term 2 LEADER]: Becoming Leader. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Running, Role: LEADER
I20260709 13:09:28.365550 25353 consensus_queue.cc:237] T b147866f4e5f4596b0b3989418cc4b0a P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } }
I20260709 13:09:28.367112 24453 raft_consensus.cc:2468] T b147866f4e5f4596b0b3989418cc4b0a P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 2.
I20260709 13:09:28.370014 24201 catalog_manager.cc:5697] T 861acffd0d804c8fbd1a63771d7937c7 P 9c2a14143eb744cf83bfa5f46284a381 reported cstate change: term changed from 1 to 2, leader changed from 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4) to 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "9c2a14143eb744cf83bfa5f46284a381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:28.387784 24201 catalog_manager.cc:5697] T b147866f4e5f4596b0b3989418cc4b0a P 9c2a14143eb744cf83bfa5f46284a381 reported cstate change: term changed from 1 to 2, leader changed from c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2) to 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "9c2a14143eb744cf83bfa5f46284a381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:28.480618 25179 raft_consensus.cc:993] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390: : Instructing follower 9c2a14143eb744cf83bfa5f46284a381 to start an election
I20260709 13:09:28.481097 25179 raft_consensus.cc:1081] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Signalling peer 9c2a14143eb744cf83bfa5f46284a381 to start an election
I20260709 13:09:28.483162 25116 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1adea9b113494bed8c9d57fdbee6e22b"
dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
 from {username='slave'} at 127.0.0.1:49148
I20260709 13:09:28.483774 25116 raft_consensus.cc:493] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:28.484181 25116 raft_consensus.cc:3060] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:28.490649 25116 raft_consensus.cc:515] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:28.492429 25116 leader_election.cc:290] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 2 election: Requested vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:28.493052 24526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1adea9b113494bed8c9d57fdbee6e22b" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:28.493368 24382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1adea9b113494bed8c9d57fdbee6e22b" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:28.493685 24526 raft_consensus.cc:3055] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:28.493937 24382 raft_consensus.cc:3060] T 1adea9b113494bed8c9d57fdbee6e22b P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:28.494086 24526 raft_consensus.cc:740] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Running, Role: LEADER
I20260709 13:09:28.494879 24526 consensus_queue.cc:260] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:28.495826 24526 raft_consensus.cc:3060] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:28.499033 24382 raft_consensus.cc:2468] T 1adea9b113494bed8c9d57fdbee6e22b P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 2.
I20260709 13:09:28.500092 25070 leader_election.cc:304] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45; no voters: 
I20260709 13:09:28.501024 25372 raft_consensus.cc:2804] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:28.501104 24526 raft_consensus.cc:2468] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 2.
I20260709 13:09:28.501498 25372 raft_consensus.cc:697] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381 [term 2 LEADER]: Becoming Leader. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Running, Role: LEADER
I20260709 13:09:28.502302 25372 consensus_queue.cc:237] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:28.510239 24201 catalog_manager.cc:5697] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381 reported cstate change: term changed from 1 to 2, leader changed from 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4) to 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "9c2a14143eb744cf83bfa5f46284a381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:28.563539 25347 raft_consensus.cc:993] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390: : Instructing follower 9c2a14143eb744cf83bfa5f46284a381 to start an election
I20260709 13:09:28.564069 25179 raft_consensus.cc:1081] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Signalling peer 9c2a14143eb744cf83bfa5f46284a381 to start an election
I20260709 13:09:28.565546 25119 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "722754340deb4df18b716dc801c35c34"
dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
 from {username='slave'} at 127.0.0.1:49148
I20260709 13:09:28.566049 25119 raft_consensus.cc:493] T 722754340deb4df18b716dc801c35c34 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:28.566407 25119 raft_consensus.cc:3060] T 722754340deb4df18b716dc801c35c34 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:28.570889 25119 raft_consensus.cc:515] T 722754340deb4df18b716dc801c35c34 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:28.572476 25119 leader_election.cc:290] T 722754340deb4df18b716dc801c35c34 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 2 election: Requested vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251)
I20260709 13:09:28.573148 24453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "722754340deb4df18b716dc801c35c34" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "32088c942fbb48239e0eed952d4df449"
I20260709 13:09:28.573379 24526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "722754340deb4df18b716dc801c35c34" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:28.574043 24453 raft_consensus.cc:3060] T 722754340deb4df18b716dc801c35c34 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:28.574137 24526 raft_consensus.cc:3055] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:28.574587 24526 raft_consensus.cc:740] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Running, Role: LEADER
I20260709 13:09:28.575289 24526 consensus_queue.cc:260] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:28.576313 24526 raft_consensus.cc:3060] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:28.579998 24453 raft_consensus.cc:2468] T 722754340deb4df18b716dc801c35c34 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 2.
I20260709 13:09:28.581359 24526 raft_consensus.cc:2468] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 2.
I20260709 13:09:28.581509 25070 leader_election.cc:304] T 722754340deb4df18b716dc801c35c34 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381; no voters: 
I20260709 13:09:28.582413 25372 raft_consensus.cc:2804] T 722754340deb4df18b716dc801c35c34 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:28.582474 24524 raft_consensus.cc:1275] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Refusing update from remote peer 9c2a14143eb744cf83bfa5f46284a381: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:28.582898 25372 raft_consensus.cc:697] T 722754340deb4df18b716dc801c35c34 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 LEADER]: Becoming Leader. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Running, Role: LEADER
I20260709 13:09:28.583922 25372 consensus_queue.cc:237] T 722754340deb4df18b716dc801c35c34 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:28.584045 25353 consensus_queue.cc:1048] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:28.596045 24201 catalog_manager.cc:5697] T 722754340deb4df18b716dc801c35c34 P 9c2a14143eb744cf83bfa5f46284a381 reported cstate change: term changed from 1 to 2, leader changed from 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4) to 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "9c2a14143eb744cf83bfa5f46284a381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:28.616384 24375 raft_consensus.cc:1275] T 40dd44ec1fb347f49c31a8b650ab1639 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Refusing update from remote peer 9c2a14143eb744cf83bfa5f46284a381: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:28.618180 25352 consensus_queue.cc:1048] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:28.676888 24526 raft_consensus.cc:1275] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [term 4 FOLLOWER]: Refusing update from remote peer 9c2a14143eb744cf83bfa5f46284a381: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:09:28.678071 25352 consensus_queue.cc:1048] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:28.687793 24375 raft_consensus.cc:1275] T 42bcfb18b77e41afa60057ccfd718fa9 P c40e7ba102224d24a22ccd1f68aaae45 [term 4 FOLLOWER]: Refusing update from remote peer 9c2a14143eb744cf83bfa5f46284a381: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:09:28.689632 25352 consensus_queue.cc:1048] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:28.774322 24382 raft_consensus.cc:1275] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Refusing update from remote peer 9c2a14143eb744cf83bfa5f46284a381: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:28.775599 25352 consensus_queue.cc:1048] T b147866f4e5f4596b0b3989418cc4b0a P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:28.787220 24453 raft_consensus.cc:1275] T b147866f4e5f4596b0b3989418cc4b0a P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Refusing update from remote peer 9c2a14143eb744cf83bfa5f46284a381: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:28.788620 25352 consensus_queue.cc:1048] T b147866f4e5f4596b0b3989418cc4b0a P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:28.811839 24526 raft_consensus.cc:1275] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Refusing update from remote peer 9c2a14143eb744cf83bfa5f46284a381: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:28.813261 25352 consensus_queue.cc:1048] T 861acffd0d804c8fbd1a63771d7937c7 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:28.828421 24453 raft_consensus.cc:1275] T 861acffd0d804c8fbd1a63771d7937c7 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Refusing update from remote peer 9c2a14143eb744cf83bfa5f46284a381: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:28.836982 25353 consensus_queue.cc:1048] T 861acffd0d804c8fbd1a63771d7937c7 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:28.957926 24526 raft_consensus.cc:1275] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Refusing update from remote peer 9c2a14143eb744cf83bfa5f46284a381: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:28.959126 25353 consensus_queue.cc:1048] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:28.968217 24375 raft_consensus.cc:1275] T 1adea9b113494bed8c9d57fdbee6e22b P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Refusing update from remote peer 9c2a14143eb744cf83bfa5f46284a381: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:28.969704 25352 consensus_queue.cc:1048] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:29.055758 24452 raft_consensus.cc:1275] T 722754340deb4df18b716dc801c35c34 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Refusing update from remote peer 9c2a14143eb744cf83bfa5f46284a381: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:29.057256 25353 consensus_queue.cc:1048] T 722754340deb4df18b716dc801c35c34 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:29.067842 24526 raft_consensus.cc:1275] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Refusing update from remote peer 9c2a14143eb744cf83bfa5f46284a381: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:29.068993 25353 consensus_queue.cc:1048] T 722754340deb4df18b716dc801c35c34 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:29.128343 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:29.185052 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:29.302421 24526 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "40cf5b30d13f4c3e86a9aa657c20c650"
dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
mode: GRACEFUL
new_leader_uuid: "9c2a14143eb744cf83bfa5f46284a381"
 from {username='slave'} at 127.0.0.1:33304
I20260709 13:09:29.303915 24526 raft_consensus.cc:606] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [term 2 LEADER]: Received request to transfer leadership to TS 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:29.307626 24375 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e2b4c67e0f6b4a1e9cfb3703e43e988f"
dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
mode: GRACEFUL
new_leader_uuid: "9c2a14143eb744cf83bfa5f46284a381"
 from {username='slave'} at 127.0.0.1:53088
I20260709 13:09:29.308238 24375 raft_consensus.cc:606] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45 [term 1 LEADER]: Received request to transfer leadership to TS 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:29.309652 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:09:29.314157 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:09:29.579324 25347 raft_consensus.cc:993] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390: : Instructing follower 9c2a14143eb744cf83bfa5f46284a381 to start an election
I20260709 13:09:29.580298 25412 raft_consensus.cc:1081] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [term 2 LEADER]: Signalling peer 9c2a14143eb744cf83bfa5f46284a381 to start an election
I20260709 13:09:29.582734 25119 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "40cf5b30d13f4c3e86a9aa657c20c650"
dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
 from {username='slave'} at 127.0.0.1:49148
I20260709 13:09:29.583390 25119 raft_consensus.cc:493] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:29.583850 25119 raft_consensus.cc:3060] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:29.592257 25119 raft_consensus.cc:515] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:29.594976 25119 leader_election.cc:290] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 3 election: Requested vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:29.597019 24375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40cf5b30d13f4c3e86a9aa657c20c650" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:29.598078 24375 raft_consensus.cc:3060] T 40cf5b30d13f4c3e86a9aa657c20c650 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:29.597992 24526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40cf5b30d13f4c3e86a9aa657c20c650" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:29.599938 24526 raft_consensus.cc:3055] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:29.600317 24526 raft_consensus.cc:740] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 838cb9997b0f437989d3f4e3e9904390, State: Running, Role: LEADER
I20260709 13:09:29.601329 24526 consensus_queue.cc:260] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:29.603147 24526 raft_consensus.cc:3060] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:29.612200 24526 raft_consensus.cc:2468] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 3.
I20260709 13:09:29.613008 24375 raft_consensus.cc:2468] T 40cf5b30d13f4c3e86a9aa657c20c650 P c40e7ba102224d24a22ccd1f68aaae45 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 3.
I20260709 13:09:29.613823 25070 leader_election.cc:304] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381; no voters: 
I20260709 13:09:29.615106 25353 raft_consensus.cc:2804] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:29.617791 25353 raft_consensus.cc:697] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [term 3 LEADER]: Becoming Leader. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Running, Role: LEADER
I20260709 13:09:29.619035 25353 consensus_queue.cc:237] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:29.632454 24204 catalog_manager.cc:5697] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 reported cstate change: term changed from 2 to 3, leader changed from 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4) to 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1). New cstate: current_term: 3 leader_uuid: "9c2a14143eb744cf83bfa5f46284a381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:29.668486 25340 raft_consensus.cc:993] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45: : Instructing follower 9c2a14143eb744cf83bfa5f46284a381 to start an election
I20260709 13:09:29.669384 25377 raft_consensus.cc:1081] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45 [term 1 LEADER]: Signalling peer 9c2a14143eb744cf83bfa5f46284a381 to start an election
I20260709 13:09:29.671069 25119 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e2b4c67e0f6b4a1e9cfb3703e43e988f"
dest_uuid: "9c2a14143eb744cf83bfa5f46284a381"
 from {username='slave'} at 127.0.0.1:49168
I20260709 13:09:29.671624 25119 raft_consensus.cc:493] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:29.672034 25119 raft_consensus.cc:3060] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:29.678838 25119 raft_consensus.cc:515] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:29.680403 25119 leader_election.cc:290] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 2 election: Requested vote from peers 838cb9997b0f437989d3f4e3e9904390 (127.14.103.4:33251), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
I20260709 13:09:29.681205 24526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2b4c67e0f6b4a1e9cfb3703e43e988f" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "838cb9997b0f437989d3f4e3e9904390"
I20260709 13:09:29.681603 24382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2b4c67e0f6b4a1e9cfb3703e43e988f" candidate_uuid: "9c2a14143eb744cf83bfa5f46284a381" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c40e7ba102224d24a22ccd1f68aaae45"
I20260709 13:09:29.681833 24526 raft_consensus.cc:3060] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:29.682219 24382 raft_consensus.cc:3055] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:29.682557 24382 raft_consensus.cc:740] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c40e7ba102224d24a22ccd1f68aaae45, State: Running, Role: LEADER
I20260709 13:09:29.683353 24382 consensus_queue.cc:260] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:29.684870 24382 raft_consensus.cc:3060] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:29.689314 24526 raft_consensus.cc:2468] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 2.
I20260709 13:09:29.690641 25070 leader_election.cc:304] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390, 9c2a14143eb744cf83bfa5f46284a381; no voters: 
I20260709 13:09:29.691694 25353 raft_consensus.cc:2804] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:29.692214 25353 raft_consensus.cc:697] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381 [term 2 LEADER]: Becoming Leader. State: Replica: 9c2a14143eb744cf83bfa5f46284a381, State: Running, Role: LEADER
I20260709 13:09:29.693125 25353 consensus_queue.cc:237] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:29.693487 24382 raft_consensus.cc:2468] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2a14143eb744cf83bfa5f46284a381 in term 2.
I20260709 13:09:29.705030 24204 catalog_manager.cc:5697] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381 reported cstate change: term changed from 1 to 2, leader changed from c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2) to 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "9c2a14143eb744cf83bfa5f46284a381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:30.161438 24526 raft_consensus.cc:1275] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [term 3 FOLLOWER]: Refusing update from remote peer 9c2a14143eb744cf83bfa5f46284a381: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:30.162809 25353 consensus_queue.cc:1048] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:30.169000 24382 raft_consensus.cc:1275] T 40cf5b30d13f4c3e86a9aa657c20c650 P c40e7ba102224d24a22ccd1f68aaae45 [term 3 FOLLOWER]: Refusing update from remote peer 9c2a14143eb744cf83bfa5f46284a381: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:30.171195 25353 consensus_queue.cc:1048] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:30.300649 24375 raft_consensus.cc:1275] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Refusing update from remote peer 9c2a14143eb744cf83bfa5f46284a381: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:30.301774 25353 consensus_queue.cc:1048] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:30.311858 24526 raft_consensus.cc:1275] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Refusing update from remote peer 9c2a14143eb744cf83bfa5f46284a381: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:30.313482 25423 consensus_queue.cc:1048] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:30.315657 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:30.368345 14748 tablet_server.cc:179] TabletServer@127.14.103.1:45351 shutting down...
W20260709 13:09:30.395880 24346 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 40 similar messages]
W20260709 13:09:30.401326 24346 consensus_peers.cc:597] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:30.421748 24346 consensus_peers.cc:597] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:30.423002 24420 consensus_peers.cc:597] T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:30.424036 24346 consensus_peers.cc:597] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:30.426011 14748 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:09:30.426951 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.427776 14748 raft_consensus.cc:2243] T b147866f4e5f4596b0b3989418cc4b0a P 9c2a14143eb744cf83bfa5f46284a381 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:30.428617 14748 raft_consensus.cc:2272] T b147866f4e5f4596b0b3989418cc4b0a P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.431398 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.432168 14748 raft_consensus.cc:2243] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:09:30.433007 14748 raft_consensus.cc:2272] T 8775bc34e96d48ae8f5598affd26efdc P 9c2a14143eb744cf83bfa5f46284a381 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.435262 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.436169 14748 raft_consensus.cc:2243] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.436653 14748 raft_consensus.cc:2272] T 2f7b6652eba4485ca0632f9fefc2e326 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.439471 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.440222 14748 raft_consensus.cc:2243] T 363cde7d4cb84353a74a18cbd9701fa7 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.440725 14748 raft_consensus.cc:2272] T 363cde7d4cb84353a74a18cbd9701fa7 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.444032 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.444950 14748 raft_consensus.cc:2243] T 4af7fc369d124452beb5fd07cf8592ec P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.445571 14748 raft_consensus.cc:2272] T 4af7fc369d124452beb5fd07cf8592ec P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.449012 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.449894 14748 raft_consensus.cc:2243] T e4dd51cda55249b48d7bfa39d5feb398 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.450315 14748 raft_consensus.cc:2272] T e4dd51cda55249b48d7bfa39d5feb398 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.453277 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.454135 14748 raft_consensus.cc:2243] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [term 3 LEADER]: Raft consensus shutting down.
W20260709 13:09:30.454298 24420 consensus_peers.cc:597] T edd24afe4d7847e3980c6e586b9dac5e P 32088c942fbb48239e0eed952d4df449 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:30.455024 14748 raft_consensus.cc:2272] T 40cf5b30d13f4c3e86a9aa657c20c650 P 9c2a14143eb744cf83bfa5f46284a381 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.458630 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.459434 14748 raft_consensus.cc:2243] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.460059 14748 raft_consensus.cc:2272] T 373185ca8ab04fdb91c7947ec0668808 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.463441 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.464377 14748 raft_consensus.cc:2243] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.464980 14748 raft_consensus.cc:2272] T 5bb19353ba334eac831ae2cce277298a P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:30.467142 24420 consensus_peers.cc:597] T 958ccd87b65945468e9247c40ca27c33 P 32088c942fbb48239e0eed952d4df449 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:30.468252 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.469147 14748 raft_consensus.cc:2243] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.469698 14748 raft_consensus.cc:2272] T dd5ac0d1debc4695a406617ec06f07a0 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.472743 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.473366 14748 raft_consensus.cc:2243] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.473794 14748 raft_consensus.cc:2272] T 5b8d15473a964a0c8b8c283c4552706e P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.475911 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.476432 14748 raft_consensus.cc:2243] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:30.477171 14748 raft_consensus.cc:2272] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.479858 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.480448 14748 raft_consensus.cc:2243] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.480917 14748 raft_consensus.cc:2272] T c60b42ac6d7e48bfae725dc113758659 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.483006 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.483632 14748 raft_consensus.cc:2243] T 86262701775a456dacd47fcdc1a5d13b P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.484196 14748 raft_consensus.cc:2272] T 86262701775a456dacd47fcdc1a5d13b P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.486793 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.487404 14748 raft_consensus.cc:2243] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.487998 14748 raft_consensus.cc:2272] T e219ba07753f49a5abe454b27cd1494e P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:30.488700 24494 consensus_peers.cc:597] T 3e71fd85d9424a08aed292991bccf5ee P 838cb9997b0f437989d3f4e3e9904390 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:30.491192 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.491941 14748 raft_consensus.cc:2243] T 3e71fd85d9424a08aed292991bccf5ee P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.492360 14748 raft_consensus.cc:2272] T 3e71fd85d9424a08aed292991bccf5ee P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.494485 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.495085 14748 raft_consensus.cc:2243] T 958ccd87b65945468e9247c40ca27c33 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.495510 14748 raft_consensus.cc:2272] T 958ccd87b65945468e9247c40ca27c33 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.498070 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.498914 14748 raft_consensus.cc:2243] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.499531 14748 raft_consensus.cc:2272] T 53b7256c3dd84f5b952d1f9d47691f4e P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:30.501314 24420 consensus_peers.cc:597] T dd5ac0d1debc4695a406617ec06f07a0 P 32088c942fbb48239e0eed952d4df449 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:30.502625 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.503358 14748 raft_consensus.cc:2243] T 861acffd0d804c8fbd1a63771d7937c7 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:30.504176 14748 raft_consensus.cc:2272] T 861acffd0d804c8fbd1a63771d7937c7 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.506493 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.507082 14748 raft_consensus.cc:2243] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.507507 14748 raft_consensus.cc:2272] T bd607d60354e4864ae5499ee1c1a05c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.509670 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.510389 14748 raft_consensus.cc:2243] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.511049 14748 raft_consensus.cc:2272] T 9711acbe7d044647bf774ee849c8a032 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.514158 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.515059 14748 raft_consensus.cc:2243] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.515794 14748 raft_consensus.cc:2272] T 72ac163496084a3e977344fa9228a3c9 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:30.516877 24346 consensus_peers.cc:597] T 4af7fc369d124452beb5fd07cf8592ec P c40e7ba102224d24a22ccd1f68aaae45 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:30.519200 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.520121 14748 raft_consensus.cc:2243] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:09:30.521328 14748 raft_consensus.cc:2272] T 42bcfb18b77e41afa60057ccfd718fa9 P 9c2a14143eb744cf83bfa5f46284a381 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.524348 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.525130 14748 raft_consensus.cc:2243] T 915d8224f8b64cdab90dff1689046548 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:30.526017 14748 raft_consensus.cc:2272] T 915d8224f8b64cdab90dff1689046548 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:30.528033 24494 consensus_peers.cc:597] T 836ed40e5bb8485cb93cb03bc2508265 P 838cb9997b0f437989d3f4e3e9904390 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:30.528837 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.529688 14748 raft_consensus.cc:2243] T 722754340deb4df18b716dc801c35c34 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:30.530478 14748 raft_consensus.cc:2272] T 722754340deb4df18b716dc801c35c34 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.533439 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.534281 14748 raft_consensus.cc:2243] T 2deba793363c4afca7d75b2fc3814304 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.534819 14748 raft_consensus.cc:2272] T 2deba793363c4afca7d75b2fc3814304 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:30.535820 24346 consensus_peers.cc:597] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:30.537771 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.538671 14748 raft_consensus.cc:2243] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:30.539902 14748 raft_consensus.cc:2272] T 1adea9b113494bed8c9d57fdbee6e22b P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.542909 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.543737 14748 raft_consensus.cc:2243] T 836ed40e5bb8485cb93cb03bc2508265 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.544305 14748 raft_consensus.cc:2272] T 836ed40e5bb8485cb93cb03bc2508265 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:30.545413 24494 consensus_peers.cc:597] T 363cde7d4cb84353a74a18cbd9701fa7 P 838cb9997b0f437989d3f4e3e9904390 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:30.546845 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.547494 14748 raft_consensus.cc:2243] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.547998 14748 raft_consensus.cc:2272] T 72d2665a42a94a82b1ceec249960fdf6 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.551028 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.551923 14748 raft_consensus.cc:2243] T c6af3489742e4539866e5a3b10761ca2 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.552474 14748 raft_consensus.cc:2272] T c6af3489742e4539866e5a3b10761ca2 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.555300 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.555862 14748 raft_consensus.cc:2243] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.556243 14748 raft_consensus.cc:2272] T 6427fa143f574eaba2ee4c979e5485fc P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.558326 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.559201 14748 raft_consensus.cc:2243] T 9b7cd923257143648cfe6a296e18b642 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.559731 14748 raft_consensus.cc:2272] T 9b7cd923257143648cfe6a296e18b642 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.562368 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.562901 14748 raft_consensus.cc:2243] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.563344 14748 raft_consensus.cc:2272] T 4f32a46373be4a29b6ac72efa35494b4 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.565382 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.565917 14748 raft_consensus.cc:2243] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:30.566689 14748 raft_consensus.cc:2272] T 40dd44ec1fb347f49c31a8b650ab1639 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.569046 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.569774 14748 raft_consensus.cc:2243] T 93b6310306174b4ca68b467ea7f5d23e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.570196 14748 raft_consensus.cc:2272] T 93b6310306174b4ca68b467ea7f5d23e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.572943 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.573532 14748 raft_consensus.cc:2243] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.574018 14748 raft_consensus.cc:2272] T 0f795acd16fd472b8c9360fb14e7e5d7 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.576076 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.576591 14748 raft_consensus.cc:2243] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:30.577394 14748 raft_consensus.cc:2272] T 5eaa917e65d9403dbb241f84ee96ba9d P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.579276 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.579829 14748 raft_consensus.cc:2243] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:09:30.580602 14748 raft_consensus.cc:2272] T b88feffc0859413482b0daf562d2f86a P 9c2a14143eb744cf83bfa5f46284a381 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.582553 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.583042 14748 raft_consensus.cc:2243] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.583498 14748 raft_consensus.cc:2272] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.585578 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.586062 14748 raft_consensus.cc:2243] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:09:30.586896 14748 raft_consensus.cc:2272] T 9c88d903166c4812a88653e389ab5776 P 9c2a14143eb744cf83bfa5f46284a381 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.589201 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.589783 14748 raft_consensus.cc:2243] T edd24afe4d7847e3980c6e586b9dac5e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.590291 14748 raft_consensus.cc:2272] T edd24afe4d7847e3980c6e586b9dac5e P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:30.594218 24346 consensus_peers.cc:597] T 142bbcbf57124963866a3967c8adebed P c40e7ba102224d24a22ccd1f68aaae45 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:30.594251 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.595422 14748 raft_consensus.cc:2243] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:09:30.596549 14748 raft_consensus.cc:2272] T 90d9e64f97544895b9b97dfdc5ceb359 P 9c2a14143eb744cf83bfa5f46284a381 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.599511 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.600423 14748 raft_consensus.cc:2243] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.601141 14748 raft_consensus.cc:2272] T 6ae3dc25b8484b6f99d8f41685229cb8 P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.604487 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.605408 14748 raft_consensus.cc:2243] T 54fa63b278394f569a9ecffed4dec233 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.605866 14748 raft_consensus.cc:2272] T 54fa63b278394f569a9ecffed4dec233 P 9c2a14143eb744cf83bfa5f46284a381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.608897 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:30.609386 14748 raft_consensus.cc:2243] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.609829 14748 raft_consensus.cc:2272] T 142bbcbf57124963866a3967c8adebed P 9c2a14143eb744cf83bfa5f46284a381 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:30.626976 24346 consensus_peers.cc:597] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:30.638517 24420 consensus_peers.cc:597] T e4dd51cda55249b48d7bfa39d5feb398 P 32088c942fbb48239e0eed952d4df449 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:30.642920 24346 consensus_peers.cc:597] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:30.661468 24420 consensus_peers.cc:597] T 93b6310306174b4ca68b467ea7f5d23e P 32088c942fbb48239e0eed952d4df449 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:30.663259 24420 consensus_peers.cc:597] T 9b7cd923257143648cfe6a296e18b642 P 32088c942fbb48239e0eed952d4df449 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:30.667403 24346 consensus_peers.cc:597] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:30.686074 24346 consensus_peers.cc:597] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:45351: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:30.696440 14748 tablet_server.cc:196] TabletServer@127.14.103.1:45351 shutdown complete.
W20260709 13:09:30.717454 24420 consensus_peers.cc:597] T 72d2665a42a94a82b1ceec249960fdf6 P 32088c942fbb48239e0eed952d4df449 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:45351: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:30.743382 24346 consensus_peers.cc:597] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:45351: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:30.748296 24420 consensus_peers.cc:597] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 32088c942fbb48239e0eed952d4df449 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:45351: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:30.762012 24420 consensus_peers.cc:597] T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:45351: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:30.768836 24494 consensus_peers.cc:597] T c6af3489742e4539866e5a3b10761ca2 P 838cb9997b0f437989d3f4e3e9904390 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:45351: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:30.787163 24420 consensus_peers.cc:597] T 2deba793363c4afca7d75b2fc3814304 P 32088c942fbb48239e0eed952d4df449 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:45351: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:30.790944 24346 consensus_peers.cc:597] T 54fa63b278394f569a9ecffed4dec233 P c40e7ba102224d24a22ccd1f68aaae45 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:45351: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:30.793681 24494 consensus_peers.cc:597] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:45351: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:30.802433 14748 tablet_server.cc:179] TabletServer@127.14.103.2:0 shutting down...
W20260709 13:09:30.821156 24494 consensus_peers.cc:597] T 86262701775a456dacd47fcdc1a5d13b P 838cb9997b0f437989d3f4e3e9904390 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:45351: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:30.830363 24493 consensus_peers.cc:597] T 84af43749e474087b0a5857a95dfb31d P 838cb9997b0f437989d3f4e3e9904390 -> Peer c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177): Couldn't send request to peer c40e7ba102224d24a22ccd1f68aaae45. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:30.859512 14748 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:09:30.860636 14748 tablet_replica.cc:333] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:30.861639 14748 raft_consensus.cc:2243] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:30.862588 14748 raft_consensus.cc:2272] T 5b8d15473a964a0c8b8c283c4552706e P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.865509 14748 tablet_replica.cc:333] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:30.866479 14748 raft_consensus.cc:2243] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:09:30.866824 24419 consensus_peers.cc:597] T aac1b62e27a44fb5b9a23cfc42a5d00b P 32088c942fbb48239e0eed952d4df449 -> Peer c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177): Couldn't send request to peer c40e7ba102224d24a22ccd1f68aaae45. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:30.867628 14748 raft_consensus.cc:2272] T e219ba07753f49a5abe454b27cd1494e P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.870532 14748 tablet_replica.cc:333] T 3cf2c3be9fd244d4b9d4703d424d3c5d P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:30.871145 14748 raft_consensus.cc:2243] T 3cf2c3be9fd244d4b9d4703d424d3c5d P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.871532 14748 raft_consensus.cc:2272] T 3cf2c3be9fd244d4b9d4703d424d3c5d P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.873620 14748 tablet_replica.cc:333] T dd5ac0d1debc4695a406617ec06f07a0 P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:30.874418 14748 raft_consensus.cc:2243] T dd5ac0d1debc4695a406617ec06f07a0 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.875005 14748 raft_consensus.cc:2272] T dd5ac0d1debc4695a406617ec06f07a0 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.877775 14748 tablet_replica.cc:333] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:30.878384 14748 raft_consensus.cc:2243] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:30.879179 14748 raft_consensus.cc:2272] T bd607d60354e4864ae5499ee1c1a05c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.881341 14748 tablet_replica.cc:333] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:30.881924 14748 raft_consensus.cc:2243] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:30.882716 14748 raft_consensus.cc:2272] T 2f7b6652eba4485ca0632f9fefc2e326 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.885171 14748 tablet_replica.cc:333] T 624f99e3165f49babbfce35effc9d627 P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:30.886023 14748 raft_consensus.cc:2243] T 624f99e3165f49babbfce35effc9d627 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.886571 14748 raft_consensus.cc:2272] T 624f99e3165f49babbfce35effc9d627 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:30.889231 24494 consensus_peers.cc:597] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390 -> Peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Couldn't send request to peer 9c2a14143eb744cf83bfa5f46284a381. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:45351: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:30.889617 14748 tablet_replica.cc:333] T a1b91f42722c4116950ce80bc02eccd0 P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:30.890288 14748 raft_consensus.cc:2243] T a1b91f42722c4116950ce80bc02eccd0 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:30.891086 14748 raft_consensus.cc:2272] T a1b91f42722c4116950ce80bc02eccd0 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.893338 14748 tablet_replica.cc:333] T 3e71fd85d9424a08aed292991bccf5ee P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:30.893952 14748 raft_consensus.cc:2243] T 3e71fd85d9424a08aed292991bccf5ee P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.894413 14748 raft_consensus.cc:2272] T 3e71fd85d9424a08aed292991bccf5ee P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.896564 14748 tablet_replica.cc:333] T 72e12f99e2084675ab3f53ffb836e464 P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:30.897142 14748 raft_consensus.cc:2243] T 72e12f99e2084675ab3f53ffb836e464 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.897568 14748 raft_consensus.cc:2272] T 72e12f99e2084675ab3f53ffb836e464 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.899518 14748 tablet_replica.cc:333] T 40cf5b30d13f4c3e86a9aa657c20c650 P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:30.900074 14748 raft_consensus.cc:2243] T 40cf5b30d13f4c3e86a9aa657c20c650 P c40e7ba102224d24a22ccd1f68aaae45 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.900498 14748 raft_consensus.cc:2272] T 40cf5b30d13f4c3e86a9aa657c20c650 P c40e7ba102224d24a22ccd1f68aaae45 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.902941 14748 tablet_replica.cc:333] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:30.903533 14748 raft_consensus.cc:2243] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:30.904389 14748 raft_consensus.cc:2272] T 373185ca8ab04fdb91c7947ec0668808 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.906404 14748 tablet_replica.cc:333] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:30.906986 14748 raft_consensus.cc:2243] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:30.907855 14748 raft_consensus.cc:2272] T 5bb19353ba334eac831ae2cce277298a P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.909730 14748 tablet_replica.cc:333] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:30.910313 14748 raft_consensus.cc:2243] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:30.911062 14748 raft_consensus.cc:2272] T 0f795acd16fd472b8c9360fb14e7e5d7 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.912889 14748 tablet_replica.cc:333] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:30.913446 14748 raft_consensus.cc:2243] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.913946 14748 raft_consensus.cc:2272] T 5eaa917e65d9403dbb241f84ee96ba9d P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.916635 14748 tablet_replica.cc:333] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:30.917135 14748 raft_consensus.cc:2243] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.917542 14748 raft_consensus.cc:2272] T e2b4c67e0f6b4a1e9cfb3703e43e988f P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.920603 14748 tablet_replica.cc:333] T db9e8fa567ab481c9b80ae8e30acd628 P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:30.921187 14748 raft_consensus.cc:2243] T db9e8fa567ab481c9b80ae8e30acd628 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:30.921957 14748 raft_consensus.cc:2272] T db9e8fa567ab481c9b80ae8e30acd628 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.924046 14748 tablet_replica.cc:333] T 90d9e64f97544895b9b97dfdc5ceb359 P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:30.924616 14748 raft_consensus.cc:2243] T 90d9e64f97544895b9b97dfdc5ceb359 P c40e7ba102224d24a22ccd1f68aaae45 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.925060 14748 raft_consensus.cc:2272] T 90d9e64f97544895b9b97dfdc5ceb359 P c40e7ba102224d24a22ccd1f68aaae45 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.926999 14748 tablet_replica.cc:333] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:30.927562 14748 raft_consensus.cc:2243] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.928047 14748 raft_consensus.cc:2272] T b147866f4e5f4596b0b3989418cc4b0a P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.930121 14748 tablet_replica.cc:333] T 40dd44ec1fb347f49c31a8b650ab1639 P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:30.930667 14748 raft_consensus.cc:2243] T 40dd44ec1fb347f49c31a8b650ab1639 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.931062 14748 raft_consensus.cc:2272] T 40dd44ec1fb347f49c31a8b650ab1639 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.933018 14748 tablet_replica.cc:333] T aac1b62e27a44fb5b9a23cfc42a5d00b P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:30.933568 14748 raft_consensus.cc:2243] T aac1b62e27a44fb5b9a23cfc42a5d00b P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.933974 14748 raft_consensus.cc:2272] T aac1b62e27a44fb5b9a23cfc42a5d00b P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.936046 14748 tablet_replica.cc:333] T ae6c1a13de5442c5bb4e58e7ce36e741 P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:30.936667 14748 raft_consensus.cc:2243] T ae6c1a13de5442c5bb4e58e7ce36e741 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.937211 14748 raft_consensus.cc:2272] T ae6c1a13de5442c5bb4e58e7ce36e741 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.940078 14748 tablet_replica.cc:333] T 05840bb250764e02998cdddc290e5139 P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:30.940685 14748 raft_consensus.cc:2243] T 05840bb250764e02998cdddc290e5139 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.941111 14748 raft_consensus.cc:2272] T 05840bb250764e02998cdddc290e5139 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.943038 14748 tablet_replica.cc:333] T f4e53ea5cf694db697fa0e474da54423 P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:30.943627 14748 raft_consensus.cc:2243] T f4e53ea5cf694db697fa0e474da54423 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.944139 14748 raft_consensus.cc:2272] T f4e53ea5cf694db697fa0e474da54423 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.946642 14748 tablet_replica.cc:333] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:30.947316 14748 raft_consensus.cc:2243] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:30.948104 14748 raft_consensus.cc:2272] T 53b7256c3dd84f5b952d1f9d47691f4e P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.949951 14748 tablet_replica.cc:333] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:30.950464 14748 raft_consensus.cc:2243] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.950896 14748 raft_consensus.cc:2272] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.952732 14748 tablet_replica.cc:333] T 4f085ebc2da04fbb87d48e4f206e61f0 P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:30.953480 14748 raft_consensus.cc:2243] T 4f085ebc2da04fbb87d48e4f206e61f0 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.953953 14748 raft_consensus.cc:2272] T 4f085ebc2da04fbb87d48e4f206e61f0 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.956714 14748 tablet_replica.cc:333] T 1adea9b113494bed8c9d57fdbee6e22b P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:30.957422 14748 raft_consensus.cc:2243] T 1adea9b113494bed8c9d57fdbee6e22b P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.957938 14748 raft_consensus.cc:2272] T 1adea9b113494bed8c9d57fdbee6e22b P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:30.960772 24493 consensus_peers.cc:597] T 2039124a6d8848149440a70100af8755 P 838cb9997b0f437989d3f4e3e9904390 -> Peer c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177): Couldn't send request to peer c40e7ba102224d24a22ccd1f68aaae45. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:30.960809 14748 tablet_replica.cc:333] T c6af3489742e4539866e5a3b10761ca2 P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:30.961629 14748 raft_consensus.cc:2243] T c6af3489742e4539866e5a3b10761ca2 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.962086 14748 raft_consensus.cc:2272] T c6af3489742e4539866e5a3b10761ca2 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.964176 14748 tablet_replica.cc:333] T 48aea62663ac40998d7f09389b93d586 P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:30.964742 14748 raft_consensus.cc:2243] T 48aea62663ac40998d7f09389b93d586 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.965154 14748 raft_consensus.cc:2272] T 48aea62663ac40998d7f09389b93d586 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.967183 14748 tablet_replica.cc:333] T 9b7cd923257143648cfe6a296e18b642 P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:30.967877 14748 raft_consensus.cc:2243] T 9b7cd923257143648cfe6a296e18b642 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.968411 14748 raft_consensus.cc:2272] T 9b7cd923257143648cfe6a296e18b642 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:30.970194 24493 consensus_peers.cc:597] T 3e71fd85d9424a08aed292991bccf5ee P 838cb9997b0f437989d3f4e3e9904390 -> Peer c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177): Couldn't send request to peer c40e7ba102224d24a22ccd1f68aaae45. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:30.970954 14748 tablet_replica.cc:333] T 4f32a46373be4a29b6ac72efa35494b4 P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:30.971603 14748 raft_consensus.cc:2243] T 4f32a46373be4a29b6ac72efa35494b4 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.972065 14748 raft_consensus.cc:2272] T 4f32a46373be4a29b6ac72efa35494b4 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.974017 14748 tablet_replica.cc:333] T 915d8224f8b64cdab90dff1689046548 P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:30.974701 14748 raft_consensus.cc:2243] T 915d8224f8b64cdab90dff1689046548 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.975220 14748 raft_consensus.cc:2272] T 915d8224f8b64cdab90dff1689046548 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.977629 14748 tablet_replica.cc:333] T 42bcfb18b77e41afa60057ccfd718fa9 P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:30.978123 14748 raft_consensus.cc:2243] T 42bcfb18b77e41afa60057ccfd718fa9 P c40e7ba102224d24a22ccd1f68aaae45 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.978492 14748 raft_consensus.cc:2272] T 42bcfb18b77e41afa60057ccfd718fa9 P c40e7ba102224d24a22ccd1f68aaae45 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.980587 14748 tablet_replica.cc:333] T 84af43749e474087b0a5857a95dfb31d P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:30.982191 14748 raft_consensus.cc:2243] T 84af43749e474087b0a5857a95dfb31d P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.983058 14748 raft_consensus.cc:2272] T 84af43749e474087b0a5857a95dfb31d P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:30.994743 14748 tablet_replica.cc:333] T 6ae3dc25b8484b6f99d8f41685229cb8 P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:30.995496 14748 raft_consensus.cc:2243] T 6ae3dc25b8484b6f99d8f41685229cb8 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:30.996101 14748 raft_consensus.cc:2272] T 6ae3dc25b8484b6f99d8f41685229cb8 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:30.996236 24493 consensus_peers.cc:597] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 838cb9997b0f437989d3f4e3e9904390 -> Peer c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177): Couldn't send request to peer c40e7ba102224d24a22ccd1f68aaae45. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:30.999580 14748 tablet_replica.cc:333] T 54fa63b278394f569a9ecffed4dec233 P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:31.000313 14748 raft_consensus.cc:2243] T 54fa63b278394f569a9ecffed4dec233 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:31.001313 14748 raft_consensus.cc:2272] T 54fa63b278394f569a9ecffed4dec233 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.005023 14748 tablet_replica.cc:333] T 8775bc34e96d48ae8f5598affd26efdc P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:31.005537 14748 raft_consensus.cc:2243] T 8775bc34e96d48ae8f5598affd26efdc P c40e7ba102224d24a22ccd1f68aaae45 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.005939 14748 raft_consensus.cc:2272] T 8775bc34e96d48ae8f5598affd26efdc P c40e7ba102224d24a22ccd1f68aaae45 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.007941 14748 tablet_replica.cc:333] T a3e3b771ce2c4165aad02954773c58d7 P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:31.008419 14748 raft_consensus.cc:2243] T a3e3b771ce2c4165aad02954773c58d7 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.008769 14748 raft_consensus.cc:2272] T a3e3b771ce2c4165aad02954773c58d7 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.010910 14748 tablet_replica.cc:333] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:31.011583 14748 raft_consensus.cc:2243] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:31.012501 14748 raft_consensus.cc:2272] T 72ac163496084a3e977344fa9228a3c9 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:31.014607 24419 consensus_peers.cc:597] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 32088c942fbb48239e0eed952d4df449 -> Peer c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177): Couldn't send request to peer c40e7ba102224d24a22ccd1f68aaae45. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:31.014983 14748 tablet_replica.cc:333] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:31.016009 14748 raft_consensus.cc:2243] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:31.017066 14748 raft_consensus.cc:2272] T 9711acbe7d044647bf774ee849c8a032 P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.019835 14748 tablet_replica.cc:333] T 4af7fc369d124452beb5fd07cf8592ec P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:31.020502 14748 raft_consensus.cc:2243] T 4af7fc369d124452beb5fd07cf8592ec P c40e7ba102224d24a22ccd1f68aaae45 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:31.021373 14748 raft_consensus.cc:2272] T 4af7fc369d124452beb5fd07cf8592ec P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.023576 14748 tablet_replica.cc:333] T 142bbcbf57124963866a3967c8adebed P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:31.024127 14748 raft_consensus.cc:2243] T 142bbcbf57124963866a3967c8adebed P c40e7ba102224d24a22ccd1f68aaae45 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:31.025115 14748 raft_consensus.cc:2272] T 142bbcbf57124963866a3967c8adebed P c40e7ba102224d24a22ccd1f68aaae45 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.027243 14748 tablet_replica.cc:333] T 2039124a6d8848149440a70100af8755 P c40e7ba102224d24a22ccd1f68aaae45: stopping tablet replica
I20260709 13:09:31.027805 14748 raft_consensus.cc:2243] T 2039124a6d8848149440a70100af8755 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.028314 14748 raft_consensus.cc:2272] T 2039124a6d8848149440a70100af8755 P c40e7ba102224d24a22ccd1f68aaae45 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:31.032922 24419 consensus_peers.cc:597] T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449 -> Peer c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177): Couldn't send request to peer c40e7ba102224d24a22ccd1f68aaae45. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:31.087486 24493 consensus_peers.cc:597] T c6af3489742e4539866e5a3b10761ca2 P 838cb9997b0f437989d3f4e3e9904390 -> Peer c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177): Couldn't send request to peer c40e7ba102224d24a22ccd1f68aaae45. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:31.119279 14748 tablet_server.cc:196] TabletServer@127.14.103.2:0 shutdown complete.
W20260709 13:09:31.152473 24419 consensus_peers.cc:597] T 48aea62663ac40998d7f09389b93d586 P 32088c942fbb48239e0eed952d4df449 -> Peer c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177): Couldn't send request to peer c40e7ba102224d24a22ccd1f68aaae45. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:46177: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:31.164129 24493 consensus_peers.cc:597] T 05840bb250764e02998cdddc290e5139 P 838cb9997b0f437989d3f4e3e9904390 -> Peer c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177): Couldn't send request to peer c40e7ba102224d24a22ccd1f68aaae45. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:46177: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:31.176183 24493 consensus_peers.cc:597] T ae6c1a13de5442c5bb4e58e7ce36e741 P 838cb9997b0f437989d3f4e3e9904390 -> Peer c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177): Couldn't send request to peer c40e7ba102224d24a22ccd1f68aaae45. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:46177: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:31.204612 14748 tablet_server.cc:179] TabletServer@127.14.103.3:0 shutting down...
W20260709 13:09:31.219889 24493 consensus_peers.cc:597] T 86262701775a456dacd47fcdc1a5d13b P 838cb9997b0f437989d3f4e3e9904390 -> Peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Couldn't send request to peer 32088c942fbb48239e0eed952d4df449. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:31.247526 14748 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:09:31.248457 14748 tablet_replica.cc:333] T e219ba07753f49a5abe454b27cd1494e P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.249187 14748 raft_consensus.cc:2243] T e219ba07753f49a5abe454b27cd1494e P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.249789 14748 raft_consensus.cc:2272] T e219ba07753f49a5abe454b27cd1494e P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.252852 14748 tablet_replica.cc:333] T 86262701775a456dacd47fcdc1a5d13b P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.253556 14748 raft_consensus.cc:2243] T 86262701775a456dacd47fcdc1a5d13b P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.254097 14748 raft_consensus.cc:2272] T 86262701775a456dacd47fcdc1a5d13b P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.256233 14748 tablet_replica.cc:333] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.256762 14748 raft_consensus.cc:2243] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.257124 14748 raft_consensus.cc:2272] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.259267 14748 tablet_replica.cc:333] T bd607d60354e4864ae5499ee1c1a05c9 P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.260033 14748 raft_consensus.cc:2243] T bd607d60354e4864ae5499ee1c1a05c9 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.260565 14748 raft_consensus.cc:2272] T bd607d60354e4864ae5499ee1c1a05c9 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.263339 14748 tablet_replica.cc:333] T edd24afe4d7847e3980c6e586b9dac5e P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.264218 14748 raft_consensus.cc:2243] T edd24afe4d7847e3980c6e586b9dac5e P 32088c942fbb48239e0eed952d4df449 [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:09:31.264376 24493 consensus_peers.cc:597] T 624f99e3165f49babbfce35effc9d627 P 838cb9997b0f437989d3f4e3e9904390 -> Peer c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177): Couldn't send request to peer c40e7ba102224d24a22ccd1f68aaae45. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:46177: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:31.264379 24419 consensus_peers.cc:597] T 4f085ebc2da04fbb87d48e4f206e61f0 P 32088c942fbb48239e0eed952d4df449 -> Peer c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177): Couldn't send request to peer c40e7ba102224d24a22ccd1f68aaae45. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:46177: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:31.265074 14748 raft_consensus.cc:2272] T edd24afe4d7847e3980c6e586b9dac5e P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.267724 14748 tablet_replica.cc:333] T dd5ac0d1debc4695a406617ec06f07a0 P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.268543 14748 raft_consensus.cc:2243] T dd5ac0d1debc4695a406617ec06f07a0 P 32088c942fbb48239e0eed952d4df449 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:31.269598 14748 raft_consensus.cc:2272] T dd5ac0d1debc4695a406617ec06f07a0 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.272593 14748 tablet_replica.cc:333] T b88feffc0859413482b0daf562d2f86a P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
W20260709 13:09:31.273268 24419 consensus_peers.cc:597] T a3e3b771ce2c4165aad02954773c58d7 P 32088c942fbb48239e0eed952d4df449 -> Peer c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177): Couldn't send request to peer c40e7ba102224d24a22ccd1f68aaae45. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:46177: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:31.273337 14748 raft_consensus.cc:2243] T b88feffc0859413482b0daf562d2f86a P 32088c942fbb48239e0eed952d4df449 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.274228 14748 raft_consensus.cc:2272] T b88feffc0859413482b0daf562d2f86a P 32088c942fbb48239e0eed952d4df449 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:31.275804 24493 consensus_peers.cc:597] T 72e12f99e2084675ab3f53ffb836e464 P 838cb9997b0f437989d3f4e3e9904390 -> Peer c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177): Couldn't send request to peer c40e7ba102224d24a22ccd1f68aaae45. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:46177: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:31.276690 14748 tablet_replica.cc:333] T 915d8224f8b64cdab90dff1689046548 P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.277307 14748 raft_consensus.cc:2243] T 915d8224f8b64cdab90dff1689046548 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.277681 14748 raft_consensus.cc:2272] T 915d8224f8b64cdab90dff1689046548 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.279739 14748 tablet_replica.cc:333] T 363cde7d4cb84353a74a18cbd9701fa7 P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.280544 14748 raft_consensus.cc:2243] T 363cde7d4cb84353a74a18cbd9701fa7 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.281064 14748 raft_consensus.cc:2272] T 363cde7d4cb84353a74a18cbd9701fa7 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:31.283370 24419 consensus_peers.cc:597] T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449 -> Peer c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177): Couldn't send request to peer c40e7ba102224d24a22ccd1f68aaae45. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:46177: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:31.283944 14748 tablet_replica.cc:333] T 2f7b6652eba4485ca0632f9fefc2e326 P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.284581 14748 raft_consensus.cc:2243] T 2f7b6652eba4485ca0632f9fefc2e326 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.285012 14748 raft_consensus.cc:2272] T 2f7b6652eba4485ca0632f9fefc2e326 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.286865 14748 tablet_replica.cc:333] T 624f99e3165f49babbfce35effc9d627 P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.287401 14748 raft_consensus.cc:2243] T 624f99e3165f49babbfce35effc9d627 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.287859 14748 raft_consensus.cc:2272] T 624f99e3165f49babbfce35effc9d627 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.289669 14748 tablet_replica.cc:333] T a1b91f42722c4116950ce80bc02eccd0 P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.290205 14748 raft_consensus.cc:2243] T a1b91f42722c4116950ce80bc02eccd0 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.290664 14748 raft_consensus.cc:2272] T a1b91f42722c4116950ce80bc02eccd0 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.293114 14748 tablet_replica.cc:333] T a3e3b771ce2c4165aad02954773c58d7 P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.293686 14748 raft_consensus.cc:2243] T a3e3b771ce2c4165aad02954773c58d7 P 32088c942fbb48239e0eed952d4df449 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:31.294421 14748 raft_consensus.cc:2272] T a3e3b771ce2c4165aad02954773c58d7 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.297056 14748 tablet_replica.cc:333] T 72ac163496084a3e977344fa9228a3c9 P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.297927 14748 raft_consensus.cc:2243] T 72ac163496084a3e977344fa9228a3c9 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.298542 14748 raft_consensus.cc:2272] T 72ac163496084a3e977344fa9228a3c9 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.301160 14748 tablet_replica.cc:333] T f4e53ea5cf694db697fa0e474da54423 P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.301867 14748 raft_consensus.cc:2243] T f4e53ea5cf694db697fa0e474da54423 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.302362 14748 raft_consensus.cc:2272] T f4e53ea5cf694db697fa0e474da54423 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:31.302767 24493 consensus_peers.cc:597] T 05840bb250764e02998cdddc290e5139 P 838cb9997b0f437989d3f4e3e9904390 -> Peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Couldn't send request to peer 32088c942fbb48239e0eed952d4df449. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:31.304610 14748 tablet_replica.cc:333] T 72e12f99e2084675ab3f53ffb836e464 P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.305279 14748 raft_consensus.cc:2243] T 72e12f99e2084675ab3f53ffb836e464 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.305704 14748 raft_consensus.cc:2272] T 72e12f99e2084675ab3f53ffb836e464 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.307639 14748 tablet_replica.cc:333] T 84af43749e474087b0a5857a95dfb31d P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.308254 14748 raft_consensus.cc:2243] T 84af43749e474087b0a5857a95dfb31d P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.308661 14748 raft_consensus.cc:2272] T 84af43749e474087b0a5857a95dfb31d P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.310595 14748 tablet_replica.cc:333] T 54fa63b278394f569a9ecffed4dec233 P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.311133 14748 raft_consensus.cc:2243] T 54fa63b278394f569a9ecffed4dec233 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.311555 14748 raft_consensus.cc:2272] T 54fa63b278394f569a9ecffed4dec233 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.313555 14748 tablet_replica.cc:333] T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.314069 14748 raft_consensus.cc:2243] T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:31.315013 14748 raft_consensus.cc:2272] T 6ae3dc25b8484b6f99d8f41685229cb8 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.317488 14748 tablet_replica.cc:333] T ae6c1a13de5442c5bb4e58e7ce36e741 P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.318185 14748 raft_consensus.cc:2243] T ae6c1a13de5442c5bb4e58e7ce36e741 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.318706 14748 raft_consensus.cc:2272] T ae6c1a13de5442c5bb4e58e7ce36e741 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:31.318948 24493 consensus_peers.cc:597] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390 -> Peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Couldn't send request to peer 32088c942fbb48239e0eed952d4df449. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:31.320919 14748 tablet_replica.cc:333] T aac1b62e27a44fb5b9a23cfc42a5d00b P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.321580 14748 raft_consensus.cc:2243] T aac1b62e27a44fb5b9a23cfc42a5d00b P 32088c942fbb48239e0eed952d4df449 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:31.322350 14748 raft_consensus.cc:2272] T aac1b62e27a44fb5b9a23cfc42a5d00b P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.324551 14748 tablet_replica.cc:333] T 93b6310306174b4ca68b467ea7f5d23e P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.325107 14748 raft_consensus.cc:2243] T 93b6310306174b4ca68b467ea7f5d23e P 32088c942fbb48239e0eed952d4df449 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:31.325853 14748 raft_consensus.cc:2272] T 93b6310306174b4ca68b467ea7f5d23e P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.328658 14748 tablet_replica.cc:333] T 05840bb250764e02998cdddc290e5139 P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.329380 14748 raft_consensus.cc:2243] T 05840bb250764e02998cdddc290e5139 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.329789 14748 raft_consensus.cc:2272] T 05840bb250764e02998cdddc290e5139 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:31.330765 24493 consensus_peers.cc:597] T 72e12f99e2084675ab3f53ffb836e464 P 838cb9997b0f437989d3f4e3e9904390 -> Peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Couldn't send request to peer 32088c942fbb48239e0eed952d4df449. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:31.332083 14748 tablet_replica.cc:333] T db9e8fa567ab481c9b80ae8e30acd628 P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.332935 14748 raft_consensus.cc:2243] T db9e8fa567ab481c9b80ae8e30acd628 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.333392 14748 raft_consensus.cc:2272] T db9e8fa567ab481c9b80ae8e30acd628 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:31.333958 24493 consensus_peers.cc:597] T f4e53ea5cf694db697fa0e474da54423 P 838cb9997b0f437989d3f4e3e9904390 -> Peer c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177): Couldn't send request to peer c40e7ba102224d24a22ccd1f68aaae45. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:46177: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:31.336239 14748 tablet_replica.cc:333] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.336903 14748 raft_consensus.cc:2243] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.337571 14748 raft_consensus.cc:2272] T 90d9e64f97544895b9b97dfdc5ceb359 P 32088c942fbb48239e0eed952d4df449 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:31.338469 24419 consensus_peers.cc:597] T 9b7cd923257143648cfe6a296e18b642 P 32088c942fbb48239e0eed952d4df449 -> Peer c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177): Couldn't send request to peer c40e7ba102224d24a22ccd1f68aaae45. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:46177: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:31.340081 14748 tablet_replica.cc:333] T b147866f4e5f4596b0b3989418cc4b0a P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.340678 14748 raft_consensus.cc:2243] T b147866f4e5f4596b0b3989418cc4b0a P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.341092 14748 raft_consensus.cc:2272] T b147866f4e5f4596b0b3989418cc4b0a P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.342914 14748 tablet_replica.cc:333] T 836ed40e5bb8485cb93cb03bc2508265 P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.343444 14748 raft_consensus.cc:2243] T 836ed40e5bb8485cb93cb03bc2508265 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.343851 14748 raft_consensus.cc:2272] T 836ed40e5bb8485cb93cb03bc2508265 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.345721 14748 tablet_replica.cc:333] T 2deba793363c4afca7d75b2fc3814304 P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.346268 14748 raft_consensus.cc:2243] T 2deba793363c4afca7d75b2fc3814304 P 32088c942fbb48239e0eed952d4df449 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:31.347021 14748 raft_consensus.cc:2272] T 2deba793363c4afca7d75b2fc3814304 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.349355 14748 tablet_replica.cc:333] T 72d2665a42a94a82b1ceec249960fdf6 P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.349983 14748 raft_consensus.cc:2243] T 72d2665a42a94a82b1ceec249960fdf6 P 32088c942fbb48239e0eed952d4df449 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:31.350682 14748 raft_consensus.cc:2272] T 72d2665a42a94a82b1ceec249960fdf6 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.352700 14748 tablet_replica.cc:333] T 6427fa143f574eaba2ee4c979e5485fc P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.353513 14748 raft_consensus.cc:2243] T 6427fa143f574eaba2ee4c979e5485fc P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.354061 14748 raft_consensus.cc:2272] T 6427fa143f574eaba2ee4c979e5485fc P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.356851 14748 tablet_replica.cc:333] T 48aea62663ac40998d7f09389b93d586 P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.357556 14748 raft_consensus.cc:2243] T 48aea62663ac40998d7f09389b93d586 P 32088c942fbb48239e0eed952d4df449 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:31.358493 14748 raft_consensus.cc:2272] T 48aea62663ac40998d7f09389b93d586 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.361014 14748 tablet_replica.cc:333] T 9b7cd923257143648cfe6a296e18b642 P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.361670 14748 raft_consensus.cc:2243] T 9b7cd923257143648cfe6a296e18b642 P 32088c942fbb48239e0eed952d4df449 [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:09:31.362293 24493 consensus_peers.cc:597] T 363cde7d4cb84353a74a18cbd9701fa7 P 838cb9997b0f437989d3f4e3e9904390 -> Peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Couldn't send request to peer 32088c942fbb48239e0eed952d4df449. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:31.362676 14748 raft_consensus.cc:2272] T 9b7cd923257143648cfe6a296e18b642 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.364790 14748 tablet_replica.cc:333] T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.365566 14748 raft_consensus.cc:2243] T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:31.366459 14748 raft_consensus.cc:2272] T 4f32a46373be4a29b6ac72efa35494b4 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.369174 14748 tablet_replica.cc:333] T c60b42ac6d7e48bfae725dc113758659 P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.370069 14748 raft_consensus.cc:2243] T c60b42ac6d7e48bfae725dc113758659 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.370586 14748 raft_consensus.cc:2272] T c60b42ac6d7e48bfae725dc113758659 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.373172 14748 tablet_replica.cc:333] T 958ccd87b65945468e9247c40ca27c33 P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.373960 14748 raft_consensus.cc:2243] T 958ccd87b65945468e9247c40ca27c33 P 32088c942fbb48239e0eed952d4df449 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:31.374935 14748 raft_consensus.cc:2272] T 958ccd87b65945468e9247c40ca27c33 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.377663 14748 tablet_replica.cc:333] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.378315 14748 raft_consensus.cc:2243] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.378818 14748 raft_consensus.cc:2272] T 9c88d903166c4812a88653e389ab5776 P 32088c942fbb48239e0eed952d4df449 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.381044 14748 tablet_replica.cc:333] T 4f085ebc2da04fbb87d48e4f206e61f0 P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.381611 14748 raft_consensus.cc:2243] T 4f085ebc2da04fbb87d48e4f206e61f0 P 32088c942fbb48239e0eed952d4df449 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:31.382390 14748 raft_consensus.cc:2272] T 4f085ebc2da04fbb87d48e4f206e61f0 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.384466 14748 tablet_replica.cc:333] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.384948 14748 raft_consensus.cc:2243] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 32088c942fbb48239e0eed952d4df449 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:31.385661 14748 raft_consensus.cc:2272] T a3b35ea0300f4b9e9afa80cf1dc9ba0f P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.387494 14748 tablet_replica.cc:333] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.388001 14748 raft_consensus.cc:2243] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.388520 14748 raft_consensus.cc:2272] T 8775bc34e96d48ae8f5598affd26efdc P 32088c942fbb48239e0eed952d4df449 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.390978 14748 tablet_replica.cc:333] T 373185ca8ab04fdb91c7947ec0668808 P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.391654 14748 raft_consensus.cc:2243] T 373185ca8ab04fdb91c7947ec0668808 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.392274 14748 raft_consensus.cc:2272] T 373185ca8ab04fdb91c7947ec0668808 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.394428 14748 tablet_replica.cc:333] T e4dd51cda55249b48d7bfa39d5feb398 P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.395073 14748 raft_consensus.cc:2243] T e4dd51cda55249b48d7bfa39d5feb398 P 32088c942fbb48239e0eed952d4df449 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:31.396040 14748 raft_consensus.cc:2272] T e4dd51cda55249b48d7bfa39d5feb398 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:31.398217 24493 consensus_peers.cc:597] T 2039124a6d8848149440a70100af8755 P 838cb9997b0f437989d3f4e3e9904390 -> Peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Couldn't send request to peer 32088c942fbb48239e0eed952d4df449. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:31.398519 14748 tablet_replica.cc:333] T 861acffd0d804c8fbd1a63771d7937c7 P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.399127 14748 raft_consensus.cc:2243] T 861acffd0d804c8fbd1a63771d7937c7 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.399554 14748 raft_consensus.cc:2272] T 861acffd0d804c8fbd1a63771d7937c7 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.401533 14748 tablet_replica.cc:333] T 2039124a6d8848149440a70100af8755 P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.402093 14748 raft_consensus.cc:2243] T 2039124a6d8848149440a70100af8755 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.402490 14748 raft_consensus.cc:2272] T 2039124a6d8848149440a70100af8755 P 32088c942fbb48239e0eed952d4df449 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.404346 14748 tablet_replica.cc:333] T 722754340deb4df18b716dc801c35c34 P 32088c942fbb48239e0eed952d4df449: stopping tablet replica
I20260709 13:09:31.404786 14748 raft_consensus.cc:2243] T 722754340deb4df18b716dc801c35c34 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.405193 14748 raft_consensus.cc:2272] T 722754340deb4df18b716dc801c35c34 P 32088c942fbb48239e0eed952d4df449 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.463006 14748 tablet_server.cc:196] TabletServer@127.14.103.3:0 shutdown complete.
I20260709 13:09:31.464320 25421 raft_consensus.cc:493] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:31.464876 25421 raft_consensus.cc:515] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:31.466668 25421 leader_election.cc:290] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
W20260709 13:09:31.470602 24494 leader_election.cc:336] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Network error: Client connection negotiation failed: client connection to 127.14.103.1:45351: connect: Connection refused (error 111)
W20260709 13:09:31.471655 24493 leader_election.cc:336] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177): Network error: Client connection negotiation failed: client connection to 127.14.103.2:46177: connect: Connection refused (error 111)
I20260709 13:09:31.472101 24493 leader_election.cc:304] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390; no voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:31.472811 25421 raft_consensus.cc:2749] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:09:31.489096 24493 consensus_peers.cc:597] T 836ed40e5bb8485cb93cb03bc2508265 P 838cb9997b0f437989d3f4e3e9904390 -> Peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Couldn't send request to peer 32088c942fbb48239e0eed952d4df449. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.3:39679: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:31.493705 24493 consensus_peers.cc:597] T f4e53ea5cf694db697fa0e474da54423 P 838cb9997b0f437989d3f4e3e9904390 -> Peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Couldn't send request to peer 32088c942fbb48239e0eed952d4df449. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.3:39679: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:31.518864 25421 raft_consensus.cc:493] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:31.519493 25421 raft_consensus.cc:515] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } } peers { permanent_uuid: "c40e7ba102224d24a22ccd1f68aaae45" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 46177 } }
I20260709 13:09:31.521929 25421 leader_election.cc:290] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351), c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177)
W20260709 13:09:31.526507 24494 leader_election.cc:336] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Network error: Client connection negotiation failed: client connection to 127.14.103.1:45351: connect: Connection refused (error 111)
W20260709 13:09:31.528481 24493 leader_election.cc:336] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c40e7ba102224d24a22ccd1f68aaae45 (127.14.103.2:46177): Network error: Client connection negotiation failed: client connection to 127.14.103.2:46177: connect: Connection refused (error 111)
I20260709 13:09:31.528843 24493 leader_election.cc:304] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390; no voters: 9c2a14143eb744cf83bfa5f46284a381, c40e7ba102224d24a22ccd1f68aaae45
I20260709 13:09:31.529340 25438 raft_consensus.cc:493] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:31.529534 25421 raft_consensus.cc:2749] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:09:31.529940 25438 raft_consensus.cc:515] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:31.532428 25438 leader_election.cc:290] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
W20260709 13:09:31.534132 24493 consensus_peers.cc:597] T 84af43749e474087b0a5857a95dfb31d P 838cb9997b0f437989d3f4e3e9904390 -> Peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Couldn't send request to peer 32088c942fbb48239e0eed952d4df449. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.3:39679: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:31.536547 24493 consensus_peers.cc:597] T 624f99e3165f49babbfce35effc9d627 P 838cb9997b0f437989d3f4e3e9904390 -> Peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Couldn't send request to peer 32088c942fbb48239e0eed952d4df449. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.3:39679: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:31.536857 24494 leader_election.cc:336] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Network error: Client connection negotiation failed: client connection to 127.14.103.1:45351: connect: Connection refused (error 111)
W20260709 13:09:31.537575 24493 leader_election.cc:336] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Network error: Client connection negotiation failed: client connection to 127.14.103.3:39679: connect: Connection refused (error 111)
I20260709 13:09:31.538069 24493 leader_election.cc:304] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390; no voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:31.538795 25438 raft_consensus.cc:2749] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:31.540563 14748 tablet_server.cc:179] TabletServer@127.14.103.4:0 shutting down...
I20260709 13:09:31.594992 14748 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:09:31.596294 14748 tablet_replica.cc:333] T 5b8d15473a964a0c8b8c283c4552706e P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.597470 14748 raft_consensus.cc:2243] T 5b8d15473a964a0c8b8c283c4552706e P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.598742 14748 raft_consensus.cc:2272] T 5b8d15473a964a0c8b8c283c4552706e P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.601668 14748 tablet_replica.cc:333] T 86262701775a456dacd47fcdc1a5d13b P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.602341 14748 raft_consensus.cc:2243] T 86262701775a456dacd47fcdc1a5d13b P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:31.603274 14748 raft_consensus.cc:2272] T 86262701775a456dacd47fcdc1a5d13b P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.605768 14748 tablet_replica.cc:333] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.606506 14748 raft_consensus.cc:2243] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:31.607483 14748 raft_consensus.cc:2272] T 3cf2c3be9fd244d4b9d4703d424d3c5d P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:31.609709 24493 consensus_peers.cc:597] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390 -> Peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Couldn't send request to peer 32088c942fbb48239e0eed952d4df449. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.3:39679: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:31.610234 14748 tablet_replica.cc:333] T edd24afe4d7847e3980c6e586b9dac5e P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.610898 14748 raft_consensus.cc:2243] T edd24afe4d7847e3980c6e586b9dac5e P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.611318 14748 raft_consensus.cc:2272] T edd24afe4d7847e3980c6e586b9dac5e P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.613356 14748 tablet_replica.cc:333] T 84af43749e474087b0a5857a95dfb31d P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.613920 14748 raft_consensus.cc:2243] T 84af43749e474087b0a5857a95dfb31d P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:31.614641 14748 raft_consensus.cc:2272] T 84af43749e474087b0a5857a95dfb31d P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.616897 14748 tablet_replica.cc:333] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.617475 14748 raft_consensus.cc:2243] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.617889 14748 raft_consensus.cc:2272] T b88feffc0859413482b0daf562d2f86a P 838cb9997b0f437989d3f4e3e9904390 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.619846 14748 tablet_replica.cc:333] T 363cde7d4cb84353a74a18cbd9701fa7 P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.620420 14748 raft_consensus.cc:2243] T 363cde7d4cb84353a74a18cbd9701fa7 P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:31.621176 14748 raft_consensus.cc:2272] T 363cde7d4cb84353a74a18cbd9701fa7 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.623171 14748 tablet_replica.cc:333] T 624f99e3165f49babbfce35effc9d627 P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.623751 14748 raft_consensus.cc:2243] T 624f99e3165f49babbfce35effc9d627 P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:31.624470 14748 raft_consensus.cc:2272] T 624f99e3165f49babbfce35effc9d627 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.626276 14748 tablet_replica.cc:333] T a1b91f42722c4116950ce80bc02eccd0 P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.626777 14748 raft_consensus.cc:2243] T a1b91f42722c4116950ce80bc02eccd0 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.627116 14748 raft_consensus.cc:2272] T a1b91f42722c4116950ce80bc02eccd0 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.629112 14748 tablet_replica.cc:333] T 72e12f99e2084675ab3f53ffb836e464 P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.629721 14748 raft_consensus.cc:2243] T 72e12f99e2084675ab3f53ffb836e464 P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:31.630437 14748 raft_consensus.cc:2272] T 72e12f99e2084675ab3f53ffb836e464 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.632257 14748 tablet_replica.cc:333] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.632782 14748 raft_consensus.cc:2243] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.633198 14748 raft_consensus.cc:2272] T 40cf5b30d13f4c3e86a9aa657c20c650 P 838cb9997b0f437989d3f4e3e9904390 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.634927 14748 tablet_replica.cc:333] T 5bb19353ba334eac831ae2cce277298a P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.635478 14748 raft_consensus.cc:2243] T 5bb19353ba334eac831ae2cce277298a P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.635921 14748 raft_consensus.cc:2272] T 5bb19353ba334eac831ae2cce277298a P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.637761 14748 tablet_replica.cc:333] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.638394 14748 raft_consensus.cc:2243] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.638803 14748 raft_consensus.cc:2272] T 40dd44ec1fb347f49c31a8b650ab1639 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.640635 14748 tablet_replica.cc:333] T 93b6310306174b4ca68b467ea7f5d23e P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.641191 14748 raft_consensus.cc:2243] T 93b6310306174b4ca68b467ea7f5d23e P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.641590 14748 raft_consensus.cc:2272] T 93b6310306174b4ca68b467ea7f5d23e P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.643446 14748 tablet_replica.cc:333] T aac1b62e27a44fb5b9a23cfc42a5d00b P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.644017 14748 raft_consensus.cc:2243] T aac1b62e27a44fb5b9a23cfc42a5d00b P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.644424 14748 raft_consensus.cc:2272] T aac1b62e27a44fb5b9a23cfc42a5d00b P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.646214 14748 tablet_replica.cc:333] T 05840bb250764e02998cdddc290e5139 P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.646777 14748 raft_consensus.cc:2243] T 05840bb250764e02998cdddc290e5139 P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:31.647562 14748 raft_consensus.cc:2272] T 05840bb250764e02998cdddc290e5139 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.649454 14748 tablet_replica.cc:333] T ae6c1a13de5442c5bb4e58e7ce36e741 P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.649991 14748 raft_consensus.cc:2243] T ae6c1a13de5442c5bb4e58e7ce36e741 P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:31.650707 14748 raft_consensus.cc:2272] T ae6c1a13de5442c5bb4e58e7ce36e741 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.652588 14748 tablet_replica.cc:333] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.653107 14748 raft_consensus.cc:2243] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.653503 14748 raft_consensus.cc:2272] T e2b4c67e0f6b4a1e9cfb3703e43e988f P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.655323 14748 tablet_replica.cc:333] T c6af3489742e4539866e5a3b10761ca2 P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.655880 14748 raft_consensus.cc:2243] T c6af3489742e4539866e5a3b10761ca2 P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:31.656682 14748 raft_consensus.cc:2272] T c6af3489742e4539866e5a3b10761ca2 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.658592 14748 tablet_replica.cc:333] T 48aea62663ac40998d7f09389b93d586 P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.659144 14748 raft_consensus.cc:2243] T 48aea62663ac40998d7f09389b93d586 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.659569 14748 raft_consensus.cc:2272] T 48aea62663ac40998d7f09389b93d586 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.661226 25444 raft_consensus.cc:493] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9c2a14143eb744cf83bfa5f46284a381)
I20260709 13:09:31.661674 14748 tablet_replica.cc:333] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.661716 25444 raft_consensus.cc:515] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32088c942fbb48239e0eed952d4df449" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 39679 } } peers { permanent_uuid: "838cb9997b0f437989d3f4e3e9904390" member_type: VOTER last_known_addr { host: "127.14.103.4" port: 33251 } } peers { permanent_uuid: "9c2a14143eb744cf83bfa5f46284a381" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 45351 } }
I20260709 13:09:31.662685 14748 raft_consensus.cc:2243] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:31.663698 14748 raft_consensus.cc:2272] T 6427fa143f574eaba2ee4c979e5485fc P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.664103 25444 leader_election.cc:290] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679), 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351)
I20260709 13:09:31.666666 14748 tablet_replica.cc:333] T 0f795acd16fd472b8c9360fb14e7e5d7 P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.667550 14748 raft_consensus.cc:2243] T 0f795acd16fd472b8c9360fb14e7e5d7 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:09:31.668037 24493 leader_election.cc:336] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 32088c942fbb48239e0eed952d4df449 (127.14.103.3:39679): Network error: Client connection negotiation failed: client connection to 127.14.103.3:39679: connect: Connection refused (error 111)
I20260709 13:09:31.668133 14748 raft_consensus.cc:2272] T 0f795acd16fd472b8c9360fb14e7e5d7 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:31.668596 24494 leader_election.cc:336] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 9c2a14143eb744cf83bfa5f46284a381 (127.14.103.1:45351): Network error: Client connection negotiation failed: client connection to 127.14.103.1:45351: connect: Connection refused (error 111)
I20260709 13:09:31.668965 24494 leader_election.cc:304] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 838cb9997b0f437989d3f4e3e9904390; no voters: 32088c942fbb48239e0eed952d4df449, 9c2a14143eb744cf83bfa5f46284a381
I20260709 13:09:31.669545 25444 raft_consensus.cc:2749] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:31.670835 14748 tablet_replica.cc:333] T 5eaa917e65d9403dbb241f84ee96ba9d P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.671401 14748 raft_consensus.cc:2243] T 5eaa917e65d9403dbb241f84ee96ba9d P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.671829 14748 raft_consensus.cc:2272] T 5eaa917e65d9403dbb241f84ee96ba9d P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.673826 14748 tablet_replica.cc:333] T f4e53ea5cf694db697fa0e474da54423 P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.674278 14748 raft_consensus.cc:2243] T f4e53ea5cf694db697fa0e474da54423 P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:31.674938 14748 raft_consensus.cc:2272] T f4e53ea5cf694db697fa0e474da54423 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.676859 14748 tablet_replica.cc:333] T 53b7256c3dd84f5b952d1f9d47691f4e P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.677305 14748 raft_consensus.cc:2243] T 53b7256c3dd84f5b952d1f9d47691f4e P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.677639 14748 raft_consensus.cc:2272] T 53b7256c3dd84f5b952d1f9d47691f4e P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.679283 14748 tablet_replica.cc:333] T 836ed40e5bb8485cb93cb03bc2508265 P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.679807 14748 raft_consensus.cc:2243] T 836ed40e5bb8485cb93cb03bc2508265 P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:31.680469 14748 raft_consensus.cc:2272] T 836ed40e5bb8485cb93cb03bc2508265 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.682149 14748 tablet_replica.cc:333] T 2deba793363c4afca7d75b2fc3814304 P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.682611 14748 raft_consensus.cc:2243] T 2deba793363c4afca7d75b2fc3814304 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.682933 14748 raft_consensus.cc:2272] T 2deba793363c4afca7d75b2fc3814304 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.684682 14748 tablet_replica.cc:333] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.685127 14748 raft_consensus.cc:2243] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.685472 14748 raft_consensus.cc:2272] T 1adea9b113494bed8c9d57fdbee6e22b P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.687242 14748 tablet_replica.cc:333] T 72d2665a42a94a82b1ceec249960fdf6 P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.687773 14748 raft_consensus.cc:2243] T 72d2665a42a94a82b1ceec249960fdf6 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.688138 14748 raft_consensus.cc:2272] T 72d2665a42a94a82b1ceec249960fdf6 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.689756 14748 tablet_replica.cc:333] T 9c88d903166c4812a88653e389ab5776 P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.690193 14748 raft_consensus.cc:2243] T 9c88d903166c4812a88653e389ab5776 P 838cb9997b0f437989d3f4e3e9904390 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.690562 14748 raft_consensus.cc:2272] T 9c88d903166c4812a88653e389ab5776 P 838cb9997b0f437989d3f4e3e9904390 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.692279 14748 tablet_replica.cc:333] T 4f085ebc2da04fbb87d48e4f206e61f0 P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.692724 14748 raft_consensus.cc:2243] T 4f085ebc2da04fbb87d48e4f206e61f0 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.693064 14748 raft_consensus.cc:2272] T 4f085ebc2da04fbb87d48e4f206e61f0 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.694842 14748 tablet_replica.cc:333] T 3e71fd85d9424a08aed292991bccf5ee P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.695336 14748 raft_consensus.cc:2243] T 3e71fd85d9424a08aed292991bccf5ee P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:31.696058 14748 raft_consensus.cc:2272] T 3e71fd85d9424a08aed292991bccf5ee P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.697881 14748 tablet_replica.cc:333] T 958ccd87b65945468e9247c40ca27c33 P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.698328 14748 raft_consensus.cc:2243] T 958ccd87b65945468e9247c40ca27c33 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.698668 14748 raft_consensus.cc:2272] T 958ccd87b65945468e9247c40ca27c33 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.700433 14748 tablet_replica.cc:333] T 142bbcbf57124963866a3967c8adebed P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.700897 14748 raft_consensus.cc:2243] T 142bbcbf57124963866a3967c8adebed P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.701243 14748 raft_consensus.cc:2272] T 142bbcbf57124963866a3967c8adebed P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.702977 14748 tablet_replica.cc:333] T 2039124a6d8848149440a70100af8755 P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.703387 14748 raft_consensus.cc:2243] T 2039124a6d8848149440a70100af8755 P 838cb9997b0f437989d3f4e3e9904390 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:31.704061 14748 raft_consensus.cc:2272] T 2039124a6d8848149440a70100af8755 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.705842 14748 tablet_replica.cc:333] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.706267 14748 raft_consensus.cc:2243] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.706627 14748 raft_consensus.cc:2272] T 722754340deb4df18b716dc801c35c34 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.708454 14748 tablet_replica.cc:333] T db9e8fa567ab481c9b80ae8e30acd628 P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.708942 14748 raft_consensus.cc:2243] T db9e8fa567ab481c9b80ae8e30acd628 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.709316 14748 raft_consensus.cc:2272] T db9e8fa567ab481c9b80ae8e30acd628 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.711331 14748 tablet_replica.cc:333] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.711793 14748 raft_consensus.cc:2243] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.712210 14748 raft_consensus.cc:2272] T 42bcfb18b77e41afa60057ccfd718fa9 P 838cb9997b0f437989d3f4e3e9904390 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.714025 14748 tablet_replica.cc:333] T 4af7fc369d124452beb5fd07cf8592ec P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.714454 14748 raft_consensus.cc:2243] T 4af7fc369d124452beb5fd07cf8592ec P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.714798 14748 raft_consensus.cc:2272] T 4af7fc369d124452beb5fd07cf8592ec P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.716737 14748 tablet_replica.cc:333] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.717157 14748 raft_consensus.cc:2243] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.717514 14748 raft_consensus.cc:2272] T 861acffd0d804c8fbd1a63771d7937c7 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.719215 14748 tablet_replica.cc:333] T e4dd51cda55249b48d7bfa39d5feb398 P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.719631 14748 raft_consensus.cc:2243] T e4dd51cda55249b48d7bfa39d5feb398 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.719995 14748 raft_consensus.cc:2272] T e4dd51cda55249b48d7bfa39d5feb398 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.721669 14748 tablet_replica.cc:333] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.722090 14748 raft_consensus.cc:2243] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:31.722762 14748 raft_consensus.cc:2272] T c60b42ac6d7e48bfae725dc113758659 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.724478 14748 tablet_replica.cc:333] T a3e3b771ce2c4165aad02954773c58d7 P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.724875 14748 raft_consensus.cc:2243] T a3e3b771ce2c4165aad02954773c58d7 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.725211 14748 raft_consensus.cc:2272] T a3e3b771ce2c4165aad02954773c58d7 P 838cb9997b0f437989d3f4e3e9904390 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.726969 14748 tablet_replica.cc:333] T 9711acbe7d044647bf774ee849c8a032 P 838cb9997b0f437989d3f4e3e9904390: stopping tablet replica
I20260709 13:09:31.727337 14748 raft_consensus.cc:2243] T 9711acbe7d044647bf774ee849c8a032 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:31.727700 14748 raft_consensus.cc:2272] T 9711acbe7d044647bf774ee849c8a032 P 838cb9997b0f437989d3f4e3e9904390 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.791736 14748 tablet_server.cc:196] TabletServer@127.14.103.4:0 shutdown complete.
I20260709 13:09:31.853925 14748 master.cc:562] Master@127.14.103.62:41611 shutting down...
I20260709 13:09:31.878284 14748 raft_consensus.cc:2243] T 00000000000000000000000000000000 P a20e5c2ad87d4af086eb0cf554c1d829 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:31.878846 14748 raft_consensus.cc:2272] T 00000000000000000000000000000000 P a20e5c2ad87d4af086eb0cf554c1d829 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:31.879128 14748 tablet_replica.cc:333] T 00000000000000000000000000000000 P a20e5c2ad87d4af086eb0cf554c1d829: stopping tablet replica
I20260709 13:09:31.901777 14748 master.cc:584] Master@127.14.103.62:41611 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (27515 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 13:09:31.996243 14748 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.103.62:33155
I20260709 13:09:31.997504 14748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:32.003590 25446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:32.003630 25445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:32.004563 25448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:32.004805 14748 server_base.cc:1061] running on GCE node
I20260709 13:09:32.005860 14748 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:32.006039 14748 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:32.006150 14748 hybrid_clock.cc:648] HybridClock initialized: now 1783602572006139 us; error 0 us; skew 500 ppm
I20260709 13:09:32.006616 14748 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:32.008971 14748 webserver.cc:533] Webserver started at http://127.14.103.62:38707/ using document root <none> and password file <none>
I20260709 13:09:32.009408 14748 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:32.009568 14748 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:32.009773 14748 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:32.010830 14748 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602247531521-14748-0/minicluster-data/master-0-root/instance:
uuid: "47dfa972db9b4c5f8973bcbc1b8e3320"
format_stamp: "Formatted at 2026-07-09 13:09:32 on dist-test-slave-fpp7"
I20260709 13:09:32.015084 14748 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 13:09:32.018352 25453 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:32.019096 14748 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:09:32.019402 14748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602247531521-14748-0/minicluster-data/master-0-root
uuid: "47dfa972db9b4c5f8973bcbc1b8e3320"
format_stamp: "Formatted at 2026-07-09 13:09:32 on dist-test-slave-fpp7"
I20260709 13:09:32.019680 14748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602247531521-14748-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602247531521-14748-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602247531521-14748-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:32.054488 14748 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:32.055768 14748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:32.093676 14748 rpc_server.cc:307] RPC server started. Bound to: 127.14.103.62:33155
I20260709 13:09:32.093783 25504 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.103.62:33155 every 8 connection(s)
I20260709 13:09:32.097522 25505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:32.107882 25505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47dfa972db9b4c5f8973bcbc1b8e3320: Bootstrap starting.
I20260709 13:09:32.112594 25505 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47dfa972db9b4c5f8973bcbc1b8e3320: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:32.116735 25505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47dfa972db9b4c5f8973bcbc1b8e3320: No bootstrap required, opened a new log
I20260709 13:09:32.118817 25505 raft_consensus.cc:359] T 00000000000000000000000000000000 P 47dfa972db9b4c5f8973bcbc1b8e3320 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47dfa972db9b4c5f8973bcbc1b8e3320" member_type: VOTER }
I20260709 13:09:32.119169 25505 raft_consensus.cc:385] T 00000000000000000000000000000000 P 47dfa972db9b4c5f8973bcbc1b8e3320 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:32.119349 25505 raft_consensus.cc:740] T 00000000000000000000000000000000 P 47dfa972db9b4c5f8973bcbc1b8e3320 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47dfa972db9b4c5f8973bcbc1b8e3320, State: Initialized, Role: FOLLOWER
I20260709 13:09:32.119867 25505 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47dfa972db9b4c5f8973bcbc1b8e3320 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47dfa972db9b4c5f8973bcbc1b8e3320" member_type: VOTER }
I20260709 13:09:32.120262 25505 raft_consensus.cc:399] T 00000000000000000000000000000000 P 47dfa972db9b4c5f8973bcbc1b8e3320 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:09:32.120457 25505 raft_consensus.cc:493] T 00000000000000000000000000000000 P 47dfa972db9b4c5f8973bcbc1b8e3320 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:09:32.120662 25505 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 47dfa972db9b4c5f8973bcbc1b8e3320 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:32.125123 25505 raft_consensus.cc:515] T 00000000000000000000000000000000 P 47dfa972db9b4c5f8973bcbc1b8e3320 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47dfa972db9b4c5f8973bcbc1b8e3320" member_type: VOTER }
I20260709 13:09:32.125586 25505 leader_election.cc:304] T 00000000000000000000000000000000 P 47dfa972db9b4c5f8973bcbc1b8e3320 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47dfa972db9b4c5f8973bcbc1b8e3320; no voters: 
I20260709 13:09:32.126653 25505 leader_election.cc:290] T 00000000000000000000000000000000 P 47dfa972db9b4c5f8973bcbc1b8e3320 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:09:32.127138 25508 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 47dfa972db9b4c5f8973bcbc1b8e3320 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:32.128692 25508 raft_consensus.cc:697] T 00000000000000000000000000000000 P 47dfa972db9b4c5f8973bcbc1b8e3320 [term 1 LEADER]: Becoming Leader. State: Replica: 47dfa972db9b4c5f8973bcbc1b8e3320, State: Running, Role: LEADER
I20260709 13:09:32.129542 25505 sys_catalog.cc:565] T 00000000000000000000000000000000 P 47dfa972db9b4c5f8973bcbc1b8e3320 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:09:32.129446 25508 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47dfa972db9b4c5f8973bcbc1b8e3320 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47dfa972db9b4c5f8973bcbc1b8e3320" member_type: VOTER }
I20260709 13:09:32.134931 25510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47dfa972db9b4c5f8973bcbc1b8e3320 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47dfa972db9b4c5f8973bcbc1b8e3320. Latest consensus state: current_term: 1 leader_uuid: "47dfa972db9b4c5f8973bcbc1b8e3320" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47dfa972db9b4c5f8973bcbc1b8e3320" member_type: VOTER } }
I20260709 13:09:32.135711 25510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47dfa972db9b4c5f8973bcbc1b8e3320 [sys.catalog]: This master's current role is: LEADER
I20260709 13:09:32.135502 25509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47dfa972db9b4c5f8973bcbc1b8e3320 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47dfa972db9b4c5f8973bcbc1b8e3320" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47dfa972db9b4c5f8973bcbc1b8e3320" member_type: VOTER } }
I20260709 13:09:32.136575 25509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47dfa972db9b4c5f8973bcbc1b8e3320 [sys.catalog]: This master's current role is: LEADER
I20260709 13:09:32.138120 25517 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:09:32.144845 25517 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:09:32.145972 14748 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:09:32.153877 25517 catalog_manager.cc:1383] Generated new cluster ID: d373dd4ffd3f49e2b852cacba730e5d9
I20260709 13:09:32.154172 25517 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:09:32.173038 25517 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:09:32.174311 25517 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:09:32.193883 25517 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 47dfa972db9b4c5f8973bcbc1b8e3320: Generated new TSK 0
I20260709 13:09:32.194497 25517 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:09:32.212875 14748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:32.218912 25526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:32.219921 25527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:32.221735 25529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:32.222041 14748 server_base.cc:1061] running on GCE node
I20260709 13:09:32.222982 14748 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:32.223203 14748 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:32.223380 14748 hybrid_clock.cc:648] HybridClock initialized: now 1783602572223361 us; error 0 us; skew 500 ppm
I20260709 13:09:32.224009 14748 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:32.226169 14748 webserver.cc:533] Webserver started at http://127.14.103.1:34333/ using document root <none> and password file <none>
I20260709 13:09:32.226655 14748 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:32.226871 14748 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:32.227137 14748 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:32.228286 14748 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602247531521-14748-0/minicluster-data/ts-0-root/instance:
uuid: "a7e4f250aa5d48759eaabbdff8c99210"
format_stamp: "Formatted at 2026-07-09 13:09:32 on dist-test-slave-fpp7"
I20260709 13:09:32.232774 14748 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:09:32.236493 25534 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:32.237270 14748 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.004s	sys 0.000s
I20260709 13:09:32.237560 14748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602247531521-14748-0/minicluster-data/ts-0-root
uuid: "a7e4f250aa5d48759eaabbdff8c99210"
format_stamp: "Formatted at 2026-07-09 13:09:32 on dist-test-slave-fpp7"
I20260709 13:09:32.237844 14748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602247531521-14748-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602247531521-14748-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602247531521-14748-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:32.259969 14748 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:32.261178 14748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:32.262602 14748 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:09:32.264808 14748 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:09:32.265035 14748 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:32.265270 14748 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:09:32.265456 14748 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:32.304611 14748 rpc_server.cc:307] RPC server started. Bound to: 127.14.103.1:43185
I20260709 13:09:32.304713 25596 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.103.1:43185 every 8 connection(s)
I20260709 13:09:32.309713 14748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:32.317186 25601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:32.318436 25602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:32.323364 14748 server_base.cc:1061] running on GCE node
W20260709 13:09:32.323287 25604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:32.324323 25597 heartbeater.cc:344] Connected to a master server at 127.14.103.62:33155
I20260709 13:09:32.324779 25597 heartbeater.cc:461] Registering TS with master...
I20260709 13:09:32.325177 14748 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:32.325435 14748 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:32.325510 25597 heartbeater.cc:507] Master 127.14.103.62:33155 requested a full tablet report, sending...
I20260709 13:09:32.325615 14748 hybrid_clock.cc:648] HybridClock initialized: now 1783602572325582 us; error 0 us; skew 500 ppm
I20260709 13:09:32.326402 14748 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:32.327814 25470 ts_manager.cc:194] Registered new tserver with Master: a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185)
I20260709 13:09:32.329141 14748 webserver.cc:533] Webserver started at http://127.14.103.2:44335/ using document root <none> and password file <none>
I20260709 13:09:32.329679 14748 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:32.329702 25470 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33708
I20260709 13:09:32.329867 14748 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:32.330312 14748 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:32.331646 14748 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602247531521-14748-0/minicluster-data/ts-1-root/instance:
uuid: "6a8699df36b74658a40f65ad30923ba9"
format_stamp: "Formatted at 2026-07-09 13:09:32 on dist-test-slave-fpp7"
I20260709 13:09:32.336180 14748 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:09:32.339632 25609 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:32.340523 14748 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:09:32.340835 14748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602247531521-14748-0/minicluster-data/ts-1-root
uuid: "6a8699df36b74658a40f65ad30923ba9"
format_stamp: "Formatted at 2026-07-09 13:09:32 on dist-test-slave-fpp7"
I20260709 13:09:32.341135 14748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602247531521-14748-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602247531521-14748-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602247531521-14748-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:32.354537 14748 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:32.355638 14748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:32.357046 14748 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:09:32.359143 14748 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:09:32.359369 14748 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:32.359602 14748 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:09:32.359833 14748 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:09:32.398236 14748 rpc_server.cc:307] RPC server started. Bound to: 127.14.103.2:39169
I20260709 13:09:32.398350 25671 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.103.2:39169 every 8 connection(s)
I20260709 13:09:32.403000 14748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:32.409199 25675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:32.411166 25676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:32.413529 25678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:32.413764 14748 server_base.cc:1061] running on GCE node
I20260709 13:09:32.414897 14748 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:32.415164 14748 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:32.415256 25672 heartbeater.cc:344] Connected to a master server at 127.14.103.62:33155
I20260709 13:09:32.415360 14748 hybrid_clock.cc:648] HybridClock initialized: now 1783602572415318 us; error 0 us; skew 500 ppm
I20260709 13:09:32.415817 25672 heartbeater.cc:461] Registering TS with master...
I20260709 13:09:32.416201 14748 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:32.416596 25672 heartbeater.cc:507] Master 127.14.103.62:33155 requested a full tablet report, sending...
I20260709 13:09:32.418953 14748 webserver.cc:533] Webserver started at http://127.14.103.3:35489/ using document root <none> and password file <none>
I20260709 13:09:32.418998 25470 ts_manager.cc:194] Registered new tserver with Master: 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2:39169)
I20260709 13:09:32.419904 14748 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:32.420148 14748 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:32.420426 14748 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:32.421073 25470 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33722
I20260709 13:09:32.421674 14748 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602247531521-14748-0/minicluster-data/ts-2-root/instance:
uuid: "99dfff6363df46ce97b027cb3bd42e11"
format_stamp: "Formatted at 2026-07-09 13:09:32 on dist-test-slave-fpp7"
I20260709 13:09:32.426070 14748 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:09:32.429237 25683 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:32.430169 14748 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:09:32.430418 14748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602247531521-14748-0/minicluster-data/ts-2-root
uuid: "99dfff6363df46ce97b027cb3bd42e11"
format_stamp: "Formatted at 2026-07-09 13:09:32 on dist-test-slave-fpp7"
I20260709 13:09:32.430663 14748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602247531521-14748-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602247531521-14748-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602247531521-14748-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:32.442629 14748 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:32.443876 14748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:32.445274 14748 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:09:32.447389 14748 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:09:32.447587 14748 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:32.447867 14748 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:09:32.448004 14748 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:32.487991 14748 rpc_server.cc:307] RPC server started. Bound to: 127.14.103.3:34359
I20260709 13:09:32.488096 25745 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.103.3:34359 every 8 connection(s)
I20260709 13:09:32.501634 25746 heartbeater.cc:344] Connected to a master server at 127.14.103.62:33155
I20260709 13:09:32.502110 25746 heartbeater.cc:461] Registering TS with master...
I20260709 13:09:32.502833 25746 heartbeater.cc:507] Master 127.14.103.62:33155 requested a full tablet report, sending...
I20260709 13:09:32.504662 25470 ts_manager.cc:194] Registered new tserver with Master: 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3:34359)
I20260709 13:09:32.504804 14748 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013690919s
I20260709 13:09:32.506011 25470 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33738
I20260709 13:09:32.530711 25470 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33746:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:09:32.533700 25470 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:09:32.613010 25711 tablet_service.cc:1511] Processing CreateTablet for tablet fa373cebfd3540728639dddbaee4cb1d (DEFAULT_TABLE table=test-workload [id=fc0080004cf646c985e2bcd4841e07df]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:09:32.614514 25711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa373cebfd3540728639dddbaee4cb1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:32.621289 25710 tablet_service.cc:1511] Processing CreateTablet for tablet bab07f367f714676a11b512f3abbea7c (DEFAULT_TABLE table=test-workload [id=fc0080004cf646c985e2bcd4841e07df]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:09:32.622843 25710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bab07f367f714676a11b512f3abbea7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:32.634477 25637 tablet_service.cc:1511] Processing CreateTablet for tablet fa373cebfd3540728639dddbaee4cb1d (DEFAULT_TABLE table=test-workload [id=fc0080004cf646c985e2bcd4841e07df]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:09:32.634555 25709 tablet_service.cc:1511] Processing CreateTablet for tablet fc82e3058d55474faed434b751f7c644 (DEFAULT_TABLE table=test-workload [id=fc0080004cf646c985e2bcd4841e07df]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:09:32.636583 25709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc82e3058d55474faed434b751f7c644. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:32.636929 25633 tablet_service.cc:1511] Processing CreateTablet for tablet 8372ab57bb904260beb2b0c69bc39ad0 (DEFAULT_TABLE table=test-workload [id=fc0080004cf646c985e2bcd4841e07df]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:09:32.642074 25636 tablet_service.cc:1511] Processing CreateTablet for tablet bab07f367f714676a11b512f3abbea7c (DEFAULT_TABLE table=test-workload [id=fc0080004cf646c985e2bcd4841e07df]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:09:32.644407 25635 tablet_service.cc:1511] Processing CreateTablet for tablet fc82e3058d55474faed434b751f7c644 (DEFAULT_TABLE table=test-workload [id=fc0080004cf646c985e2bcd4841e07df]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:09:32.645911 25708 tablet_service.cc:1511] Processing CreateTablet for tablet 0d81fc45e39443b9b33953f8c66437eb (DEFAULT_TABLE table=test-workload [id=fc0080004cf646c985e2bcd4841e07df]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:09:32.648239 25708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d81fc45e39443b9b33953f8c66437eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:32.638758 25633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8372ab57bb904260beb2b0c69bc39ad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:32.658046 25635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc82e3058d55474faed434b751f7c644. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:32.658700 25637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa373cebfd3540728639dddbaee4cb1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:32.659392 25636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bab07f367f714676a11b512f3abbea7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:32.656219 25632 tablet_service.cc:1511] Processing CreateTablet for tablet e1f6c52711174eacb83ea8fd2ad1b57b (DEFAULT_TABLE table=test-workload [id=fc0080004cf646c985e2bcd4841e07df]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:09:32.677778 25632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1f6c52711174eacb83ea8fd2ad1b57b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:32.678848 25704 tablet_service.cc:1511] Processing CreateTablet for tablet 6d5ba3150425499f9cfa64eb06d6ba91 (DEFAULT_TABLE table=test-workload [id=fc0080004cf646c985e2bcd4841e07df]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:09:32.657881 25634 tablet_service.cc:1511] Processing CreateTablet for tablet 0d81fc45e39443b9b33953f8c66437eb (DEFAULT_TABLE table=test-workload [id=fc0080004cf646c985e2bcd4841e07df]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:09:32.661340 25631 tablet_service.cc:1511] Processing CreateTablet for tablet 2a14348b4c6a4d3cbbf18beffcd03814 (DEFAULT_TABLE table=test-workload [id=fc0080004cf646c985e2bcd4841e07df]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:09:32.680682 25704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d5ba3150425499f9cfa64eb06d6ba91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:32.662540 25707 tablet_service.cc:1511] Processing CreateTablet for tablet 8372ab57bb904260beb2b0c69bc39ad0 (DEFAULT_TABLE table=test-workload [id=fc0080004cf646c985e2bcd4841e07df]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:09:32.682410 25707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8372ab57bb904260beb2b0c69bc39ad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:32.662786 25630 tablet_service.cc:1511] Processing CreateTablet for tablet 6d5ba3150425499f9cfa64eb06d6ba91 (DEFAULT_TABLE table=test-workload [id=fc0080004cf646c985e2bcd4841e07df]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:09:32.664243 25629 tablet_service.cc:1511] Processing CreateTablet for tablet 64becd02c9434edeaf5824c5a455b0e0 (DEFAULT_TABLE table=test-workload [id=fc0080004cf646c985e2bcd4841e07df]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:09:32.665625 25628 tablet_service.cc:1511] Processing CreateTablet for tablet 377fe3ce2b474722b845c07b3f74b830 (DEFAULT_TABLE table=test-workload [id=fc0080004cf646c985e2bcd4841e07df]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:09:32.665855 25706 tablet_service.cc:1511] Processing CreateTablet for tablet e1f6c52711174eacb83ea8fd2ad1b57b (DEFAULT_TABLE table=test-workload [id=fc0080004cf646c985e2bcd4841e07df]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:09:32.686712 25631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a14348b4c6a4d3cbbf18beffcd03814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:32.686829 25706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1f6c52711174eacb83ea8fd2ad1b57b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:32.669157 25705 tablet_service.cc:1511] Processing CreateTablet for tablet 2a14348b4c6a4d3cbbf18beffcd03814 (DEFAULT_TABLE table=test-workload [id=fc0080004cf646c985e2bcd4841e07df]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:09:32.689146 25705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a14348b4c6a4d3cbbf18beffcd03814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:32.692812 25634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d81fc45e39443b9b33953f8c66437eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:32.696461 25628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 377fe3ce2b474722b845c07b3f74b830. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:32.708657 25629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64becd02c9434edeaf5824c5a455b0e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:32.727731 25761 tablet_bootstrap.cc:492] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11: Bootstrap starting.
I20260709 13:09:32.732545 25703 tablet_service.cc:1511] Processing CreateTablet for tablet 64becd02c9434edeaf5824c5a455b0e0 (DEFAULT_TABLE table=test-workload [id=fc0080004cf646c985e2bcd4841e07df]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:09:32.734000 25703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64becd02c9434edeaf5824c5a455b0e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:32.725852 25630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d5ba3150425499f9cfa64eb06d6ba91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:32.747304 25562 tablet_service.cc:1511] Processing CreateTablet for tablet fa373cebfd3540728639dddbaee4cb1d (DEFAULT_TABLE table=test-workload [id=fc0080004cf646c985e2bcd4841e07df]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:09:32.748745 25562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa373cebfd3540728639dddbaee4cb1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:32.752875 25561 tablet_service.cc:1511] Processing CreateTablet for tablet bab07f367f714676a11b512f3abbea7c (DEFAULT_TABLE table=test-workload [id=fc0080004cf646c985e2bcd4841e07df]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:09:32.754356 25561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bab07f367f714676a11b512f3abbea7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:32.756939 25560 tablet_service.cc:1511] Processing CreateTablet for tablet fc82e3058d55474faed434b751f7c644 (DEFAULT_TABLE table=test-workload [id=fc0080004cf646c985e2bcd4841e07df]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:09:32.758315 25560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc82e3058d55474faed434b751f7c644. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:32.760833 25559 tablet_service.cc:1511] Processing CreateTablet for tablet 0d81fc45e39443b9b33953f8c66437eb (DEFAULT_TABLE table=test-workload [id=fc0080004cf646c985e2bcd4841e07df]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:09:32.762187 25559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d81fc45e39443b9b33953f8c66437eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:32.764858 25558 tablet_service.cc:1511] Processing CreateTablet for tablet 8372ab57bb904260beb2b0c69bc39ad0 (DEFAULT_TABLE table=test-workload [id=fc0080004cf646c985e2bcd4841e07df]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:09:32.766217 25558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8372ab57bb904260beb2b0c69bc39ad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:32.768894 25557 tablet_service.cc:1511] Processing CreateTablet for tablet e1f6c52711174eacb83ea8fd2ad1b57b (DEFAULT_TABLE table=test-workload [id=fc0080004cf646c985e2bcd4841e07df]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:09:32.770272 25557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1f6c52711174eacb83ea8fd2ad1b57b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:32.772842 25556 tablet_service.cc:1511] Processing CreateTablet for tablet 2a14348b4c6a4d3cbbf18beffcd03814 (DEFAULT_TABLE table=test-workload [id=fc0080004cf646c985e2bcd4841e07df]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:09:32.774140 25556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a14348b4c6a4d3cbbf18beffcd03814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:32.776161 25761 tablet_bootstrap.cc:654] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:32.776889 25555 tablet_service.cc:1511] Processing CreateTablet for tablet 6d5ba3150425499f9cfa64eb06d6ba91 (DEFAULT_TABLE table=test-workload [id=fc0080004cf646c985e2bcd4841e07df]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:09:32.779181 25554 tablet_service.cc:1511] Processing CreateTablet for tablet 64becd02c9434edeaf5824c5a455b0e0 (DEFAULT_TABLE table=test-workload [id=fc0080004cf646c985e2bcd4841e07df]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:09:32.781317 25702 tablet_service.cc:1511] Processing CreateTablet for tablet 377fe3ce2b474722b845c07b3f74b830 (DEFAULT_TABLE table=test-workload [id=fc0080004cf646c985e2bcd4841e07df]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:09:32.781829 25553 tablet_service.cc:1511] Processing CreateTablet for tablet 377fe3ce2b474722b845c07b3f74b830 (DEFAULT_TABLE table=test-workload [id=fc0080004cf646c985e2bcd4841e07df]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:09:32.782642 25702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 377fe3ce2b474722b845c07b3f74b830. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:32.785162 25762 tablet_bootstrap.cc:492] T fa373cebfd3540728639dddbaee4cb1d P 6a8699df36b74658a40f65ad30923ba9: Bootstrap starting.
I20260709 13:09:32.796561 25762 tablet_bootstrap.cc:654] T fa373cebfd3540728639dddbaee4cb1d P 6a8699df36b74658a40f65ad30923ba9: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:32.856578 25554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64becd02c9434edeaf5824c5a455b0e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:32.864163 25553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 377fe3ce2b474722b845c07b3f74b830. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:32.892920 25761 tablet_bootstrap.cc:492] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11: No bootstrap required, opened a new log
I20260709 13:09:32.893520 25761 ts_tablet_manager.cc:1403] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11: Time spent bootstrapping tablet: real 0.166s	user 0.035s	sys 0.028s
I20260709 13:09:32.883239 25555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d5ba3150425499f9cfa64eb06d6ba91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:32.897384 25762 tablet_bootstrap.cc:492] T fa373cebfd3540728639dddbaee4cb1d P 6a8699df36b74658a40f65ad30923ba9: No bootstrap required, opened a new log
I20260709 13:09:32.898713 25761 raft_consensus.cc:359] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } } peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } }
I20260709 13:09:32.907166 25762 ts_tablet_manager.cc:1403] T fa373cebfd3540728639dddbaee4cb1d P 6a8699df36b74658a40f65ad30923ba9: Time spent bootstrapping tablet: real 0.122s	user 0.019s	sys 0.037s
I20260709 13:09:32.910686 25762 raft_consensus.cc:359] T fa373cebfd3540728639dddbaee4cb1d P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } } peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } }
I20260709 13:09:32.911371 25762 raft_consensus.cc:385] T fa373cebfd3540728639dddbaee4cb1d P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:32.907114 25761 raft_consensus.cc:385] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:32.914063 25762 raft_consensus.cc:740] T fa373cebfd3540728639dddbaee4cb1d P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a8699df36b74658a40f65ad30923ba9, State: Initialized, Role: FOLLOWER
I20260709 13:09:32.914690 25761 raft_consensus.cc:740] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99dfff6363df46ce97b027cb3bd42e11, State: Initialized, Role: FOLLOWER
I20260709 13:09:32.915906 25762 consensus_queue.cc:260] T fa373cebfd3540728639dddbaee4cb1d P 6a8699df36b74658a40f65ad30923ba9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } } peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } }
I20260709 13:09:32.916983 25761 consensus_queue.cc:260] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } } peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } }
I20260709 13:09:32.938088 25672 heartbeater.cc:499] Master 127.14.103.62:33155 was elected leader, sending a full tablet report...
I20260709 13:09:32.940369 25765 tablet_bootstrap.cc:492] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap starting.
I20260709 13:09:32.940794 25746 heartbeater.cc:499] Master 127.14.103.62:33155 was elected leader, sending a full tablet report...
I20260709 13:09:32.941370 25762 ts_tablet_manager.cc:1434] T fa373cebfd3540728639dddbaee4cb1d P 6a8699df36b74658a40f65ad30923ba9: Time spent starting tablet: real 0.034s	user 0.014s	sys 0.006s
I20260709 13:09:32.942528 25762 tablet_bootstrap.cc:492] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9: Bootstrap starting.
I20260709 13:09:32.948686 25762 tablet_bootstrap.cc:654] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:32.948686 25765 tablet_bootstrap.cc:654] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:32.959976 25761 ts_tablet_manager.cc:1434] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11: Time spent starting tablet: real 0.066s	user 0.016s	sys 0.008s
I20260709 13:09:32.961059 25761 tablet_bootstrap.cc:492] T bab07f367f714676a11b512f3abbea7c P 99dfff6363df46ce97b027cb3bd42e11: Bootstrap starting.
I20260709 13:09:32.961490 25762 tablet_bootstrap.cc:492] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9: No bootstrap required, opened a new log
I20260709 13:09:32.962057 25762 ts_tablet_manager.cc:1403] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9: Time spent bootstrapping tablet: real 0.020s	user 0.006s	sys 0.011s
I20260709 13:09:32.965197 25762 raft_consensus.cc:359] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:32.965926 25762 raft_consensus.cc:385] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:32.966225 25762 raft_consensus.cc:740] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a8699df36b74658a40f65ad30923ba9, State: Initialized, Role: FOLLOWER
I20260709 13:09:32.967476 25761 tablet_bootstrap.cc:654] T bab07f367f714676a11b512f3abbea7c P 99dfff6363df46ce97b027cb3bd42e11: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:32.969565 25765 tablet_bootstrap.cc:492] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210: No bootstrap required, opened a new log
I20260709 13:09:32.966986 25762 consensus_queue.cc:260] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:32.970088 25765 ts_tablet_manager.cc:1403] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210: Time spent bootstrapping tablet: real 0.030s	user 0.009s	sys 0.007s
I20260709 13:09:32.971640 25762 ts_tablet_manager.cc:1434] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
I20260709 13:09:32.972661 25762 tablet_bootstrap.cc:492] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9: Bootstrap starting.
I20260709 13:09:32.972959 25765 raft_consensus.cc:359] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:32.973729 25765 raft_consensus.cc:385] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:32.973937 25761 tablet_bootstrap.cc:492] T bab07f367f714676a11b512f3abbea7c P 99dfff6363df46ce97b027cb3bd42e11: No bootstrap required, opened a new log
I20260709 13:09:32.974035 25765 raft_consensus.cc:740] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7e4f250aa5d48759eaabbdff8c99210, State: Initialized, Role: FOLLOWER
I20260709 13:09:32.974730 25761 ts_tablet_manager.cc:1403] T bab07f367f714676a11b512f3abbea7c P 99dfff6363df46ce97b027cb3bd42e11: Time spent bootstrapping tablet: real 0.014s	user 0.011s	sys 0.001s
I20260709 13:09:32.975065 25765 consensus_queue.cc:260] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:32.977906 25761 raft_consensus.cc:359] T bab07f367f714676a11b512f3abbea7c P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:32.978680 25761 raft_consensus.cc:385] T bab07f367f714676a11b512f3abbea7c P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:32.979029 25761 raft_consensus.cc:740] T bab07f367f714676a11b512f3abbea7c P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99dfff6363df46ce97b027cb3bd42e11, State: Initialized, Role: FOLLOWER
I20260709 13:09:32.980018 25761 consensus_queue.cc:260] T bab07f367f714676a11b512f3abbea7c P 99dfff6363df46ce97b027cb3bd42e11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:32.980799 25762 tablet_bootstrap.cc:654] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:32.982563 25761 ts_tablet_manager.cc:1434] T bab07f367f714676a11b512f3abbea7c P 99dfff6363df46ce97b027cb3bd42e11: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:09:32.983701 25761 tablet_bootstrap.cc:492] T fc82e3058d55474faed434b751f7c644 P 99dfff6363df46ce97b027cb3bd42e11: Bootstrap starting.
I20260709 13:09:32.990356 25761 tablet_bootstrap.cc:654] T fc82e3058d55474faed434b751f7c644 P 99dfff6363df46ce97b027cb3bd42e11: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:33.011137 25765 ts_tablet_manager.cc:1434] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210: Time spent starting tablet: real 0.041s	user 0.012s	sys 0.027s
I20260709 13:09:33.012085 25597 heartbeater.cc:499] Master 127.14.103.62:33155 was elected leader, sending a full tablet report...
I20260709 13:09:33.013288 25762 tablet_bootstrap.cc:492] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9: No bootstrap required, opened a new log
I20260709 13:09:33.013777 25762 ts_tablet_manager.cc:1403] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9: Time spent bootstrapping tablet: real 0.041s	user 0.016s	sys 0.021s
I20260709 13:09:33.014403 25761 tablet_bootstrap.cc:492] T fc82e3058d55474faed434b751f7c644 P 99dfff6363df46ce97b027cb3bd42e11: No bootstrap required, opened a new log
I20260709 13:09:33.014947 25761 ts_tablet_manager.cc:1403] T fc82e3058d55474faed434b751f7c644 P 99dfff6363df46ce97b027cb3bd42e11: Time spent bootstrapping tablet: real 0.031s	user 0.006s	sys 0.023s
I20260709 13:09:33.016577 25765 tablet_bootstrap.cc:492] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap starting.
I20260709 13:09:33.016259 25762 raft_consensus.cc:359] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.017064 25762 raft_consensus.cc:385] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:33.017392 25762 raft_consensus.cc:740] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a8699df36b74658a40f65ad30923ba9, State: Initialized, Role: FOLLOWER
I20260709 13:09:33.017908 25761 raft_consensus.cc:359] T fc82e3058d55474faed434b751f7c644 P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.018673 25761 raft_consensus.cc:385] T fc82e3058d55474faed434b751f7c644 P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:33.018282 25762 consensus_queue.cc:260] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.018976 25761 raft_consensus.cc:740] T fc82e3058d55474faed434b751f7c644 P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99dfff6363df46ce97b027cb3bd42e11, State: Initialized, Role: FOLLOWER
I20260709 13:09:33.019904 25761 consensus_queue.cc:260] T fc82e3058d55474faed434b751f7c644 P 99dfff6363df46ce97b027cb3bd42e11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.023059 25762 ts_tablet_manager.cc:1434] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:09:33.024377 25762 tablet_bootstrap.cc:492] T fc82e3058d55474faed434b751f7c644 P 6a8699df36b74658a40f65ad30923ba9: Bootstrap starting.
I20260709 13:09:33.026540 25761 ts_tablet_manager.cc:1434] T fc82e3058d55474faed434b751f7c644 P 99dfff6363df46ce97b027cb3bd42e11: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.004s
I20260709 13:09:33.026925 25765 tablet_bootstrap.cc:654] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:33.027596 25761 tablet_bootstrap.cc:492] T 0d81fc45e39443b9b33953f8c66437eb P 99dfff6363df46ce97b027cb3bd42e11: Bootstrap starting.
I20260709 13:09:33.033785 25762 tablet_bootstrap.cc:654] T fc82e3058d55474faed434b751f7c644 P 6a8699df36b74658a40f65ad30923ba9: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:33.036314 25761 tablet_bootstrap.cc:654] T 0d81fc45e39443b9b33953f8c66437eb P 99dfff6363df46ce97b027cb3bd42e11: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:33.039887 25765 tablet_bootstrap.cc:492] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210: No bootstrap required, opened a new log
I20260709 13:09:33.040551 25765 ts_tablet_manager.cc:1403] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210: Time spent bootstrapping tablet: real 0.024s	user 0.019s	sys 0.000s
I20260709 13:09:33.043462 25762 tablet_bootstrap.cc:492] T fc82e3058d55474faed434b751f7c644 P 6a8699df36b74658a40f65ad30923ba9: No bootstrap required, opened a new log
I20260709 13:09:33.044037 25762 ts_tablet_manager.cc:1403] T fc82e3058d55474faed434b751f7c644 P 6a8699df36b74658a40f65ad30923ba9: Time spent bootstrapping tablet: real 0.020s	user 0.005s	sys 0.009s
I20260709 13:09:33.044447 25765 raft_consensus.cc:359] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.045526 25765 raft_consensus.cc:385] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:33.045938 25765 raft_consensus.cc:740] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7e4f250aa5d48759eaabbdff8c99210, State: Initialized, Role: FOLLOWER
I20260709 13:09:33.047046 25765 consensus_queue.cc:260] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.048102 25762 raft_consensus.cc:359] T fc82e3058d55474faed434b751f7c644 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.048913 25762 raft_consensus.cc:385] T fc82e3058d55474faed434b751f7c644 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:33.049208 25762 raft_consensus.cc:740] T fc82e3058d55474faed434b751f7c644 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a8699df36b74658a40f65ad30923ba9, State: Initialized, Role: FOLLOWER
I20260709 13:09:33.050168 25762 consensus_queue.cc:260] T fc82e3058d55474faed434b751f7c644 P 6a8699df36b74658a40f65ad30923ba9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.050275 25765 ts_tablet_manager.cc:1434] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:09:33.053804 25765 tablet_bootstrap.cc:492] T fc82e3058d55474faed434b751f7c644 P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap starting.
I20260709 13:09:33.054445 25761 tablet_bootstrap.cc:492] T 0d81fc45e39443b9b33953f8c66437eb P 99dfff6363df46ce97b027cb3bd42e11: No bootstrap required, opened a new log
I20260709 13:09:33.054983 25761 ts_tablet_manager.cc:1403] T 0d81fc45e39443b9b33953f8c66437eb P 99dfff6363df46ce97b027cb3bd42e11: Time spent bootstrapping tablet: real 0.028s	user 0.014s	sys 0.011s
I20260709 13:09:33.056306 25762 ts_tablet_manager.cc:1434] T fc82e3058d55474faed434b751f7c644 P 6a8699df36b74658a40f65ad30923ba9: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.003s
I20260709 13:09:33.057458 25762 tablet_bootstrap.cc:492] T 6d5ba3150425499f9cfa64eb06d6ba91 P 6a8699df36b74658a40f65ad30923ba9: Bootstrap starting.
I20260709 13:09:33.058089 25761 raft_consensus.cc:359] T 0d81fc45e39443b9b33953f8c66437eb P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.058866 25761 raft_consensus.cc:385] T 0d81fc45e39443b9b33953f8c66437eb P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:33.059162 25761 raft_consensus.cc:740] T 0d81fc45e39443b9b33953f8c66437eb P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99dfff6363df46ce97b027cb3bd42e11, State: Initialized, Role: FOLLOWER
I20260709 13:09:33.060024 25761 consensus_queue.cc:260] T 0d81fc45e39443b9b33953f8c66437eb P 99dfff6363df46ce97b027cb3bd42e11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.060853 25765 tablet_bootstrap.cc:654] T fc82e3058d55474faed434b751f7c644 P a7e4f250aa5d48759eaabbdff8c99210: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:33.064595 25762 tablet_bootstrap.cc:654] T 6d5ba3150425499f9cfa64eb06d6ba91 P 6a8699df36b74658a40f65ad30923ba9: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:33.067695 25761 ts_tablet_manager.cc:1434] T 0d81fc45e39443b9b33953f8c66437eb P 99dfff6363df46ce97b027cb3bd42e11: Time spent starting tablet: real 0.012s	user 0.002s	sys 0.006s
I20260709 13:09:33.068791 25761 tablet_bootstrap.cc:492] T 6d5ba3150425499f9cfa64eb06d6ba91 P 99dfff6363df46ce97b027cb3bd42e11: Bootstrap starting.
I20260709 13:09:33.069921 25765 tablet_bootstrap.cc:492] T fc82e3058d55474faed434b751f7c644 P a7e4f250aa5d48759eaabbdff8c99210: No bootstrap required, opened a new log
I20260709 13:09:33.070415 25765 ts_tablet_manager.cc:1403] T fc82e3058d55474faed434b751f7c644 P a7e4f250aa5d48759eaabbdff8c99210: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.000s
I20260709 13:09:33.073567 25765 raft_consensus.cc:359] T fc82e3058d55474faed434b751f7c644 P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.074285 25765 raft_consensus.cc:385] T fc82e3058d55474faed434b751f7c644 P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:33.074565 25765 raft_consensus.cc:740] T fc82e3058d55474faed434b751f7c644 P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7e4f250aa5d48759eaabbdff8c99210, State: Initialized, Role: FOLLOWER
I20260709 13:09:33.075003 25761 tablet_bootstrap.cc:654] T 6d5ba3150425499f9cfa64eb06d6ba91 P 99dfff6363df46ce97b027cb3bd42e11: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:33.075415 25765 consensus_queue.cc:260] T fc82e3058d55474faed434b751f7c644 P a7e4f250aa5d48759eaabbdff8c99210 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.082742 25765 ts_tablet_manager.cc:1434] T fc82e3058d55474faed434b751f7c644 P a7e4f250aa5d48759eaabbdff8c99210: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:09:33.084158 25765 tablet_bootstrap.cc:492] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap starting.
I20260709 13:09:33.092561 25765 tablet_bootstrap.cc:654] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:33.097496 25762 tablet_bootstrap.cc:492] T 6d5ba3150425499f9cfa64eb06d6ba91 P 6a8699df36b74658a40f65ad30923ba9: No bootstrap required, opened a new log
I20260709 13:09:33.097970 25762 ts_tablet_manager.cc:1403] T 6d5ba3150425499f9cfa64eb06d6ba91 P 6a8699df36b74658a40f65ad30923ba9: Time spent bootstrapping tablet: real 0.041s	user 0.016s	sys 0.019s
I20260709 13:09:33.100682 25762 raft_consensus.cc:359] T 6d5ba3150425499f9cfa64eb06d6ba91 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.101245 25762 raft_consensus.cc:385] T 6d5ba3150425499f9cfa64eb06d6ba91 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:33.101486 25762 raft_consensus.cc:740] T 6d5ba3150425499f9cfa64eb06d6ba91 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a8699df36b74658a40f65ad30923ba9, State: Initialized, Role: FOLLOWER
I20260709 13:09:33.102146 25761 tablet_bootstrap.cc:492] T 6d5ba3150425499f9cfa64eb06d6ba91 P 99dfff6363df46ce97b027cb3bd42e11: No bootstrap required, opened a new log
I20260709 13:09:33.102136 25762 consensus_queue.cc:260] T 6d5ba3150425499f9cfa64eb06d6ba91 P 6a8699df36b74658a40f65ad30923ba9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.102463 25765 tablet_bootstrap.cc:492] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210: No bootstrap required, opened a new log
I20260709 13:09:33.102771 25761 ts_tablet_manager.cc:1403] T 6d5ba3150425499f9cfa64eb06d6ba91 P 99dfff6363df46ce97b027cb3bd42e11: Time spent bootstrapping tablet: real 0.034s	user 0.018s	sys 0.016s
I20260709 13:09:33.103045 25765 ts_tablet_manager.cc:1403] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.004s
I20260709 13:09:33.105315 25761 raft_consensus.cc:359] T 6d5ba3150425499f9cfa64eb06d6ba91 P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.105962 25761 raft_consensus.cc:385] T 6d5ba3150425499f9cfa64eb06d6ba91 P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:33.106195 25761 raft_consensus.cc:740] T 6d5ba3150425499f9cfa64eb06d6ba91 P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99dfff6363df46ce97b027cb3bd42e11, State: Initialized, Role: FOLLOWER
I20260709 13:09:33.106215 25765 raft_consensus.cc:359] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.106997 25765 raft_consensus.cc:385] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:33.106920 25761 consensus_queue.cc:260] T 6d5ba3150425499f9cfa64eb06d6ba91 P 99dfff6363df46ce97b027cb3bd42e11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.107388 25765 raft_consensus.cc:740] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7e4f250aa5d48759eaabbdff8c99210, State: Initialized, Role: FOLLOWER
I20260709 13:09:33.108311 25765 consensus_queue.cc:260] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.110551 25765 ts_tablet_manager.cc:1434] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.006s
I20260709 13:09:33.111068 25762 ts_tablet_manager.cc:1434] T 6d5ba3150425499f9cfa64eb06d6ba91 P 6a8699df36b74658a40f65ad30923ba9: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.002s
I20260709 13:09:33.111990 25765 tablet_bootstrap.cc:492] T fa373cebfd3540728639dddbaee4cb1d P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap starting.
I20260709 13:09:33.109112 25761 ts_tablet_manager.cc:1434] T 6d5ba3150425499f9cfa64eb06d6ba91 P 99dfff6363df46ce97b027cb3bd42e11: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:09:33.116609 25761 tablet_bootstrap.cc:492] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11: Bootstrap starting.
I20260709 13:09:33.118949 25762 tablet_bootstrap.cc:492] T e1f6c52711174eacb83ea8fd2ad1b57b P 6a8699df36b74658a40f65ad30923ba9: Bootstrap starting.
I20260709 13:09:33.119240 25765 tablet_bootstrap.cc:654] T fa373cebfd3540728639dddbaee4cb1d P a7e4f250aa5d48759eaabbdff8c99210: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:33.126247 25767 raft_consensus.cc:493] T fc82e3058d55474faed434b751f7c644 P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:33.126797 25767 raft_consensus.cc:515] T fc82e3058d55474faed434b751f7c644 P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.128396 25761 tablet_bootstrap.cc:654] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:33.128470 25762 tablet_bootstrap.cc:654] T e1f6c52711174eacb83ea8fd2ad1b57b P 6a8699df36b74658a40f65ad30923ba9: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:33.131960 25767 leader_election.cc:290] T fc82e3058d55474faed434b751f7c644 P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2:39169)
W20260709 13:09:33.140900 25518 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:33.147244 25765 tablet_bootstrap.cc:492] T fa373cebfd3540728639dddbaee4cb1d P a7e4f250aa5d48759eaabbdff8c99210: No bootstrap required, opened a new log
I20260709 13:09:33.147943 25765 ts_tablet_manager.cc:1403] T fa373cebfd3540728639dddbaee4cb1d P a7e4f250aa5d48759eaabbdff8c99210: Time spent bootstrapping tablet: real 0.036s	user 0.014s	sys 0.020s
I20260709 13:09:33.153017 25765 raft_consensus.cc:359] T fa373cebfd3540728639dddbaee4cb1d P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } } peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } }
I20260709 13:09:33.153720 25765 raft_consensus.cc:385] T fa373cebfd3540728639dddbaee4cb1d P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:33.154021 25765 raft_consensus.cc:740] T fa373cebfd3540728639dddbaee4cb1d P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7e4f250aa5d48759eaabbdff8c99210, State: Initialized, Role: FOLLOWER
I20260709 13:09:33.154855 25765 consensus_queue.cc:260] T fa373cebfd3540728639dddbaee4cb1d P a7e4f250aa5d48759eaabbdff8c99210 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } } peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } }
I20260709 13:09:33.157526 25765 ts_tablet_manager.cc:1434] T fa373cebfd3540728639dddbaee4cb1d P a7e4f250aa5d48759eaabbdff8c99210: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:09:33.158245 25762 tablet_bootstrap.cc:492] T e1f6c52711174eacb83ea8fd2ad1b57b P 6a8699df36b74658a40f65ad30923ba9: No bootstrap required, opened a new log
I20260709 13:09:33.158556 25765 tablet_bootstrap.cc:492] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap starting.
I20260709 13:09:33.158782 25762 ts_tablet_manager.cc:1403] T e1f6c52711174eacb83ea8fd2ad1b57b P 6a8699df36b74658a40f65ad30923ba9: Time spent bootstrapping tablet: real 0.040s	user 0.009s	sys 0.009s
I20260709 13:09:33.161895 25762 raft_consensus.cc:359] T e1f6c52711174eacb83ea8fd2ad1b57b P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.162663 25762 raft_consensus.cc:385] T e1f6c52711174eacb83ea8fd2ad1b57b P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:33.162945 25762 raft_consensus.cc:740] T e1f6c52711174eacb83ea8fd2ad1b57b P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a8699df36b74658a40f65ad30923ba9, State: Initialized, Role: FOLLOWER
I20260709 13:09:33.166146 25765 tablet_bootstrap.cc:654] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:33.170181 25762 consensus_queue.cc:260] T e1f6c52711174eacb83ea8fd2ad1b57b P 6a8699df36b74658a40f65ad30923ba9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.184775 25766 raft_consensus.cc:493] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:33.185320 25766 raft_consensus.cc:515] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.186388 25761 tablet_bootstrap.cc:492] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11: No bootstrap required, opened a new log
I20260709 13:09:33.186991 25761 ts_tablet_manager.cc:1403] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11: Time spent bootstrapping tablet: real 0.071s	user 0.031s	sys 0.031s
I20260709 13:09:33.187487 25765 tablet_bootstrap.cc:492] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210: No bootstrap required, opened a new log
I20260709 13:09:33.188074 25765 ts_tablet_manager.cc:1403] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210: Time spent bootstrapping tablet: real 0.030s	user 0.015s	sys 0.005s
I20260709 13:09:33.191320 25761 raft_consensus.cc:359] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.191545 25765 raft_consensus.cc:359] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.192338 25761 raft_consensus.cc:385] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:33.192631 25765 raft_consensus.cc:385] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:33.192801 25761 raft_consensus.cc:740] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99dfff6363df46ce97b027cb3bd42e11, State: Initialized, Role: FOLLOWER
I20260709 13:09:33.193192 25765 raft_consensus.cc:740] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7e4f250aa5d48759eaabbdff8c99210, State: Initialized, Role: FOLLOWER
I20260709 13:09:33.193960 25761 consensus_queue.cc:260] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.194943 25789 raft_consensus.cc:493] T e1f6c52711174eacb83ea8fd2ad1b57b P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:33.195439 25789 raft_consensus.cc:515] T e1f6c52711174eacb83ea8fd2ad1b57b P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.196772 25761 ts_tablet_manager.cc:1434] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11: Time spent starting tablet: real 0.009s	user 0.011s	sys 0.000s
I20260709 13:09:33.197516 25789 leader_election.cc:290] T e1f6c52711174eacb83ea8fd2ad1b57b P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3:34359)
I20260709 13:09:33.197933 25761 tablet_bootstrap.cc:492] T e1f6c52711174eacb83ea8fd2ad1b57b P 99dfff6363df46ce97b027cb3bd42e11: Bootstrap starting.
I20260709 13:09:33.198153 25766 leader_election.cc:290] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3:34359)
I20260709 13:09:33.198202 25765 consensus_queue.cc:260] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.200006 25762 ts_tablet_manager.cc:1434] T e1f6c52711174eacb83ea8fd2ad1b57b P 6a8699df36b74658a40f65ad30923ba9: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.002s
I20260709 13:09:33.201062 25762 tablet_bootstrap.cc:492] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9: Bootstrap starting.
I20260709 13:09:33.203570 25765 ts_tablet_manager.cc:1434] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:09:33.204715 25765 tablet_bootstrap.cc:492] T 8372ab57bb904260beb2b0c69bc39ad0 P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap starting.
I20260709 13:09:33.206521 25761 tablet_bootstrap.cc:654] T e1f6c52711174eacb83ea8fd2ad1b57b P 99dfff6363df46ce97b027cb3bd42e11: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:33.214372 25762 tablet_bootstrap.cc:654] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:33.214372 25765 tablet_bootstrap.cc:654] T 8372ab57bb904260beb2b0c69bc39ad0 P a7e4f250aa5d48759eaabbdff8c99210: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:33.217042 25766 raft_consensus.cc:493] T 6d5ba3150425499f9cfa64eb06d6ba91 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:33.217859 25766 raft_consensus.cc:515] T 6d5ba3150425499f9cfa64eb06d6ba91 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.220877 25766 leader_election.cc:290] T 6d5ba3150425499f9cfa64eb06d6ba91 P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3:34359)
I20260709 13:09:33.229437 25765 tablet_bootstrap.cc:492] T 8372ab57bb904260beb2b0c69bc39ad0 P a7e4f250aa5d48759eaabbdff8c99210: No bootstrap required, opened a new log
I20260709 13:09:33.230201 25765 ts_tablet_manager.cc:1403] T 8372ab57bb904260beb2b0c69bc39ad0 P a7e4f250aa5d48759eaabbdff8c99210: Time spent bootstrapping tablet: real 0.026s	user 0.003s	sys 0.013s
I20260709 13:09:33.234088 25765 raft_consensus.cc:359] T 8372ab57bb904260beb2b0c69bc39ad0 P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.235072 25765 raft_consensus.cc:385] T 8372ab57bb904260beb2b0c69bc39ad0 P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:33.235456 25765 raft_consensus.cc:740] T 8372ab57bb904260beb2b0c69bc39ad0 P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7e4f250aa5d48759eaabbdff8c99210, State: Initialized, Role: FOLLOWER
I20260709 13:09:33.236624 25765 consensus_queue.cc:260] T 8372ab57bb904260beb2b0c69bc39ad0 P a7e4f250aa5d48759eaabbdff8c99210 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.238914 25647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc82e3058d55474faed434b751f7c644" candidate_uuid: "99dfff6363df46ce97b027cb3bd42e11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a8699df36b74658a40f65ad30923ba9" is_pre_election: true
I20260709 13:09:33.239936 25647 raft_consensus.cc:2468] T fc82e3058d55474faed434b751f7c644 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99dfff6363df46ce97b027cb3bd42e11 in term 0.
I20260709 13:09:33.245958 25765 ts_tablet_manager.cc:1434] T 8372ab57bb904260beb2b0c69bc39ad0 P a7e4f250aa5d48759eaabbdff8c99210: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.003s
I20260709 13:09:33.246676 25572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc82e3058d55474faed434b751f7c644" candidate_uuid: "99dfff6363df46ce97b027cb3bd42e11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7e4f250aa5d48759eaabbdff8c99210" is_pre_election: true
I20260709 13:09:33.247125 25765 tablet_bootstrap.cc:492] T bab07f367f714676a11b512f3abbea7c P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap starting.
I20260709 13:09:33.247331 25572 raft_consensus.cc:2468] T fc82e3058d55474faed434b751f7c644 P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99dfff6363df46ce97b027cb3bd42e11 in term 0.
I20260709 13:09:33.252354 25686 leader_election.cc:304] T fc82e3058d55474faed434b751f7c644 P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99dfff6363df46ce97b027cb3bd42e11, a7e4f250aa5d48759eaabbdff8c99210; no voters: 
I20260709 13:09:33.253647 25767 raft_consensus.cc:2804] T fc82e3058d55474faed434b751f7c644 P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:33.254174 25767 raft_consensus.cc:493] T fc82e3058d55474faed434b751f7c644 P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:33.254194 25765 tablet_bootstrap.cc:654] T bab07f367f714676a11b512f3abbea7c P a7e4f250aa5d48759eaabbdff8c99210: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:33.254647 25767 raft_consensus.cc:3060] T fc82e3058d55474faed434b751f7c644 P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:33.266615 25765 tablet_bootstrap.cc:492] T bab07f367f714676a11b512f3abbea7c P a7e4f250aa5d48759eaabbdff8c99210: No bootstrap required, opened a new log
I20260709 13:09:33.267235 25765 ts_tablet_manager.cc:1403] T bab07f367f714676a11b512f3abbea7c P a7e4f250aa5d48759eaabbdff8c99210: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.004s
I20260709 13:09:33.268069 25767 raft_consensus.cc:515] T fc82e3058d55474faed434b751f7c644 P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.270803 25765 raft_consensus.cc:359] T bab07f367f714676a11b512f3abbea7c P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.275802 25765 raft_consensus.cc:385] T bab07f367f714676a11b512f3abbea7c P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:33.276198 25765 raft_consensus.cc:740] T bab07f367f714676a11b512f3abbea7c P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7e4f250aa5d48759eaabbdff8c99210, State: Initialized, Role: FOLLOWER
I20260709 13:09:33.277834 25765 consensus_queue.cc:260] T bab07f367f714676a11b512f3abbea7c P a7e4f250aa5d48759eaabbdff8c99210 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.283802 25771 raft_consensus.cc:493] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:33.284250 25771 raft_consensus.cc:515] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.286940 25771 leader_election.cc:290] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2:39169), 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3:34359)
I20260709 13:09:33.284268 25572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc82e3058d55474faed434b751f7c644" candidate_uuid: "99dfff6363df46ce97b027cb3bd42e11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7e4f250aa5d48759eaabbdff8c99210"
I20260709 13:09:33.289556 25765 ts_tablet_manager.cc:1434] T bab07f367f714676a11b512f3abbea7c P a7e4f250aa5d48759eaabbdff8c99210: Time spent starting tablet: real 0.022s	user 0.010s	sys 0.000s
I20260709 13:09:33.289449 25767 leader_election.cc:290] T fc82e3058d55474faed434b751f7c644 P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 1 election: Requested vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2:39169)
I20260709 13:09:33.288662 25572 raft_consensus.cc:3060] T fc82e3058d55474faed434b751f7c644 P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:33.290694 25765 tablet_bootstrap.cc:492] T 6d5ba3150425499f9cfa64eb06d6ba91 P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap starting.
I20260709 13:09:33.305281 25762 tablet_bootstrap.cc:492] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9: No bootstrap required, opened a new log
I20260709 13:09:33.305863 25762 ts_tablet_manager.cc:1403] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9: Time spent bootstrapping tablet: real 0.105s	user 0.042s	sys 0.025s
I20260709 13:09:33.308743 25762 raft_consensus.cc:359] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.309440 25762 raft_consensus.cc:385] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:33.309743 25762 raft_consensus.cc:740] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a8699df36b74658a40f65ad30923ba9, State: Initialized, Role: FOLLOWER
I20260709 13:09:33.310557 25762 consensus_queue.cc:260] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.313809 25762 ts_tablet_manager.cc:1434] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 13:09:33.317934 25762 tablet_bootstrap.cc:492] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9: Bootstrap starting.
I20260709 13:09:33.318044 25765 tablet_bootstrap.cc:654] T 6d5ba3150425499f9cfa64eb06d6ba91 P a7e4f250aa5d48759eaabbdff8c99210: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:33.298216 25767 raft_consensus.cc:493] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:33.318339 25572 raft_consensus.cc:2468] T fc82e3058d55474faed434b751f7c644 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99dfff6363df46ce97b027cb3bd42e11 in term 1.
I20260709 13:09:33.324191 25647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc82e3058d55474faed434b751f7c644" candidate_uuid: "99dfff6363df46ce97b027cb3bd42e11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a8699df36b74658a40f65ad30923ba9"
I20260709 13:09:33.324851 25647 raft_consensus.cc:3060] T fc82e3058d55474faed434b751f7c644 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:33.328083 25686 leader_election.cc:304] T fc82e3058d55474faed434b751f7c644 P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99dfff6363df46ce97b027cb3bd42e11, a7e4f250aa5d48759eaabbdff8c99210; no voters: 
I20260709 13:09:33.321964 25767 raft_consensus.cc:515] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } } peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } }
I20260709 13:09:33.329048 25798 raft_consensus.cc:2804] T fc82e3058d55474faed434b751f7c644 P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:33.324882 25771 raft_consensus.cc:493] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:33.331530 25771 raft_consensus.cc:515] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.332094 25572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa373cebfd3540728639dddbaee4cb1d" candidate_uuid: "99dfff6363df46ce97b027cb3bd42e11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7e4f250aa5d48759eaabbdff8c99210" is_pre_election: true
I20260709 13:09:33.332772 25572 raft_consensus.cc:2468] T fa373cebfd3540728639dddbaee4cb1d P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99dfff6363df46ce97b027cb3bd42e11 in term 0.
I20260709 13:09:33.333207 25646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa373cebfd3540728639dddbaee4cb1d" candidate_uuid: "99dfff6363df46ce97b027cb3bd42e11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a8699df36b74658a40f65ad30923ba9" is_pre_election: true
I20260709 13:09:33.333702 25771 leader_election.cc:290] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2:39169), 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3:34359)
I20260709 13:09:33.333909 25646 raft_consensus.cc:2468] T fa373cebfd3540728639dddbaee4cb1d P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99dfff6363df46ce97b027cb3bd42e11 in term 0.
I20260709 13:09:33.334225 25771 raft_consensus.cc:493] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:33.334672 25771 raft_consensus.cc:515] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.335270 25686 leader_election.cc:304] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99dfff6363df46ce97b027cb3bd42e11, a7e4f250aa5d48759eaabbdff8c99210; no voters: 
I20260709 13:09:33.342489 25771 leader_election.cc:290] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2:39169), 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3:34359)
I20260709 13:09:33.351081 25766 raft_consensus.cc:493] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:33.351701 25766 raft_consensus.cc:515] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.352303 25761 tablet_bootstrap.cc:492] T e1f6c52711174eacb83ea8fd2ad1b57b P 99dfff6363df46ce97b027cb3bd42e11: No bootstrap required, opened a new log
I20260709 13:09:33.352962 25761 ts_tablet_manager.cc:1403] T e1f6c52711174eacb83ea8fd2ad1b57b P 99dfff6363df46ce97b027cb3bd42e11: Time spent bootstrapping tablet: real 0.155s	user 0.049s	sys 0.052s
I20260709 13:09:33.353989 25766 leader_election.cc:290] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3:34359)
I20260709 13:09:33.340678 25767 leader_election.cc:290] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2:39169)
I20260709 13:09:33.356959 25767 raft_consensus.cc:2804] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:33.357337 25767 raft_consensus.cc:493] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:33.357764 25767 raft_consensus.cc:3060] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:33.358523 25647 raft_consensus.cc:2468] T fc82e3058d55474faed434b751f7c644 P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99dfff6363df46ce97b027cb3bd42e11 in term 1.
I20260709 13:09:33.360203 25798 raft_consensus.cc:697] T fc82e3058d55474faed434b751f7c644 P 99dfff6363df46ce97b027cb3bd42e11 [term 1 LEADER]: Becoming Leader. State: Replica: 99dfff6363df46ce97b027cb3bd42e11, State: Running, Role: LEADER
I20260709 13:09:33.363094 25572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab07f367f714676a11b512f3abbea7c" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7e4f250aa5d48759eaabbdff8c99210" is_pre_election: true
I20260709 13:09:33.363781 25572 raft_consensus.cc:2468] T bab07f367f714676a11b512f3abbea7c P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a8699df36b74658a40f65ad30923ba9 in term 0.
I20260709 13:09:33.364270 25569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a14348b4c6a4d3cbbf18beffcd03814" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7e4f250aa5d48759eaabbdff8c99210" is_pre_election: true
I20260709 13:09:33.364679 25720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1f6c52711174eacb83ea8fd2ad1b57b" candidate_uuid: "a7e4f250aa5d48759eaabbdff8c99210" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99dfff6363df46ce97b027cb3bd42e11" is_pre_election: true
I20260709 13:09:33.364956 25571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1f6c52711174eacb83ea8fd2ad1b57b" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7e4f250aa5d48759eaabbdff8c99210" is_pre_election: true
I20260709 13:09:33.365473 25612 leader_election.cc:304] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a8699df36b74658a40f65ad30923ba9, a7e4f250aa5d48759eaabbdff8c99210; no voters: 
I20260709 13:09:33.365711 25571 raft_consensus.cc:2468] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a8699df36b74658a40f65ad30923ba9 in term 0.
I20260709 13:09:33.365893 25719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a14348b4c6a4d3cbbf18beffcd03814" candidate_uuid: "a7e4f250aa5d48759eaabbdff8c99210" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99dfff6363df46ce97b027cb3bd42e11" is_pre_election: true
I20260709 13:09:33.366417 25766 raft_consensus.cc:2804] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:33.366648 25719 raft_consensus.cc:2468] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7e4f250aa5d48759eaabbdff8c99210 in term 0.
I20260709 13:09:33.366868 25766 raft_consensus.cc:493] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:33.367408 25766 raft_consensus.cc:3060] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:33.367625 25721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d81fc45e39443b9b33953f8c66437eb" candidate_uuid: "a7e4f250aa5d48759eaabbdff8c99210" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99dfff6363df46ce97b027cb3bd42e11" is_pre_election: true
I20260709 13:09:33.368292 25721 raft_consensus.cc:2468] T 0d81fc45e39443b9b33953f8c66437eb P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7e4f250aa5d48759eaabbdff8c99210 in term 0.
I20260709 13:09:33.368536 25537 leader_election.cc:304] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99dfff6363df46ce97b027cb3bd42e11, a7e4f250aa5d48759eaabbdff8c99210; no voters: 
I20260709 13:09:33.365015 25569 raft_consensus.cc:2468] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a8699df36b74658a40f65ad30923ba9 in term 0.
I20260709 13:09:33.372552 25771 raft_consensus.cc:2804] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:33.372887 25767 raft_consensus.cc:515] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } } peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } }
I20260709 13:09:33.366786 25570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d5ba3150425499f9cfa64eb06d6ba91" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7e4f250aa5d48759eaabbdff8c99210" is_pre_election: true
I20260709 13:09:33.360852 25761 raft_consensus.cc:359] T e1f6c52711174eacb83ea8fd2ad1b57b P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.375105 25766 raft_consensus.cc:515] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.375540 25761 raft_consensus.cc:385] T e1f6c52711174eacb83ea8fd2ad1b57b P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:33.376166 25761 raft_consensus.cc:740] T e1f6c52711174eacb83ea8fd2ad1b57b P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99dfff6363df46ce97b027cb3bd42e11, State: Initialized, Role: FOLLOWER
I20260709 13:09:33.377434 25766 leader_election.cc:290] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 1 election: Requested vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3:34359)
I20260709 13:09:33.377970 25771 raft_consensus.cc:493] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:33.378067 25718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d5ba3150425499f9cfa64eb06d6ba91" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99dfff6363df46ce97b027cb3bd42e11" is_pre_election: true
I20260709 13:09:33.378446 25719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab07f367f714676a11b512f3abbea7c" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99dfff6363df46ce97b027cb3bd42e11" is_pre_election: true
I20260709 13:09:33.377056 25761 consensus_queue.cc:260] T e1f6c52711174eacb83ea8fd2ad1b57b P 99dfff6363df46ce97b027cb3bd42e11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.378836 25718 raft_consensus.cc:2468] T 6d5ba3150425499f9cfa64eb06d6ba91 P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a8699df36b74658a40f65ad30923ba9 in term 0.
I20260709 13:09:33.380194 25570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab07f367f714676a11b512f3abbea7c" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7e4f250aa5d48759eaabbdff8c99210"
I20260709 13:09:33.380431 25720 raft_consensus.cc:2468] T e1f6c52711174eacb83ea8fd2ad1b57b P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7e4f250aa5d48759eaabbdff8c99210 in term 0.
I20260709 13:09:33.380836 25537 leader_election.cc:304] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99dfff6363df46ce97b027cb3bd42e11, a7e4f250aa5d48759eaabbdff8c99210; no voters: 
I20260709 13:09:33.380853 25570 raft_consensus.cc:3060] T bab07f367f714676a11b512f3abbea7c P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:33.381636 25716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab07f367f714676a11b512f3abbea7c" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99dfff6363df46ce97b027cb3bd42e11"
I20260709 13:09:33.379570 25612 leader_election.cc:304] T e1f6c52711174eacb83ea8fd2ad1b57b P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a8699df36b74658a40f65ad30923ba9, a7e4f250aa5d48759eaabbdff8c99210; no voters: 
I20260709 13:09:33.361508 25798 consensus_queue.cc:237] T fc82e3058d55474faed434b751f7c644 P 99dfff6363df46ce97b027cb3bd42e11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.382923 25612 leader_election.cc:304] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a8699df36b74658a40f65ad30923ba9, a7e4f250aa5d48759eaabbdff8c99210; no voters: 
I20260709 13:09:33.379310 25717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a14348b4c6a4d3cbbf18beffcd03814" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99dfff6363df46ce97b027cb3bd42e11" is_pre_election: true
I20260709 13:09:33.379040 25719 raft_consensus.cc:2468] T bab07f367f714676a11b512f3abbea7c P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a8699df36b74658a40f65ad30923ba9 in term 0.
I20260709 13:09:33.380213 25721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1f6c52711174eacb83ea8fd2ad1b57b" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99dfff6363df46ce97b027cb3bd42e11" is_pre_election: true
I20260709 13:09:33.383981 25717 raft_consensus.cc:2468] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a8699df36b74658a40f65ad30923ba9 in term 0.
I20260709 13:09:33.378505 25771 raft_consensus.cc:3060] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:33.379577 25765 tablet_bootstrap.cc:492] T 6d5ba3150425499f9cfa64eb06d6ba91 P a7e4f250aa5d48759eaabbdff8c99210: No bootstrap required, opened a new log
I20260709 13:09:33.384859 25789 raft_consensus.cc:2804] T e1f6c52711174eacb83ea8fd2ad1b57b P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:33.385031 25721 raft_consensus.cc:2468] T e1f6c52711174eacb83ea8fd2ad1b57b P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a8699df36b74658a40f65ad30923ba9 in term 0.
W20260709 13:09:33.385330 25612 leader_election.cc:343] T 6d5ba3150425499f9cfa64eb06d6ba91 P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:33.390719 25569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa373cebfd3540728639dddbaee4cb1d" candidate_uuid: "99dfff6363df46ce97b027cb3bd42e11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7e4f250aa5d48759eaabbdff8c99210"
I20260709 13:09:33.391367 25569 raft_consensus.cc:3060] T fa373cebfd3540728639dddbaee4cb1d P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:33.389266 25765 ts_tablet_manager.cc:1403] T 6d5ba3150425499f9cfa64eb06d6ba91 P a7e4f250aa5d48759eaabbdff8c99210: Time spent bootstrapping tablet: real 0.099s	user 0.012s	sys 0.001s
I20260709 13:09:33.389767 25789 raft_consensus.cc:493] T e1f6c52711174eacb83ea8fd2ad1b57b P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:33.393023 25789 raft_consensus.cc:3060] T e1f6c52711174eacb83ea8fd2ad1b57b P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:33.394420 25771 raft_consensus.cc:515] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.396770 25765 raft_consensus.cc:359] T 6d5ba3150425499f9cfa64eb06d6ba91 P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.397284 25765 raft_consensus.cc:385] T 6d5ba3150425499f9cfa64eb06d6ba91 P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:33.397511 25765 raft_consensus.cc:740] T 6d5ba3150425499f9cfa64eb06d6ba91 P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7e4f250aa5d48759eaabbdff8c99210, State: Initialized, Role: FOLLOWER
I20260709 13:09:33.398041 25765 consensus_queue.cc:260] T 6d5ba3150425499f9cfa64eb06d6ba91 P a7e4f250aa5d48759eaabbdff8c99210 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.385522 25761 ts_tablet_manager.cc:1434] T e1f6c52711174eacb83ea8fd2ad1b57b P 99dfff6363df46ce97b027cb3bd42e11: Time spent starting tablet: real 0.032s	user 0.009s	sys 0.000s
I20260709 13:09:33.387905 25762 tablet_bootstrap.cc:654] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:33.384972 25801 raft_consensus.cc:2804] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:33.387226 25647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa373cebfd3540728639dddbaee4cb1d" candidate_uuid: "99dfff6363df46ce97b027cb3bd42e11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a8699df36b74658a40f65ad30923ba9"
I20260709 13:09:33.403482 25761 tablet_bootstrap.cc:492] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11: Bootstrap starting.
I20260709 13:09:33.403638 25771 leader_election.cc:290] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210 [CANDIDATE]: Term 1 election: Requested vote from peers 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2:39169), 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3:34359)
I20260709 13:09:33.386092 25767 leader_election.cc:290] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 1 election: Requested vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2:39169)
I20260709 13:09:33.389622 25537 leader_election.cc:304] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99dfff6363df46ce97b027cb3bd42e11, a7e4f250aa5d48759eaabbdff8c99210; no voters: 
I20260709 13:09:33.391985 25766 raft_consensus.cc:2804] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:33.404022 25612 leader_election.cc:304] T 6d5ba3150425499f9cfa64eb06d6ba91 P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a8699df36b74658a40f65ad30923ba9, 99dfff6363df46ce97b027cb3bd42e11; no voters: a7e4f250aa5d48759eaabbdff8c99210
I20260709 13:09:33.405120 25801 raft_consensus.cc:493] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:33.416817 25765 ts_tablet_manager.cc:1434] T 6d5ba3150425499f9cfa64eb06d6ba91 P a7e4f250aa5d48759eaabbdff8c99210: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.004s
I20260709 13:09:33.415433 25766 raft_consensus.cc:493] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:33.418838 25801 raft_consensus.cc:3060] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:33.414353 25647 raft_consensus.cc:3060] T fa373cebfd3540728639dddbaee4cb1d P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:33.415048 25570 raft_consensus.cc:2468] T bab07f367f714676a11b512f3abbea7c P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a8699df36b74658a40f65ad30923ba9 in term 1.
I20260709 13:09:33.422964 25805 raft_consensus.cc:493] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:33.423466 25805 raft_consensus.cc:515] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.425510 25805 leader_election.cc:290] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3:34359)
I20260709 13:09:33.425904 25569 raft_consensus.cc:2468] T fa373cebfd3540728639dddbaee4cb1d P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99dfff6363df46ce97b027cb3bd42e11 in term 1.
I20260709 13:09:33.426445 25771 raft_consensus.cc:2804] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:33.426857 25771 raft_consensus.cc:493] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:33.427310 25771 raft_consensus.cc:3060] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:33.414728 25789 raft_consensus.cc:515] T e1f6c52711174eacb83ea8fd2ad1b57b P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.429520 25789 leader_election.cc:290] T e1f6c52711174eacb83ea8fd2ad1b57b P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 1 election: Requested vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3:34359)
I20260709 13:09:33.417408 25766 raft_consensus.cc:3060] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:33.431250 25686 leader_election.cc:304] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99dfff6363df46ce97b027cb3bd42e11, a7e4f250aa5d48759eaabbdff8c99210; no voters: 
I20260709 13:09:33.429323 25470 catalog_manager.cc:5697] T fc82e3058d55474faed434b751f7c644 P 99dfff6363df46ce97b027cb3bd42e11 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3). New cstate: current_term: 1 leader_uuid: "99dfff6363df46ce97b027cb3bd42e11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:33.433162 25798 raft_consensus.cc:2804] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:33.433683 25798 raft_consensus.cc:697] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [term 1 LEADER]: Becoming Leader. State: Replica: 99dfff6363df46ce97b027cb3bd42e11, State: Running, Role: LEADER
I20260709 13:09:33.434706 25798 consensus_queue.cc:237] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } } peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } }
I20260709 13:09:33.426043 25805 raft_consensus.cc:2804] T 6d5ba3150425499f9cfa64eb06d6ba91 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:33.441272 25765 tablet_bootstrap.cc:492] T 64becd02c9434edeaf5824c5a455b0e0 P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap starting.
I20260709 13:09:33.439570 25805 raft_consensus.cc:493] T 6d5ba3150425499f9cfa64eb06d6ba91 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:33.443475 25805 raft_consensus.cc:3060] T 6d5ba3150425499f9cfa64eb06d6ba91 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:33.443219 25801 raft_consensus.cc:515] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.445981 25646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d81fc45e39443b9b33953f8c66437eb" candidate_uuid: "a7e4f250aa5d48759eaabbdff8c99210" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a8699df36b74658a40f65ad30923ba9" is_pre_election: true
I20260709 13:09:33.445992 25801 leader_election.cc:290] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210 [CANDIDATE]: Term 1 election: Requested vote from peers 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2:39169), 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3:34359)
I20260709 13:09:33.448909 25646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1f6c52711174eacb83ea8fd2ad1b57b" candidate_uuid: "a7e4f250aa5d48759eaabbdff8c99210" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a8699df36b74658a40f65ad30923ba9" is_pre_election: true
I20260709 13:09:33.450007 25646 raft_consensus.cc:2393] T e1f6c52711174eacb83ea8fd2ad1b57b P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a7e4f250aa5d48759eaabbdff8c99210 in current term 1: Already voted for candidate 6a8699df36b74658a40f65ad30923ba9 in this term.
I20260709 13:09:33.450677 25761 tablet_bootstrap.cc:654] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:33.451759 25646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a14348b4c6a4d3cbbf18beffcd03814" candidate_uuid: "a7e4f250aa5d48759eaabbdff8c99210" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a8699df36b74658a40f65ad30923ba9" is_pre_election: true
I20260709 13:09:33.455705 25645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a14348b4c6a4d3cbbf18beffcd03814" candidate_uuid: "a7e4f250aa5d48759eaabbdff8c99210" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a8699df36b74658a40f65ad30923ba9"
I20260709 13:09:33.457186 25771 raft_consensus.cc:515] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.459307 25771 leader_election.cc:290] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210 [CANDIDATE]: Term 1 election: Requested vote from peers 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2:39169), 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3:34359)
I20260709 13:09:33.459520 25765 tablet_bootstrap.cc:654] T 64becd02c9434edeaf5824c5a455b0e0 P a7e4f250aa5d48759eaabbdff8c99210: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:33.459093 25805 raft_consensus.cc:515] T 6d5ba3150425499f9cfa64eb06d6ba91 P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.461951 25647 raft_consensus.cc:2468] T fa373cebfd3540728639dddbaee4cb1d P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99dfff6363df46ce97b027cb3bd42e11 in term 1.
I20260709 13:09:33.462649 25762 tablet_bootstrap.cc:492] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9: No bootstrap required, opened a new log
I20260709 13:09:33.463122 25762 ts_tablet_manager.cc:1403] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9: Time spent bootstrapping tablet: real 0.145s	user 0.010s	sys 0.010s
W20260709 13:09:33.455067 25537 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:34359, user_credentials={real_user=slave}} blocked reactor thread for 65527.6us
I20260709 13:09:33.467603 25721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a14348b4c6a4d3cbbf18beffcd03814" candidate_uuid: "a7e4f250aa5d48759eaabbdff8c99210" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99dfff6363df46ce97b027cb3bd42e11"
I20260709 13:09:33.467481 25762 raft_consensus.cc:359] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.468257 25721 raft_consensus.cc:3060] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:33.468950 25647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d81fc45e39443b9b33953f8c66437eb" candidate_uuid: "a7e4f250aa5d48759eaabbdff8c99210" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a8699df36b74658a40f65ad30923ba9"
I20260709 13:09:33.470743 25569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8372ab57bb904260beb2b0c69bc39ad0" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7e4f250aa5d48759eaabbdff8c99210" is_pre_election: true
I20260709 13:09:33.468295 25762 raft_consensus.cc:385] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:33.472025 25762 raft_consensus.cc:740] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a8699df36b74658a40f65ad30923ba9, State: Initialized, Role: FOLLOWER
I20260709 13:09:33.471406 25569 raft_consensus.cc:2468] T 8372ab57bb904260beb2b0c69bc39ad0 P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a8699df36b74658a40f65ad30923ba9 in term 0.
I20260709 13:09:33.472378 25717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8372ab57bb904260beb2b0c69bc39ad0" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99dfff6363df46ce97b027cb3bd42e11" is_pre_election: true
I20260709 13:09:33.473529 25762 consensus_queue.cc:260] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.475864 25719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d81fc45e39443b9b33953f8c66437eb" candidate_uuid: "a7e4f250aa5d48759eaabbdff8c99210" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99dfff6363df46ce97b027cb3bd42e11"
I20260709 13:09:33.477937 25720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1f6c52711174eacb83ea8fd2ad1b57b" candidate_uuid: "a7e4f250aa5d48759eaabbdff8c99210" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99dfff6363df46ce97b027cb3bd42e11"
I20260709 13:09:33.476462 25719 raft_consensus.cc:3060] T 0d81fc45e39443b9b33953f8c66437eb P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:09:33.478935 25535 leader_election.cc:343] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2:39169): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:33.479187 25717 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1f6c52711174eacb83ea8fd2ad1b57b" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99dfff6363df46ce97b027cb3bd42e11"
I20260709 13:09:33.479998 25612 leader_election.cc:304] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a8699df36b74658a40f65ad30923ba9, a7e4f250aa5d48759eaabbdff8c99210; no voters: 99dfff6363df46ce97b027cb3bd42e11
I20260709 13:09:33.478149 25766 raft_consensus.cc:515] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.476424 25570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1f6c52711174eacb83ea8fd2ad1b57b" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7e4f250aa5d48759eaabbdff8c99210"
I20260709 13:09:33.478524 25720 raft_consensus.cc:3060] T e1f6c52711174eacb83ea8fd2ad1b57b P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:33.482784 25789 raft_consensus.cc:2804] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:33.483475 25789 raft_consensus.cc:697] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [term 1 LEADER]: Becoming Leader. State: Replica: 6a8699df36b74658a40f65ad30923ba9, State: Running, Role: LEADER
I20260709 13:09:33.484431 25789 consensus_queue.cc:237] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.488375 25721 raft_consensus.cc:2468] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7e4f250aa5d48759eaabbdff8c99210 in term 1.
I20260709 13:09:33.489353 25646 raft_consensus.cc:2393] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a7e4f250aa5d48759eaabbdff8c99210 in current term 1: Already voted for candidate 6a8699df36b74658a40f65ad30923ba9 in this term.
I20260709 13:09:33.476823 25647 raft_consensus.cc:3060] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:33.489790 25537 leader_election.cc:304] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 99dfff6363df46ce97b027cb3bd42e11, a7e4f250aa5d48759eaabbdff8c99210; no voters: 6a8699df36b74658a40f65ad30923ba9
I20260709 13:09:33.490777 25809 raft_consensus.cc:2804] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:33.491053 25612 leader_election.cc:304] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a8699df36b74658a40f65ad30923ba9, a7e4f250aa5d48759eaabbdff8c99210; no voters: 
I20260709 13:09:33.481637 25570 raft_consensus.cc:2393] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a8699df36b74658a40f65ad30923ba9 in current term 1: Already voted for candidate a7e4f250aa5d48759eaabbdff8c99210 in this term.
I20260709 13:09:33.492573 25766 leader_election.cc:290] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 1 election: Requested vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3:34359)
I20260709 13:09:33.496074 25762 ts_tablet_manager.cc:1434] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.000s
I20260709 13:09:33.495365 25646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1f6c52711174eacb83ea8fd2ad1b57b" candidate_uuid: "a7e4f250aa5d48759eaabbdff8c99210" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a8699df36b74658a40f65ad30923ba9"
I20260709 13:09:33.496824 25720 raft_consensus.cc:2468] T e1f6c52711174eacb83ea8fd2ad1b57b P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7e4f250aa5d48759eaabbdff8c99210 in term 1.
I20260709 13:09:33.497535 25646 raft_consensus.cc:2393] T e1f6c52711174eacb83ea8fd2ad1b57b P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a7e4f250aa5d48759eaabbdff8c99210 in current term 1: Already voted for candidate 6a8699df36b74658a40f65ad30923ba9 in this term.
I20260709 13:09:33.498025 25537 leader_election.cc:304] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99dfff6363df46ce97b027cb3bd42e11, a7e4f250aa5d48759eaabbdff8c99210; no voters: 
I20260709 13:09:33.498127 25719 raft_consensus.cc:2468] T 0d81fc45e39443b9b33953f8c66437eb P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7e4f250aa5d48759eaabbdff8c99210 in term 1.
I20260709 13:09:33.491356 25809 raft_consensus.cc:697] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 LEADER]: Becoming Leader. State: Replica: a7e4f250aa5d48759eaabbdff8c99210, State: Running, Role: LEADER
I20260709 13:09:33.497076 25762 tablet_bootstrap.cc:492] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9: Bootstrap starting.
I20260709 13:09:33.498979 25771 raft_consensus.cc:2804] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:33.500640 25766 raft_consensus.cc:2804] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:33.502547 25765 tablet_bootstrap.cc:492] T 64becd02c9434edeaf5824c5a455b0e0 P a7e4f250aa5d48759eaabbdff8c99210: No bootstrap required, opened a new log
I20260709 13:09:33.502929 25537 leader_election.cc:304] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99dfff6363df46ce97b027cb3bd42e11, a7e4f250aa5d48759eaabbdff8c99210; no voters: 
I20260709 13:09:33.503396 25771 raft_consensus.cc:697] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210 [term 1 LEADER]: Becoming Leader. State: Replica: a7e4f250aa5d48759eaabbdff8c99210, State: Running, Role: LEADER
I20260709 13:09:33.503504 25765 ts_tablet_manager.cc:1403] T 64becd02c9434edeaf5824c5a455b0e0 P a7e4f250aa5d48759eaabbdff8c99210: Time spent bootstrapping tablet: real 0.062s	user 0.008s	sys 0.008s
I20260709 13:09:33.504621 25766 raft_consensus.cc:493] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:33.504457 25807 raft_consensus.cc:2804] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:33.504326 25771 consensus_queue.cc:237] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.505419 25807 raft_consensus.cc:697] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210 [term 1 LEADER]: Becoming Leader. State: Replica: a7e4f250aa5d48759eaabbdff8c99210, State: Running, Role: LEADER
I20260709 13:09:33.505499 25809 consensus_queue.cc:237] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.506314 25807 consensus_queue.cc:237] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.506458 25765 raft_consensus.cc:359] T 64becd02c9434edeaf5824c5a455b0e0 P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.507259 25765 raft_consensus.cc:385] T 64becd02c9434edeaf5824c5a455b0e0 P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:33.507534 25765 raft_consensus.cc:740] T 64becd02c9434edeaf5824c5a455b0e0 P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7e4f250aa5d48759eaabbdff8c99210, State: Initialized, Role: FOLLOWER
I20260709 13:09:33.508320 25761 tablet_bootstrap.cc:492] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11: No bootstrap required, opened a new log
I20260709 13:09:33.508157 25765 consensus_queue.cc:260] T 64becd02c9434edeaf5824c5a455b0e0 P a7e4f250aa5d48759eaabbdff8c99210 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.508821 25761 ts_tablet_manager.cc:1403] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11: Time spent bootstrapping tablet: real 0.106s	user 0.014s	sys 0.000s
I20260709 13:09:33.511287 25765 ts_tablet_manager.cc:1434] T 64becd02c9434edeaf5824c5a455b0e0 P a7e4f250aa5d48759eaabbdff8c99210: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:09:33.512133 25766 raft_consensus.cc:3060] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:33.516322 25570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a14348b4c6a4d3cbbf18beffcd03814" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7e4f250aa5d48759eaabbdff8c99210"
I20260709 13:09:33.517920 25805 leader_election.cc:290] T 6d5ba3150425499f9cfa64eb06d6ba91 P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 1 election: Requested vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3:34359)
I20260709 13:09:33.526877 25761 raft_consensus.cc:359] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.527419 25761 raft_consensus.cc:385] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:33.527637 25761 raft_consensus.cc:740] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99dfff6363df46ce97b027cb3bd42e11, State: Initialized, Role: FOLLOWER
I20260709 13:09:33.530644 25720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a14348b4c6a4d3cbbf18beffcd03814" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99dfff6363df46ce97b027cb3bd42e11"
I20260709 13:09:33.531440 25720 raft_consensus.cc:2393] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a8699df36b74658a40f65ad30923ba9 in current term 1: Already voted for candidate a7e4f250aa5d48759eaabbdff8c99210 in this term.
I20260709 13:09:33.533212 25647 raft_consensus.cc:2468] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7e4f250aa5d48759eaabbdff8c99210 in term 1.
I20260709 13:09:33.535298 25766 raft_consensus.cc:515] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.537986 25771 raft_consensus.cc:493] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:33.538537 25771 raft_consensus.cc:515] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
W20260709 13:09:33.539654 25612 leader_election.cc:343] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3:34359): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:33.541057 25762 tablet_bootstrap.cc:654] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:33.540692 25761 consensus_queue.cc:260] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.542982 25761 ts_tablet_manager.cc:1434] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11: Time spent starting tablet: real 0.034s	user 0.005s	sys 0.000s
I20260709 13:09:33.544153 25761 tablet_bootstrap.cc:492] T 8372ab57bb904260beb2b0c69bc39ad0 P 99dfff6363df46ce97b027cb3bd42e11: Bootstrap starting.
I20260709 13:09:33.542840 25612 leader_election.cc:304] T e1f6c52711174eacb83ea8fd2ad1b57b P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a8699df36b74658a40f65ad30923ba9; no voters: 99dfff6363df46ce97b027cb3bd42e11, a7e4f250aa5d48759eaabbdff8c99210
I20260709 13:09:33.545295 25762 tablet_bootstrap.cc:492] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9: No bootstrap required, opened a new log
I20260709 13:09:33.545220 25789 raft_consensus.cc:2749] T e1f6c52711174eacb83ea8fd2ad1b57b P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:09:33.545678 25762 ts_tablet_manager.cc:1403] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9: Time spent bootstrapping tablet: real 0.049s	user 0.007s	sys 0.004s
I20260709 13:09:33.547595 25762 raft_consensus.cc:359] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.548296 25762 raft_consensus.cc:385] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:33.548568 25762 raft_consensus.cc:740] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a8699df36b74658a40f65ad30923ba9, State: Initialized, Role: FOLLOWER
I20260709 13:09:33.549188 25762 consensus_queue.cc:260] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.549906 25720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "377fe3ce2b474722b845c07b3f74b830" candidate_uuid: "a7e4f250aa5d48759eaabbdff8c99210" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99dfff6363df46ce97b027cb3bd42e11" is_pre_election: true
I20260709 13:09:33.550235 25766 leader_election.cc:290] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 1 election: Requested vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3:34359)
I20260709 13:09:33.550561 25720 raft_consensus.cc:2468] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7e4f250aa5d48759eaabbdff8c99210 in term 0.
I20260709 13:09:33.549679 25771 leader_election.cc:290] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2:39169), 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3:34359)
I20260709 13:09:33.550436 25469 catalog_manager.cc:5697] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210 reported cstate change: term changed from 0 to 1, leader changed from <none> to a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "a7e4f250aa5d48759eaabbdff8c99210" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:33.551745 25762 ts_tablet_manager.cc:1434] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:09:33.552033 25537 leader_election.cc:304] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99dfff6363df46ce97b027cb3bd42e11, a7e4f250aa5d48759eaabbdff8c99210; no voters: 
I20260709 13:09:33.555202 25762 tablet_bootstrap.cc:492] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9: Bootstrap starting.
I20260709 13:09:33.546402 25612 leader_election.cc:304] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a8699df36b74658a40f65ad30923ba9; no voters: 99dfff6363df46ce97b027cb3bd42e11, a7e4f250aa5d48759eaabbdff8c99210
I20260709 13:09:33.556255 25789 raft_consensus.cc:2749] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:09:33.558094 25570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d5ba3150425499f9cfa64eb06d6ba91" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7e4f250aa5d48759eaabbdff8c99210"
I20260709 13:09:33.558652 25570 raft_consensus.cc:3060] T 6d5ba3150425499f9cfa64eb06d6ba91 P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:33.560585 25762 tablet_bootstrap.cc:654] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:33.561883 25470 catalog_manager.cc:5697] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3). New cstate: current_term: 1 leader_uuid: "99dfff6363df46ce97b027cb3bd42e11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:33.567888 25771 raft_consensus.cc:2804] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:33.568132 25570 raft_consensus.cc:2468] T 6d5ba3150425499f9cfa64eb06d6ba91 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a8699df36b74658a40f65ad30923ba9 in term 1.
I20260709 13:09:33.569227 25771 raft_consensus.cc:493] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:33.569846 25771 raft_consensus.cc:3060] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:33.570370 25720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d5ba3150425499f9cfa64eb06d6ba91" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99dfff6363df46ce97b027cb3bd42e11"
I20260709 13:09:33.571014 25720 raft_consensus.cc:3060] T 6d5ba3150425499f9cfa64eb06d6ba91 P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:33.577327 25720 raft_consensus.cc:2468] T 6d5ba3150425499f9cfa64eb06d6ba91 P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a8699df36b74658a40f65ad30923ba9 in term 1.
I20260709 13:09:33.580145 25570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8372ab57bb904260beb2b0c69bc39ad0" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7e4f250aa5d48759eaabbdff8c99210"
I20260709 13:09:33.580638 25570 raft_consensus.cc:3060] T 8372ab57bb904260beb2b0c69bc39ad0 P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:33.553521 25761 tablet_bootstrap.cc:654] T 8372ab57bb904260beb2b0c69bc39ad0 P 99dfff6363df46ce97b027cb3bd42e11: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:33.587182 25570 raft_consensus.cc:2468] T 8372ab57bb904260beb2b0c69bc39ad0 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a8699df36b74658a40f65ad30923ba9 in term 1.
I20260709 13:09:33.586726 25469 catalog_manager.cc:5697] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210 reported cstate change: term changed from 0 to 1, leader changed from <none> to a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "a7e4f250aa5d48759eaabbdff8c99210" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:33.589221 25720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8372ab57bb904260beb2b0c69bc39ad0" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99dfff6363df46ce97b027cb3bd42e11"
I20260709 13:09:33.588809 25469 catalog_manager.cc:5697] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210 reported cstate change: term changed from 0 to 1, leader changed from <none> to a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "a7e4f250aa5d48759eaabbdff8c99210" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:33.590412 25771 raft_consensus.cc:515] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.591162 25647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "377fe3ce2b474722b845c07b3f74b830" candidate_uuid: "a7e4f250aa5d48759eaabbdff8c99210" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a8699df36b74658a40f65ad30923ba9" is_pre_election: true
W20260709 13:09:33.598482 25535 leader_election.cc:343] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2:39169): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:33.600353 25761 tablet_bootstrap.cc:492] T 8372ab57bb904260beb2b0c69bc39ad0 P 99dfff6363df46ce97b027cb3bd42e11: No bootstrap required, opened a new log
I20260709 13:09:33.601089 25761 ts_tablet_manager.cc:1403] T 8372ab57bb904260beb2b0c69bc39ad0 P 99dfff6363df46ce97b027cb3bd42e11: Time spent bootstrapping tablet: real 0.057s	user 0.012s	sys 0.000s
I20260709 13:09:33.601853 25612 leader_election.cc:304] T 6d5ba3150425499f9cfa64eb06d6ba91 P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a8699df36b74658a40f65ad30923ba9, 99dfff6363df46ce97b027cb3bd42e11; no voters: 
I20260709 13:09:33.602677 25789 raft_consensus.cc:2804] T 6d5ba3150425499f9cfa64eb06d6ba91 P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:33.603170 25789 raft_consensus.cc:697] T 6d5ba3150425499f9cfa64eb06d6ba91 P 6a8699df36b74658a40f65ad30923ba9 [term 1 LEADER]: Becoming Leader. State: Replica: 6a8699df36b74658a40f65ad30923ba9, State: Running, Role: LEADER
I20260709 13:09:33.604017 25789 consensus_queue.cc:237] T 6d5ba3150425499f9cfa64eb06d6ba91 P 6a8699df36b74658a40f65ad30923ba9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.605859 25771 leader_election.cc:290] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210 [CANDIDATE]: Term 1 election: Requested vote from peers 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2:39169), 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3:34359)
I20260709 13:09:33.607427 25720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "377fe3ce2b474722b845c07b3f74b830" candidate_uuid: "a7e4f250aa5d48759eaabbdff8c99210" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99dfff6363df46ce97b027cb3bd42e11"
I20260709 13:09:33.608076 25720 raft_consensus.cc:3060] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:33.609196 25761 raft_consensus.cc:359] T 8372ab57bb904260beb2b0c69bc39ad0 P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.609935 25761 raft_consensus.cc:385] T 8372ab57bb904260beb2b0c69bc39ad0 P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:33.610198 25761 raft_consensus.cc:740] T 8372ab57bb904260beb2b0c69bc39ad0 P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99dfff6363df46ce97b027cb3bd42e11, State: Initialized, Role: FOLLOWER
I20260709 13:09:33.610313 25762 tablet_bootstrap.cc:492] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9: No bootstrap required, opened a new log
I20260709 13:09:33.610889 25762 ts_tablet_manager.cc:1403] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9: Time spent bootstrapping tablet: real 0.056s	user 0.037s	sys 0.018s
I20260709 13:09:33.611017 25761 consensus_queue.cc:260] T 8372ab57bb904260beb2b0c69bc39ad0 P 99dfff6363df46ce97b027cb3bd42e11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.613550 25762 raft_consensus.cc:359] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
W20260709 13:09:33.613607 25612 leader_election.cc:343] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3:34359): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:33.614236 25762 raft_consensus.cc:385] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:33.614547 25762 raft_consensus.cc:740] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a8699df36b74658a40f65ad30923ba9, State: Initialized, Role: FOLLOWER
I20260709 13:09:33.615496 25761 ts_tablet_manager.cc:1434] T 8372ab57bb904260beb2b0c69bc39ad0 P 99dfff6363df46ce97b027cb3bd42e11: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:09:33.615288 25762 consensus_queue.cc:260] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.616519 25761 tablet_bootstrap.cc:492] T 64becd02c9434edeaf5824c5a455b0e0 P 99dfff6363df46ce97b027cb3bd42e11: Bootstrap starting.
I20260709 13:09:33.615283 25612 leader_election.cc:304] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a8699df36b74658a40f65ad30923ba9, a7e4f250aa5d48759eaabbdff8c99210; no voters: 99dfff6363df46ce97b027cb3bd42e11
I20260709 13:09:33.617745 25762 ts_tablet_manager.cc:1434] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:09:33.620411 25766 raft_consensus.cc:2804] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:33.621027 25766 raft_consensus.cc:697] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9 [term 1 LEADER]: Becoming Leader. State: Replica: 6a8699df36b74658a40f65ad30923ba9, State: Running, Role: LEADER
I20260709 13:09:33.621955 25766 consensus_queue.cc:237] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.622597 25761 tablet_bootstrap.cc:654] T 64becd02c9434edeaf5824c5a455b0e0 P 99dfff6363df46ce97b027cb3bd42e11: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:33.625195 25647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "377fe3ce2b474722b845c07b3f74b830" candidate_uuid: "a7e4f250aa5d48759eaabbdff8c99210" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a8699df36b74658a40f65ad30923ba9"
I20260709 13:09:33.625838 25647 raft_consensus.cc:3060] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:33.629684 25761 tablet_bootstrap.cc:492] T 64becd02c9434edeaf5824c5a455b0e0 P 99dfff6363df46ce97b027cb3bd42e11: No bootstrap required, opened a new log
I20260709 13:09:33.630120 25761 ts_tablet_manager.cc:1403] T 64becd02c9434edeaf5824c5a455b0e0 P 99dfff6363df46ce97b027cb3bd42e11: Time spent bootstrapping tablet: real 0.014s	user 0.007s	sys 0.004s
I20260709 13:09:33.631570 25647 raft_consensus.cc:2468] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7e4f250aa5d48759eaabbdff8c99210 in term 1.
I20260709 13:09:33.632890 25535 leader_election.cc:304] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a8699df36b74658a40f65ad30923ba9, a7e4f250aa5d48759eaabbdff8c99210; no voters: 
I20260709 13:09:33.632726 25761 raft_consensus.cc:359] T 64becd02c9434edeaf5824c5a455b0e0 P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.633282 25720 raft_consensus.cc:2468] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7e4f250aa5d48759eaabbdff8c99210 in term 1.
I20260709 13:09:33.633865 25771 raft_consensus.cc:2804] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:33.634392 25771 raft_consensus.cc:697] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 LEADER]: Becoming Leader. State: Replica: a7e4f250aa5d48759eaabbdff8c99210, State: Running, Role: LEADER
I20260709 13:09:33.635288 25771 consensus_queue.cc:237] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.633677 25761 raft_consensus.cc:385] T 64becd02c9434edeaf5824c5a455b0e0 P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:33.636130 25761 raft_consensus.cc:740] T 64becd02c9434edeaf5824c5a455b0e0 P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99dfff6363df46ce97b027cb3bd42e11, State: Initialized, Role: FOLLOWER
I20260709 13:09:33.636966 25761 consensus_queue.cc:260] T 64becd02c9434edeaf5824c5a455b0e0 P 99dfff6363df46ce97b027cb3bd42e11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.639218 25761 ts_tablet_manager.cc:1434] T 64becd02c9434edeaf5824c5a455b0e0 P 99dfff6363df46ce97b027cb3bd42e11: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
I20260709 13:09:33.642802 25469 catalog_manager.cc:5697] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2). New cstate: current_term: 1 leader_uuid: "6a8699df36b74658a40f65ad30923ba9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:33.644304 25469 catalog_manager.cc:5697] T 6d5ba3150425499f9cfa64eb06d6ba91 P 6a8699df36b74658a40f65ad30923ba9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2). New cstate: current_term: 1 leader_uuid: "6a8699df36b74658a40f65ad30923ba9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:33.645747 25469 catalog_manager.cc:5697] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2). New cstate: current_term: 1 leader_uuid: "6a8699df36b74658a40f65ad30923ba9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:33.655113 25470 catalog_manager.cc:5697] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210 reported cstate change: term changed from 0 to 1, leader changed from <none> to a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "a7e4f250aa5d48759eaabbdff8c99210" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:33.826785 25798 consensus_queue.cc:1048] T fc82e3058d55474faed434b751f7c644 P 99dfff6363df46ce97b027cb3bd42e11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:33.838657 25798 consensus_queue.cc:1048] T fc82e3058d55474faed434b751f7c644 P 99dfff6363df46ce97b027cb3bd42e11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:33.874292 25766 raft_consensus.cc:493] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:33.874922 25766 raft_consensus.cc:515] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.876605 25766 leader_election.cc:290] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3:34359)
I20260709 13:09:33.877554 25570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64becd02c9434edeaf5824c5a455b0e0" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7e4f250aa5d48759eaabbdff8c99210" is_pre_election: true
I20260709 13:09:33.877867 25720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64becd02c9434edeaf5824c5a455b0e0" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99dfff6363df46ce97b027cb3bd42e11" is_pre_election: true
I20260709 13:09:33.878185 25570 raft_consensus.cc:2468] T 64becd02c9434edeaf5824c5a455b0e0 P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a8699df36b74658a40f65ad30923ba9 in term 0.
I20260709 13:09:33.878499 25720 raft_consensus.cc:2468] T 64becd02c9434edeaf5824c5a455b0e0 P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a8699df36b74658a40f65ad30923ba9 in term 0.
I20260709 13:09:33.879171 25612 leader_election.cc:304] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a8699df36b74658a40f65ad30923ba9, a7e4f250aa5d48759eaabbdff8c99210; no voters: 
I20260709 13:09:33.880039 25766 raft_consensus.cc:2804] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:33.880333 25766 raft_consensus.cc:493] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:33.880602 25766 raft_consensus.cc:3060] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:33.885725 25766 raft_consensus.cc:515] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.887179 25766 leader_election.cc:290] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 1 election: Requested vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3:34359)
I20260709 13:09:33.888000 25570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64becd02c9434edeaf5824c5a455b0e0" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7e4f250aa5d48759eaabbdff8c99210"
I20260709 13:09:33.888260 25720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64becd02c9434edeaf5824c5a455b0e0" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99dfff6363df46ce97b027cb3bd42e11"
I20260709 13:09:33.888537 25570 raft_consensus.cc:3060] T 64becd02c9434edeaf5824c5a455b0e0 P a7e4f250aa5d48759eaabbdff8c99210 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:33.888852 25720 raft_consensus.cc:3060] T 64becd02c9434edeaf5824c5a455b0e0 P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:33.893366 25570 raft_consensus.cc:2468] T 64becd02c9434edeaf5824c5a455b0e0 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a8699df36b74658a40f65ad30923ba9 in term 1.
I20260709 13:09:33.893364 25720 raft_consensus.cc:2468] T 64becd02c9434edeaf5824c5a455b0e0 P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a8699df36b74658a40f65ad30923ba9 in term 1.
I20260709 13:09:33.894444 25612 leader_election.cc:304] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a8699df36b74658a40f65ad30923ba9, a7e4f250aa5d48759eaabbdff8c99210; no voters: 
I20260709 13:09:33.895059 25766 raft_consensus.cc:2804] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:33.895447 25766 raft_consensus.cc:697] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9 [term 1 LEADER]: Becoming Leader. State: Replica: 6a8699df36b74658a40f65ad30923ba9, State: Running, Role: LEADER
I20260709 13:09:33.896098 25766 consensus_queue.cc:237] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:33.902921 25470 catalog_manager.cc:5697] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2). New cstate: current_term: 1 leader_uuid: "6a8699df36b74658a40f65ad30923ba9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:33.918845 25771 consensus_queue.cc:1048] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:33.919521 25766 consensus_queue.cc:1048] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:33.930981 25719 raft_consensus.cc:3060] T bab07f367f714676a11b512f3abbea7c P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:33.932489 25809 consensus_queue.cc:1048] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:33.943109 25810 consensus_queue.cc:1048] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:33.946718 25807 consensus_queue.cc:1048] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:33.960140 25809 consensus_queue.cc:1048] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:33.985581 25809 consensus_queue.cc:1048] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:33.995486 25807 consensus_queue.cc:1048] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:34.000239 25810 consensus_queue.cc:1048] T 6d5ba3150425499f9cfa64eb06d6ba91 P 6a8699df36b74658a40f65ad30923ba9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:34.005640 25802 raft_consensus.cc:493] T 8372ab57bb904260beb2b0c69bc39ad0 P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:34.006413 25802 raft_consensus.cc:515] T 8372ab57bb904260beb2b0c69bc39ad0 P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:34.007995 25767 consensus_queue.cc:1048] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:34.010154 25569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8372ab57bb904260beb2b0c69bc39ad0" candidate_uuid: "99dfff6363df46ce97b027cb3bd42e11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7e4f250aa5d48759eaabbdff8c99210" is_pre_election: true
I20260709 13:09:34.011197 25569 raft_consensus.cc:2393] T 8372ab57bb904260beb2b0c69bc39ad0 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99dfff6363df46ce97b027cb3bd42e11 in current term 1: Already voted for candidate 6a8699df36b74658a40f65ad30923ba9 in this term.
I20260709 13:09:34.014578 25802 leader_election.cc:290] T 8372ab57bb904260beb2b0c69bc39ad0 P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2:39169)
I20260709 13:09:34.015511 25646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8372ab57bb904260beb2b0c69bc39ad0" candidate_uuid: "99dfff6363df46ce97b027cb3bd42e11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a8699df36b74658a40f65ad30923ba9" is_pre_election: true
I20260709 13:09:34.019375 25684 leader_election.cc:304] T 8372ab57bb904260beb2b0c69bc39ad0 P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99dfff6363df46ce97b027cb3bd42e11; no voters: 6a8699df36b74658a40f65ad30923ba9, a7e4f250aa5d48759eaabbdff8c99210
I20260709 13:09:34.020805 25767 raft_consensus.cc:3060] T 8372ab57bb904260beb2b0c69bc39ad0 P 99dfff6363df46ce97b027cb3bd42e11 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:34.024475 25805 consensus_queue.cc:1048] T 6d5ba3150425499f9cfa64eb06d6ba91 P 6a8699df36b74658a40f65ad30923ba9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:34.033818 25802 consensus_queue.cc:1048] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:34.035540 25767 raft_consensus.cc:2749] T 8372ab57bb904260beb2b0c69bc39ad0 P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:34.048417 14748 ts_manager.cc:295] Set tserver state for a7e4f250aa5d48759eaabbdff8c99210 to MAINTENANCE_MODE
I20260709 13:09:34.049306 14748 tablet_server.cc:179] TabletServer@127.14.103.1:0 shutting down...
I20260709 13:09:34.049085 25807 consensus_queue.cc:1048] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:34.048295 25805 consensus_queue.cc:1048] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:09:34.101924 25686 consensus_peers.cc:597] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 -> Peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Couldn't send request to peer a7e4f250aa5d48759eaabbdff8c99210. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:34.107559 25809 consensus_queue.cc:1048] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:09:34.110654 25612 consensus_peers.cc:597] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9 -> Peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Couldn't send request to peer a7e4f250aa5d48759eaabbdff8c99210. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:34.130266 14748 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:09:34.130978 14748 tablet_replica.cc:333] T bab07f367f714676a11b512f3abbea7c P a7e4f250aa5d48759eaabbdff8c99210: stopping tablet replica
I20260709 13:09:34.131577 14748 raft_consensus.cc:2243] T bab07f367f714676a11b512f3abbea7c P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:34.132067 14748 raft_consensus.cc:2272] T bab07f367f714676a11b512f3abbea7c P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:34.134923 14748 tablet_replica.cc:333] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210: stopping tablet replica
I20260709 13:09:34.135461 14748 raft_consensus.cc:2243] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:34.136258 14748 raft_consensus.cc:2272] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:34.138320 14748 tablet_replica.cc:333] T fa373cebfd3540728639dddbaee4cb1d P a7e4f250aa5d48759eaabbdff8c99210: stopping tablet replica
I20260709 13:09:34.138851 14748 raft_consensus.cc:2243] T fa373cebfd3540728639dddbaee4cb1d P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:34.139236 14748 raft_consensus.cc:2272] T fa373cebfd3540728639dddbaee4cb1d P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:34.141309 14748 tablet_replica.cc:333] T 6d5ba3150425499f9cfa64eb06d6ba91 P a7e4f250aa5d48759eaabbdff8c99210: stopping tablet replica
I20260709 13:09:34.141880 14748 raft_consensus.cc:2243] T 6d5ba3150425499f9cfa64eb06d6ba91 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:09:34.141980 25518 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:09:34.142293 14748 raft_consensus.cc:2272] T 6d5ba3150425499f9cfa64eb06d6ba91 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:34.144544 14748 tablet_replica.cc:333] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210: stopping tablet replica
I20260709 13:09:34.145033 14748 raft_consensus.cc:2243] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:34.145746 14748 raft_consensus.cc:2272] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:34.147595 14748 tablet_replica.cc:333] T fc82e3058d55474faed434b751f7c644 P a7e4f250aa5d48759eaabbdff8c99210: stopping tablet replica
I20260709 13:09:34.148116 14748 raft_consensus.cc:2243] T fc82e3058d55474faed434b751f7c644 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:34.148447 14748 raft_consensus.cc:2272] T fc82e3058d55474faed434b751f7c644 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:34.150365 14748 tablet_replica.cc:333] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210: stopping tablet replica
I20260709 13:09:34.150822 14748 raft_consensus.cc:2243] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:34.151526 14748 raft_consensus.cc:2272] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:34.153452 14748 tablet_replica.cc:333] T 64becd02c9434edeaf5824c5a455b0e0 P a7e4f250aa5d48759eaabbdff8c99210: stopping tablet replica
I20260709 13:09:34.153905 14748 raft_consensus.cc:2243] T 64becd02c9434edeaf5824c5a455b0e0 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:34.154197 14748 raft_consensus.cc:2272] T 64becd02c9434edeaf5824c5a455b0e0 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:34.155694 14748 tablet_replica.cc:333] T 8372ab57bb904260beb2b0c69bc39ad0 P a7e4f250aa5d48759eaabbdff8c99210: stopping tablet replica
I20260709 13:09:34.156177 14748 raft_consensus.cc:2243] T 8372ab57bb904260beb2b0c69bc39ad0 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:34.156468 14748 raft_consensus.cc:2272] T 8372ab57bb904260beb2b0c69bc39ad0 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:34.157979 14748 tablet_replica.cc:333] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210: stopping tablet replica
I20260709 13:09:34.158357 14748 raft_consensus.cc:2243] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:34.159044 14748 raft_consensus.cc:2272] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:34.188215 14748 tablet_server.cc:196] TabletServer@127.14.103.1:0 shutdown complete.
I20260709 13:09:34.282410 25810 consensus_queue.cc:1048] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:09:34.295472 25612 consensus_peers.cc:597] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9 -> Peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Couldn't send request to peer a7e4f250aa5d48759eaabbdff8c99210. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:34.347500 25686 consensus_peers.cc:597] T fc82e3058d55474faed434b751f7c644 P 99dfff6363df46ce97b027cb3bd42e11 -> Peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Couldn't send request to peer a7e4f250aa5d48759eaabbdff8c99210. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:34.421423 25612 consensus_peers.cc:597] T 6d5ba3150425499f9cfa64eb06d6ba91 P 6a8699df36b74658a40f65ad30923ba9 -> Peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Couldn't send request to peer a7e4f250aa5d48759eaabbdff8c99210. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:34.440855 25612 consensus_peers.cc:597] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 -> Peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Couldn't send request to peer a7e4f250aa5d48759eaabbdff8c99210. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:35.143237 25518 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:09:35.412539 25686 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111) [suppressed 110 similar messages]
I20260709 13:09:35.434562 25844 raft_consensus.cc:493] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a7e4f250aa5d48759eaabbdff8c99210)
I20260709 13:09:35.435076 25844 raft_consensus.cc:515] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:35.436329 25789 raft_consensus.cc:493] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a7e4f250aa5d48759eaabbdff8c99210)
I20260709 13:09:35.436919 25789 raft_consensus.cc:515] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:35.437196 25844 leader_election.cc:290] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2:39169)
I20260709 13:09:35.438349 25647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a14348b4c6a4d3cbbf18beffcd03814" candidate_uuid: "99dfff6363df46ce97b027cb3bd42e11" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a8699df36b74658a40f65ad30923ba9" is_pre_election: true
I20260709 13:09:35.439023 25789 leader_election.cc:290] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3:34359)
I20260709 13:09:35.439025 25647 raft_consensus.cc:2468] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99dfff6363df46ce97b027cb3bd42e11 in term 1.
I20260709 13:09:35.440951 25684 leader_election.cc:304] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a8699df36b74658a40f65ad30923ba9, 99dfff6363df46ce97b027cb3bd42e11; no voters: 
I20260709 13:09:35.441023 25719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a14348b4c6a4d3cbbf18beffcd03814" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99dfff6363df46ce97b027cb3bd42e11" is_pre_election: true
I20260709 13:09:35.441781 25719 raft_consensus.cc:2468] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a8699df36b74658a40f65ad30923ba9 in term 1.
W20260709 13:09:35.441828 25686 leader_election.cc:336] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111)
I20260709 13:09:35.442384 25844 raft_consensus.cc:2804] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:35.442817 25844 raft_consensus.cc:493] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a7e4f250aa5d48759eaabbdff8c99210)
I20260709 13:09:35.443207 25844 raft_consensus.cc:3060] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:35.443293 25612 leader_election.cc:304] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a8699df36b74658a40f65ad30923ba9, 99dfff6363df46ce97b027cb3bd42e11; no voters: 
I20260709 13:09:35.444187 25789 raft_consensus.cc:2804] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:09:35.444267 25612 leader_election.cc:336] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111)
I20260709 13:09:35.444628 25789 raft_consensus.cc:493] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a7e4f250aa5d48759eaabbdff8c99210)
I20260709 13:09:35.445026 25789 raft_consensus.cc:3060] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:35.448792 25844 raft_consensus.cc:515] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:35.449874 25789 raft_consensus.cc:515] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:35.450508 25844 leader_election.cc:290] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 2 election: Requested vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2:39169)
I20260709 13:09:35.451606 25647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a14348b4c6a4d3cbbf18beffcd03814" candidate_uuid: "99dfff6363df46ce97b027cb3bd42e11" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a8699df36b74658a40f65ad30923ba9"
I20260709 13:09:35.451790 25789 leader_election.cc:290] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 2 election: Requested vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3:34359)
I20260709 13:09:35.452396 25647 raft_consensus.cc:2393] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 99dfff6363df46ce97b027cb3bd42e11 in current term 2: Already voted for candidate 6a8699df36b74658a40f65ad30923ba9 in this term.
I20260709 13:09:35.453063 25719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a14348b4c6a4d3cbbf18beffcd03814" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99dfff6363df46ce97b027cb3bd42e11"
I20260709 13:09:35.453815 25719 raft_consensus.cc:2393] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a8699df36b74658a40f65ad30923ba9 in current term 2: Already voted for candidate 99dfff6363df46ce97b027cb3bd42e11 in this term.
W20260709 13:09:35.454710 25686 leader_election.cc:336] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111)
I20260709 13:09:35.455108 25686 leader_election.cc:304] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99dfff6363df46ce97b027cb3bd42e11; no voters: 6a8699df36b74658a40f65ad30923ba9, a7e4f250aa5d48759eaabbdff8c99210
I20260709 13:09:35.455765 25844 raft_consensus.cc:2749] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:09:35.456944 25612 leader_election.cc:336] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111)
I20260709 13:09:35.457357 25612 leader_election.cc:304] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a8699df36b74658a40f65ad30923ba9; no voters: 99dfff6363df46ce97b027cb3bd42e11, a7e4f250aa5d48759eaabbdff8c99210
I20260709 13:09:35.457932 25789 raft_consensus.cc:2749] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:09:35.463258 25789 raft_consensus.cc:493] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a7e4f250aa5d48759eaabbdff8c99210)
I20260709 13:09:35.463663 25844 raft_consensus.cc:493] T 0d81fc45e39443b9b33953f8c66437eb P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a7e4f250aa5d48759eaabbdff8c99210)
I20260709 13:09:35.463799 25789 raft_consensus.cc:515] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:35.464221 25844 raft_consensus.cc:515] T 0d81fc45e39443b9b33953f8c66437eb P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:35.465639 25789 leader_election.cc:290] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3:34359)
I20260709 13:09:35.466944 25647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d81fc45e39443b9b33953f8c66437eb" candidate_uuid: "99dfff6363df46ce97b027cb3bd42e11" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a8699df36b74658a40f65ad30923ba9" is_pre_election: true
I20260709 13:09:35.467357 25844 leader_election.cc:290] T 0d81fc45e39443b9b33953f8c66437eb P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2:39169)
I20260709 13:09:35.467602 25647 raft_consensus.cc:2468] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99dfff6363df46ce97b027cb3bd42e11 in term 1.
I20260709 13:09:35.467387 25719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d81fc45e39443b9b33953f8c66437eb" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99dfff6363df46ce97b027cb3bd42e11" is_pre_election: true
I20260709 13:09:35.468894 25719 raft_consensus.cc:2468] T 0d81fc45e39443b9b33953f8c66437eb P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a8699df36b74658a40f65ad30923ba9 in term 1.
I20260709 13:09:35.469019 25684 leader_election.cc:304] T 0d81fc45e39443b9b33953f8c66437eb P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a8699df36b74658a40f65ad30923ba9, 99dfff6363df46ce97b027cb3bd42e11; no voters: 
I20260709 13:09:35.470187 25844 raft_consensus.cc:2804] T 0d81fc45e39443b9b33953f8c66437eb P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:35.470142 25612 leader_election.cc:304] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a8699df36b74658a40f65ad30923ba9, 99dfff6363df46ce97b027cb3bd42e11; no voters: 
W20260709 13:09:35.470582 25686 leader_election.cc:336] T 0d81fc45e39443b9b33953f8c66437eb P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111)
I20260709 13:09:35.470752 25844 raft_consensus.cc:493] T 0d81fc45e39443b9b33953f8c66437eb P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a7e4f250aa5d48759eaabbdff8c99210)
I20260709 13:09:35.471185 25789 raft_consensus.cc:2804] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:35.471307 25844 raft_consensus.cc:3060] T 0d81fc45e39443b9b33953f8c66437eb P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:09:35.471246 25612 leader_election.cc:336] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111)
I20260709 13:09:35.471768 25789 raft_consensus.cc:493] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a7e4f250aa5d48759eaabbdff8c99210)
I20260709 13:09:35.472230 25789 raft_consensus.cc:3060] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:35.479045 25844 raft_consensus.cc:515] T 0d81fc45e39443b9b33953f8c66437eb P 99dfff6363df46ce97b027cb3bd42e11 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:35.479094 25789 raft_consensus.cc:515] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:35.481389 25789 leader_election.cc:290] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 2 election: Requested vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3:34359)
I20260709 13:09:35.481990 25844 leader_election.cc:290] T 0d81fc45e39443b9b33953f8c66437eb P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 2 election: Requested vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2:39169)
I20260709 13:09:35.482591 25647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d81fc45e39443b9b33953f8c66437eb" candidate_uuid: "99dfff6363df46ce97b027cb3bd42e11" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a8699df36b74658a40f65ad30923ba9"
I20260709 13:09:35.483433 25647 raft_consensus.cc:2393] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 99dfff6363df46ce97b027cb3bd42e11 in current term 2: Already voted for candidate 6a8699df36b74658a40f65ad30923ba9 in this term.
I20260709 13:09:35.483852 25719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d81fc45e39443b9b33953f8c66437eb" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99dfff6363df46ce97b027cb3bd42e11"
I20260709 13:09:35.484606 25719 raft_consensus.cc:2393] T 0d81fc45e39443b9b33953f8c66437eb P 99dfff6363df46ce97b027cb3bd42e11 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a8699df36b74658a40f65ad30923ba9 in current term 2: Already voted for candidate 99dfff6363df46ce97b027cb3bd42e11 in this term.
W20260709 13:09:35.486137 25686 leader_election.cc:336] T 0d81fc45e39443b9b33953f8c66437eb P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111)
I20260709 13:09:35.486632 25686 leader_election.cc:304] T 0d81fc45e39443b9b33953f8c66437eb P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99dfff6363df46ce97b027cb3bd42e11; no voters: 6a8699df36b74658a40f65ad30923ba9, a7e4f250aa5d48759eaabbdff8c99210
W20260709 13:09:35.487288 25612 leader_election.cc:336] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111)
I20260709 13:09:35.487381 25844 raft_consensus.cc:2749] T 0d81fc45e39443b9b33953f8c66437eb P 99dfff6363df46ce97b027cb3bd42e11 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:09:35.487938 25612 leader_election.cc:304] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a8699df36b74658a40f65ad30923ba9; no voters: 99dfff6363df46ce97b027cb3bd42e11, a7e4f250aa5d48759eaabbdff8c99210
I20260709 13:09:35.488636 25789 raft_consensus.cc:2749] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:09:35.494781 25844 raft_consensus.cc:493] T e1f6c52711174eacb83ea8fd2ad1b57b P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a7e4f250aa5d48759eaabbdff8c99210)
I20260709 13:09:35.495301 25844 raft_consensus.cc:515] T e1f6c52711174eacb83ea8fd2ad1b57b P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:35.497172 25844 leader_election.cc:290] T e1f6c52711174eacb83ea8fd2ad1b57b P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2:39169)
I20260709 13:09:35.498308 25647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1f6c52711174eacb83ea8fd2ad1b57b" candidate_uuid: "99dfff6363df46ce97b027cb3bd42e11" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a8699df36b74658a40f65ad30923ba9" is_pre_election: true
I20260709 13:09:35.499015 25647 raft_consensus.cc:2468] T e1f6c52711174eacb83ea8fd2ad1b57b P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99dfff6363df46ce97b027cb3bd42e11 in term 1.
I20260709 13:09:35.499483 25789 raft_consensus.cc:493] T e1f6c52711174eacb83ea8fd2ad1b57b P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a7e4f250aa5d48759eaabbdff8c99210)
I20260709 13:09:35.499993 25789 raft_consensus.cc:515] T e1f6c52711174eacb83ea8fd2ad1b57b P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:35.500154 25684 leader_election.cc:304] T e1f6c52711174eacb83ea8fd2ad1b57b P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a8699df36b74658a40f65ad30923ba9, 99dfff6363df46ce97b027cb3bd42e11; no voters: 
W20260709 13:09:35.500972 25686 leader_election.cc:336] T e1f6c52711174eacb83ea8fd2ad1b57b P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111)
I20260709 13:09:35.501001 25844 raft_consensus.cc:2804] T e1f6c52711174eacb83ea8fd2ad1b57b P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:35.501616 25844 raft_consensus.cc:493] T e1f6c52711174eacb83ea8fd2ad1b57b P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a7e4f250aa5d48759eaabbdff8c99210)
I20260709 13:09:35.502027 25844 raft_consensus.cc:3060] T e1f6c52711174eacb83ea8fd2ad1b57b P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:35.502698 25789 leader_election.cc:290] T e1f6c52711174eacb83ea8fd2ad1b57b P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3:34359)
I20260709 13:09:35.503706 25719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1f6c52711174eacb83ea8fd2ad1b57b" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99dfff6363df46ce97b027cb3bd42e11" is_pre_election: true
W20260709 13:09:35.505816 25612 leader_election.cc:336] T e1f6c52711174eacb83ea8fd2ad1b57b P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111)
I20260709 13:09:35.508428 25844 raft_consensus.cc:515] T e1f6c52711174eacb83ea8fd2ad1b57b P 99dfff6363df46ce97b027cb3bd42e11 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:35.509399 25719 raft_consensus.cc:2393] T e1f6c52711174eacb83ea8fd2ad1b57b P 99dfff6363df46ce97b027cb3bd42e11 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a8699df36b74658a40f65ad30923ba9 in current term 2: Already voted for candidate 99dfff6363df46ce97b027cb3bd42e11 in this term.
I20260709 13:09:35.510313 25844 leader_election.cc:290] T e1f6c52711174eacb83ea8fd2ad1b57b P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 2 election: Requested vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2:39169)
I20260709 13:09:35.511420 25647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1f6c52711174eacb83ea8fd2ad1b57b" candidate_uuid: "99dfff6363df46ce97b027cb3bd42e11" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a8699df36b74658a40f65ad30923ba9"
I20260709 13:09:35.511592 25612 leader_election.cc:304] T e1f6c52711174eacb83ea8fd2ad1b57b P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a8699df36b74658a40f65ad30923ba9; no voters: 99dfff6363df46ce97b027cb3bd42e11, a7e4f250aa5d48759eaabbdff8c99210
I20260709 13:09:35.512097 25647 raft_consensus.cc:3060] T e1f6c52711174eacb83ea8fd2ad1b57b P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:09:35.514016 25686 leader_election.cc:336] T e1f6c52711174eacb83ea8fd2ad1b57b P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111)
I20260709 13:09:35.517710 25647 raft_consensus.cc:2468] T e1f6c52711174eacb83ea8fd2ad1b57b P 6a8699df36b74658a40f65ad30923ba9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99dfff6363df46ce97b027cb3bd42e11 in term 2.
I20260709 13:09:35.518232 25789 raft_consensus.cc:2749] T e1f6c52711174eacb83ea8fd2ad1b57b P 6a8699df36b74658a40f65ad30923ba9 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:35.518800 25684 leader_election.cc:304] T e1f6c52711174eacb83ea8fd2ad1b57b P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a8699df36b74658a40f65ad30923ba9, 99dfff6363df46ce97b027cb3bd42e11; no voters: a7e4f250aa5d48759eaabbdff8c99210
I20260709 13:09:35.519568 25844 raft_consensus.cc:2804] T e1f6c52711174eacb83ea8fd2ad1b57b P 99dfff6363df46ce97b027cb3bd42e11 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:35.520419 25844 raft_consensus.cc:697] T e1f6c52711174eacb83ea8fd2ad1b57b P 99dfff6363df46ce97b027cb3bd42e11 [term 2 LEADER]: Becoming Leader. State: Replica: 99dfff6363df46ce97b027cb3bd42e11, State: Running, Role: LEADER
I20260709 13:09:35.521242 25844 consensus_queue.cc:237] T e1f6c52711174eacb83ea8fd2ad1b57b P 99dfff6363df46ce97b027cb3bd42e11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:35.528299 25470 catalog_manager.cc:5697] T e1f6c52711174eacb83ea8fd2ad1b57b P 99dfff6363df46ce97b027cb3bd42e11 reported cstate change: term changed from 1 to 2, leader changed from a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1) to 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3). New cstate: current_term: 2 leader_uuid: "99dfff6363df46ce97b027cb3bd42e11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:35.601272 25855 raft_consensus.cc:493] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a7e4f250aa5d48759eaabbdff8c99210)
I20260709 13:09:35.601791 25855 raft_consensus.cc:515] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:35.603389 25855 leader_election.cc:290] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2:39169)
I20260709 13:09:35.604461 25647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "377fe3ce2b474722b845c07b3f74b830" candidate_uuid: "99dfff6363df46ce97b027cb3bd42e11" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a8699df36b74658a40f65ad30923ba9" is_pre_election: true
W20260709 13:09:35.607000 25686 leader_election.cc:336] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111)
I20260709 13:09:35.607383 25686 leader_election.cc:304] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99dfff6363df46ce97b027cb3bd42e11; no voters: 6a8699df36b74658a40f65ad30923ba9, a7e4f250aa5d48759eaabbdff8c99210
I20260709 13:09:35.607982 25855 raft_consensus.cc:2749] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:35.613812 25789 raft_consensus.cc:493] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a7e4f250aa5d48759eaabbdff8c99210)
I20260709 13:09:35.614346 25789 raft_consensus.cc:515] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:35.616449 25789 leader_election.cc:290] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3:34359)
I20260709 13:09:35.617950 25719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "377fe3ce2b474722b845c07b3f74b830" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99dfff6363df46ce97b027cb3bd42e11" is_pre_election: true
I20260709 13:09:35.618688 25719 raft_consensus.cc:2468] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a8699df36b74658a40f65ad30923ba9 in term 1.
I20260709 13:09:35.619619 25612 leader_election.cc:304] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a8699df36b74658a40f65ad30923ba9, 99dfff6363df46ce97b027cb3bd42e11; no voters: 
I20260709 13:09:35.620330 25789 raft_consensus.cc:2804] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:09:35.620399 25612 leader_election.cc:336] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111)
I20260709 13:09:35.620723 25789 raft_consensus.cc:493] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a7e4f250aa5d48759eaabbdff8c99210)
I20260709 13:09:35.621098 25789 raft_consensus.cc:3060] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:35.627128 25789 raft_consensus.cc:515] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:35.628840 25789 leader_election.cc:290] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 2 election: Requested vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3:34359)
I20260709 13:09:35.630261 25719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "377fe3ce2b474722b845c07b3f74b830" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99dfff6363df46ce97b027cb3bd42e11"
I20260709 13:09:35.630962 25719 raft_consensus.cc:3060] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:09:35.633198 25612 leader_election.cc:336] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111)
I20260709 13:09:35.636943 25719 raft_consensus.cc:2468] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a8699df36b74658a40f65ad30923ba9 in term 2.
I20260709 13:09:35.637871 25612 leader_election.cc:304] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a8699df36b74658a40f65ad30923ba9, 99dfff6363df46ce97b027cb3bd42e11; no voters: a7e4f250aa5d48759eaabbdff8c99210
I20260709 13:09:35.638506 25789 raft_consensus.cc:2804] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:35.638967 25789 raft_consensus.cc:697] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [term 2 LEADER]: Becoming Leader. State: Replica: 6a8699df36b74658a40f65ad30923ba9, State: Running, Role: LEADER
I20260709 13:09:35.639582 25789 consensus_queue.cc:237] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:35.646672 25470 catalog_manager.cc:5697] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 reported cstate change: term changed from 1 to 2, leader changed from a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1) to 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2). New cstate: current_term: 2 leader_uuid: "6a8699df36b74658a40f65ad30923ba9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:35.974689 25647 raft_consensus.cc:1275] T e1f6c52711174eacb83ea8fd2ad1b57b P 6a8699df36b74658a40f65ad30923ba9 [term 2 FOLLOWER]: Refusing update from remote peer 99dfff6363df46ce97b027cb3bd42e11: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:35.975991 25855 consensus_queue.cc:1048] T e1f6c52711174eacb83ea8fd2ad1b57b P 99dfff6363df46ce97b027cb3bd42e11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:09:35.992724 25686 consensus_peers.cc:597] T e1f6c52711174eacb83ea8fd2ad1b57b P 99dfff6363df46ce97b027cb3bd42e11 -> Peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Couldn't send request to peer a7e4f250aa5d48759eaabbdff8c99210. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:36.048369 25719 raft_consensus.cc:1275] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11 [term 2 FOLLOWER]: Refusing update from remote peer 6a8699df36b74658a40f65ad30923ba9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:36.049652 25789 consensus_queue.cc:1048] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:09:36.061688 25612 consensus_peers.cc:597] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 -> Peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Couldn't send request to peer a7e4f250aa5d48759eaabbdff8c99210. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:36.144289 25518 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:09:36.621084 25612 consensus_peers.cc:597] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9 -> Peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Couldn't send request to peer a7e4f250aa5d48759eaabbdff8c99210. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:36.639889 25686 consensus_peers.cc:597] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 -> Peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Couldn't send request to peer a7e4f250aa5d48759eaabbdff8c99210. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:36.649366 25612 consensus_peers.cc:597] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9 -> Peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Couldn't send request to peer a7e4f250aa5d48759eaabbdff8c99210. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:36.769203 25612 consensus_peers.cc:597] T 6d5ba3150425499f9cfa64eb06d6ba91 P 6a8699df36b74658a40f65ad30923ba9 -> Peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Couldn't send request to peer a7e4f250aa5d48759eaabbdff8c99210. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:36.922838 25686 consensus_peers.cc:597] T fc82e3058d55474faed434b751f7c644 P 99dfff6363df46ce97b027cb3bd42e11 -> Peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Couldn't send request to peer a7e4f250aa5d48759eaabbdff8c99210. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:37.145269 25518 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:09:37.210850 25612 consensus_peers.cc:597] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 -> Peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Couldn't send request to peer a7e4f250aa5d48759eaabbdff8c99210. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:09:37.293183 25789 raft_consensus.cc:493] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:37.293675 25789 raft_consensus.cc:515] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:37.295361 25789 leader_election.cc:290] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3:34359)
I20260709 13:09:37.296768 25719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d81fc45e39443b9b33953f8c66437eb" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99dfff6363df46ce97b027cb3bd42e11" is_pre_election: true
I20260709 13:09:37.297435 25719 raft_consensus.cc:2468] T 0d81fc45e39443b9b33953f8c66437eb P 99dfff6363df46ce97b027cb3bd42e11 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a8699df36b74658a40f65ad30923ba9 in term 2.
I20260709 13:09:37.298557 25612 leader_election.cc:304] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a8699df36b74658a40f65ad30923ba9, 99dfff6363df46ce97b027cb3bd42e11; no voters: 
I20260709 13:09:37.299145 25789 raft_consensus.cc:2804] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:09:37.299197 25612 leader_election.cc:336] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111)
I20260709 13:09:37.299567 25789 raft_consensus.cc:493] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:37.299954 25789 raft_consensus.cc:3060] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:37.304736 25789 raft_consensus.cc:515] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:37.306267 25789 leader_election.cc:290] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 3 election: Requested vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3:34359)
I20260709 13:09:37.307531 25719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d81fc45e39443b9b33953f8c66437eb" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99dfff6363df46ce97b027cb3bd42e11"
I20260709 13:09:37.308175 25719 raft_consensus.cc:3060] T 0d81fc45e39443b9b33953f8c66437eb P 99dfff6363df46ce97b027cb3bd42e11 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:09:37.311638 25612 leader_election.cc:336] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111)
I20260709 13:09:37.313967 25719 raft_consensus.cc:2468] T 0d81fc45e39443b9b33953f8c66437eb P 99dfff6363df46ce97b027cb3bd42e11 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a8699df36b74658a40f65ad30923ba9 in term 3.
I20260709 13:09:37.314917 25612 leader_election.cc:304] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a8699df36b74658a40f65ad30923ba9, 99dfff6363df46ce97b027cb3bd42e11; no voters: a7e4f250aa5d48759eaabbdff8c99210
I20260709 13:09:37.315517 25789 raft_consensus.cc:2804] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:37.316326 25789 raft_consensus.cc:697] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [term 3 LEADER]: Becoming Leader. State: Replica: 6a8699df36b74658a40f65ad30923ba9, State: Running, Role: LEADER
I20260709 13:09:37.317013 25789 consensus_queue.cc:237] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:37.323822 25469 catalog_manager.cc:5697] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 reported cstate change: term changed from 1 to 3, leader changed from a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1) to 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2). New cstate: current_term: 3 leader_uuid: "6a8699df36b74658a40f65ad30923ba9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:37.393079 25855 raft_consensus.cc:493] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:37.393541 25855 raft_consensus.cc:515] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:37.395123 25855 leader_election.cc:290] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2:39169)
I20260709 13:09:37.396222 25647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a14348b4c6a4d3cbbf18beffcd03814" candidate_uuid: "99dfff6363df46ce97b027cb3bd42e11" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a8699df36b74658a40f65ad30923ba9" is_pre_election: true
I20260709 13:09:37.396889 25647 raft_consensus.cc:2468] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99dfff6363df46ce97b027cb3bd42e11 in term 2.
I20260709 13:09:37.397989 25684 leader_election.cc:304] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a8699df36b74658a40f65ad30923ba9, 99dfff6363df46ce97b027cb3bd42e11; no voters: 
I20260709 13:09:37.398692 25855 raft_consensus.cc:2804] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:09:37.398732 25686 leader_election.cc:336] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111)
I20260709 13:09:37.399035 25855 raft_consensus.cc:493] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:37.399384 25855 raft_consensus.cc:3060] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:37.404241 25855 raft_consensus.cc:515] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:37.405771 25855 leader_election.cc:290] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 3 election: Requested vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2:39169)
I20260709 13:09:37.406839 25647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a14348b4c6a4d3cbbf18beffcd03814" candidate_uuid: "99dfff6363df46ce97b027cb3bd42e11" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6a8699df36b74658a40f65ad30923ba9"
I20260709 13:09:37.407380 25647 raft_consensus.cc:3060] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:09:37.409175 25686 leader_election.cc:336] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111)
I20260709 13:09:37.412276 25647 raft_consensus.cc:2468] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99dfff6363df46ce97b027cb3bd42e11 in term 3.
I20260709 13:09:37.413283 25684 leader_election.cc:304] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a8699df36b74658a40f65ad30923ba9, 99dfff6363df46ce97b027cb3bd42e11; no voters: a7e4f250aa5d48759eaabbdff8c99210
I20260709 13:09:37.413988 25855 raft_consensus.cc:2804] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:37.415380 25855 raft_consensus.cc:697] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 3 LEADER]: Becoming Leader. State: Replica: 99dfff6363df46ce97b027cb3bd42e11, State: Running, Role: LEADER
I20260709 13:09:37.416358 25855 consensus_queue.cc:237] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:37.424480 25469 catalog_manager.cc:5697] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 reported cstate change: term changed from 1 to 3, leader changed from a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1) to 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3). New cstate: current_term: 3 leader_uuid: "99dfff6363df46ce97b027cb3bd42e11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:37.703900 25719 raft_consensus.cc:1275] T 0d81fc45e39443b9b33953f8c66437eb P 99dfff6363df46ce97b027cb3bd42e11 [term 3 FOLLOWER]: Refusing update from remote peer 6a8699df36b74658a40f65ad30923ba9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:09:37.705034 25789 consensus_queue.cc:1048] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:09:37.724465 25612 consensus_peers.cc:597] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 -> Peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Couldn't send request to peer a7e4f250aa5d48759eaabbdff8c99210. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:37.806355 25647 raft_consensus.cc:1275] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [term 3 FOLLOWER]: Refusing update from remote peer 99dfff6363df46ce97b027cb3bd42e11: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:09:37.807593 25855 consensus_queue.cc:1048] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:09:37.819638 25686 consensus_peers.cc:597] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 -> Peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Couldn't send request to peer a7e4f250aa5d48759eaabbdff8c99210. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:38.146154 25518 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:09:38.199959 25686 consensus_peers.cc:597] T e1f6c52711174eacb83ea8fd2ad1b57b P 99dfff6363df46ce97b027cb3bd42e11 -> Peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Couldn't send request to peer a7e4f250aa5d48759eaabbdff8c99210. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:38.610538 25612 consensus_peers.cc:597] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 -> Peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Couldn't send request to peer a7e4f250aa5d48759eaabbdff8c99210. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:09:39.002790 25612 consensus_peers.cc:597] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9 -> Peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Couldn't send request to peer a7e4f250aa5d48759eaabbdff8c99210. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:09:39.147056 25518 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:09:39.193122 25612 consensus_peers.cc:597] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9 -> Peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Couldn't send request to peer a7e4f250aa5d48759eaabbdff8c99210. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:09:39.193630 25686 consensus_peers.cc:597] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 -> Peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Couldn't send request to peer a7e4f250aa5d48759eaabbdff8c99210. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:09:39.206889 14748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:39.213207 25885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:39.213819 25886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:39.216938 14748 server_base.cc:1061] running on GCE node
W20260709 13:09:39.216964 25888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:39.218163 14748 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:39.218393 14748 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:39.218642 14748 hybrid_clock.cc:648] HybridClock initialized: now 1783602579218622 us; error 0 us; skew 500 ppm
I20260709 13:09:39.219182 14748 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:39.221589 14748 webserver.cc:533] Webserver started at http://127.14.103.1:34333/ using document root <none> and password file <none>
I20260709 13:09:39.222160 14748 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:39.222347 14748 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:09:39.225786 14748 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:09:39.229615 25893 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:39.230540 14748 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:09:39.230805 14748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602247531521-14748-0/minicluster-data/ts-0-root
uuid: "a7e4f250aa5d48759eaabbdff8c99210"
format_stamp: "Formatted at 2026-07-09 13:09:32 on dist-test-slave-fpp7"
I20260709 13:09:39.231038 14748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602247531521-14748-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602247531521-14748-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602247531521-14748-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:39.243595 14748 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:39.244674 14748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:39.246479 14748 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:09:39.250437 25900 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
W20260709 13:09:39.359254 25612 consensus_peers.cc:597] T 6d5ba3150425499f9cfa64eb06d6ba91 P 6a8699df36b74658a40f65ad30923ba9 -> Peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Couldn't send request to peer a7e4f250aa5d48759eaabbdff8c99210. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:09:39.364745 14748 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 13:09:39.365136 14748 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.116s	user 0.003s	sys 0.000s
I20260709 13:09:39.365541 14748 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 13:09:39.372186 25900 tablet_bootstrap.cc:492] T 64becd02c9434edeaf5824c5a455b0e0 P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap starting.
I20260709 13:09:39.388052 25900 tablet_bootstrap.cc:492] T 64becd02c9434edeaf5824c5a455b0e0 P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:39.388965 25900 tablet_bootstrap.cc:492] T 64becd02c9434edeaf5824c5a455b0e0 P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap complete.
I20260709 13:09:39.389667 25900 ts_tablet_manager.cc:1403] T 64becd02c9434edeaf5824c5a455b0e0 P a7e4f250aa5d48759eaabbdff8c99210: Time spent bootstrapping tablet: real 0.018s	user 0.015s	sys 0.000s
I20260709 13:09:39.393250 25900 raft_consensus.cc:359] T 64becd02c9434edeaf5824c5a455b0e0 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:39.394367 25900 raft_consensus.cc:740] T 64becd02c9434edeaf5824c5a455b0e0 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7e4f250aa5d48759eaabbdff8c99210, State: Initialized, Role: FOLLOWER
I20260709 13:09:39.395213 25900 consensus_queue.cc:260] T 64becd02c9434edeaf5824c5a455b0e0 P a7e4f250aa5d48759eaabbdff8c99210 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:39.398238 25900 ts_tablet_manager.cc:1434] T 64becd02c9434edeaf5824c5a455b0e0 P a7e4f250aa5d48759eaabbdff8c99210: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:09:39.399096 25900 tablet_bootstrap.cc:492] T fa373cebfd3540728639dddbaee4cb1d P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap starting.
I20260709 13:09:39.414489 25900 tablet_bootstrap.cc:492] T fa373cebfd3540728639dddbaee4cb1d P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:39.415392 25900 tablet_bootstrap.cc:492] T fa373cebfd3540728639dddbaee4cb1d P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap complete.
I20260709 13:09:39.416100 25900 ts_tablet_manager.cc:1403] T fa373cebfd3540728639dddbaee4cb1d P a7e4f250aa5d48759eaabbdff8c99210: Time spent bootstrapping tablet: real 0.017s	user 0.009s	sys 0.007s
I20260709 13:09:39.418762 25900 raft_consensus.cc:359] T fa373cebfd3540728639dddbaee4cb1d P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } } peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } }
I20260709 13:09:39.419461 25900 raft_consensus.cc:740] T fa373cebfd3540728639dddbaee4cb1d P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7e4f250aa5d48759eaabbdff8c99210, State: Initialized, Role: FOLLOWER
I20260709 13:09:39.420221 25900 consensus_queue.cc:260] T fa373cebfd3540728639dddbaee4cb1d P a7e4f250aa5d48759eaabbdff8c99210 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } } peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } }
I20260709 13:09:39.422344 25900 ts_tablet_manager.cc:1434] T fa373cebfd3540728639dddbaee4cb1d P a7e4f250aa5d48759eaabbdff8c99210: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.004s
I20260709 13:09:39.423130 25900 tablet_bootstrap.cc:492] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap starting.
I20260709 13:09:39.429031 14748 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 13:09:39.429409 14748 ts_tablet_manager.cc:595] Time spent register tablets: real 0.064s	user 0.055s	sys 0.008s
I20260709 13:09:39.441344 25900 tablet_bootstrap.cc:492] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:39.442698 25900 tablet_bootstrap.cc:492] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap complete.
I20260709 13:09:39.443584 25900 ts_tablet_manager.cc:1403] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.007s
I20260709 13:09:39.446964 25900 raft_consensus.cc:359] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:39.447860 25900 raft_consensus.cc:740] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7e4f250aa5d48759eaabbdff8c99210, State: Initialized, Role: FOLLOWER
I20260709 13:09:39.448808 25900 consensus_queue.cc:260] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:39.451383 25900 ts_tablet_manager.cc:1434] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.004s
I20260709 13:09:39.452404 25900 tablet_bootstrap.cc:492] T 8372ab57bb904260beb2b0c69bc39ad0 P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap starting.
I20260709 13:09:39.469008 25900 tablet_bootstrap.cc:492] T 8372ab57bb904260beb2b0c69bc39ad0 P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:39.469988 25900 tablet_bootstrap.cc:492] T 8372ab57bb904260beb2b0c69bc39ad0 P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap complete.
I20260709 13:09:39.470772 25900 ts_tablet_manager.cc:1403] T 8372ab57bb904260beb2b0c69bc39ad0 P a7e4f250aa5d48759eaabbdff8c99210: Time spent bootstrapping tablet: real 0.019s	user 0.010s	sys 0.005s
I20260709 13:09:39.473865 25900 raft_consensus.cc:359] T 8372ab57bb904260beb2b0c69bc39ad0 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:39.474705 25900 raft_consensus.cc:740] T 8372ab57bb904260beb2b0c69bc39ad0 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7e4f250aa5d48759eaabbdff8c99210, State: Initialized, Role: FOLLOWER
I20260709 13:09:39.475570 25900 consensus_queue.cc:260] T 8372ab57bb904260beb2b0c69bc39ad0 P a7e4f250aa5d48759eaabbdff8c99210 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:39.477640 25900 ts_tablet_manager.cc:1434] T 8372ab57bb904260beb2b0c69bc39ad0 P a7e4f250aa5d48759eaabbdff8c99210: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:09:39.478504 25900 tablet_bootstrap.cc:492] T fc82e3058d55474faed434b751f7c644 P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap starting.
I20260709 13:09:39.495028 14748 rpc_server.cc:307] RPC server started. Bound to: 127.14.103.1:43185
I20260709 13:09:39.495273 25966 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.103.1:43185 every 8 connection(s)
I20260709 13:09:39.497591 25900 tablet_bootstrap.cc:492] T fc82e3058d55474faed434b751f7c644 P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:39.498329 25900 tablet_bootstrap.cc:492] T fc82e3058d55474faed434b751f7c644 P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap complete.
I20260709 13:09:39.498912 25900 ts_tablet_manager.cc:1403] T fc82e3058d55474faed434b751f7c644 P a7e4f250aa5d48759eaabbdff8c99210: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.009s
W20260709 13:09:39.500065 25968 tablet.cc:2406] T 64becd02c9434edeaf5824c5a455b0e0 P a7e4f250aa5d48759eaabbdff8c99210: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:09:39.501138 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:39.502002 25900 raft_consensus.cc:359] T fc82e3058d55474faed434b751f7c644 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:39.502898 25900 raft_consensus.cc:740] T fc82e3058d55474faed434b751f7c644 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7e4f250aa5d48759eaabbdff8c99210, State: Initialized, Role: FOLLOWER
I20260709 13:09:39.504175 25900 consensus_queue.cc:260] T fc82e3058d55474faed434b751f7c644 P a7e4f250aa5d48759eaabbdff8c99210 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:39.507293 25900 ts_tablet_manager.cc:1434] T fc82e3058d55474faed434b751f7c644 P a7e4f250aa5d48759eaabbdff8c99210: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.001s
I20260709 13:09:39.508482 25900 tablet_bootstrap.cc:492] T bab07f367f714676a11b512f3abbea7c P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap starting.
I20260709 13:09:39.534518 25967 heartbeater.cc:344] Connected to a master server at 127.14.103.62:33155
I20260709 13:09:39.535094 25967 heartbeater.cc:461] Registering TS with master...
I20260709 13:09:39.536201 25967 heartbeater.cc:507] Master 127.14.103.62:33155 requested a full tablet report, sending...
I20260709 13:09:39.542694 25900 tablet_bootstrap.cc:492] T bab07f367f714676a11b512f3abbea7c P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:39.544086 25900 tablet_bootstrap.cc:492] T bab07f367f714676a11b512f3abbea7c P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap complete.
I20260709 13:09:39.544771 25900 ts_tablet_manager.cc:1403] T bab07f367f714676a11b512f3abbea7c P a7e4f250aa5d48759eaabbdff8c99210: Time spent bootstrapping tablet: real 0.037s	user 0.014s	sys 0.013s
I20260709 13:09:39.544893 25470 ts_manager.cc:194] Re-registered known tserver with Master: a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185)
I20260709 13:09:39.547475 25900 raft_consensus.cc:359] T bab07f367f714676a11b512f3abbea7c P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:39.550227 25900 raft_consensus.cc:740] T bab07f367f714676a11b512f3abbea7c P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7e4f250aa5d48759eaabbdff8c99210, State: Initialized, Role: FOLLOWER
I20260709 13:09:39.551007 25900 consensus_queue.cc:260] T bab07f367f714676a11b512f3abbea7c P a7e4f250aa5d48759eaabbdff8c99210 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:39.553164 25900 ts_tablet_manager.cc:1434] T bab07f367f714676a11b512f3abbea7c P a7e4f250aa5d48759eaabbdff8c99210: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:09:39.554947 25900 tablet_bootstrap.cc:492] T 6d5ba3150425499f9cfa64eb06d6ba91 P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap starting.
I20260709 13:09:39.558274 25470 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54414
I20260709 13:09:39.564564 25967 heartbeater.cc:499] Master 127.14.103.62:33155 was elected leader, sending a full tablet report...
I20260709 13:09:39.565769 25647 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "377fe3ce2b474722b845c07b3f74b830"
dest_uuid: "6a8699df36b74658a40f65ad30923ba9"
mode: GRACEFUL
new_leader_uuid: "99dfff6363df46ce97b027cb3bd42e11"
 from {username='slave'} at 127.0.0.1:39170
I20260709 13:09:39.566571 25647 raft_consensus.cc:606] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [term 2 LEADER]: Received request to transfer leadership to TS 99dfff6363df46ce97b027cb3bd42e11
I20260709 13:09:39.570482 25647 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "64becd02c9434edeaf5824c5a455b0e0"
dest_uuid: "6a8699df36b74658a40f65ad30923ba9"
mode: GRACEFUL
new_leader_uuid: "99dfff6363df46ce97b027cb3bd42e11"
 from {username='slave'} at 127.0.0.1:39170
I20260709 13:09:39.571482 25647 raft_consensus.cc:606] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9 [term 1 LEADER]: Received request to transfer leadership to TS 99dfff6363df46ce97b027cb3bd42e11
I20260709 13:09:39.573611 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:09:39.574352 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:09:39.575351 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:39.581010 25900 tablet_bootstrap.cc:492] T 6d5ba3150425499f9cfa64eb06d6ba91 P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:39.582027 25900 tablet_bootstrap.cc:492] T 6d5ba3150425499f9cfa64eb06d6ba91 P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap complete.
I20260709 13:09:39.583210 25900 ts_tablet_manager.cc:1403] T 6d5ba3150425499f9cfa64eb06d6ba91 P a7e4f250aa5d48759eaabbdff8c99210: Time spent bootstrapping tablet: real 0.030s	user 0.019s	sys 0.007s
I20260709 13:09:39.585558 25900 raft_consensus.cc:359] T 6d5ba3150425499f9cfa64eb06d6ba91 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:39.586079 25900 raft_consensus.cc:740] T 6d5ba3150425499f9cfa64eb06d6ba91 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7e4f250aa5d48759eaabbdff8c99210, State: Initialized, Role: FOLLOWER
I20260709 13:09:39.586702 25900 consensus_queue.cc:260] T 6d5ba3150425499f9cfa64eb06d6ba91 P a7e4f250aa5d48759eaabbdff8c99210 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:39.588235 25900 ts_tablet_manager.cc:1434] T 6d5ba3150425499f9cfa64eb06d6ba91 P a7e4f250aa5d48759eaabbdff8c99210: Time spent starting tablet: real 0.005s	user 0.001s	sys 0.003s
I20260709 13:09:39.588866 25900 tablet_bootstrap.cc:492] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap starting.
I20260709 13:09:39.608026 25900 tablet_bootstrap.cc:492] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:39.609148 25900 tablet_bootstrap.cc:492] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap complete.
I20260709 13:09:39.609810 25900 ts_tablet_manager.cc:1403] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.004s
I20260709 13:09:39.611871 25900 raft_consensus.cc:359] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:39.612378 25900 raft_consensus.cc:740] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7e4f250aa5d48759eaabbdff8c99210, State: Initialized, Role: FOLLOWER
I20260709 13:09:39.612917 25900 consensus_queue.cc:260] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:39.614476 25900 ts_tablet_manager.cc:1434] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210: Time spent starting tablet: real 0.004s	user 0.002s	sys 0.000s
I20260709 13:09:39.615298 25900 tablet_bootstrap.cc:492] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap starting.
I20260709 13:09:39.628127 25900 tablet_bootstrap.cc:492] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:39.628862 25900 tablet_bootstrap.cc:492] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap complete.
I20260709 13:09:39.629405 25900 ts_tablet_manager.cc:1403] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210: Time spent bootstrapping tablet: real 0.014s	user 0.012s	sys 0.000s
I20260709 13:09:39.631436 25900 raft_consensus.cc:359] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:39.632023 25900 raft_consensus.cc:740] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7e4f250aa5d48759eaabbdff8c99210, State: Initialized, Role: FOLLOWER
I20260709 13:09:39.632642 25900 consensus_queue.cc:260] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:39.634200 25900 ts_tablet_manager.cc:1434] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:09:39.634980 25900 tablet_bootstrap.cc:492] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap starting.
I20260709 13:09:39.647549 25900 tablet_bootstrap.cc:492] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:39.648321 25900 tablet_bootstrap.cc:492] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210: Bootstrap complete.
I20260709 13:09:39.648859 25900 ts_tablet_manager.cc:1403] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210: Time spent bootstrapping tablet: real 0.014s	user 0.012s	sys 0.000s
I20260709 13:09:39.650692 25900 raft_consensus.cc:359] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:39.651187 25900 raft_consensus.cc:740] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7e4f250aa5d48759eaabbdff8c99210, State: Initialized, Role: FOLLOWER
I20260709 13:09:39.651892 25900 consensus_queue.cc:260] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:39.653384 25900 ts_tablet_manager.cc:1434] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210: Time spent starting tablet: real 0.004s	user 0.001s	sys 0.003s
I20260709 13:09:39.679811 25904 raft_consensus.cc:993] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9: : Instructing follower 99dfff6363df46ce97b027cb3bd42e11 to start an election
I20260709 13:09:39.680364 25903 raft_consensus.cc:1081] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9 [term 1 LEADER]: Signalling peer 99dfff6363df46ce97b027cb3bd42e11 to start an election
I20260709 13:09:39.681931 25720 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "64becd02c9434edeaf5824c5a455b0e0"
dest_uuid: "99dfff6363df46ce97b027cb3bd42e11"
 from {username='slave'} at 127.0.0.1:44300
I20260709 13:09:39.682442 25720 raft_consensus.cc:493] T 64becd02c9434edeaf5824c5a455b0e0 P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:39.682775 25720 raft_consensus.cc:3060] T 64becd02c9434edeaf5824c5a455b0e0 P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:39.689277 25720 raft_consensus.cc:515] T 64becd02c9434edeaf5824c5a455b0e0 P 99dfff6363df46ce97b027cb3bd42e11 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:39.691041 25720 leader_election.cc:290] T 64becd02c9434edeaf5824c5a455b0e0 P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 2 election: Requested vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2:39169)
I20260709 13:09:39.692042 25941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64becd02c9434edeaf5824c5a455b0e0" candidate_uuid: "99dfff6363df46ce97b027cb3bd42e11" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a7e4f250aa5d48759eaabbdff8c99210"
I20260709 13:09:39.692260 25647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64becd02c9434edeaf5824c5a455b0e0" candidate_uuid: "99dfff6363df46ce97b027cb3bd42e11" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6a8699df36b74658a40f65ad30923ba9"
I20260709 13:09:39.692739 25941 raft_consensus.cc:3060] T 64becd02c9434edeaf5824c5a455b0e0 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:39.692883 25647 raft_consensus.cc:3055] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:39.693289 25647 raft_consensus.cc:740] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6a8699df36b74658a40f65ad30923ba9, State: Running, Role: LEADER
I20260709 13:09:39.694214 25647 consensus_queue.cc:260] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:39.695335 25647 raft_consensus.cc:3060] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:39.698836 25941 raft_consensus.cc:2468] T 64becd02c9434edeaf5824c5a455b0e0 P a7e4f250aa5d48759eaabbdff8c99210 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99dfff6363df46ce97b027cb3bd42e11 in term 2.
I20260709 13:09:39.700098 25686 leader_election.cc:304] T 64becd02c9434edeaf5824c5a455b0e0 P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99dfff6363df46ce97b027cb3bd42e11, a7e4f250aa5d48759eaabbdff8c99210; no voters: 
I20260709 13:09:39.701114 25855 raft_consensus.cc:2804] T 64becd02c9434edeaf5824c5a455b0e0 P 99dfff6363df46ce97b027cb3bd42e11 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:39.701758 25647 raft_consensus.cc:2468] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99dfff6363df46ce97b027cb3bd42e11 in term 2.
I20260709 13:09:39.702260 25855 raft_consensus.cc:697] T 64becd02c9434edeaf5824c5a455b0e0 P 99dfff6363df46ce97b027cb3bd42e11 [term 2 LEADER]: Becoming Leader. State: Replica: 99dfff6363df46ce97b027cb3bd42e11, State: Running, Role: LEADER
I20260709 13:09:39.703583 25855 consensus_queue.cc:237] T 64becd02c9434edeaf5824c5a455b0e0 P 99dfff6363df46ce97b027cb3bd42e11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:39.710861 25470 catalog_manager.cc:5697] T 64becd02c9434edeaf5824c5a455b0e0 P 99dfff6363df46ce97b027cb3bd42e11 reported cstate change: term changed from 1 to 2, leader changed from 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2) to 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3). New cstate: current_term: 2 leader_uuid: "99dfff6363df46ce97b027cb3bd42e11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:39.848832 25941 raft_consensus.cc:3060] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:39.907439 25941 raft_consensus.cc:3060] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:39.961608 25941 raft_consensus.cc:3060] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:09:40.025369 25904 raft_consensus.cc:993] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9: : Instructing follower 99dfff6363df46ce97b027cb3bd42e11 to start an election
I20260709 13:09:40.025830 25903 raft_consensus.cc:1081] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [term 2 LEADER]: Signalling peer 99dfff6363df46ce97b027cb3bd42e11 to start an election
I20260709 13:09:40.027295 25720 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "377fe3ce2b474722b845c07b3f74b830"
dest_uuid: "99dfff6363df46ce97b027cb3bd42e11"
 from {username='slave'} at 127.0.0.1:44300
I20260709 13:09:40.027786 25720 raft_consensus.cc:493] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:40.028040 25720 raft_consensus.cc:3060] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:40.038437 25720 raft_consensus.cc:515] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:40.040252 25720 leader_election.cc:290] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 3 election: Requested vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2:39169)
I20260709 13:09:40.040782 25941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "377fe3ce2b474722b845c07b3f74b830" candidate_uuid: "99dfff6363df46ce97b027cb3bd42e11" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a7e4f250aa5d48759eaabbdff8c99210"
I20260709 13:09:40.041215 25647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "377fe3ce2b474722b845c07b3f74b830" candidate_uuid: "99dfff6363df46ce97b027cb3bd42e11" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6a8699df36b74658a40f65ad30923ba9"
I20260709 13:09:40.041488 25941 raft_consensus.cc:3060] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:40.041877 25647 raft_consensus.cc:3055] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:40.042198 25647 raft_consensus.cc:740] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6a8699df36b74658a40f65ad30923ba9, State: Running, Role: LEADER
I20260709 13:09:40.043059 25647 consensus_queue.cc:260] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:40.044459 25647 raft_consensus.cc:3060] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:40.049074 25941 raft_consensus.cc:2468] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99dfff6363df46ce97b027cb3bd42e11 in term 3.
I20260709 13:09:40.050472 25686 leader_election.cc:304] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99dfff6363df46ce97b027cb3bd42e11, a7e4f250aa5d48759eaabbdff8c99210; no voters: 
I20260709 13:09:40.051396 25994 raft_consensus.cc:2804] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:40.051877 25994 raft_consensus.cc:697] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11 [term 3 LEADER]: Becoming Leader. State: Replica: 99dfff6363df46ce97b027cb3bd42e11, State: Running, Role: LEADER
I20260709 13:09:40.052035 25647 raft_consensus.cc:2468] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99dfff6363df46ce97b027cb3bd42e11 in term 3.
I20260709 13:09:40.053061 25994 consensus_queue.cc:237] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:40.060410 25470 catalog_manager.cc:5697] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11 reported cstate change: term changed from 2 to 3, leader changed from 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2) to 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3). New cstate: current_term: 3 leader_uuid: "99dfff6363df46ce97b027cb3bd42e11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:40.064739 25941 raft_consensus.cc:3060] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Advancing to term 3
W20260709 13:09:40.148327 25518 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:09:40.190225 25647 raft_consensus.cc:1275] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9 [term 2 FOLLOWER]: Refusing update from remote peer 99dfff6363df46ce97b027cb3bd42e11: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:40.191435 25994 consensus_queue.cc:1048] T 64becd02c9434edeaf5824c5a455b0e0 P 99dfff6363df46ce97b027cb3bd42e11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:40.201035 25941 raft_consensus.cc:1275] T 64becd02c9434edeaf5824c5a455b0e0 P a7e4f250aa5d48759eaabbdff8c99210 [term 2 FOLLOWER]: Refusing update from remote peer 99dfff6363df46ce97b027cb3bd42e11: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:40.202632 25994 consensus_queue.cc:1048] T 64becd02c9434edeaf5824c5a455b0e0 P 99dfff6363df46ce97b027cb3bd42e11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:40.462433 25941 raft_consensus.cc:1275] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210 [term 3 FOLLOWER]: Refusing update from remote peer 99dfff6363df46ce97b027cb3bd42e11: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:40.463738 25994 consensus_queue.cc:1048] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:40.470957 25647 raft_consensus.cc:1275] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [term 3 FOLLOWER]: Refusing update from remote peer 99dfff6363df46ce97b027cb3bd42e11: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:40.473644 25855 consensus_queue.cc:1048] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:40.586053 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:40.604097 25720 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "377fe3ce2b474722b845c07b3f74b830"
dest_uuid: "99dfff6363df46ce97b027cb3bd42e11"
mode: GRACEFUL
new_leader_uuid: "6a8699df36b74658a40f65ad30923ba9"
 from {username='slave'} at 127.0.0.1:44938
I20260709 13:09:40.604641 25720 raft_consensus.cc:606] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11 [term 3 LEADER]: Received request to transfer leadership to TS 6a8699df36b74658a40f65ad30923ba9
I20260709 13:09:40.607246 25720 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fa373cebfd3540728639dddbaee4cb1d"
dest_uuid: "99dfff6363df46ce97b027cb3bd42e11"
mode: GRACEFUL
new_leader_uuid: "6a8699df36b74658a40f65ad30923ba9"
 from {username='slave'} at 127.0.0.1:44938
I20260709 13:09:40.607802 25720 raft_consensus.cc:606] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [term 1 LEADER]: Received request to transfer leadership to TS 6a8699df36b74658a40f65ad30923ba9
I20260709 13:09:40.608925 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:09:40.609483 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:09:40.610116 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:40.943923 25855 raft_consensus.cc:993] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11: : Instructing follower 6a8699df36b74658a40f65ad30923ba9 to start an election
I20260709 13:09:40.944375 25984 raft_consensus.cc:1081] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [term 1 LEADER]: Signalling peer 6a8699df36b74658a40f65ad30923ba9 to start an election
I20260709 13:09:40.945704 25647 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fa373cebfd3540728639dddbaee4cb1d"
dest_uuid: "6a8699df36b74658a40f65ad30923ba9"
 from {username='slave'} at 127.0.0.1:33576
I20260709 13:09:40.946084 25647 raft_consensus.cc:493] T fa373cebfd3540728639dddbaee4cb1d P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:40.946316 25647 raft_consensus.cc:3060] T fa373cebfd3540728639dddbaee4cb1d P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:40.951695 25647 raft_consensus.cc:515] T fa373cebfd3540728639dddbaee4cb1d P 6a8699df36b74658a40f65ad30923ba9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } } peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } }
I20260709 13:09:40.953174 25647 leader_election.cc:290] T fa373cebfd3540728639dddbaee4cb1d P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 2 election: Requested vote from peers 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3:34359), a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185)
I20260709 13:09:40.953832 25720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa373cebfd3540728639dddbaee4cb1d" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "99dfff6363df46ce97b027cb3bd42e11"
I20260709 13:09:40.954103 25941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa373cebfd3540728639dddbaee4cb1d" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a7e4f250aa5d48759eaabbdff8c99210"
I20260709 13:09:40.954496 25720 raft_consensus.cc:3055] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:40.954785 25941 raft_consensus.cc:3060] T fa373cebfd3540728639dddbaee4cb1d P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:40.954972 25720 raft_consensus.cc:740] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 99dfff6363df46ce97b027cb3bd42e11, State: Running, Role: LEADER
I20260709 13:09:40.955991 25720 consensus_queue.cc:260] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } } peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } }
I20260709 13:09:40.957130 25720 raft_consensus.cc:3060] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:40.961645 25941 raft_consensus.cc:2468] T fa373cebfd3540728639dddbaee4cb1d P a7e4f250aa5d48759eaabbdff8c99210 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a8699df36b74658a40f65ad30923ba9 in term 2.
I20260709 13:09:40.962348 25984 raft_consensus.cc:993] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11: : Instructing follower 6a8699df36b74658a40f65ad30923ba9 to start an election
I20260709 13:09:40.962857 25984 raft_consensus.cc:1081] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11 [term 3 LEADER]: Signalling peer 6a8699df36b74658a40f65ad30923ba9 to start an election
I20260709 13:09:40.963112 25612 leader_election.cc:304] T fa373cebfd3540728639dddbaee4cb1d P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a8699df36b74658a40f65ad30923ba9, a7e4f250aa5d48759eaabbdff8c99210; no voters: 
I20260709 13:09:40.964299 25997 raft_consensus.cc:2804] T fa373cebfd3540728639dddbaee4cb1d P 6a8699df36b74658a40f65ad30923ba9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:40.964850 25720 raft_consensus.cc:2468] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a8699df36b74658a40f65ad30923ba9 in term 2.
I20260709 13:09:40.965214 25647 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "377fe3ce2b474722b845c07b3f74b830"
dest_uuid: "6a8699df36b74658a40f65ad30923ba9"
 from {username='slave'} at 127.0.0.1:33576
I20260709 13:09:40.965528 25997 raft_consensus.cc:697] T fa373cebfd3540728639dddbaee4cb1d P 6a8699df36b74658a40f65ad30923ba9 [term 2 LEADER]: Becoming Leader. State: Replica: 6a8699df36b74658a40f65ad30923ba9, State: Running, Role: LEADER
I20260709 13:09:40.965811 25647 raft_consensus.cc:493] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:40.966207 25647 raft_consensus.cc:3060] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:40.966671 25997 consensus_queue.cc:237] T fa373cebfd3540728639dddbaee4cb1d P 6a8699df36b74658a40f65ad30923ba9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } } peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } }
I20260709 13:09:40.972544 25647 raft_consensus.cc:515] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:40.974467 25647 leader_election.cc:290] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 4 election: Requested vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3:34359)
I20260709 13:09:40.975083 25941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "377fe3ce2b474722b845c07b3f74b830" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a7e4f250aa5d48759eaabbdff8c99210"
I20260709 13:09:40.975373 25720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "377fe3ce2b474722b845c07b3f74b830" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "99dfff6363df46ce97b027cb3bd42e11"
I20260709 13:09:40.975903 25941 raft_consensus.cc:3060] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:40.975142 25470 catalog_manager.cc:5697] T fa373cebfd3540728639dddbaee4cb1d P 6a8699df36b74658a40f65ad30923ba9 reported cstate change: term changed from 1 to 2, leader changed from 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3) to 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2). New cstate: current_term: 2 leader_uuid: "6a8699df36b74658a40f65ad30923ba9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:40.976051 25720 raft_consensus.cc:3055] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:09:40.976543 25720 raft_consensus.cc:740] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 99dfff6363df46ce97b027cb3bd42e11, State: Running, Role: LEADER
I20260709 13:09:40.977294 25720 consensus_queue.cc:260] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:40.978220 25720 raft_consensus.cc:3060] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:40.982431 25941 raft_consensus.cc:2468] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a8699df36b74658a40f65ad30923ba9 in term 4.
I20260709 13:09:40.983830 25612 leader_election.cc:304] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a8699df36b74658a40f65ad30923ba9, a7e4f250aa5d48759eaabbdff8c99210; no voters: 
I20260709 13:09:40.984476 25997 raft_consensus.cc:2804] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:09:40.984818 25720 raft_consensus.cc:2468] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a8699df36b74658a40f65ad30923ba9 in term 4.
I20260709 13:09:40.985030 25997 raft_consensus.cc:697] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [term 4 LEADER]: Becoming Leader. State: Replica: 6a8699df36b74658a40f65ad30923ba9, State: Running, Role: LEADER
I20260709 13:09:40.986156 25997 consensus_queue.cc:237] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:40.991950 25470 catalog_manager.cc:5697] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 reported cstate change: term changed from 3 to 4, leader changed from 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3) to 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2). New cstate: current_term: 4 leader_uuid: "6a8699df36b74658a40f65ad30923ba9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } health_report { overall_health: UNKNOWN } } }
W20260709 13:09:41.149298 25518 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:09:41.396903 25720 raft_consensus.cc:1275] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11 [term 4 FOLLOWER]: Refusing update from remote peer 6a8699df36b74658a40f65ad30923ba9: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:09:41.398518 25997 consensus_queue.cc:1048] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:09:41.407271 25941 raft_consensus.cc:1275] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210 [term 4 FOLLOWER]: Refusing update from remote peer 6a8699df36b74658a40f65ad30923ba9: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:09:41.408875 25997 consensus_queue.cc:1048] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:09:41.563133 25941 raft_consensus.cc:1275] T fa373cebfd3540728639dddbaee4cb1d P a7e4f250aa5d48759eaabbdff8c99210 [term 2 FOLLOWER]: Refusing update from remote peer 6a8699df36b74658a40f65ad30923ba9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:41.564371 25997 consensus_queue.cc:1048] T fa373cebfd3540728639dddbaee4cb1d P 6a8699df36b74658a40f65ad30923ba9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:41.573089 25720 raft_consensus.cc:1275] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [term 2 FOLLOWER]: Refusing update from remote peer 6a8699df36b74658a40f65ad30923ba9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:41.574998 26013 consensus_queue.cc:1048] T fa373cebfd3540728639dddbaee4cb1d P 6a8699df36b74658a40f65ad30923ba9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:41.617182 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:41.625128 25647 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8372ab57bb904260beb2b0c69bc39ad0"
dest_uuid: "6a8699df36b74658a40f65ad30923ba9"
mode: GRACEFUL
new_leader_uuid: "99dfff6363df46ce97b027cb3bd42e11"
 from {username='slave'} at 127.0.0.1:39170
I20260709 13:09:41.625617 25647 raft_consensus.cc:606] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9 [term 1 LEADER]: Received request to transfer leadership to TS 99dfff6363df46ce97b027cb3bd42e11
I20260709 13:09:41.627825 25647 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bab07f367f714676a11b512f3abbea7c"
dest_uuid: "6a8699df36b74658a40f65ad30923ba9"
mode: GRACEFUL
new_leader_uuid: "99dfff6363df46ce97b027cb3bd42e11"
 from {username='slave'} at 127.0.0.1:39170
I20260709 13:09:41.628257 25647 raft_consensus.cc:606] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [term 1 LEADER]: Received request to transfer leadership to TS 99dfff6363df46ce97b027cb3bd42e11
I20260709 13:09:41.629176 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:09:41.629680 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:09:41.630218 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:41.964506 26015 raft_consensus.cc:993] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9: : Instructing follower 99dfff6363df46ce97b027cb3bd42e11 to start an election
I20260709 13:09:41.964962 26013 raft_consensus.cc:1081] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9 [term 1 LEADER]: Signalling peer 99dfff6363df46ce97b027cb3bd42e11 to start an election
I20260709 13:09:41.966256 25720 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8372ab57bb904260beb2b0c69bc39ad0"
dest_uuid: "99dfff6363df46ce97b027cb3bd42e11"
 from {username='slave'} at 127.0.0.1:44300
I20260709 13:09:41.966665 25720 raft_consensus.cc:493] T 8372ab57bb904260beb2b0c69bc39ad0 P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:41.966909 25720 raft_consensus.cc:3060] T 8372ab57bb904260beb2b0c69bc39ad0 P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:41.971907 25720 raft_consensus.cc:515] T 8372ab57bb904260beb2b0c69bc39ad0 P 99dfff6363df46ce97b027cb3bd42e11 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:41.973515 25720 leader_election.cc:290] T 8372ab57bb904260beb2b0c69bc39ad0 P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 2 election: Requested vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2:39169)
I20260709 13:09:41.974251 25940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8372ab57bb904260beb2b0c69bc39ad0" candidate_uuid: "99dfff6363df46ce97b027cb3bd42e11" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a7e4f250aa5d48759eaabbdff8c99210"
I20260709 13:09:41.974481 25647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8372ab57bb904260beb2b0c69bc39ad0" candidate_uuid: "99dfff6363df46ce97b027cb3bd42e11" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6a8699df36b74658a40f65ad30923ba9"
I20260709 13:09:41.974857 25940 raft_consensus.cc:3060] T 8372ab57bb904260beb2b0c69bc39ad0 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:41.975045 25647 raft_consensus.cc:3055] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:41.975425 25647 raft_consensus.cc:740] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6a8699df36b74658a40f65ad30923ba9, State: Running, Role: LEADER
I20260709 13:09:41.976083 25647 consensus_queue.cc:260] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:41.976989 25647 raft_consensus.cc:3060] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:41.980695 25940 raft_consensus.cc:2468] T 8372ab57bb904260beb2b0c69bc39ad0 P a7e4f250aa5d48759eaabbdff8c99210 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99dfff6363df46ce97b027cb3bd42e11 in term 2.
I20260709 13:09:41.981328 25647 raft_consensus.cc:2468] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99dfff6363df46ce97b027cb3bd42e11 in term 2.
I20260709 13:09:41.981915 25686 leader_election.cc:304] T 8372ab57bb904260beb2b0c69bc39ad0 P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99dfff6363df46ce97b027cb3bd42e11, a7e4f250aa5d48759eaabbdff8c99210; no voters: 
I20260709 13:09:41.982750 25855 raft_consensus.cc:2804] T 8372ab57bb904260beb2b0c69bc39ad0 P 99dfff6363df46ce97b027cb3bd42e11 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:41.983850 25855 raft_consensus.cc:697] T 8372ab57bb904260beb2b0c69bc39ad0 P 99dfff6363df46ce97b027cb3bd42e11 [term 2 LEADER]: Becoming Leader. State: Replica: 99dfff6363df46ce97b027cb3bd42e11, State: Running, Role: LEADER
I20260709 13:09:41.984776 25855 consensus_queue.cc:237] T 8372ab57bb904260beb2b0c69bc39ad0 P 99dfff6363df46ce97b027cb3bd42e11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:41.993100 25470 catalog_manager.cc:5697] T 8372ab57bb904260beb2b0c69bc39ad0 P 99dfff6363df46ce97b027cb3bd42e11 reported cstate change: term changed from 1 to 2, leader changed from 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2) to 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3). New cstate: current_term: 2 leader_uuid: "99dfff6363df46ce97b027cb3bd42e11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:42.025918 25903 raft_consensus.cc:993] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9: : Instructing follower 99dfff6363df46ce97b027cb3bd42e11 to start an election
I20260709 13:09:42.026372 25903 raft_consensus.cc:1081] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [term 1 LEADER]: Signalling peer 99dfff6363df46ce97b027cb3bd42e11 to start an election
I20260709 13:09:42.027815 25720 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bab07f367f714676a11b512f3abbea7c"
dest_uuid: "99dfff6363df46ce97b027cb3bd42e11"
 from {username='slave'} at 127.0.0.1:44300
I20260709 13:09:42.028292 25720 raft_consensus.cc:493] T bab07f367f714676a11b512f3abbea7c P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:42.028630 25720 raft_consensus.cc:3060] T bab07f367f714676a11b512f3abbea7c P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:42.033788 25720 raft_consensus.cc:515] T bab07f367f714676a11b512f3abbea7c P 99dfff6363df46ce97b027cb3bd42e11 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:42.035176 25720 leader_election.cc:290] T bab07f367f714676a11b512f3abbea7c P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 2 election: Requested vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2:39169)
I20260709 13:09:42.035986 25940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab07f367f714676a11b512f3abbea7c" candidate_uuid: "99dfff6363df46ce97b027cb3bd42e11" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a7e4f250aa5d48759eaabbdff8c99210"
I20260709 13:09:42.036144 25647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab07f367f714676a11b512f3abbea7c" candidate_uuid: "99dfff6363df46ce97b027cb3bd42e11" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6a8699df36b74658a40f65ad30923ba9"
I20260709 13:09:42.036739 25940 raft_consensus.cc:3060] T bab07f367f714676a11b512f3abbea7c P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:42.036813 25647 raft_consensus.cc:3055] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:42.037225 25647 raft_consensus.cc:740] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6a8699df36b74658a40f65ad30923ba9, State: Running, Role: LEADER
I20260709 13:09:42.037870 25647 consensus_queue.cc:260] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:42.038849 25647 raft_consensus.cc:3060] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:42.042322 25940 raft_consensus.cc:2468] T bab07f367f714676a11b512f3abbea7c P a7e4f250aa5d48759eaabbdff8c99210 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99dfff6363df46ce97b027cb3bd42e11 in term 2.
I20260709 13:09:42.043251 25647 raft_consensus.cc:2468] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99dfff6363df46ce97b027cb3bd42e11 in term 2.
I20260709 13:09:42.043371 25686 leader_election.cc:304] T bab07f367f714676a11b512f3abbea7c P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99dfff6363df46ce97b027cb3bd42e11, a7e4f250aa5d48759eaabbdff8c99210; no voters: 
I20260709 13:09:42.044289 25855 raft_consensus.cc:2804] T bab07f367f714676a11b512f3abbea7c P 99dfff6363df46ce97b027cb3bd42e11 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:42.044706 25855 raft_consensus.cc:697] T bab07f367f714676a11b512f3abbea7c P 99dfff6363df46ce97b027cb3bd42e11 [term 2 LEADER]: Becoming Leader. State: Replica: 99dfff6363df46ce97b027cb3bd42e11, State: Running, Role: LEADER
I20260709 13:09:42.045493 25855 consensus_queue.cc:237] T bab07f367f714676a11b512f3abbea7c P 99dfff6363df46ce97b027cb3bd42e11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:42.051826 25470 catalog_manager.cc:5697] T bab07f367f714676a11b512f3abbea7c P 99dfff6363df46ce97b027cb3bd42e11 reported cstate change: term changed from 1 to 2, leader changed from 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2) to 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3). New cstate: current_term: 2 leader_uuid: "99dfff6363df46ce97b027cb3bd42e11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } health_report { overall_health: HEALTHY } } }
W20260709 13:09:42.150151 25518 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:09:42.516244 25940 raft_consensus.cc:1275] T bab07f367f714676a11b512f3abbea7c P a7e4f250aa5d48759eaabbdff8c99210 [term 2 FOLLOWER]: Refusing update from remote peer 99dfff6363df46ce97b027cb3bd42e11: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:42.517462 25855 consensus_queue.cc:1048] T bab07f367f714676a11b512f3abbea7c P 99dfff6363df46ce97b027cb3bd42e11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:42.529675 25647 raft_consensus.cc:1275] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [term 2 FOLLOWER]: Refusing update from remote peer 99dfff6363df46ce97b027cb3bd42e11: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:42.531088 25855 consensus_queue.cc:1048] T bab07f367f714676a11b512f3abbea7c P 99dfff6363df46ce97b027cb3bd42e11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:42.553006 25647 raft_consensus.cc:1275] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9 [term 2 FOLLOWER]: Refusing update from remote peer 99dfff6363df46ce97b027cb3bd42e11: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:42.554373 26016 consensus_queue.cc:1048] T 8372ab57bb904260beb2b0c69bc39ad0 P 99dfff6363df46ce97b027cb3bd42e11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:42.563911 25940 raft_consensus.cc:1275] T 8372ab57bb904260beb2b0c69bc39ad0 P a7e4f250aa5d48759eaabbdff8c99210 [term 2 FOLLOWER]: Refusing update from remote peer 99dfff6363df46ce97b027cb3bd42e11: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:42.565411 26026 consensus_queue.cc:1048] T 8372ab57bb904260beb2b0c69bc39ad0 P 99dfff6363df46ce97b027cb3bd42e11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:42.637960 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:42.645464 25720 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2a14348b4c6a4d3cbbf18beffcd03814"
dest_uuid: "99dfff6363df46ce97b027cb3bd42e11"
mode: GRACEFUL
new_leader_uuid: "6a8699df36b74658a40f65ad30923ba9"
 from {username='slave'} at 127.0.0.1:44938
I20260709 13:09:42.645985 25720 raft_consensus.cc:606] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 3 LEADER]: Received request to transfer leadership to TS 6a8699df36b74658a40f65ad30923ba9
I20260709 13:09:42.648559 25720 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bab07f367f714676a11b512f3abbea7c"
dest_uuid: "99dfff6363df46ce97b027cb3bd42e11"
mode: GRACEFUL
new_leader_uuid: "6a8699df36b74658a40f65ad30923ba9"
 from {username='slave'} at 127.0.0.1:44938
I20260709 13:09:42.649022 25720 raft_consensus.cc:606] T bab07f367f714676a11b512f3abbea7c P 99dfff6363df46ce97b027cb3bd42e11 [term 2 LEADER]: Received request to transfer leadership to TS 6a8699df36b74658a40f65ad30923ba9
I20260709 13:09:42.650091 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:09:42.650566 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:09:42.651144 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:42.927908 26016 raft_consensus.cc:993] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11: : Instructing follower 6a8699df36b74658a40f65ad30923ba9 to start an election
I20260709 13:09:42.928561 26026 raft_consensus.cc:1081] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 3 LEADER]: Signalling peer 6a8699df36b74658a40f65ad30923ba9 to start an election
I20260709 13:09:42.930470 25647 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2a14348b4c6a4d3cbbf18beffcd03814"
dest_uuid: "6a8699df36b74658a40f65ad30923ba9"
 from {username='slave'} at 127.0.0.1:33576
I20260709 13:09:42.931115 25647 raft_consensus.cc:493] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:42.931474 25647 raft_consensus.cc:3060] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:42.937199 25647 raft_consensus.cc:515] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:42.938799 25647 leader_election.cc:290] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 4 election: Requested vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3:34359)
I20260709 13:09:42.939982 25720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a14348b4c6a4d3cbbf18beffcd03814" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "99dfff6363df46ce97b027cb3bd42e11"
I20260709 13:09:42.939752 25940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a14348b4c6a4d3cbbf18beffcd03814" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "a7e4f250aa5d48759eaabbdff8c99210"
I20260709 13:09:42.940652 25720 raft_consensus.cc:3055] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:09:42.940660 25940 raft_consensus.cc:3060] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:42.941124 25720 raft_consensus.cc:740] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 99dfff6363df46ce97b027cb3bd42e11, State: Running, Role: LEADER
I20260709 13:09:42.942052 25720 consensus_queue.cc:260] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:42.943392 25720 raft_consensus.cc:3060] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:42.948604 25940 raft_consensus.cc:2468] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a8699df36b74658a40f65ad30923ba9 in term 4.
I20260709 13:09:42.949890 25720 raft_consensus.cc:2468] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a8699df36b74658a40f65ad30923ba9 in term 4.
I20260709 13:09:42.950142 25612 leader_election.cc:304] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a8699df36b74658a40f65ad30923ba9, a7e4f250aa5d48759eaabbdff8c99210; no voters: 
I20260709 13:09:42.951131 25903 raft_consensus.cc:2804] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:09:42.952188 25903 raft_consensus.cc:697] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [term 4 LEADER]: Becoming Leader. State: Replica: 6a8699df36b74658a40f65ad30923ba9, State: Running, Role: LEADER
I20260709 13:09:42.953120 25903 consensus_queue.cc:237] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:42.961747 25470 catalog_manager.cc:5697] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 reported cstate change: term changed from 3 to 4, leader changed from 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3) to 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2). New cstate: current_term: 4 leader_uuid: "6a8699df36b74658a40f65ad30923ba9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } health_report { overall_health: UNKNOWN } } }
W20260709 13:09:43.151064 25518 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:09:43.158692 26026 raft_consensus.cc:993] T bab07f367f714676a11b512f3abbea7c P 99dfff6363df46ce97b027cb3bd42e11: : Instructing follower 6a8699df36b74658a40f65ad30923ba9 to start an election
I20260709 13:09:43.159318 26036 raft_consensus.cc:1081] T bab07f367f714676a11b512f3abbea7c P 99dfff6363df46ce97b027cb3bd42e11 [term 2 LEADER]: Signalling peer 6a8699df36b74658a40f65ad30923ba9 to start an election
I20260709 13:09:43.160844 25647 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bab07f367f714676a11b512f3abbea7c"
dest_uuid: "6a8699df36b74658a40f65ad30923ba9"
 from {username='slave'} at 127.0.0.1:33576
I20260709 13:09:43.161332 25647 raft_consensus.cc:493] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:43.161691 25647 raft_consensus.cc:3060] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:43.167904 25647 raft_consensus.cc:515] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:43.169421 25647 leader_election.cc:290] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 3 election: Requested vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3:34359)
I20260709 13:09:43.170300 25940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab07f367f714676a11b512f3abbea7c" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a7e4f250aa5d48759eaabbdff8c99210"
I20260709 13:09:43.170477 25720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab07f367f714676a11b512f3abbea7c" candidate_uuid: "6a8699df36b74658a40f65ad30923ba9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "99dfff6363df46ce97b027cb3bd42e11"
I20260709 13:09:43.171140 25940 raft_consensus.cc:3060] T bab07f367f714676a11b512f3abbea7c P a7e4f250aa5d48759eaabbdff8c99210 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:43.171226 25720 raft_consensus.cc:3055] T bab07f367f714676a11b512f3abbea7c P 99dfff6363df46ce97b027cb3bd42e11 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:43.171751 25720 raft_consensus.cc:740] T bab07f367f714676a11b512f3abbea7c P 99dfff6363df46ce97b027cb3bd42e11 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 99dfff6363df46ce97b027cb3bd42e11, State: Running, Role: LEADER
I20260709 13:09:43.172489 25720 consensus_queue.cc:260] T bab07f367f714676a11b512f3abbea7c P 99dfff6363df46ce97b027cb3bd42e11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:43.173403 25720 raft_consensus.cc:3060] T bab07f367f714676a11b512f3abbea7c P 99dfff6363df46ce97b027cb3bd42e11 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:43.176864 25940 raft_consensus.cc:2468] T bab07f367f714676a11b512f3abbea7c P a7e4f250aa5d48759eaabbdff8c99210 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a8699df36b74658a40f65ad30923ba9 in term 3.
I20260709 13:09:43.178077 25612 leader_election.cc:304] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a8699df36b74658a40f65ad30923ba9, a7e4f250aa5d48759eaabbdff8c99210; no voters: 
I20260709 13:09:43.178232 25720 raft_consensus.cc:2468] T bab07f367f714676a11b512f3abbea7c P 99dfff6363df46ce97b027cb3bd42e11 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a8699df36b74658a40f65ad30923ba9 in term 3.
I20260709 13:09:43.178838 25903 raft_consensus.cc:2804] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:43.179342 25903 raft_consensus.cc:697] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [term 3 LEADER]: Becoming Leader. State: Replica: 6a8699df36b74658a40f65ad30923ba9, State: Running, Role: LEADER
I20260709 13:09:43.180168 25903 consensus_queue.cc:237] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:43.186278 25470 catalog_manager.cc:5697] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 reported cstate change: term changed from 2 to 3, leader changed from 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3) to 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2). New cstate: current_term: 3 leader_uuid: "6a8699df36b74658a40f65ad30923ba9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:43.486280 25720 raft_consensus.cc:1275] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 4 FOLLOWER]: Refusing update from remote peer 6a8699df36b74658a40f65ad30923ba9: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:09:43.487867 26031 consensus_queue.cc:1048] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:43.498198 25940 raft_consensus.cc:1275] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210 [term 4 FOLLOWER]: Refusing update from remote peer 6a8699df36b74658a40f65ad30923ba9: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:09:43.500384 25903 consensus_queue.cc:1048] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:43.634406 25940 raft_consensus.cc:1275] T bab07f367f714676a11b512f3abbea7c P a7e4f250aa5d48759eaabbdff8c99210 [term 3 FOLLOWER]: Refusing update from remote peer 6a8699df36b74658a40f65ad30923ba9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:43.635818 25903 consensus_queue.cc:1048] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:43.645308 25720 raft_consensus.cc:1275] T bab07f367f714676a11b512f3abbea7c P 99dfff6363df46ce97b027cb3bd42e11 [term 3 FOLLOWER]: Refusing update from remote peer 6a8699df36b74658a40f65ad30923ba9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:43.646817 26031 consensus_queue.cc:1048] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:43.657792 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:43.666431 25647 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2a14348b4c6a4d3cbbf18beffcd03814"
dest_uuid: "6a8699df36b74658a40f65ad30923ba9"
mode: GRACEFUL
new_leader_uuid: "99dfff6363df46ce97b027cb3bd42e11"
 from {username='slave'} at 127.0.0.1:39170
I20260709 13:09:43.666936 25647 raft_consensus.cc:606] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [term 4 LEADER]: Received request to transfer leadership to TS 99dfff6363df46ce97b027cb3bd42e11
I20260709 13:09:43.669394 25647 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fa373cebfd3540728639dddbaee4cb1d"
dest_uuid: "6a8699df36b74658a40f65ad30923ba9"
mode: GRACEFUL
new_leader_uuid: "99dfff6363df46ce97b027cb3bd42e11"
 from {username='slave'} at 127.0.0.1:39170
I20260709 13:09:43.669826 25647 raft_consensus.cc:606] T fa373cebfd3540728639dddbaee4cb1d P 6a8699df36b74658a40f65ad30923ba9 [term 2 LEADER]: Received request to transfer leadership to TS 99dfff6363df46ce97b027cb3bd42e11
I20260709 13:09:43.670787 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:09:43.671357 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:09:43.672081 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:43.860638 26031 raft_consensus.cc:993] T fa373cebfd3540728639dddbaee4cb1d P 6a8699df36b74658a40f65ad30923ba9: : Instructing follower 99dfff6363df46ce97b027cb3bd42e11 to start an election
I20260709 13:09:43.861198 26039 raft_consensus.cc:1081] T fa373cebfd3540728639dddbaee4cb1d P 6a8699df36b74658a40f65ad30923ba9 [term 2 LEADER]: Signalling peer 99dfff6363df46ce97b027cb3bd42e11 to start an election
I20260709 13:09:43.862728 25720 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fa373cebfd3540728639dddbaee4cb1d"
dest_uuid: "99dfff6363df46ce97b027cb3bd42e11"
 from {username='slave'} at 127.0.0.1:44300
I20260709 13:09:43.863436 25720 raft_consensus.cc:493] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:43.863988 25720 raft_consensus.cc:3060] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:43.871830 25720 raft_consensus.cc:515] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } } peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } }
I20260709 13:09:43.873656 25720 leader_election.cc:290] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 3 election: Requested vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2:39169)
I20260709 13:09:43.874442 25940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa373cebfd3540728639dddbaee4cb1d" candidate_uuid: "99dfff6363df46ce97b027cb3bd42e11" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a7e4f250aa5d48759eaabbdff8c99210"
I20260709 13:09:43.874655 25647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa373cebfd3540728639dddbaee4cb1d" candidate_uuid: "99dfff6363df46ce97b027cb3bd42e11" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6a8699df36b74658a40f65ad30923ba9"
I20260709 13:09:43.875129 25940 raft_consensus.cc:3060] T fa373cebfd3540728639dddbaee4cb1d P a7e4f250aa5d48759eaabbdff8c99210 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:43.875347 25647 raft_consensus.cc:3055] T fa373cebfd3540728639dddbaee4cb1d P 6a8699df36b74658a40f65ad30923ba9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:43.875790 25647 raft_consensus.cc:740] T fa373cebfd3540728639dddbaee4cb1d P 6a8699df36b74658a40f65ad30923ba9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6a8699df36b74658a40f65ad30923ba9, State: Running, Role: LEADER
I20260709 13:09:43.876514 25647 consensus_queue.cc:260] T fa373cebfd3540728639dddbaee4cb1d P 6a8699df36b74658a40f65ad30923ba9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } } peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } }
I20260709 13:09:43.877571 25647 raft_consensus.cc:3060] T fa373cebfd3540728639dddbaee4cb1d P 6a8699df36b74658a40f65ad30923ba9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:43.880860 25940 raft_consensus.cc:2468] T fa373cebfd3540728639dddbaee4cb1d P a7e4f250aa5d48759eaabbdff8c99210 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99dfff6363df46ce97b027cb3bd42e11 in term 3.
I20260709 13:09:43.882048 25686 leader_election.cc:304] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99dfff6363df46ce97b027cb3bd42e11, a7e4f250aa5d48759eaabbdff8c99210; no voters: 
I20260709 13:09:43.882961 26036 raft_consensus.cc:2804] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:43.883883 25647 raft_consensus.cc:2468] T fa373cebfd3540728639dddbaee4cb1d P 6a8699df36b74658a40f65ad30923ba9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99dfff6363df46ce97b027cb3bd42e11 in term 3.
I20260709 13:09:43.884143 26036 raft_consensus.cc:697] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [term 3 LEADER]: Becoming Leader. State: Replica: 99dfff6363df46ce97b027cb3bd42e11, State: Running, Role: LEADER
I20260709 13:09:43.885221 26036 consensus_queue.cc:237] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } } peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } }
I20260709 13:09:43.892555 25470 catalog_manager.cc:5697] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 reported cstate change: term changed from 2 to 3, leader changed from 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2) to 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3). New cstate: current_term: 3 leader_uuid: "99dfff6363df46ce97b027cb3bd42e11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:43.907927 26031 raft_consensus.cc:993] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9: : Instructing follower 99dfff6363df46ce97b027cb3bd42e11 to start an election
I20260709 13:09:43.908454 26039 raft_consensus.cc:1081] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [term 4 LEADER]: Signalling peer 99dfff6363df46ce97b027cb3bd42e11 to start an election
I20260709 13:09:43.910163 25720 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2a14348b4c6a4d3cbbf18beffcd03814"
dest_uuid: "99dfff6363df46ce97b027cb3bd42e11"
 from {username='slave'} at 127.0.0.1:44300
I20260709 13:09:43.910706 25720 raft_consensus.cc:493] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:43.911011 25720 raft_consensus.cc:3060] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:43.916316 25720 raft_consensus.cc:515] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:43.917886 25720 leader_election.cc:290] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 5 election: Requested vote from peers a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185), 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2:39169)
I20260709 13:09:43.918794 25940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a14348b4c6a4d3cbbf18beffcd03814" candidate_uuid: "99dfff6363df46ce97b027cb3bd42e11" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "a7e4f250aa5d48759eaabbdff8c99210"
I20260709 13:09:43.918968 25647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a14348b4c6a4d3cbbf18beffcd03814" candidate_uuid: "99dfff6363df46ce97b027cb3bd42e11" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "6a8699df36b74658a40f65ad30923ba9"
I20260709 13:09:43.919508 25940 raft_consensus.cc:3060] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:43.919631 25647 raft_consensus.cc:3055] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:09:43.920156 25647 raft_consensus.cc:740] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6a8699df36b74658a40f65ad30923ba9, State: Running, Role: LEADER
I20260709 13:09:43.920821 25647 consensus_queue.cc:260] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:43.921797 25647 raft_consensus.cc:3060] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:43.925871 25940 raft_consensus.cc:2468] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99dfff6363df46ce97b027cb3bd42e11 in term 5.
I20260709 13:09:43.926333 25647 raft_consensus.cc:2468] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99dfff6363df46ce97b027cb3bd42e11 in term 5.
I20260709 13:09:43.927078 25686 leader_election.cc:304] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99dfff6363df46ce97b027cb3bd42e11, a7e4f250aa5d48759eaabbdff8c99210; no voters: 
I20260709 13:09:43.927863 26036 raft_consensus.cc:2804] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:09:43.928391 26036 raft_consensus.cc:697] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 5 LEADER]: Becoming Leader. State: Replica: 99dfff6363df46ce97b027cb3bd42e11, State: Running, Role: LEADER
I20260709 13:09:43.929306 26036 consensus_queue.cc:237] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } }
I20260709 13:09:43.936095 25470 catalog_manager.cc:5697] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 reported cstate change: term changed from 4 to 5, leader changed from 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2) to 99dfff6363df46ce97b027cb3bd42e11 (127.14.103.3). New cstate: current_term: 5 leader_uuid: "99dfff6363df46ce97b027cb3bd42e11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99dfff6363df46ce97b027cb3bd42e11" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 34359 } health_report { overall_health: HEALTHY } } }
W20260709 13:09:44.152138 25518 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:09:44.265376 25647 raft_consensus.cc:1275] T fa373cebfd3540728639dddbaee4cb1d P 6a8699df36b74658a40f65ad30923ba9 [term 3 FOLLOWER]: Refusing update from remote peer 99dfff6363df46ce97b027cb3bd42e11: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:44.266933 26036 consensus_queue.cc:1048] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:44.276916 25940 raft_consensus.cc:1275] T fa373cebfd3540728639dddbaee4cb1d P a7e4f250aa5d48759eaabbdff8c99210 [term 3 FOLLOWER]: Refusing update from remote peer 99dfff6363df46ce97b027cb3bd42e11: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:44.279018 26036 consensus_queue.cc:1048] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:44.463361 25941 raft_consensus.cc:1275] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210 [term 5 FOLLOWER]: Refusing update from remote peer 99dfff6363df46ce97b027cb3bd42e11: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:09:44.464736 26036 consensus_queue.cc:1048] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7e4f250aa5d48759eaabbdff8c99210" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 43185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:09:44.473762 25647 raft_consensus.cc:1275] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [term 5 FOLLOWER]: Refusing update from remote peer 99dfff6363df46ce97b027cb3bd42e11: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:09:44.475865 26058 consensus_queue.cc:1048] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a8699df36b74658a40f65ad30923ba9" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 39169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:09:44.678591 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:44.710237 14748 tablet_server.cc:179] TabletServer@127.14.103.1:43185 shutting down...
I20260709 13:09:44.730927 14748 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:09:44.731845 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:44.732538 14748 raft_consensus.cc:2243] T bab07f367f714676a11b512f3abbea7c P a7e4f250aa5d48759eaabbdff8c99210 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:44.733013 14748 raft_consensus.cc:2272] T bab07f367f714676a11b512f3abbea7c P a7e4f250aa5d48759eaabbdff8c99210 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:44.734874 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:44.735442 14748 raft_consensus.cc:2243] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:44.735896 14748 raft_consensus.cc:2272] T 2a14348b4c6a4d3cbbf18beffcd03814 P a7e4f250aa5d48759eaabbdff8c99210 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:44.738119 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:44.738755 14748 raft_consensus.cc:2243] T 6d5ba3150425499f9cfa64eb06d6ba91 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:44.739176 14748 raft_consensus.cc:2272] T 6d5ba3150425499f9cfa64eb06d6ba91 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:44.740975 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:44.741504 14748 raft_consensus.cc:2243] T fc82e3058d55474faed434b751f7c644 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:44.741858 14748 raft_consensus.cc:2272] T fc82e3058d55474faed434b751f7c644 P a7e4f250aa5d48759eaabbdff8c99210 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:44.743597 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:44.744146 14748 raft_consensus.cc:2243] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:44.744539 14748 raft_consensus.cc:2272] T e1f6c52711174eacb83ea8fd2ad1b57b P a7e4f250aa5d48759eaabbdff8c99210 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:44.746225 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:44.746716 14748 raft_consensus.cc:2243] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:44.747104 14748 raft_consensus.cc:2272] T 377fe3ce2b474722b845c07b3f74b830 P a7e4f250aa5d48759eaabbdff8c99210 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:44.748903 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:44.749409 14748 raft_consensus.cc:2243] T fa373cebfd3540728639dddbaee4cb1d P a7e4f250aa5d48759eaabbdff8c99210 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:44.749799 14748 raft_consensus.cc:2272] T fa373cebfd3540728639dddbaee4cb1d P a7e4f250aa5d48759eaabbdff8c99210 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:44.751971 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:44.752576 14748 raft_consensus.cc:2243] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:44.753000 14748 raft_consensus.cc:2272] T 0d81fc45e39443b9b33953f8c66437eb P a7e4f250aa5d48759eaabbdff8c99210 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:44.754681 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:44.755152 14748 raft_consensus.cc:2243] T 8372ab57bb904260beb2b0c69bc39ad0 P a7e4f250aa5d48759eaabbdff8c99210 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:44.755545 14748 raft_consensus.cc:2272] T 8372ab57bb904260beb2b0c69bc39ad0 P a7e4f250aa5d48759eaabbdff8c99210 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:44.757272 14748 tablet_replica.cc:333] stopping tablet replica
I20260709 13:09:44.757730 14748 raft_consensus.cc:2243] T 64becd02c9434edeaf5824c5a455b0e0 P a7e4f250aa5d48759eaabbdff8c99210 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:44.758129 14748 raft_consensus.cc:2272] T 64becd02c9434edeaf5824c5a455b0e0 P a7e4f250aa5d48759eaabbdff8c99210 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:44.786707 14748 tablet_server.cc:196] TabletServer@127.14.103.1:43185 shutdown complete.
I20260709 13:09:44.808887 14748 tablet_server.cc:179] TabletServer@127.14.103.2:0 shutting down...
W20260709 13:09:44.835820 25686 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111) [suppressed 89 similar messages]
I20260709 13:09:44.840898 14748 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:09:44.841130 25686 consensus_peers.cc:597] T e1f6c52711174eacb83ea8fd2ad1b57b P 99dfff6363df46ce97b027cb3bd42e11 -> Peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Couldn't send request to peer a7e4f250aa5d48759eaabbdff8c99210. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:44.841853 14748 tablet_replica.cc:333] T e1f6c52711174eacb83ea8fd2ad1b57b P 6a8699df36b74658a40f65ad30923ba9: stopping tablet replica
I20260709 13:09:44.842518 14748 raft_consensus.cc:2243] T e1f6c52711174eacb83ea8fd2ad1b57b P 6a8699df36b74658a40f65ad30923ba9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:44.842976 14748 raft_consensus.cc:2272] T e1f6c52711174eacb83ea8fd2ad1b57b P 6a8699df36b74658a40f65ad30923ba9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:44.845332 14748 tablet_replica.cc:333] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9: stopping tablet replica
I20260709 13:09:44.846005 14748 raft_consensus.cc:2243] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:44.846442 14748 raft_consensus.cc:2272] T 64becd02c9434edeaf5824c5a455b0e0 P 6a8699df36b74658a40f65ad30923ba9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:44.848515 14748 tablet_replica.cc:333] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9: stopping tablet replica
I20260709 13:09:44.849283 14748 raft_consensus.cc:2243] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:09:44.850186 14748 raft_consensus.cc:2272] T 0d81fc45e39443b9b33953f8c66437eb P 6a8699df36b74658a40f65ad30923ba9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:44.852384 14748 tablet_replica.cc:333] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9: stopping tablet replica
I20260709 13:09:44.852929 14748 raft_consensus.cc:2243] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:44.853339 14748 raft_consensus.cc:2272] T 8372ab57bb904260beb2b0c69bc39ad0 P 6a8699df36b74658a40f65ad30923ba9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:44.855765 14748 tablet_replica.cc:333] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9: stopping tablet replica
I20260709 13:09:44.856266 14748 raft_consensus.cc:2243] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:44.856686 14748 raft_consensus.cc:2272] T 2a14348b4c6a4d3cbbf18beffcd03814 P 6a8699df36b74658a40f65ad30923ba9 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:44.861662 14748 tablet_replica.cc:333] T fc82e3058d55474faed434b751f7c644 P 6a8699df36b74658a40f65ad30923ba9: stopping tablet replica
I20260709 13:09:44.862699 14748 raft_consensus.cc:2243] T fc82e3058d55474faed434b751f7c644 P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:44.863343 14748 raft_consensus.cc:2272] T fc82e3058d55474faed434b751f7c644 P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:44.866659 14748 tablet_replica.cc:333] T 6d5ba3150425499f9cfa64eb06d6ba91 P 6a8699df36b74658a40f65ad30923ba9: stopping tablet replica
W20260709 13:09:44.867077 25684 consensus_peers.cc:597] T 8372ab57bb904260beb2b0c69bc39ad0 P 99dfff6363df46ce97b027cb3bd42e11 -> Peer 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2:39169): Couldn't send request to peer 6a8699df36b74658a40f65ad30923ba9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:44.867547 14748 raft_consensus.cc:2243] T 6d5ba3150425499f9cfa64eb06d6ba91 P 6a8699df36b74658a40f65ad30923ba9 [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:09:44.867609 25686 consensus_peers.cc:597] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 -> Peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Couldn't send request to peer a7e4f250aa5d48759eaabbdff8c99210. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:44.868805 14748 raft_consensus.cc:2272] T 6d5ba3150425499f9cfa64eb06d6ba91 P 6a8699df36b74658a40f65ad30923ba9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:44.870911 14748 tablet_replica.cc:333] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9: stopping tablet replica
I20260709 13:09:44.871418 14748 raft_consensus.cc:2243] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:09:44.872210 14748 raft_consensus.cc:2272] T bab07f367f714676a11b512f3abbea7c P 6a8699df36b74658a40f65ad30923ba9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:44.874235 14748 tablet_replica.cc:333] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9: stopping tablet replica
I20260709 13:09:44.874728 14748 raft_consensus.cc:2243] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:09:44.875394 14748 raft_consensus.cc:2272] T 377fe3ce2b474722b845c07b3f74b830 P 6a8699df36b74658a40f65ad30923ba9 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:44.877447 14748 tablet_replica.cc:333] T fa373cebfd3540728639dddbaee4cb1d P 6a8699df36b74658a40f65ad30923ba9: stopping tablet replica
I20260709 13:09:44.877882 14748 raft_consensus.cc:2243] T fa373cebfd3540728639dddbaee4cb1d P 6a8699df36b74658a40f65ad30923ba9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:44.878232 14748 raft_consensus.cc:2272] T fa373cebfd3540728639dddbaee4cb1d P 6a8699df36b74658a40f65ad30923ba9 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:44.905509 25684 consensus_peers.cc:597] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 -> Peer 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2:39169): Couldn't send request to peer 6a8699df36b74658a40f65ad30923ba9. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:39169: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:44.909890 14748 tablet_server.cc:196] TabletServer@127.14.103.2:0 shutdown complete.
W20260709 13:09:44.920882 25684 consensus_peers.cc:597] T fc82e3058d55474faed434b751f7c644 P 99dfff6363df46ce97b027cb3bd42e11 -> Peer 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2:39169): Couldn't send request to peer 6a8699df36b74658a40f65ad30923ba9. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:39169: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:44.928560 25686 consensus_peers.cc:597] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 -> Peer a7e4f250aa5d48759eaabbdff8c99210 (127.14.103.1:43185): Couldn't send request to peer a7e4f250aa5d48759eaabbdff8c99210. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:43185: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:44.940416 14748 tablet_server.cc:179] TabletServer@127.14.103.3:0 shutting down...
W20260709 13:09:44.961241 25684 consensus_peers.cc:597] T 64becd02c9434edeaf5824c5a455b0e0 P 99dfff6363df46ce97b027cb3bd42e11 -> Peer 6a8699df36b74658a40f65ad30923ba9 (127.14.103.2:39169): Couldn't send request to peer 6a8699df36b74658a40f65ad30923ba9. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:39169: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:44.969193 14748 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:09:44.969995 14748 tablet_replica.cc:333] T e1f6c52711174eacb83ea8fd2ad1b57b P 99dfff6363df46ce97b027cb3bd42e11: stopping tablet replica
I20260709 13:09:44.971115 14748 raft_consensus.cc:2243] T e1f6c52711174eacb83ea8fd2ad1b57b P 99dfff6363df46ce97b027cb3bd42e11 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:44.972324 14748 raft_consensus.cc:2272] T e1f6c52711174eacb83ea8fd2ad1b57b P 99dfff6363df46ce97b027cb3bd42e11 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:44.974781 14748 tablet_replica.cc:333] T 64becd02c9434edeaf5824c5a455b0e0 P 99dfff6363df46ce97b027cb3bd42e11: stopping tablet replica
I20260709 13:09:44.975376 14748 raft_consensus.cc:2243] T 64becd02c9434edeaf5824c5a455b0e0 P 99dfff6363df46ce97b027cb3bd42e11 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:44.976203 14748 raft_consensus.cc:2272] T 64becd02c9434edeaf5824c5a455b0e0 P 99dfff6363df46ce97b027cb3bd42e11 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:44.978739 14748 tablet_replica.cc:333] T 8372ab57bb904260beb2b0c69bc39ad0 P 99dfff6363df46ce97b027cb3bd42e11: stopping tablet replica
I20260709 13:09:44.979437 14748 raft_consensus.cc:2243] T 8372ab57bb904260beb2b0c69bc39ad0 P 99dfff6363df46ce97b027cb3bd42e11 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:44.980227 14748 raft_consensus.cc:2272] T 8372ab57bb904260beb2b0c69bc39ad0 P 99dfff6363df46ce97b027cb3bd42e11 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:44.982242 14748 tablet_replica.cc:333] T 0d81fc45e39443b9b33953f8c66437eb P 99dfff6363df46ce97b027cb3bd42e11: stopping tablet replica
I20260709 13:09:44.982776 14748 raft_consensus.cc:2243] T 0d81fc45e39443b9b33953f8c66437eb P 99dfff6363df46ce97b027cb3bd42e11 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:44.983254 14748 raft_consensus.cc:2272] T 0d81fc45e39443b9b33953f8c66437eb P 99dfff6363df46ce97b027cb3bd42e11 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:44.985379 14748 tablet_replica.cc:333] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11: stopping tablet replica
I20260709 13:09:44.985886 14748 raft_consensus.cc:2243] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:09:44.986658 14748 raft_consensus.cc:2272] T 2a14348b4c6a4d3cbbf18beffcd03814 P 99dfff6363df46ce97b027cb3bd42e11 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:44.989132 14748 tablet_replica.cc:333] T fc82e3058d55474faed434b751f7c644 P 99dfff6363df46ce97b027cb3bd42e11: stopping tablet replica
I20260709 13:09:44.989593 14748 raft_consensus.cc:2243] T fc82e3058d55474faed434b751f7c644 P 99dfff6363df46ce97b027cb3bd42e11 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:44.990284 14748 raft_consensus.cc:2272] T fc82e3058d55474faed434b751f7c644 P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:44.992363 14748 tablet_replica.cc:333] T 6d5ba3150425499f9cfa64eb06d6ba91 P 99dfff6363df46ce97b027cb3bd42e11: stopping tablet replica
I20260709 13:09:44.992873 14748 raft_consensus.cc:2243] T 6d5ba3150425499f9cfa64eb06d6ba91 P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:44.993283 14748 raft_consensus.cc:2272] T 6d5ba3150425499f9cfa64eb06d6ba91 P 99dfff6363df46ce97b027cb3bd42e11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:44.995127 14748 tablet_replica.cc:333] T bab07f367f714676a11b512f3abbea7c P 99dfff6363df46ce97b027cb3bd42e11: stopping tablet replica
I20260709 13:09:44.995573 14748 raft_consensus.cc:2243] T bab07f367f714676a11b512f3abbea7c P 99dfff6363df46ce97b027cb3bd42e11 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:44.995983 14748 raft_consensus.cc:2272] T bab07f367f714676a11b512f3abbea7c P 99dfff6363df46ce97b027cb3bd42e11 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:44.997889 14748 tablet_replica.cc:333] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11: stopping tablet replica
I20260709 13:09:44.998306 14748 raft_consensus.cc:2243] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:44.998654 14748 raft_consensus.cc:2272] T 377fe3ce2b474722b845c07b3f74b830 P 99dfff6363df46ce97b027cb3bd42e11 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:45.000581 14748 tablet_replica.cc:333] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11: stopping tablet replica
I20260709 13:09:45.001011 14748 raft_consensus.cc:2243] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:09:45.001710 14748 raft_consensus.cc:2272] T fa373cebfd3540728639dddbaee4cb1d P 99dfff6363df46ce97b027cb3bd42e11 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:45.034638 14748 tablet_server.cc:196] TabletServer@127.14.103.3:0 shutdown complete.
I20260709 13:09:45.059729 14748 master.cc:562] Master@127.14.103.62:33155 shutting down...
I20260709 13:09:45.084673 14748 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 47dfa972db9b4c5f8973bcbc1b8e3320 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:45.085184 14748 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 47dfa972db9b4c5f8973bcbc1b8e3320 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:45.085469 14748 tablet_replica.cc:333] T 00000000000000000000000000000000 P 47dfa972db9b4c5f8973bcbc1b8e3320: stopping tablet replica
I20260709 13:09:45.104230 14748 master.cc:584] Master@127.14.103.62:33155 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (13155 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 13:09:45.151248 14748 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.103.62:38415
I20260709 13:09:45.152401 14748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:45.158362 26065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:45.159082 26064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:45.161573 26067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:45.162954 14748 server_base.cc:1061] running on GCE node
I20260709 13:09:45.164104 14748 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:45.164408 14748 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:45.164605 14748 hybrid_clock.cc:648] HybridClock initialized: now 1783602585164587 us; error 0 us; skew 500 ppm
I20260709 13:09:45.165428 14748 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:45.168921 14748 webserver.cc:533] Webserver started at http://127.14.103.62:42993/ using document root <none> and password file <none>
I20260709 13:09:45.169616 14748 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:45.169857 14748 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:45.170228 14748 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:45.172067 14748 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602247531521-14748-0/minicluster-data/master-0-root/instance:
uuid: "634899637e364ac4944da5a00a221d3f"
format_stamp: "Formatted at 2026-07-09 13:09:45 on dist-test-slave-fpp7"
I20260709 13:09:45.178395 14748 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.002s	sys 0.004s
I20260709 13:09:45.183537 26072 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:45.184577 14748 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:09:45.184955 14748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602247531521-14748-0/minicluster-data/master-0-root
uuid: "634899637e364ac4944da5a00a221d3f"
format_stamp: "Formatted at 2026-07-09 13:09:45 on dist-test-slave-fpp7"
I20260709 13:09:45.185346 14748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602247531521-14748-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602247531521-14748-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602247531521-14748-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:45.205178 14748 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:45.206394 14748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:45.245590 14748 rpc_server.cc:307] RPC server started. Bound to: 127.14.103.62:38415
I20260709 13:09:45.245715 26123 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.103.62:38415 every 8 connection(s)
I20260709 13:09:45.249282 26124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:45.260325 26124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 634899637e364ac4944da5a00a221d3f: Bootstrap starting.
I20260709 13:09:45.265085 26124 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 634899637e364ac4944da5a00a221d3f: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:45.269459 26124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 634899637e364ac4944da5a00a221d3f: No bootstrap required, opened a new log
I20260709 13:09:45.271603 26124 raft_consensus.cc:359] T 00000000000000000000000000000000 P 634899637e364ac4944da5a00a221d3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "634899637e364ac4944da5a00a221d3f" member_type: VOTER }
I20260709 13:09:45.271996 26124 raft_consensus.cc:385] T 00000000000000000000000000000000 P 634899637e364ac4944da5a00a221d3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:45.272265 26124 raft_consensus.cc:740] T 00000000000000000000000000000000 P 634899637e364ac4944da5a00a221d3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 634899637e364ac4944da5a00a221d3f, State: Initialized, Role: FOLLOWER
I20260709 13:09:45.272935 26124 consensus_queue.cc:260] T 00000000000000000000000000000000 P 634899637e364ac4944da5a00a221d3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "634899637e364ac4944da5a00a221d3f" member_type: VOTER }
I20260709 13:09:45.273375 26124 raft_consensus.cc:399] T 00000000000000000000000000000000 P 634899637e364ac4944da5a00a221d3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:09:45.273591 26124 raft_consensus.cc:493] T 00000000000000000000000000000000 P 634899637e364ac4944da5a00a221d3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:09:45.273885 26124 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 634899637e364ac4944da5a00a221d3f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:45.278651 26124 raft_consensus.cc:515] T 00000000000000000000000000000000 P 634899637e364ac4944da5a00a221d3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "634899637e364ac4944da5a00a221d3f" member_type: VOTER }
I20260709 13:09:45.279176 26124 leader_election.cc:304] T 00000000000000000000000000000000 P 634899637e364ac4944da5a00a221d3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 634899637e364ac4944da5a00a221d3f; no voters: 
I20260709 13:09:45.280376 26124 leader_election.cc:290] T 00000000000000000000000000000000 P 634899637e364ac4944da5a00a221d3f [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:09:45.280735 26127 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 634899637e364ac4944da5a00a221d3f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:45.282037 26127 raft_consensus.cc:697] T 00000000000000000000000000000000 P 634899637e364ac4944da5a00a221d3f [term 1 LEADER]: Becoming Leader. State: Replica: 634899637e364ac4944da5a00a221d3f, State: Running, Role: LEADER
I20260709 13:09:45.282697 26127 consensus_queue.cc:237] T 00000000000000000000000000000000 P 634899637e364ac4944da5a00a221d3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "634899637e364ac4944da5a00a221d3f" member_type: VOTER }
I20260709 13:09:45.283252 26124 sys_catalog.cc:565] T 00000000000000000000000000000000 P 634899637e364ac4944da5a00a221d3f [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:09:45.287220 26129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 634899637e364ac4944da5a00a221d3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 634899637e364ac4944da5a00a221d3f. Latest consensus state: current_term: 1 leader_uuid: "634899637e364ac4944da5a00a221d3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "634899637e364ac4944da5a00a221d3f" member_type: VOTER } }
I20260709 13:09:45.287774 26128 sys_catalog.cc:455] T 00000000000000000000000000000000 P 634899637e364ac4944da5a00a221d3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "634899637e364ac4944da5a00a221d3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "634899637e364ac4944da5a00a221d3f" member_type: VOTER } }
I20260709 13:09:45.288190 26129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 634899637e364ac4944da5a00a221d3f [sys.catalog]: This master's current role is: LEADER
I20260709 13:09:45.288578 26128 sys_catalog.cc:458] T 00000000000000000000000000000000 P 634899637e364ac4944da5a00a221d3f [sys.catalog]: This master's current role is: LEADER
I20260709 13:09:45.290434 26134 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:09:45.296679 26134 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:09:45.299118 14748 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:09:45.306015 26134 catalog_manager.cc:1383] Generated new cluster ID: 8ca10bc89fa243d6bdb13037c35a307a
I20260709 13:09:45.306329 26134 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:09:45.327500 26134 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:09:45.328809 26134 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:09:45.343042 26134 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 634899637e364ac4944da5a00a221d3f: Generated new TSK 0
I20260709 13:09:45.343633 26134 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:09:45.365569 14748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:45.372365 26145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:45.373200 26146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:45.380096 26148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:45.380846 14748 server_base.cc:1061] running on GCE node
I20260709 13:09:45.381784 14748 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:45.382015 14748 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:45.382189 14748 hybrid_clock.cc:648] HybridClock initialized: now 1783602585382166 us; error 0 us; skew 500 ppm
I20260709 13:09:45.382725 14748 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:45.385098 14748 webserver.cc:533] Webserver started at http://127.14.103.1:35379/ using document root <none> and password file <none>
I20260709 13:09:45.385569 14748 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:45.385774 14748 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:45.386031 14748 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:45.387104 14748 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602247531521-14748-0/minicluster-data/ts-0-root/instance:
uuid: "15a9f50c757f4a1db9dd325d60af9e5a"
format_stamp: "Formatted at 2026-07-09 13:09:45 on dist-test-slave-fpp7"
I20260709 13:09:45.392169 14748 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.002s
I20260709 13:09:45.395507 26153 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:45.396250 14748 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:09:45.396575 14748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602247531521-14748-0/minicluster-data/ts-0-root
uuid: "15a9f50c757f4a1db9dd325d60af9e5a"
format_stamp: "Formatted at 2026-07-09 13:09:45 on dist-test-slave-fpp7"
I20260709 13:09:45.396873 14748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602247531521-14748-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602247531521-14748-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602247531521-14748-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:45.429999 14748 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:45.431171 14748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:45.432674 14748 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:09:45.434892 14748 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:09:45.435115 14748 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:45.435367 14748 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:09:45.435549 14748 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:45.476437 14748 rpc_server.cc:307] RPC server started. Bound to: 127.14.103.1:36999
I20260709 13:09:45.476583 26215 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.103.1:36999 every 8 connection(s)
I20260709 13:09:45.481348 14748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:45.489714 26221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:45.489934 26220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:45.492961 14748 server_base.cc:1061] running on GCE node
W20260709 13:09:45.493502 26224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:45.494318 26216 heartbeater.cc:344] Connected to a master server at 127.14.103.62:38415
I20260709 13:09:45.494627 14748 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:09:45.494727 26216 heartbeater.cc:461] Registering TS with master...
W20260709 13:09:45.494936 14748 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:45.495173 14748 hybrid_clock.cc:648] HybridClock initialized: now 1783602585495155 us; error 0 us; skew 500 ppm
I20260709 13:09:45.495735 26216 heartbeater.cc:507] Master 127.14.103.62:38415 requested a full tablet report, sending...
I20260709 13:09:45.495823 14748 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:45.497805 26089 ts_manager.cc:194] Registered new tserver with Master: 15a9f50c757f4a1db9dd325d60af9e5a (127.14.103.1:36999)
I20260709 13:09:45.498720 14748 webserver.cc:533] Webserver started at http://127.14.103.2:36023/ using document root <none> and password file <none>
I20260709 13:09:45.499279 14748 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:45.499486 14748 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:45.499786 14748 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:45.499806 26089 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50436
I20260709 13:09:45.501127 14748 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602247531521-14748-0/minicluster-data/ts-1-root/instance:
uuid: "791483bf50164b81bc3aafff482349b3"
format_stamp: "Formatted at 2026-07-09 13:09:45 on dist-test-slave-fpp7"
I20260709 13:09:45.505438 14748 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 13:09:45.508704 26228 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:45.509598 14748 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:09:45.509830 14748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602247531521-14748-0/minicluster-data/ts-1-root
uuid: "791483bf50164b81bc3aafff482349b3"
format_stamp: "Formatted at 2026-07-09 13:09:45 on dist-test-slave-fpp7"
I20260709 13:09:45.510063 14748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602247531521-14748-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602247531521-14748-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602247531521-14748-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:45.525209 14748 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:45.526396 14748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:45.528370 14748 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:09:45.530573 14748 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:09:45.530745 14748 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:45.531032 14748 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:09:45.531188 14748 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:45.569804 14748 rpc_server.cc:307] RPC server started. Bound to: 127.14.103.2:34761
I20260709 13:09:45.569885 26290 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.103.2:34761 every 8 connection(s)
I20260709 13:09:45.574621 14748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:45.581809 26295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:45.582937 26294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:45.584205 26297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:45.585669 14748 server_base.cc:1061] running on GCE node
I20260709 13:09:45.585855 26291 heartbeater.cc:344] Connected to a master server at 127.14.103.62:38415
I20260709 13:09:45.586355 26291 heartbeater.cc:461] Registering TS with master...
I20260709 13:09:45.586730 14748 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:45.586999 14748 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:45.587155 26291 heartbeater.cc:507] Master 127.14.103.62:38415 requested a full tablet report, sending...
I20260709 13:09:45.587230 14748 hybrid_clock.cc:648] HybridClock initialized: now 1783602585587211 us; error 0 us; skew 500 ppm
I20260709 13:09:45.588128 14748 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:45.589608 26089 ts_manager.cc:194] Registered new tserver with Master: 791483bf50164b81bc3aafff482349b3 (127.14.103.2:34761)
I20260709 13:09:45.590847 14748 webserver.cc:533] Webserver started at http://127.14.103.3:46759/ using document root <none> and password file <none>
I20260709 13:09:45.591531 14748 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:45.591436 26089 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50450
I20260709 13:09:45.591951 14748 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:45.592301 14748 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:45.593852 14748 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602247531521-14748-0/minicluster-data/ts-2-root/instance:
uuid: "140554b577e8402e9e8f538c126ffeaf"
format_stamp: "Formatted at 2026-07-09 13:09:45 on dist-test-slave-fpp7"
I20260709 13:09:45.600128 14748 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:09:45.603195 26302 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:45.604018 14748 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:09:45.604264 14748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602247531521-14748-0/minicluster-data/ts-2-root
uuid: "140554b577e8402e9e8f538c126ffeaf"
format_stamp: "Formatted at 2026-07-09 13:09:45 on dist-test-slave-fpp7"
I20260709 13:09:45.604496 14748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602247531521-14748-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602247531521-14748-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602247531521-14748-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:45.615837 14748 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:45.617010 14748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:45.618410 14748 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:09:45.620625 14748 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:09:45.620822 14748 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:45.621078 14748 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:09:45.621236 14748 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:45.660590 14748 rpc_server.cc:307] RPC server started. Bound to: 127.14.103.3:46569
I20260709 13:09:45.660723 26364 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.103.3:46569 every 8 connection(s)
I20260709 13:09:45.673890 26365 heartbeater.cc:344] Connected to a master server at 127.14.103.62:38415
I20260709 13:09:45.674329 26365 heartbeater.cc:461] Registering TS with master...
I20260709 13:09:45.674998 26365 heartbeater.cc:507] Master 127.14.103.62:38415 requested a full tablet report, sending...
I20260709 13:09:45.676909 26089 ts_manager.cc:194] Registered new tserver with Master: 140554b577e8402e9e8f538c126ffeaf (127.14.103.3:46569)
I20260709 13:09:45.677377 14748 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013521145s
I20260709 13:09:45.678256 26089 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50462
I20260709 13:09:45.702272 26089 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50470:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:09:45.704469 26089 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:09:45.760936 26256 tablet_service.cc:1511] Processing CreateTablet for tablet 99c3d927a4974e28a84534996d38c853 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=5bca3ca302e24d3ebd187ae1ddb2699a]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:09:45.762483 26256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99c3d927a4974e28a84534996d38c853. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:45.762241 26255 tablet_service.cc:1511] Processing CreateTablet for tablet 1ad0179ad6ba487a8044f56744eb86db (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=5bca3ca302e24d3ebd187ae1ddb2699a]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:09:45.763581 26255 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ad0179ad6ba487a8044f56744eb86db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:45.765738 26330 tablet_service.cc:1511] Processing CreateTablet for tablet 99c3d927a4974e28a84534996d38c853 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=5bca3ca302e24d3ebd187ae1ddb2699a]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:09:45.767316 26330 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99c3d927a4974e28a84534996d38c853. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:45.765846 26181 tablet_service.cc:1511] Processing CreateTablet for tablet 99c3d927a4974e28a84534996d38c853 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=5bca3ca302e24d3ebd187ae1ddb2699a]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:09:45.768268 26180 tablet_service.cc:1511] Processing CreateTablet for tablet 1ad0179ad6ba487a8044f56744eb86db (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=5bca3ca302e24d3ebd187ae1ddb2699a]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:09:45.768826 26181 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99c3d927a4974e28a84534996d38c853. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:45.769559 26180 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ad0179ad6ba487a8044f56744eb86db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:45.770881 26329 tablet_service.cc:1511] Processing CreateTablet for tablet 1ad0179ad6ba487a8044f56744eb86db (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=5bca3ca302e24d3ebd187ae1ddb2699a]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:09:45.772610 26329 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ad0179ad6ba487a8044f56744eb86db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:45.794680 26380 tablet_bootstrap.cc:492] T 99c3d927a4974e28a84534996d38c853 P 791483bf50164b81bc3aafff482349b3: Bootstrap starting.
I20260709 13:09:45.804479 26380 tablet_bootstrap.cc:654] T 99c3d927a4974e28a84534996d38c853 P 791483bf50164b81bc3aafff482349b3: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:45.809437 26381 tablet_bootstrap.cc:492] T 1ad0179ad6ba487a8044f56744eb86db P 140554b577e8402e9e8f538c126ffeaf: Bootstrap starting.
I20260709 13:09:45.814059 26382 tablet_bootstrap.cc:492] T 1ad0179ad6ba487a8044f56744eb86db P 15a9f50c757f4a1db9dd325d60af9e5a: Bootstrap starting.
I20260709 13:09:45.816517 26381 tablet_bootstrap.cc:654] T 1ad0179ad6ba487a8044f56744eb86db P 140554b577e8402e9e8f538c126ffeaf: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:45.817224 26380 tablet_bootstrap.cc:492] T 99c3d927a4974e28a84534996d38c853 P 791483bf50164b81bc3aafff482349b3: No bootstrap required, opened a new log
I20260709 13:09:45.817670 26380 ts_tablet_manager.cc:1403] T 99c3d927a4974e28a84534996d38c853 P 791483bf50164b81bc3aafff482349b3: Time spent bootstrapping tablet: real 0.023s	user 0.004s	sys 0.011s
I20260709 13:09:45.820099 26380 raft_consensus.cc:359] T 99c3d927a4974e28a84534996d38c853 P 791483bf50164b81bc3aafff482349b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:45.820844 26380 raft_consensus.cc:385] T 99c3d927a4974e28a84534996d38c853 P 791483bf50164b81bc3aafff482349b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:45.821102 26380 raft_consensus.cc:740] T 99c3d927a4974e28a84534996d38c853 P 791483bf50164b81bc3aafff482349b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 791483bf50164b81bc3aafff482349b3, State: Initialized, Role: FOLLOWER
I20260709 13:09:45.821527 26382 tablet_bootstrap.cc:654] T 1ad0179ad6ba487a8044f56744eb86db P 15a9f50c757f4a1db9dd325d60af9e5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:45.821756 26380 consensus_queue.cc:260] T 99c3d927a4974e28a84534996d38c853 P 791483bf50164b81bc3aafff482349b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:45.822134 26381 tablet_bootstrap.cc:492] T 1ad0179ad6ba487a8044f56744eb86db P 140554b577e8402e9e8f538c126ffeaf: No bootstrap required, opened a new log
I20260709 13:09:45.822788 26381 ts_tablet_manager.cc:1403] T 1ad0179ad6ba487a8044f56744eb86db P 140554b577e8402e9e8f538c126ffeaf: Time spent bootstrapping tablet: real 0.014s	user 0.013s	sys 0.000s
I20260709 13:09:45.824913 26380 ts_tablet_manager.cc:1434] T 99c3d927a4974e28a84534996d38c853 P 791483bf50164b81bc3aafff482349b3: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.007s
I20260709 13:09:45.825726 26291 heartbeater.cc:499] Master 127.14.103.62:38415 was elected leader, sending a full tablet report...
I20260709 13:09:45.826368 26380 tablet_bootstrap.cc:492] T 1ad0179ad6ba487a8044f56744eb86db P 791483bf50164b81bc3aafff482349b3: Bootstrap starting.
I20260709 13:09:45.827379 26381 raft_consensus.cc:359] T 1ad0179ad6ba487a8044f56744eb86db P 140554b577e8402e9e8f538c126ffeaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:45.828167 26381 raft_consensus.cc:385] T 1ad0179ad6ba487a8044f56744eb86db P 140554b577e8402e9e8f538c126ffeaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:45.828446 26381 raft_consensus.cc:740] T 1ad0179ad6ba487a8044f56744eb86db P 140554b577e8402e9e8f538c126ffeaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 140554b577e8402e9e8f538c126ffeaf, State: Initialized, Role: FOLLOWER
I20260709 13:09:45.829398 26381 consensus_queue.cc:260] T 1ad0179ad6ba487a8044f56744eb86db P 140554b577e8402e9e8f538c126ffeaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:45.830639 26382 tablet_bootstrap.cc:492] T 1ad0179ad6ba487a8044f56744eb86db P 15a9f50c757f4a1db9dd325d60af9e5a: No bootstrap required, opened a new log
I20260709 13:09:45.831291 26382 ts_tablet_manager.cc:1403] T 1ad0179ad6ba487a8044f56744eb86db P 15a9f50c757f4a1db9dd325d60af9e5a: Time spent bootstrapping tablet: real 0.017s	user 0.007s	sys 0.004s
I20260709 13:09:45.833778 26380 tablet_bootstrap.cc:654] T 1ad0179ad6ba487a8044f56744eb86db P 791483bf50164b81bc3aafff482349b3: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:45.835414 26382 raft_consensus.cc:359] T 1ad0179ad6ba487a8044f56744eb86db P 15a9f50c757f4a1db9dd325d60af9e5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:45.836511 26382 raft_consensus.cc:385] T 1ad0179ad6ba487a8044f56744eb86db P 15a9f50c757f4a1db9dd325d60af9e5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:45.836844 26382 raft_consensus.cc:740] T 1ad0179ad6ba487a8044f56744eb86db P 15a9f50c757f4a1db9dd325d60af9e5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15a9f50c757f4a1db9dd325d60af9e5a, State: Initialized, Role: FOLLOWER
I20260709 13:09:45.837823 26365 heartbeater.cc:499] Master 127.14.103.62:38415 was elected leader, sending a full tablet report...
I20260709 13:09:45.837743 26382 consensus_queue.cc:260] T 1ad0179ad6ba487a8044f56744eb86db P 15a9f50c757f4a1db9dd325d60af9e5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:45.838897 26381 ts_tablet_manager.cc:1434] T 1ad0179ad6ba487a8044f56744eb86db P 140554b577e8402e9e8f538c126ffeaf: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.004s
I20260709 13:09:45.844702 26381 tablet_bootstrap.cc:492] T 99c3d927a4974e28a84534996d38c853 P 140554b577e8402e9e8f538c126ffeaf: Bootstrap starting.
I20260709 13:09:45.846494 26380 tablet_bootstrap.cc:492] T 1ad0179ad6ba487a8044f56744eb86db P 791483bf50164b81bc3aafff482349b3: No bootstrap required, opened a new log
I20260709 13:09:45.848654 26380 ts_tablet_manager.cc:1403] T 1ad0179ad6ba487a8044f56744eb86db P 791483bf50164b81bc3aafff482349b3: Time spent bootstrapping tablet: real 0.022s	user 0.018s	sys 0.000s
I20260709 13:09:45.851747 26380 raft_consensus.cc:359] T 1ad0179ad6ba487a8044f56744eb86db P 791483bf50164b81bc3aafff482349b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:45.852451 26380 raft_consensus.cc:385] T 1ad0179ad6ba487a8044f56744eb86db P 791483bf50164b81bc3aafff482349b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:45.852759 26380 raft_consensus.cc:740] T 1ad0179ad6ba487a8044f56744eb86db P 791483bf50164b81bc3aafff482349b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 791483bf50164b81bc3aafff482349b3, State: Initialized, Role: FOLLOWER
I20260709 13:09:45.853438 26380 consensus_queue.cc:260] T 1ad0179ad6ba487a8044f56744eb86db P 791483bf50164b81bc3aafff482349b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:45.854590 26381 tablet_bootstrap.cc:654] T 99c3d927a4974e28a84534996d38c853 P 140554b577e8402e9e8f538c126ffeaf: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:45.857784 26216 heartbeater.cc:499] Master 127.14.103.62:38415 was elected leader, sending a full tablet report...
I20260709 13:09:45.857801 26380 ts_tablet_manager.cc:1434] T 1ad0179ad6ba487a8044f56744eb86db P 791483bf50164b81bc3aafff482349b3: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:09:45.858387 26382 ts_tablet_manager.cc:1434] T 1ad0179ad6ba487a8044f56744eb86db P 15a9f50c757f4a1db9dd325d60af9e5a: Time spent starting tablet: real 0.027s	user 0.016s	sys 0.012s
I20260709 13:09:45.859428 26382 tablet_bootstrap.cc:492] T 99c3d927a4974e28a84534996d38c853 P 15a9f50c757f4a1db9dd325d60af9e5a: Bootstrap starting.
I20260709 13:09:45.865736 26382 tablet_bootstrap.cc:654] T 99c3d927a4974e28a84534996d38c853 P 15a9f50c757f4a1db9dd325d60af9e5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:45.866896 26381 tablet_bootstrap.cc:492] T 99c3d927a4974e28a84534996d38c853 P 140554b577e8402e9e8f538c126ffeaf: No bootstrap required, opened a new log
I20260709 13:09:45.867313 26381 ts_tablet_manager.cc:1403] T 99c3d927a4974e28a84534996d38c853 P 140554b577e8402e9e8f538c126ffeaf: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.000s
I20260709 13:09:45.870249 26381 raft_consensus.cc:359] T 99c3d927a4974e28a84534996d38c853 P 140554b577e8402e9e8f538c126ffeaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:45.871001 26381 raft_consensus.cc:385] T 99c3d927a4974e28a84534996d38c853 P 140554b577e8402e9e8f538c126ffeaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:45.871294 26381 raft_consensus.cc:740] T 99c3d927a4974e28a84534996d38c853 P 140554b577e8402e9e8f538c126ffeaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 140554b577e8402e9e8f538c126ffeaf, State: Initialized, Role: FOLLOWER
I20260709 13:09:45.872303 26382 tablet_bootstrap.cc:492] T 99c3d927a4974e28a84534996d38c853 P 15a9f50c757f4a1db9dd325d60af9e5a: No bootstrap required, opened a new log
I20260709 13:09:45.872301 26381 consensus_queue.cc:260] T 99c3d927a4974e28a84534996d38c853 P 140554b577e8402e9e8f538c126ffeaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:45.873060 26382 ts_tablet_manager.cc:1403] T 99c3d927a4974e28a84534996d38c853 P 15a9f50c757f4a1db9dd325d60af9e5a: Time spent bootstrapping tablet: real 0.014s	user 0.013s	sys 0.000s
I20260709 13:09:45.874524 26381 ts_tablet_manager.cc:1434] T 99c3d927a4974e28a84534996d38c853 P 140554b577e8402e9e8f538c126ffeaf: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:09:45.876042 26382 raft_consensus.cc:359] T 99c3d927a4974e28a84534996d38c853 P 15a9f50c757f4a1db9dd325d60af9e5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:45.876626 26382 raft_consensus.cc:385] T 99c3d927a4974e28a84534996d38c853 P 15a9f50c757f4a1db9dd325d60af9e5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:45.876868 26382 raft_consensus.cc:740] T 99c3d927a4974e28a84534996d38c853 P 15a9f50c757f4a1db9dd325d60af9e5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15a9f50c757f4a1db9dd325d60af9e5a, State: Initialized, Role: FOLLOWER
I20260709 13:09:45.877485 26382 consensus_queue.cc:260] T 99c3d927a4974e28a84534996d38c853 P 15a9f50c757f4a1db9dd325d60af9e5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:45.879387 26382 ts_tablet_manager.cc:1434] T 99c3d927a4974e28a84534996d38c853 P 15a9f50c757f4a1db9dd325d60af9e5a: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 13:09:45.898845 26385 raft_consensus.cc:493] T 1ad0179ad6ba487a8044f56744eb86db P 791483bf50164b81bc3aafff482349b3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.899392 26385 raft_consensus.cc:515] T 1ad0179ad6ba487a8044f56744eb86db P 791483bf50164b81bc3aafff482349b3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:45.901422 26385 leader_election.cc:290] T 1ad0179ad6ba487a8044f56744eb86db P 791483bf50164b81bc3aafff482349b3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15a9f50c757f4a1db9dd325d60af9e5a (127.14.103.1:36999), 140554b577e8402e9e8f538c126ffeaf (127.14.103.3:46569)
I20260709 13:09:45.911304 26191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ad0179ad6ba487a8044f56744eb86db" candidate_uuid: "791483bf50164b81bc3aafff482349b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" is_pre_election: true
I20260709 13:09:45.912210 26191 raft_consensus.cc:2468] T 1ad0179ad6ba487a8044f56744eb86db P 15a9f50c757f4a1db9dd325d60af9e5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 791483bf50164b81bc3aafff482349b3 in term 0.
I20260709 13:09:45.912328 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ad0179ad6ba487a8044f56744eb86db" candidate_uuid: "791483bf50164b81bc3aafff482349b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "140554b577e8402e9e8f538c126ffeaf" is_pre_election: true
I20260709 13:09:45.913034 26340 raft_consensus.cc:2468] T 1ad0179ad6ba487a8044f56744eb86db P 140554b577e8402e9e8f538c126ffeaf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 791483bf50164b81bc3aafff482349b3 in term 0.
I20260709 13:09:45.913358 26229 leader_election.cc:304] T 1ad0179ad6ba487a8044f56744eb86db P 791483bf50164b81bc3aafff482349b3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15a9f50c757f4a1db9dd325d60af9e5a, 791483bf50164b81bc3aafff482349b3; no voters: 
I20260709 13:09:45.914181 26385 raft_consensus.cc:2804] T 1ad0179ad6ba487a8044f56744eb86db P 791483bf50164b81bc3aafff482349b3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:45.914554 26385 raft_consensus.cc:493] T 1ad0179ad6ba487a8044f56744eb86db P 791483bf50164b81bc3aafff482349b3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:45.914821 26385 raft_consensus.cc:3060] T 1ad0179ad6ba487a8044f56744eb86db P 791483bf50164b81bc3aafff482349b3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:45.919807 26385 raft_consensus.cc:515] T 1ad0179ad6ba487a8044f56744eb86db P 791483bf50164b81bc3aafff482349b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:45.921315 26385 leader_election.cc:290] T 1ad0179ad6ba487a8044f56744eb86db P 791483bf50164b81bc3aafff482349b3 [CANDIDATE]: Term 1 election: Requested vote from peers 15a9f50c757f4a1db9dd325d60af9e5a (127.14.103.1:36999), 140554b577e8402e9e8f538c126ffeaf (127.14.103.3:46569)
I20260709 13:09:45.922261 26191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ad0179ad6ba487a8044f56744eb86db" candidate_uuid: "791483bf50164b81bc3aafff482349b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15a9f50c757f4a1db9dd325d60af9e5a"
I20260709 13:09:45.922264 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ad0179ad6ba487a8044f56744eb86db" candidate_uuid: "791483bf50164b81bc3aafff482349b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "140554b577e8402e9e8f538c126ffeaf"
I20260709 13:09:45.923105 26191 raft_consensus.cc:3060] T 1ad0179ad6ba487a8044f56744eb86db P 15a9f50c757f4a1db9dd325d60af9e5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:45.923177 26340 raft_consensus.cc:3060] T 1ad0179ad6ba487a8044f56744eb86db P 140554b577e8402e9e8f538c126ffeaf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:45.930475 26340 raft_consensus.cc:2468] T 1ad0179ad6ba487a8044f56744eb86db P 140554b577e8402e9e8f538c126ffeaf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 791483bf50164b81bc3aafff482349b3 in term 1.
I20260709 13:09:45.930475 26191 raft_consensus.cc:2468] T 1ad0179ad6ba487a8044f56744eb86db P 15a9f50c757f4a1db9dd325d60af9e5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 791483bf50164b81bc3aafff482349b3 in term 1.
I20260709 13:09:45.931854 26230 leader_election.cc:304] T 1ad0179ad6ba487a8044f56744eb86db P 791483bf50164b81bc3aafff482349b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 140554b577e8402e9e8f538c126ffeaf, 791483bf50164b81bc3aafff482349b3; no voters: 
I20260709 13:09:45.932554 26385 raft_consensus.cc:2804] T 1ad0179ad6ba487a8044f56744eb86db P 791483bf50164b81bc3aafff482349b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:45.933568 26385 raft_consensus.cc:697] T 1ad0179ad6ba487a8044f56744eb86db P 791483bf50164b81bc3aafff482349b3 [term 1 LEADER]: Becoming Leader. State: Replica: 791483bf50164b81bc3aafff482349b3, State: Running, Role: LEADER
I20260709 13:09:45.934347 26385 consensus_queue.cc:237] T 1ad0179ad6ba487a8044f56744eb86db P 791483bf50164b81bc3aafff482349b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:45.941577 26089 catalog_manager.cc:5697] T 1ad0179ad6ba487a8044f56744eb86db P 791483bf50164b81bc3aafff482349b3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 791483bf50164b81bc3aafff482349b3 (127.14.103.2). New cstate: current_term: 1 leader_uuid: "791483bf50164b81bc3aafff482349b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:45.993439 26385 raft_consensus.cc:493] T 99c3d927a4974e28a84534996d38c853 P 791483bf50164b81bc3aafff482349b3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.993947 26385 raft_consensus.cc:515] T 99c3d927a4974e28a84534996d38c853 P 791483bf50164b81bc3aafff482349b3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:45.995627 26385 leader_election.cc:290] T 99c3d927a4974e28a84534996d38c853 P 791483bf50164b81bc3aafff482349b3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15a9f50c757f4a1db9dd325d60af9e5a (127.14.103.1:36999), 140554b577e8402e9e8f538c126ffeaf (127.14.103.3:46569)
I20260709 13:09:45.996477 26191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c3d927a4974e28a84534996d38c853" candidate_uuid: "791483bf50164b81bc3aafff482349b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" is_pre_election: true
I20260709 13:09:45.996634 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c3d927a4974e28a84534996d38c853" candidate_uuid: "791483bf50164b81bc3aafff482349b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "140554b577e8402e9e8f538c126ffeaf" is_pre_election: true
I20260709 13:09:45.997124 26191 raft_consensus.cc:2468] T 99c3d927a4974e28a84534996d38c853 P 15a9f50c757f4a1db9dd325d60af9e5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 791483bf50164b81bc3aafff482349b3 in term 0.
I20260709 13:09:45.997251 26340 raft_consensus.cc:2468] T 99c3d927a4974e28a84534996d38c853 P 140554b577e8402e9e8f538c126ffeaf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 791483bf50164b81bc3aafff482349b3 in term 0.
I20260709 13:09:45.998139 26229 leader_election.cc:304] T 99c3d927a4974e28a84534996d38c853 P 791483bf50164b81bc3aafff482349b3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15a9f50c757f4a1db9dd325d60af9e5a, 791483bf50164b81bc3aafff482349b3; no voters: 
I20260709 13:09:45.998926 26385 raft_consensus.cc:2804] T 99c3d927a4974e28a84534996d38c853 P 791483bf50164b81bc3aafff482349b3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:45.999243 26385 raft_consensus.cc:493] T 99c3d927a4974e28a84534996d38c853 P 791483bf50164b81bc3aafff482349b3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:45.999505 26385 raft_consensus.cc:3060] T 99c3d927a4974e28a84534996d38c853 P 791483bf50164b81bc3aafff482349b3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:46.004612 26385 raft_consensus.cc:515] T 99c3d927a4974e28a84534996d38c853 P 791483bf50164b81bc3aafff482349b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:46.006224 26385 leader_election.cc:290] T 99c3d927a4974e28a84534996d38c853 P 791483bf50164b81bc3aafff482349b3 [CANDIDATE]: Term 1 election: Requested vote from peers 15a9f50c757f4a1db9dd325d60af9e5a (127.14.103.1:36999), 140554b577e8402e9e8f538c126ffeaf (127.14.103.3:46569)
I20260709 13:09:46.007237 26191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c3d927a4974e28a84534996d38c853" candidate_uuid: "791483bf50164b81bc3aafff482349b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15a9f50c757f4a1db9dd325d60af9e5a"
I20260709 13:09:46.007287 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c3d927a4974e28a84534996d38c853" candidate_uuid: "791483bf50164b81bc3aafff482349b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "140554b577e8402e9e8f538c126ffeaf"
I20260709 13:09:46.007855 26191 raft_consensus.cc:3060] T 99c3d927a4974e28a84534996d38c853 P 15a9f50c757f4a1db9dd325d60af9e5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:46.008108 26340 raft_consensus.cc:3060] T 99c3d927a4974e28a84534996d38c853 P 140554b577e8402e9e8f538c126ffeaf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:46.013191 26191 raft_consensus.cc:2468] T 99c3d927a4974e28a84534996d38c853 P 15a9f50c757f4a1db9dd325d60af9e5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 791483bf50164b81bc3aafff482349b3 in term 1.
I20260709 13:09:46.013718 26340 raft_consensus.cc:2468] T 99c3d927a4974e28a84534996d38c853 P 140554b577e8402e9e8f538c126ffeaf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 791483bf50164b81bc3aafff482349b3 in term 1.
I20260709 13:09:46.014312 26229 leader_election.cc:304] T 99c3d927a4974e28a84534996d38c853 P 791483bf50164b81bc3aafff482349b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15a9f50c757f4a1db9dd325d60af9e5a, 791483bf50164b81bc3aafff482349b3; no voters: 
I20260709 13:09:46.015163 26385 raft_consensus.cc:2804] T 99c3d927a4974e28a84534996d38c853 P 791483bf50164b81bc3aafff482349b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:46.015591 26385 raft_consensus.cc:697] T 99c3d927a4974e28a84534996d38c853 P 791483bf50164b81bc3aafff482349b3 [term 1 LEADER]: Becoming Leader. State: Replica: 791483bf50164b81bc3aafff482349b3, State: Running, Role: LEADER
I20260709 13:09:46.016446 26385 consensus_queue.cc:237] T 99c3d927a4974e28a84534996d38c853 P 791483bf50164b81bc3aafff482349b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:46.022540 26089 catalog_manager.cc:5697] T 99c3d927a4974e28a84534996d38c853 P 791483bf50164b81bc3aafff482349b3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 791483bf50164b81bc3aafff482349b3 (127.14.103.2). New cstate: current_term: 1 leader_uuid: "791483bf50164b81bc3aafff482349b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:46.073626 26089 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50484:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:09:46.075958 26089 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:09:46.098192 26180 tablet_service.cc:1511] Processing CreateTablet for tablet b6f3ad902a4f418388c74309b0021cb6 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=5a78c881d09d425594ac46fdb81d10c6]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:09:46.099483 26180 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6f3ad902a4f418388c74309b0021cb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:46.099337 26256 tablet_service.cc:1511] Processing CreateTablet for tablet b6f3ad902a4f418388c74309b0021cb6 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=5a78c881d09d425594ac46fdb81d10c6]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:09:46.100689 26256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6f3ad902a4f418388c74309b0021cb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:46.100610 26329 tablet_service.cc:1511] Processing CreateTablet for tablet b6f3ad902a4f418388c74309b0021cb6 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=5a78c881d09d425594ac46fdb81d10c6]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:09:46.101909 26329 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6f3ad902a4f418388c74309b0021cb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:46.107288 26181 tablet_service.cc:1511] Processing CreateTablet for tablet f865408f7c5a4a2cbf1e74dea076c516 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=5a78c881d09d425594ac46fdb81d10c6]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:09:46.108578 26181 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f865408f7c5a4a2cbf1e74dea076c516. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:46.111279 26255 tablet_service.cc:1511] Processing CreateTablet for tablet f865408f7c5a4a2cbf1e74dea076c516 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=5a78c881d09d425594ac46fdb81d10c6]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:09:46.112581 26255 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f865408f7c5a4a2cbf1e74dea076c516. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:46.112577 26330 tablet_service.cc:1511] Processing CreateTablet for tablet f865408f7c5a4a2cbf1e74dea076c516 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=5a78c881d09d425594ac46fdb81d10c6]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:09:46.113873 26330 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f865408f7c5a4a2cbf1e74dea076c516. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:46.115199 26382 tablet_bootstrap.cc:492] T b6f3ad902a4f418388c74309b0021cb6 P 15a9f50c757f4a1db9dd325d60af9e5a: Bootstrap starting.
I20260709 13:09:46.120349 26381 tablet_bootstrap.cc:492] T b6f3ad902a4f418388c74309b0021cb6 P 140554b577e8402e9e8f538c126ffeaf: Bootstrap starting.
I20260709 13:09:46.122080 26382 tablet_bootstrap.cc:654] T b6f3ad902a4f418388c74309b0021cb6 P 15a9f50c757f4a1db9dd325d60af9e5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:46.127862 26381 tablet_bootstrap.cc:654] T b6f3ad902a4f418388c74309b0021cb6 P 140554b577e8402e9e8f538c126ffeaf: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:46.138362 26380 tablet_bootstrap.cc:492] T f865408f7c5a4a2cbf1e74dea076c516 P 791483bf50164b81bc3aafff482349b3: Bootstrap starting.
I20260709 13:09:46.139223 26382 tablet_bootstrap.cc:492] T b6f3ad902a4f418388c74309b0021cb6 P 15a9f50c757f4a1db9dd325d60af9e5a: No bootstrap required, opened a new log
I20260709 13:09:46.141381 26381 tablet_bootstrap.cc:492] T b6f3ad902a4f418388c74309b0021cb6 P 140554b577e8402e9e8f538c126ffeaf: No bootstrap required, opened a new log
I20260709 13:09:46.141959 26381 ts_tablet_manager.cc:1403] T b6f3ad902a4f418388c74309b0021cb6 P 140554b577e8402e9e8f538c126ffeaf: Time spent bootstrapping tablet: real 0.022s	user 0.003s	sys 0.009s
I20260709 13:09:46.143947 26382 ts_tablet_manager.cc:1403] T b6f3ad902a4f418388c74309b0021cb6 P 15a9f50c757f4a1db9dd325d60af9e5a: Time spent bootstrapping tablet: real 0.029s	user 0.014s	sys 0.000s
I20260709 13:09:46.145308 26380 tablet_bootstrap.cc:654] T f865408f7c5a4a2cbf1e74dea076c516 P 791483bf50164b81bc3aafff482349b3: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:46.144971 26381 raft_consensus.cc:359] T b6f3ad902a4f418388c74309b0021cb6 P 140554b577e8402e9e8f538c126ffeaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:46.145838 26381 raft_consensus.cc:385] T b6f3ad902a4f418388c74309b0021cb6 P 140554b577e8402e9e8f538c126ffeaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:46.146166 26381 raft_consensus.cc:740] T b6f3ad902a4f418388c74309b0021cb6 P 140554b577e8402e9e8f538c126ffeaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 140554b577e8402e9e8f538c126ffeaf, State: Initialized, Role: FOLLOWER
I20260709 13:09:46.146966 26381 consensus_queue.cc:260] T b6f3ad902a4f418388c74309b0021cb6 P 140554b577e8402e9e8f538c126ffeaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:46.148458 26382 raft_consensus.cc:359] T b6f3ad902a4f418388c74309b0021cb6 P 15a9f50c757f4a1db9dd325d60af9e5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:46.149282 26382 raft_consensus.cc:385] T b6f3ad902a4f418388c74309b0021cb6 P 15a9f50c757f4a1db9dd325d60af9e5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:46.149601 26382 raft_consensus.cc:740] T b6f3ad902a4f418388c74309b0021cb6 P 15a9f50c757f4a1db9dd325d60af9e5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15a9f50c757f4a1db9dd325d60af9e5a, State: Initialized, Role: FOLLOWER
I20260709 13:09:46.149861 26381 ts_tablet_manager.cc:1434] T b6f3ad902a4f418388c74309b0021cb6 P 140554b577e8402e9e8f538c126ffeaf: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:09:46.151124 26381 tablet_bootstrap.cc:492] T f865408f7c5a4a2cbf1e74dea076c516 P 140554b577e8402e9e8f538c126ffeaf: Bootstrap starting.
I20260709 13:09:46.150436 26382 consensus_queue.cc:260] T b6f3ad902a4f418388c74309b0021cb6 P 15a9f50c757f4a1db9dd325d60af9e5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:46.154835 26382 ts_tablet_manager.cc:1434] T b6f3ad902a4f418388c74309b0021cb6 P 15a9f50c757f4a1db9dd325d60af9e5a: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:09:46.156349 26382 tablet_bootstrap.cc:492] T f865408f7c5a4a2cbf1e74dea076c516 P 15a9f50c757f4a1db9dd325d60af9e5a: Bootstrap starting.
I20260709 13:09:46.159004 26381 tablet_bootstrap.cc:654] T f865408f7c5a4a2cbf1e74dea076c516 P 140554b577e8402e9e8f538c126ffeaf: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:46.162973 26382 tablet_bootstrap.cc:654] T f865408f7c5a4a2cbf1e74dea076c516 P 15a9f50c757f4a1db9dd325d60af9e5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:46.165237 26380 tablet_bootstrap.cc:492] T f865408f7c5a4a2cbf1e74dea076c516 P 791483bf50164b81bc3aafff482349b3: No bootstrap required, opened a new log
I20260709 13:09:46.165951 26380 ts_tablet_manager.cc:1403] T f865408f7c5a4a2cbf1e74dea076c516 P 791483bf50164b81bc3aafff482349b3: Time spent bootstrapping tablet: real 0.028s	user 0.007s	sys 0.010s
I20260709 13:09:46.170562 26380 raft_consensus.cc:359] T f865408f7c5a4a2cbf1e74dea076c516 P 791483bf50164b81bc3aafff482349b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:46.171663 26380 raft_consensus.cc:385] T f865408f7c5a4a2cbf1e74dea076c516 P 791483bf50164b81bc3aafff482349b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:46.172252 26380 raft_consensus.cc:740] T f865408f7c5a4a2cbf1e74dea076c516 P 791483bf50164b81bc3aafff482349b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 791483bf50164b81bc3aafff482349b3, State: Initialized, Role: FOLLOWER
I20260709 13:09:46.174309 26380 consensus_queue.cc:260] T f865408f7c5a4a2cbf1e74dea076c516 P 791483bf50164b81bc3aafff482349b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:46.185134 26382 tablet_bootstrap.cc:492] T f865408f7c5a4a2cbf1e74dea076c516 P 15a9f50c757f4a1db9dd325d60af9e5a: No bootstrap required, opened a new log
I20260709 13:09:46.185706 26382 ts_tablet_manager.cc:1403] T f865408f7c5a4a2cbf1e74dea076c516 P 15a9f50c757f4a1db9dd325d60af9e5a: Time spent bootstrapping tablet: real 0.030s	user 0.014s	sys 0.012s
I20260709 13:09:46.186470 26380 ts_tablet_manager.cc:1434] T f865408f7c5a4a2cbf1e74dea076c516 P 791483bf50164b81bc3aafff482349b3: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.004s
I20260709 13:09:46.187626 26380 tablet_bootstrap.cc:492] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3: Bootstrap starting.
I20260709 13:09:46.188236 26381 tablet_bootstrap.cc:492] T f865408f7c5a4a2cbf1e74dea076c516 P 140554b577e8402e9e8f538c126ffeaf: No bootstrap required, opened a new log
I20260709 13:09:46.188760 26381 ts_tablet_manager.cc:1403] T f865408f7c5a4a2cbf1e74dea076c516 P 140554b577e8402e9e8f538c126ffeaf: Time spent bootstrapping tablet: real 0.038s	user 0.019s	sys 0.017s
I20260709 13:09:46.188427 26382 raft_consensus.cc:359] T f865408f7c5a4a2cbf1e74dea076c516 P 15a9f50c757f4a1db9dd325d60af9e5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:46.189146 26382 raft_consensus.cc:385] T f865408f7c5a4a2cbf1e74dea076c516 P 15a9f50c757f4a1db9dd325d60af9e5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:46.189450 26382 raft_consensus.cc:740] T f865408f7c5a4a2cbf1e74dea076c516 P 15a9f50c757f4a1db9dd325d60af9e5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15a9f50c757f4a1db9dd325d60af9e5a, State: Initialized, Role: FOLLOWER
I20260709 13:09:46.190315 26382 consensus_queue.cc:260] T f865408f7c5a4a2cbf1e74dea076c516 P 15a9f50c757f4a1db9dd325d60af9e5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:46.191813 26381 raft_consensus.cc:359] T f865408f7c5a4a2cbf1e74dea076c516 P 140554b577e8402e9e8f538c126ffeaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:46.192561 26381 raft_consensus.cc:385] T f865408f7c5a4a2cbf1e74dea076c516 P 140554b577e8402e9e8f538c126ffeaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:46.192860 26381 raft_consensus.cc:740] T f865408f7c5a4a2cbf1e74dea076c516 P 140554b577e8402e9e8f538c126ffeaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 140554b577e8402e9e8f538c126ffeaf, State: Initialized, Role: FOLLOWER
I20260709 13:09:46.195533 26380 tablet_bootstrap.cc:654] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:46.193732 26381 consensus_queue.cc:260] T f865408f7c5a4a2cbf1e74dea076c516 P 140554b577e8402e9e8f538c126ffeaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:46.197775 26381 ts_tablet_manager.cc:1434] T f865408f7c5a4a2cbf1e74dea076c516 P 140554b577e8402e9e8f538c126ffeaf: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.002s
I20260709 13:09:46.199334 26382 ts_tablet_manager.cc:1434] T f865408f7c5a4a2cbf1e74dea076c516 P 15a9f50c757f4a1db9dd325d60af9e5a: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:09:46.205489 26380 tablet_bootstrap.cc:492] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3: No bootstrap required, opened a new log
I20260709 13:09:46.205998 26380 ts_tablet_manager.cc:1403] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.000s
I20260709 13:09:46.208528 26380 raft_consensus.cc:359] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:46.209095 26380 raft_consensus.cc:385] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:46.209352 26380 raft_consensus.cc:740] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 791483bf50164b81bc3aafff482349b3, State: Initialized, Role: FOLLOWER
I20260709 13:09:46.210072 26380 consensus_queue.cc:260] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:46.212086 26380 ts_tablet_manager.cc:1434] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:09:46.280128 26385 raft_consensus.cc:493] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:46.280548 26385 raft_consensus.cc:515] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:46.282205 26385 leader_election.cc:290] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15a9f50c757f4a1db9dd325d60af9e5a (127.14.103.1:36999), 140554b577e8402e9e8f538c126ffeaf (127.14.103.3:46569)
I20260709 13:09:46.282987 26191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f3ad902a4f418388c74309b0021cb6" candidate_uuid: "791483bf50164b81bc3aafff482349b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" is_pre_election: true
I20260709 13:09:46.283180 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f3ad902a4f418388c74309b0021cb6" candidate_uuid: "791483bf50164b81bc3aafff482349b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "140554b577e8402e9e8f538c126ffeaf" is_pre_election: true
I20260709 13:09:46.283600 26191 raft_consensus.cc:2468] T b6f3ad902a4f418388c74309b0021cb6 P 15a9f50c757f4a1db9dd325d60af9e5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 791483bf50164b81bc3aafff482349b3 in term 0.
I20260709 13:09:46.283824 26340 raft_consensus.cc:2468] T b6f3ad902a4f418388c74309b0021cb6 P 140554b577e8402e9e8f538c126ffeaf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 791483bf50164b81bc3aafff482349b3 in term 0.
I20260709 13:09:46.284634 26229 leader_election.cc:304] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15a9f50c757f4a1db9dd325d60af9e5a, 791483bf50164b81bc3aafff482349b3; no voters: 
I20260709 13:09:46.285400 26385 raft_consensus.cc:2804] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:46.285714 26385 raft_consensus.cc:493] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:46.285977 26385 raft_consensus.cc:3060] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:46.291168 26385 raft_consensus.cc:515] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:46.292639 26385 leader_election.cc:290] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 [CANDIDATE]: Term 1 election: Requested vote from peers 15a9f50c757f4a1db9dd325d60af9e5a (127.14.103.1:36999), 140554b577e8402e9e8f538c126ffeaf (127.14.103.3:46569)
I20260709 13:09:46.293417 26191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f3ad902a4f418388c74309b0021cb6" candidate_uuid: "791483bf50164b81bc3aafff482349b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15a9f50c757f4a1db9dd325d60af9e5a"
I20260709 13:09:46.293650 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f3ad902a4f418388c74309b0021cb6" candidate_uuid: "791483bf50164b81bc3aafff482349b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "140554b577e8402e9e8f538c126ffeaf"
I20260709 13:09:46.294050 26191 raft_consensus.cc:3060] T b6f3ad902a4f418388c74309b0021cb6 P 15a9f50c757f4a1db9dd325d60af9e5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:46.294198 26340 raft_consensus.cc:3060] T b6f3ad902a4f418388c74309b0021cb6 P 140554b577e8402e9e8f538c126ffeaf [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:09:46.294615 26137 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:46.300942 26191 raft_consensus.cc:2468] T b6f3ad902a4f418388c74309b0021cb6 P 15a9f50c757f4a1db9dd325d60af9e5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 791483bf50164b81bc3aafff482349b3 in term 1.
I20260709 13:09:46.300946 26340 raft_consensus.cc:2468] T b6f3ad902a4f418388c74309b0021cb6 P 140554b577e8402e9e8f538c126ffeaf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 791483bf50164b81bc3aafff482349b3 in term 1.
I20260709 13:09:46.302347 26230 leader_election.cc:304] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 140554b577e8402e9e8f538c126ffeaf, 791483bf50164b81bc3aafff482349b3; no voters: 
I20260709 13:09:46.303063 26385 raft_consensus.cc:2804] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:46.303479 26385 raft_consensus.cc:697] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 [term 1 LEADER]: Becoming Leader. State: Replica: 791483bf50164b81bc3aafff482349b3, State: Running, Role: LEADER
I20260709 13:09:46.304265 26385 consensus_queue.cc:237] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:46.310694 26089 catalog_manager.cc:5697] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 791483bf50164b81bc3aafff482349b3 (127.14.103.2). New cstate: current_term: 1 leader_uuid: "791483bf50164b81bc3aafff482349b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:46.430051 26385 consensus_queue.cc:1048] T 1ad0179ad6ba487a8044f56744eb86db P 791483bf50164b81bc3aafff482349b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:46.442436 26393 consensus_queue.cc:1048] T 1ad0179ad6ba487a8044f56744eb86db P 791483bf50164b81bc3aafff482349b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:46.444870 26410 consensus_queue.cc:1048] T 99c3d927a4974e28a84534996d38c853 P 791483bf50164b81bc3aafff482349b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:46.460429 26393 consensus_queue.cc:1048] T 99c3d927a4974e28a84534996d38c853 P 791483bf50164b81bc3aafff482349b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:46.532086 26389 raft_consensus.cc:493] T f865408f7c5a4a2cbf1e74dea076c516 P 15a9f50c757f4a1db9dd325d60af9e5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:46.532608 26389 raft_consensus.cc:515] T f865408f7c5a4a2cbf1e74dea076c516 P 15a9f50c757f4a1db9dd325d60af9e5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:46.534574 26389 leader_election.cc:290] T f865408f7c5a4a2cbf1e74dea076c516 P 15a9f50c757f4a1db9dd325d60af9e5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 791483bf50164b81bc3aafff482349b3 (127.14.103.2:34761), 140554b577e8402e9e8f538c126ffeaf (127.14.103.3:46569)
I20260709 13:09:46.552938 26266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f865408f7c5a4a2cbf1e74dea076c516" candidate_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "791483bf50164b81bc3aafff482349b3" is_pre_election: true
I20260709 13:09:46.553697 26266 raft_consensus.cc:2468] T f865408f7c5a4a2cbf1e74dea076c516 P 791483bf50164b81bc3aafff482349b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15a9f50c757f4a1db9dd325d60af9e5a in term 0.
I20260709 13:09:46.554208 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f865408f7c5a4a2cbf1e74dea076c516" candidate_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "140554b577e8402e9e8f538c126ffeaf" is_pre_election: true
I20260709 13:09:46.554826 26339 raft_consensus.cc:2468] T f865408f7c5a4a2cbf1e74dea076c516 P 140554b577e8402e9e8f538c126ffeaf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15a9f50c757f4a1db9dd325d60af9e5a in term 0.
I20260709 13:09:46.554860 26157 leader_election.cc:304] T f865408f7c5a4a2cbf1e74dea076c516 P 15a9f50c757f4a1db9dd325d60af9e5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15a9f50c757f4a1db9dd325d60af9e5a, 791483bf50164b81bc3aafff482349b3; no voters: 
I20260709 13:09:46.555753 26389 raft_consensus.cc:2804] T f865408f7c5a4a2cbf1e74dea076c516 P 15a9f50c757f4a1db9dd325d60af9e5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:46.556095 26389 raft_consensus.cc:493] T f865408f7c5a4a2cbf1e74dea076c516 P 15a9f50c757f4a1db9dd325d60af9e5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:46.556358 26389 raft_consensus.cc:3060] T f865408f7c5a4a2cbf1e74dea076c516 P 15a9f50c757f4a1db9dd325d60af9e5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:46.561076 26389 raft_consensus.cc:515] T f865408f7c5a4a2cbf1e74dea076c516 P 15a9f50c757f4a1db9dd325d60af9e5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:46.562491 26389 leader_election.cc:290] T f865408f7c5a4a2cbf1e74dea076c516 P 15a9f50c757f4a1db9dd325d60af9e5a [CANDIDATE]: Term 1 election: Requested vote from peers 791483bf50164b81bc3aafff482349b3 (127.14.103.2:34761), 140554b577e8402e9e8f538c126ffeaf (127.14.103.3:46569)
I20260709 13:09:46.563158 26266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f865408f7c5a4a2cbf1e74dea076c516" candidate_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "791483bf50164b81bc3aafff482349b3"
I20260709 13:09:46.563467 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f865408f7c5a4a2cbf1e74dea076c516" candidate_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "140554b577e8402e9e8f538c126ffeaf"
I20260709 13:09:46.563771 26266 raft_consensus.cc:3060] T f865408f7c5a4a2cbf1e74dea076c516 P 791483bf50164b81bc3aafff482349b3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:46.564109 26339 raft_consensus.cc:3060] T f865408f7c5a4a2cbf1e74dea076c516 P 140554b577e8402e9e8f538c126ffeaf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:46.571316 26266 raft_consensus.cc:2468] T f865408f7c5a4a2cbf1e74dea076c516 P 791483bf50164b81bc3aafff482349b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15a9f50c757f4a1db9dd325d60af9e5a in term 1.
I20260709 13:09:46.571316 26339 raft_consensus.cc:2468] T f865408f7c5a4a2cbf1e74dea076c516 P 140554b577e8402e9e8f538c126ffeaf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15a9f50c757f4a1db9dd325d60af9e5a in term 1.
I20260709 13:09:46.572604 26157 leader_election.cc:304] T f865408f7c5a4a2cbf1e74dea076c516 P 15a9f50c757f4a1db9dd325d60af9e5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15a9f50c757f4a1db9dd325d60af9e5a, 791483bf50164b81bc3aafff482349b3; no voters: 
I20260709 13:09:46.573239 26389 raft_consensus.cc:2804] T f865408f7c5a4a2cbf1e74dea076c516 P 15a9f50c757f4a1db9dd325d60af9e5a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:46.574133 26389 raft_consensus.cc:697] T f865408f7c5a4a2cbf1e74dea076c516 P 15a9f50c757f4a1db9dd325d60af9e5a [term 1 LEADER]: Becoming Leader. State: Replica: 15a9f50c757f4a1db9dd325d60af9e5a, State: Running, Role: LEADER
I20260709 13:09:46.574779 26389 consensus_queue.cc:237] T f865408f7c5a4a2cbf1e74dea076c516 P 15a9f50c757f4a1db9dd325d60af9e5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:46.581156 26088 catalog_manager.cc:5697] T f865408f7c5a4a2cbf1e74dea076c516 P 15a9f50c757f4a1db9dd325d60af9e5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 15a9f50c757f4a1db9dd325d60af9e5a (127.14.103.1). New cstate: current_term: 1 leader_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:46.607115 26266 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "99c3d927a4974e28a84534996d38c853"
dest_uuid: "791483bf50164b81bc3aafff482349b3"
mode: GRACEFUL
new_leader_uuid: "15a9f50c757f4a1db9dd325d60af9e5a"
 from {username='slave'} at 127.0.0.1:37566
I20260709 13:09:46.607591 26266 raft_consensus.cc:606] T 99c3d927a4974e28a84534996d38c853 P 791483bf50164b81bc3aafff482349b3 [term 1 LEADER]: Received request to transfer leadership to TS 15a9f50c757f4a1db9dd325d60af9e5a
I20260709 13:09:46.612358 26266 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1ad0179ad6ba487a8044f56744eb86db"
dest_uuid: "791483bf50164b81bc3aafff482349b3"
mode: GRACEFUL
new_leader_uuid: "140554b577e8402e9e8f538c126ffeaf"
 from {username='slave'} at 127.0.0.1:37566
I20260709 13:09:46.612860 26266 raft_consensus.cc:606] T 1ad0179ad6ba487a8044f56744eb86db P 791483bf50164b81bc3aafff482349b3 [term 1 LEADER]: Received request to transfer leadership to TS 140554b577e8402e9e8f538c126ffeaf
I20260709 13:09:46.618558 26266 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b6f3ad902a4f418388c74309b0021cb6"
dest_uuid: "791483bf50164b81bc3aafff482349b3"
mode: GRACEFUL
new_leader_uuid: "15a9f50c757f4a1db9dd325d60af9e5a"
 from {username='slave'} at 127.0.0.1:37566
I20260709 13:09:46.618954 26266 raft_consensus.cc:606] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 [term 1 LEADER]: Received request to transfer leadership to TS 15a9f50c757f4a1db9dd325d60af9e5a
I20260709 13:09:46.901398 26393 consensus_queue.cc:1048] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:46.907023 26410 raft_consensus.cc:993] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3: : Instructing follower 15a9f50c757f4a1db9dd325d60af9e5a to start an election
I20260709 13:09:46.907491 26385 raft_consensus.cc:1081] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 [term 1 LEADER]: Signalling peer 15a9f50c757f4a1db9dd325d60af9e5a to start an election
I20260709 13:09:46.909257 26190 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b6f3ad902a4f418388c74309b0021cb6"
dest_uuid: "15a9f50c757f4a1db9dd325d60af9e5a"
 from {username='slave'} at 127.0.0.1:38038
I20260709 13:09:46.909830 26190 raft_consensus.cc:493] T b6f3ad902a4f418388c74309b0021cb6 P 15a9f50c757f4a1db9dd325d60af9e5a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:46.910207 26190 raft_consensus.cc:3060] T b6f3ad902a4f418388c74309b0021cb6 P 15a9f50c757f4a1db9dd325d60af9e5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:46.912523 26385 consensus_queue.cc:1048] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:46.918376 26190 raft_consensus.cc:515] T b6f3ad902a4f418388c74309b0021cb6 P 15a9f50c757f4a1db9dd325d60af9e5a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:46.921059 26266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f3ad902a4f418388c74309b0021cb6" candidate_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "791483bf50164b81bc3aafff482349b3"
I20260709 13:09:46.921609 26266 raft_consensus.cc:3055] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:46.921918 26266 raft_consensus.cc:740] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 791483bf50164b81bc3aafff482349b3, State: Running, Role: LEADER
I20260709 13:09:46.921941 26190 leader_election.cc:290] T b6f3ad902a4f418388c74309b0021cb6 P 15a9f50c757f4a1db9dd325d60af9e5a [CANDIDATE]: Term 2 election: Requested vote from peers 791483bf50164b81bc3aafff482349b3 (127.14.103.2:34761), 140554b577e8402e9e8f538c126ffeaf (127.14.103.3:46569)
I20260709 13:09:46.922464 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f3ad902a4f418388c74309b0021cb6" candidate_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "140554b577e8402e9e8f538c126ffeaf"
I20260709 13:09:46.923054 26339 raft_consensus.cc:3060] T b6f3ad902a4f418388c74309b0021cb6 P 140554b577e8402e9e8f538c126ffeaf [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:46.922833 26266 consensus_queue.cc:260] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:46.924204 26266 raft_consensus.cc:3060] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:46.930780 26266 raft_consensus.cc:2468] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15a9f50c757f4a1db9dd325d60af9e5a in term 2.
I20260709 13:09:46.930727 26339 raft_consensus.cc:2468] T b6f3ad902a4f418388c74309b0021cb6 P 140554b577e8402e9e8f538c126ffeaf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15a9f50c757f4a1db9dd325d60af9e5a in term 2.
I20260709 13:09:46.932111 26155 leader_election.cc:304] T b6f3ad902a4f418388c74309b0021cb6 P 15a9f50c757f4a1db9dd325d60af9e5a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 140554b577e8402e9e8f538c126ffeaf, 15a9f50c757f4a1db9dd325d60af9e5a; no voters: 
I20260709 13:09:46.932804 26389 raft_consensus.cc:2804] T b6f3ad902a4f418388c74309b0021cb6 P 15a9f50c757f4a1db9dd325d60af9e5a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:46.933274 26389 raft_consensus.cc:697] T b6f3ad902a4f418388c74309b0021cb6 P 15a9f50c757f4a1db9dd325d60af9e5a [term 2 LEADER]: Becoming Leader. State: Replica: 15a9f50c757f4a1db9dd325d60af9e5a, State: Running, Role: LEADER
I20260709 13:09:46.934053 26389 consensus_queue.cc:237] T b6f3ad902a4f418388c74309b0021cb6 P 15a9f50c757f4a1db9dd325d60af9e5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:46.940454 26087 catalog_manager.cc:5697] T b6f3ad902a4f418388c74309b0021cb6 P 15a9f50c757f4a1db9dd325d60af9e5a reported cstate change: term changed from 1 to 2, leader changed from 791483bf50164b81bc3aafff482349b3 (127.14.103.2) to 15a9f50c757f4a1db9dd325d60af9e5a (127.14.103.1). New cstate: current_term: 2 leader_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:46.992616 26410 raft_consensus.cc:993] T 99c3d927a4974e28a84534996d38c853 P 791483bf50164b81bc3aafff482349b3: : Instructing follower 15a9f50c757f4a1db9dd325d60af9e5a to start an election
I20260709 13:09:46.993084 26385 raft_consensus.cc:1081] T 99c3d927a4974e28a84534996d38c853 P 791483bf50164b81bc3aafff482349b3 [term 1 LEADER]: Signalling peer 15a9f50c757f4a1db9dd325d60af9e5a to start an election
I20260709 13:09:46.994488 26190 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "99c3d927a4974e28a84534996d38c853"
dest_uuid: "15a9f50c757f4a1db9dd325d60af9e5a"
 from {username='slave'} at 127.0.0.1:38038
I20260709 13:09:46.994941 26190 raft_consensus.cc:493] T 99c3d927a4974e28a84534996d38c853 P 15a9f50c757f4a1db9dd325d60af9e5a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:46.995252 26190 raft_consensus.cc:3060] T 99c3d927a4974e28a84534996d38c853 P 15a9f50c757f4a1db9dd325d60af9e5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:46.999184 26389 consensus_queue.cc:1048] T f865408f7c5a4a2cbf1e74dea076c516 P 15a9f50c757f4a1db9dd325d60af9e5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:47.001863 26190 raft_consensus.cc:515] T 99c3d927a4974e28a84534996d38c853 P 15a9f50c757f4a1db9dd325d60af9e5a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:47.004834 26265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c3d927a4974e28a84534996d38c853" candidate_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "791483bf50164b81bc3aafff482349b3"
I20260709 13:09:47.005117 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c3d927a4974e28a84534996d38c853" candidate_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "140554b577e8402e9e8f538c126ffeaf"
I20260709 13:09:47.005223 26190 leader_election.cc:290] T 99c3d927a4974e28a84534996d38c853 P 15a9f50c757f4a1db9dd325d60af9e5a [CANDIDATE]: Term 2 election: Requested vote from peers 791483bf50164b81bc3aafff482349b3 (127.14.103.2:34761), 140554b577e8402e9e8f538c126ffeaf (127.14.103.3:46569)
I20260709 13:09:47.005462 26265 raft_consensus.cc:3055] T 99c3d927a4974e28a84534996d38c853 P 791483bf50164b81bc3aafff482349b3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:47.005749 26339 raft_consensus.cc:3060] T 99c3d927a4974e28a84534996d38c853 P 140554b577e8402e9e8f538c126ffeaf [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:47.005904 26265 raft_consensus.cc:740] T 99c3d927a4974e28a84534996d38c853 P 791483bf50164b81bc3aafff482349b3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 791483bf50164b81bc3aafff482349b3, State: Running, Role: LEADER
I20260709 13:09:47.006768 26265 consensus_queue.cc:260] T 99c3d927a4974e28a84534996d38c853 P 791483bf50164b81bc3aafff482349b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:47.008020 26265 raft_consensus.cc:3060] T 99c3d927a4974e28a84534996d38c853 P 791483bf50164b81bc3aafff482349b3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:47.012959 26339 raft_consensus.cc:2468] T 99c3d927a4974e28a84534996d38c853 P 140554b577e8402e9e8f538c126ffeaf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15a9f50c757f4a1db9dd325d60af9e5a in term 2.
I20260709 13:09:47.014704 26420 consensus_queue.cc:1048] T f865408f7c5a4a2cbf1e74dea076c516 P 15a9f50c757f4a1db9dd325d60af9e5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:47.014729 26155 leader_election.cc:304] T 99c3d927a4974e28a84534996d38c853 P 15a9f50c757f4a1db9dd325d60af9e5a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 140554b577e8402e9e8f538c126ffeaf, 15a9f50c757f4a1db9dd325d60af9e5a; no voters: 
I20260709 13:09:47.015751 26265 raft_consensus.cc:2468] T 99c3d927a4974e28a84534996d38c853 P 791483bf50164b81bc3aafff482349b3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15a9f50c757f4a1db9dd325d60af9e5a in term 2.
I20260709 13:09:47.016074 26420 raft_consensus.cc:2804] T 99c3d927a4974e28a84534996d38c853 P 15a9f50c757f4a1db9dd325d60af9e5a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:47.016566 26420 raft_consensus.cc:697] T 99c3d927a4974e28a84534996d38c853 P 15a9f50c757f4a1db9dd325d60af9e5a [term 2 LEADER]: Becoming Leader. State: Replica: 15a9f50c757f4a1db9dd325d60af9e5a, State: Running, Role: LEADER
I20260709 13:09:47.017354 26420 consensus_queue.cc:237] T 99c3d927a4974e28a84534996d38c853 P 15a9f50c757f4a1db9dd325d60af9e5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:47.029711 26088 catalog_manager.cc:5697] T 99c3d927a4974e28a84534996d38c853 P 15a9f50c757f4a1db9dd325d60af9e5a reported cstate change: term changed from 1 to 2, leader changed from 791483bf50164b81bc3aafff482349b3 (127.14.103.2) to 15a9f50c757f4a1db9dd325d60af9e5a (127.14.103.1). New cstate: current_term: 2 leader_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:47.047849 26410 raft_consensus.cc:993] T 1ad0179ad6ba487a8044f56744eb86db P 791483bf50164b81bc3aafff482349b3: : Instructing follower 140554b577e8402e9e8f538c126ffeaf to start an election
I20260709 13:09:47.048262 26385 raft_consensus.cc:1081] T 1ad0179ad6ba487a8044f56744eb86db P 791483bf50164b81bc3aafff482349b3 [term 1 LEADER]: Signalling peer 140554b577e8402e9e8f538c126ffeaf to start an election
I20260709 13:09:47.049500 26339 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1ad0179ad6ba487a8044f56744eb86db"
dest_uuid: "140554b577e8402e9e8f538c126ffeaf"
 from {username='slave'} at 127.0.0.1:37840
I20260709 13:09:47.049889 26339 raft_consensus.cc:493] T 1ad0179ad6ba487a8044f56744eb86db P 140554b577e8402e9e8f538c126ffeaf [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:47.050132 26339 raft_consensus.cc:3060] T 1ad0179ad6ba487a8044f56744eb86db P 140554b577e8402e9e8f538c126ffeaf [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:47.055215 26339 raft_consensus.cc:515] T 1ad0179ad6ba487a8044f56744eb86db P 140554b577e8402e9e8f538c126ffeaf [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:47.057192 26339 leader_election.cc:290] T 1ad0179ad6ba487a8044f56744eb86db P 140554b577e8402e9e8f538c126ffeaf [CANDIDATE]: Term 2 election: Requested vote from peers 15a9f50c757f4a1db9dd325d60af9e5a (127.14.103.1:36999), 791483bf50164b81bc3aafff482349b3 (127.14.103.2:34761)
I20260709 13:09:47.068737 26266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ad0179ad6ba487a8044f56744eb86db" candidate_uuid: "140554b577e8402e9e8f538c126ffeaf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "791483bf50164b81bc3aafff482349b3"
I20260709 13:09:47.069346 26266 raft_consensus.cc:3055] T 1ad0179ad6ba487a8044f56744eb86db P 791483bf50164b81bc3aafff482349b3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:47.069430 26190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ad0179ad6ba487a8044f56744eb86db" candidate_uuid: "140554b577e8402e9e8f538c126ffeaf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "15a9f50c757f4a1db9dd325d60af9e5a"
I20260709 13:09:47.069602 26266 raft_consensus.cc:740] T 1ad0179ad6ba487a8044f56744eb86db P 791483bf50164b81bc3aafff482349b3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 791483bf50164b81bc3aafff482349b3, State: Running, Role: LEADER
I20260709 13:09:47.070139 26190 raft_consensus.cc:3060] T 1ad0179ad6ba487a8044f56744eb86db P 15a9f50c757f4a1db9dd325d60af9e5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:47.070668 26266 consensus_queue.cc:260] T 1ad0179ad6ba487a8044f56744eb86db P 791483bf50164b81bc3aafff482349b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:47.071630 26266 raft_consensus.cc:3060] T 1ad0179ad6ba487a8044f56744eb86db P 791483bf50164b81bc3aafff482349b3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:47.075033 26190 raft_consensus.cc:2468] T 1ad0179ad6ba487a8044f56744eb86db P 15a9f50c757f4a1db9dd325d60af9e5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 140554b577e8402e9e8f538c126ffeaf in term 2.
I20260709 13:09:47.076169 26303 leader_election.cc:304] T 1ad0179ad6ba487a8044f56744eb86db P 140554b577e8402e9e8f538c126ffeaf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 140554b577e8402e9e8f538c126ffeaf, 15a9f50c757f4a1db9dd325d60af9e5a; no voters: 
I20260709 13:09:47.076296 26266 raft_consensus.cc:2468] T 1ad0179ad6ba487a8044f56744eb86db P 791483bf50164b81bc3aafff482349b3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 140554b577e8402e9e8f538c126ffeaf in term 2.
I20260709 13:09:47.077073 26387 raft_consensus.cc:2804] T 1ad0179ad6ba487a8044f56744eb86db P 140554b577e8402e9e8f538c126ffeaf [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:47.078308 26387 raft_consensus.cc:697] T 1ad0179ad6ba487a8044f56744eb86db P 140554b577e8402e9e8f538c126ffeaf [term 2 LEADER]: Becoming Leader. State: Replica: 140554b577e8402e9e8f538c126ffeaf, State: Running, Role: LEADER
I20260709 13:09:47.079018 26387 consensus_queue.cc:237] T 1ad0179ad6ba487a8044f56744eb86db P 140554b577e8402e9e8f538c126ffeaf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:47.085739 26087 catalog_manager.cc:5697] T 1ad0179ad6ba487a8044f56744eb86db P 140554b577e8402e9e8f538c126ffeaf reported cstate change: term changed from 1 to 2, leader changed from 791483bf50164b81bc3aafff482349b3 (127.14.103.2) to 140554b577e8402e9e8f538c126ffeaf (127.14.103.3). New cstate: current_term: 2 leader_uuid: "140554b577e8402e9e8f538c126ffeaf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:47.320740 14748 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 13:09:47.322252 14748 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 13:09:47.322625 14748 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 13:09:47.333760 26190 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b6f3ad902a4f418388c74309b0021cb6"
dest_uuid: "15a9f50c757f4a1db9dd325d60af9e5a"
mode: GRACEFUL
new_leader_uuid: "791483bf50164b81bc3aafff482349b3"
 from {username='slave'} at 127.0.0.1:60228
I20260709 13:09:47.334276 26190 raft_consensus.cc:606] T b6f3ad902a4f418388c74309b0021cb6 P 15a9f50c757f4a1db9dd325d60af9e5a [term 2 LEADER]: Received request to transfer leadership to TS 791483bf50164b81bc3aafff482349b3
I20260709 13:09:47.335340 14748 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 13:09:47.363885 26339 raft_consensus.cc:1275] T b6f3ad902a4f418388c74309b0021cb6 P 140554b577e8402e9e8f538c126ffeaf [term 2 FOLLOWER]: Refusing update from remote peer 15a9f50c757f4a1db9dd325d60af9e5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:47.365103 26389 consensus_queue.cc:1048] T b6f3ad902a4f418388c74309b0021cb6 P 15a9f50c757f4a1db9dd325d60af9e5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:47.372653 26266 raft_consensus.cc:1275] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 [term 2 FOLLOWER]: Refusing update from remote peer 15a9f50c757f4a1db9dd325d60af9e5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:47.374162 26389 consensus_queue.cc:1048] T b6f3ad902a4f418388c74309b0021cb6 P 15a9f50c757f4a1db9dd325d60af9e5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:47.380987 26426 raft_consensus.cc:993] T b6f3ad902a4f418388c74309b0021cb6 P 15a9f50c757f4a1db9dd325d60af9e5a: : Instructing follower 791483bf50164b81bc3aafff482349b3 to start an election
I20260709 13:09:47.381414 26426 raft_consensus.cc:1081] T b6f3ad902a4f418388c74309b0021cb6 P 15a9f50c757f4a1db9dd325d60af9e5a [term 2 LEADER]: Signalling peer 791483bf50164b81bc3aafff482349b3 to start an election
I20260709 13:09:47.383391 26266 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b6f3ad902a4f418388c74309b0021cb6"
dest_uuid: "791483bf50164b81bc3aafff482349b3"
 from {username='slave'} at 127.0.0.1:37564
I20260709 13:09:47.383972 26266 raft_consensus.cc:493] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:47.384332 26266 raft_consensus.cc:3060] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:47.389420 26266 raft_consensus.cc:515] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:47.391716 26266 leader_election.cc:290] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 [CANDIDATE]: Term 3 election: Requested vote from peers 15a9f50c757f4a1db9dd325d60af9e5a (127.14.103.1:36999), 140554b577e8402e9e8f538c126ffeaf (127.14.103.3:46569)
I20260709 13:09:47.391948 26190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f3ad902a4f418388c74309b0021cb6" candidate_uuid: "791483bf50164b81bc3aafff482349b3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "15a9f50c757f4a1db9dd325d60af9e5a"
I20260709 13:09:47.392313 26339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f3ad902a4f418388c74309b0021cb6" candidate_uuid: "791483bf50164b81bc3aafff482349b3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "140554b577e8402e9e8f538c126ffeaf"
I20260709 13:09:47.392616 26190 raft_consensus.cc:3055] T b6f3ad902a4f418388c74309b0021cb6 P 15a9f50c757f4a1db9dd325d60af9e5a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:47.392887 26339 raft_consensus.cc:3060] T b6f3ad902a4f418388c74309b0021cb6 P 140554b577e8402e9e8f538c126ffeaf [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:47.392982 26190 raft_consensus.cc:740] T b6f3ad902a4f418388c74309b0021cb6 P 15a9f50c757f4a1db9dd325d60af9e5a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 15a9f50c757f4a1db9dd325d60af9e5a, State: Running, Role: LEADER
I20260709 13:09:47.393921 26190 consensus_queue.cc:260] T b6f3ad902a4f418388c74309b0021cb6 P 15a9f50c757f4a1db9dd325d60af9e5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:47.395526 26190 raft_consensus.cc:3060] T b6f3ad902a4f418388c74309b0021cb6 P 15a9f50c757f4a1db9dd325d60af9e5a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:47.399933 26339 raft_consensus.cc:2468] T b6f3ad902a4f418388c74309b0021cb6 P 140554b577e8402e9e8f538c126ffeaf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 791483bf50164b81bc3aafff482349b3 in term 3.
I20260709 13:09:47.401067 26230 leader_election.cc:304] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 140554b577e8402e9e8f538c126ffeaf, 791483bf50164b81bc3aafff482349b3; no voters: 
I20260709 13:09:47.401798 26385 raft_consensus.cc:2804] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:47.402266 26385 raft_consensus.cc:697] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 [term 3 LEADER]: Becoming Leader. State: Replica: 791483bf50164b81bc3aafff482349b3, State: Running, Role: LEADER
I20260709 13:09:47.402621 26190 raft_consensus.cc:2468] T b6f3ad902a4f418388c74309b0021cb6 P 15a9f50c757f4a1db9dd325d60af9e5a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 791483bf50164b81bc3aafff482349b3 in term 3.
I20260709 13:09:47.403100 26385 consensus_queue.cc:237] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } }
I20260709 13:09:47.409332 26087 catalog_manager.cc:5697] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 reported cstate change: term changed from 2 to 3, leader changed from 15a9f50c757f4a1db9dd325d60af9e5a (127.14.103.1) to 791483bf50164b81bc3aafff482349b3 (127.14.103.2). New cstate: current_term: 3 leader_uuid: "791483bf50164b81bc3aafff482349b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a9f50c757f4a1db9dd325d60af9e5a" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 36999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:47.424100 26266 raft_consensus.cc:1275] T 99c3d927a4974e28a84534996d38c853 P 791483bf50164b81bc3aafff482349b3 [term 2 FOLLOWER]: Refusing update from remote peer 15a9f50c757f4a1db9dd325d60af9e5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:47.425550 26389 consensus_queue.cc:1048] T 99c3d927a4974e28a84534996d38c853 P 15a9f50c757f4a1db9dd325d60af9e5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:47.433251 26339 raft_consensus.cc:1275] T 99c3d927a4974e28a84534996d38c853 P 140554b577e8402e9e8f538c126ffeaf [term 2 FOLLOWER]: Refusing update from remote peer 15a9f50c757f4a1db9dd325d60af9e5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:47.434813 26426 consensus_queue.cc:1048] T 99c3d927a4974e28a84534996d38c853 P 15a9f50c757f4a1db9dd325d60af9e5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "140554b577e8402e9e8f538c126ffeaf" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 46569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:47.438383 14748 tablet_server.cc:179] TabletServer@127.14.103.1:0 shutting down...
I20260709 13:09:47.467288 14748 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:09:47.467933 14748 tablet_replica.cc:333] T f865408f7c5a4a2cbf1e74dea076c516 P 15a9f50c757f4a1db9dd325d60af9e5a: stopping tablet replica
I20260709 13:09:47.468477 14748 raft_consensus.cc:2243] T f865408f7c5a4a2cbf1e74dea076c516 P 15a9f50c757f4a1db9dd325d60af9e5a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:47.469264 14748 raft_consensus.cc:2272] T f865408f7c5a4a2cbf1e74dea076c516 P 15a9f50c757f4a1db9dd325d60af9e5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:47.471660 14748 tablet_replica.cc:333] T b6f3ad902a4f418388c74309b0021cb6 P 15a9f50c757f4a1db9dd325d60af9e5a: stopping tablet replica
I20260709 13:09:47.472216 14748 raft_consensus.cc:2243] T b6f3ad902a4f418388c74309b0021cb6 P 15a9f50c757f4a1db9dd325d60af9e5a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:47.472599 14748 raft_consensus.cc:2272] T b6f3ad902a4f418388c74309b0021cb6 P 15a9f50c757f4a1db9dd325d60af9e5a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:47.474604 14748 tablet_replica.cc:333] T 99c3d927a4974e28a84534996d38c853 P 15a9f50c757f4a1db9dd325d60af9e5a: stopping tablet replica
I20260709 13:09:47.475103 14748 raft_consensus.cc:2243] T 99c3d927a4974e28a84534996d38c853 P 15a9f50c757f4a1db9dd325d60af9e5a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:47.475983 14748 raft_consensus.cc:2272] T 99c3d927a4974e28a84534996d38c853 P 15a9f50c757f4a1db9dd325d60af9e5a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:47.477989 14748 tablet_replica.cc:333] T 1ad0179ad6ba487a8044f56744eb86db P 15a9f50c757f4a1db9dd325d60af9e5a: stopping tablet replica
I20260709 13:09:47.478448 14748 raft_consensus.cc:2243] T 1ad0179ad6ba487a8044f56744eb86db P 15a9f50c757f4a1db9dd325d60af9e5a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:47.478821 14748 raft_consensus.cc:2272] T 1ad0179ad6ba487a8044f56744eb86db P 15a9f50c757f4a1db9dd325d60af9e5a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:47.500324 14748 tablet_server.cc:196] TabletServer@127.14.103.1:0 shutdown complete.
I20260709 13:09:47.511757 26266 raft_consensus.cc:1275] T 1ad0179ad6ba487a8044f56744eb86db P 791483bf50164b81bc3aafff482349b3 [term 2 FOLLOWER]: Refusing update from remote peer 140554b577e8402e9e8f538c126ffeaf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:47.512040 14748 tablet_server.cc:179] TabletServer@127.14.103.2:0 shutting down...
I20260709 13:09:47.513558 26387 consensus_queue.cc:1048] T 1ad0179ad6ba487a8044f56744eb86db P 140554b577e8402e9e8f538c126ffeaf [LEADER]: Connected to new peer: Peer: permanent_uuid: "791483bf50164b81bc3aafff482349b3" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:09:47.519089 26306 consensus_peers.cc:597] T 1ad0179ad6ba487a8044f56744eb86db P 140554b577e8402e9e8f538c126ffeaf -> Peer 791483bf50164b81bc3aafff482349b3 (127.14.103.2:34761): Couldn't send request to peer 791483bf50164b81bc3aafff482349b3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:47.532233 14748 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:09:47.532888 14748 tablet_replica.cc:333] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3: stopping tablet replica
I20260709 13:09:47.533417 14748 raft_consensus.cc:2243] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:09:47.534044 14748 pending_rounds.cc:70] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3: Trying to abort 1 pending ops.
I20260709 13:09:47.534245 14748 pending_rounds.cc:77] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305636198012612608 op_type: NO_OP noop_request { }
I20260709 13:09:47.534536 14748 raft_consensus.cc:2889] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:09:47.534837 14748 raft_consensus.cc:2272] T b6f3ad902a4f418388c74309b0021cb6 P 791483bf50164b81bc3aafff482349b3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:47.537456 14748 tablet_replica.cc:333] T f865408f7c5a4a2cbf1e74dea076c516 P 791483bf50164b81bc3aafff482349b3: stopping tablet replica
I20260709 13:09:47.538137 14748 raft_consensus.cc:2243] T f865408f7c5a4a2cbf1e74dea076c516 P 791483bf50164b81bc3aafff482349b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:47.538609 14748 raft_consensus.cc:2272] T f865408f7c5a4a2cbf1e74dea076c516 P 791483bf50164b81bc3aafff482349b3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:47.539044 26303 consensus_peers.cc:597] T 1ad0179ad6ba487a8044f56744eb86db P 140554b577e8402e9e8f538c126ffeaf -> Peer 15a9f50c757f4a1db9dd325d60af9e5a (127.14.103.1:36999): Couldn't send request to peer 15a9f50c757f4a1db9dd325d60af9e5a. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:36999: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:47.540777 14748 tablet_replica.cc:333] T 1ad0179ad6ba487a8044f56744eb86db P 791483bf50164b81bc3aafff482349b3: stopping tablet replica
I20260709 13:09:47.541280 14748 raft_consensus.cc:2243] T 1ad0179ad6ba487a8044f56744eb86db P 791483bf50164b81bc3aafff482349b3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:47.541591 14748 raft_consensus.cc:2272] T 1ad0179ad6ba487a8044f56744eb86db P 791483bf50164b81bc3aafff482349b3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:47.543212 14748 tablet_replica.cc:333] T 99c3d927a4974e28a84534996d38c853 P 791483bf50164b81bc3aafff482349b3: stopping tablet replica
I20260709 13:09:47.543617 14748 raft_consensus.cc:2243] T 99c3d927a4974e28a84534996d38c853 P 791483bf50164b81bc3aafff482349b3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:47.544027 14748 raft_consensus.cc:2272] T 99c3d927a4974e28a84534996d38c853 P 791483bf50164b81bc3aafff482349b3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:47.565845 14748 tablet_server.cc:196] TabletServer@127.14.103.2:0 shutdown complete.
I20260709 13:09:47.577442 14748 tablet_server.cc:179] TabletServer@127.14.103.3:0 shutting down...
I20260709 13:09:47.599318 14748 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:09:47.600201 14748 tablet_replica.cc:333] T f865408f7c5a4a2cbf1e74dea076c516 P 140554b577e8402e9e8f538c126ffeaf: stopping tablet replica
I20260709 13:09:47.601025 14748 raft_consensus.cc:2243] T f865408f7c5a4a2cbf1e74dea076c516 P 140554b577e8402e9e8f538c126ffeaf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:47.601508 14748 raft_consensus.cc:2272] T f865408f7c5a4a2cbf1e74dea076c516 P 140554b577e8402e9e8f538c126ffeaf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:47.603590 14748 tablet_replica.cc:333] T b6f3ad902a4f418388c74309b0021cb6 P 140554b577e8402e9e8f538c126ffeaf: stopping tablet replica
I20260709 13:09:47.604143 14748 raft_consensus.cc:2243] T b6f3ad902a4f418388c74309b0021cb6 P 140554b577e8402e9e8f538c126ffeaf [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:47.604563 14748 raft_consensus.cc:2272] T b6f3ad902a4f418388c74309b0021cb6 P 140554b577e8402e9e8f538c126ffeaf [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:47.606606 14748 tablet_replica.cc:333] T 99c3d927a4974e28a84534996d38c853 P 140554b577e8402e9e8f538c126ffeaf: stopping tablet replica
I20260709 13:09:47.607088 14748 raft_consensus.cc:2243] T 99c3d927a4974e28a84534996d38c853 P 140554b577e8402e9e8f538c126ffeaf [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:47.607483 14748 raft_consensus.cc:2272] T 99c3d927a4974e28a84534996d38c853 P 140554b577e8402e9e8f538c126ffeaf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:47.609460 14748 tablet_replica.cc:333] T 1ad0179ad6ba487a8044f56744eb86db P 140554b577e8402e9e8f538c126ffeaf: stopping tablet replica
I20260709 13:09:47.609911 14748 raft_consensus.cc:2243] T 1ad0179ad6ba487a8044f56744eb86db P 140554b577e8402e9e8f538c126ffeaf [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:47.610617 14748 pending_rounds.cc:70] T 1ad0179ad6ba487a8044f56744eb86db P 140554b577e8402e9e8f538c126ffeaf: Trying to abort 1 pending ops.
I20260709 13:09:47.610816 14748 pending_rounds.cc:77] T 1ad0179ad6ba487a8044f56744eb86db P 140554b577e8402e9e8f538c126ffeaf: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305636196684222464 op_type: NO_OP noop_request { }
I20260709 13:09:47.611080 14748 raft_consensus.cc:2889] T 1ad0179ad6ba487a8044f56744eb86db P 140554b577e8402e9e8f538c126ffeaf [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:09:47.611337 14748 raft_consensus.cc:2272] T 1ad0179ad6ba487a8044f56744eb86db P 140554b577e8402e9e8f538c126ffeaf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:47.633885 14748 tablet_server.cc:196] TabletServer@127.14.103.3:0 shutdown complete.
I20260709 13:09:47.645174 14748 master.cc:562] Master@127.14.103.62:38415 shutting down...
I20260709 13:09:47.659883 14748 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 634899637e364ac4944da5a00a221d3f [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:47.660414 14748 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 634899637e364ac4944da5a00a221d3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:47.660725 14748 tablet_replica.cc:333] T 00000000000000000000000000000000 P 634899637e364ac4944da5a00a221d3f: stopping tablet replica
I20260709 13:09:47.678256 14748 master.cc:584] Master@127.14.103.62:38415 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (2552 ms)
[----------] 6 tests from LeaderRebalancerTest (340006 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 13:09:47.703949 14748 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.103.62:39907
I20260709 13:09:47.704974 14748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:47.709635 26434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:47.709863 26435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:47.711289 26437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:47.712019 14748 server_base.cc:1061] running on GCE node
I20260709 13:09:47.712822 14748 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:47.713028 14748 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:47.713197 14748 hybrid_clock.cc:648] HybridClock initialized: now 1783602587713180 us; error 0 us; skew 500 ppm
I20260709 13:09:47.713734 14748 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:47.716056 14748 webserver.cc:533] Webserver started at http://127.14.103.62:45139/ using document root <none> and password file <none>
I20260709 13:09:47.716528 14748 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:47.716724 14748 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:47.716985 14748 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:47.718035 14748 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602247531521-14748-0/minicluster-data/master-0-root/instance:
uuid: "e68faef5ba744349a655384e3cc32fd8"
format_stamp: "Formatted at 2026-07-09 13:09:47 on dist-test-slave-fpp7"
I20260709 13:09:47.722122 14748 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.003s
I20260709 13:09:47.725245 26442 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:47.725894 14748 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:09:47.726167 14748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602247531521-14748-0/minicluster-data/master-0-root
uuid: "e68faef5ba744349a655384e3cc32fd8"
format_stamp: "Formatted at 2026-07-09 13:09:47 on dist-test-slave-fpp7"
I20260709 13:09:47.726423 14748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602247531521-14748-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602247531521-14748-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602247531521-14748-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:47.751165 14748 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:47.752158 14748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:47.785766 14748 rpc_server.cc:307] RPC server started. Bound to: 127.14.103.62:39907
I20260709 13:09:47.785879 26493 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.103.62:39907 every 8 connection(s)
I20260709 13:09:47.789371 26494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:47.799249 26494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e68faef5ba744349a655384e3cc32fd8: Bootstrap starting.
I20260709 13:09:47.803396 26494 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e68faef5ba744349a655384e3cc32fd8: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:47.806939 26494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e68faef5ba744349a655384e3cc32fd8: No bootstrap required, opened a new log
I20260709 13:09:47.808830 26494 raft_consensus.cc:359] T 00000000000000000000000000000000 P e68faef5ba744349a655384e3cc32fd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e68faef5ba744349a655384e3cc32fd8" member_type: VOTER }
I20260709 13:09:47.809165 26494 raft_consensus.cc:385] T 00000000000000000000000000000000 P e68faef5ba744349a655384e3cc32fd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:47.809335 26494 raft_consensus.cc:740] T 00000000000000000000000000000000 P e68faef5ba744349a655384e3cc32fd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e68faef5ba744349a655384e3cc32fd8, State: Initialized, Role: FOLLOWER
I20260709 13:09:47.809815 26494 consensus_queue.cc:260] T 00000000000000000000000000000000 P e68faef5ba744349a655384e3cc32fd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e68faef5ba744349a655384e3cc32fd8" member_type: VOTER }
I20260709 13:09:47.810192 26494 raft_consensus.cc:399] T 00000000000000000000000000000000 P e68faef5ba744349a655384e3cc32fd8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:09:47.810374 26494 raft_consensus.cc:493] T 00000000000000000000000000000000 P e68faef5ba744349a655384e3cc32fd8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:09:47.810580 26494 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e68faef5ba744349a655384e3cc32fd8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:47.814996 26494 raft_consensus.cc:515] T 00000000000000000000000000000000 P e68faef5ba744349a655384e3cc32fd8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e68faef5ba744349a655384e3cc32fd8" member_type: VOTER }
I20260709 13:09:47.815418 26494 leader_election.cc:304] T 00000000000000000000000000000000 P e68faef5ba744349a655384e3cc32fd8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e68faef5ba744349a655384e3cc32fd8; no voters: 
I20260709 13:09:47.816505 26494 leader_election.cc:290] T 00000000000000000000000000000000 P e68faef5ba744349a655384e3cc32fd8 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:09:47.816833 26497 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e68faef5ba744349a655384e3cc32fd8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:47.818287 26497 raft_consensus.cc:697] T 00000000000000000000000000000000 P e68faef5ba744349a655384e3cc32fd8 [term 1 LEADER]: Becoming Leader. State: Replica: e68faef5ba744349a655384e3cc32fd8, State: Running, Role: LEADER
I20260709 13:09:47.818885 26497 consensus_queue.cc:237] T 00000000000000000000000000000000 P e68faef5ba744349a655384e3cc32fd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e68faef5ba744349a655384e3cc32fd8" member_type: VOTER }
I20260709 13:09:47.819311 26494 sys_catalog.cc:565] T 00000000000000000000000000000000 P e68faef5ba744349a655384e3cc32fd8 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:09:47.821842 26499 sys_catalog.cc:455] T 00000000000000000000000000000000 P e68faef5ba744349a655384e3cc32fd8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e68faef5ba744349a655384e3cc32fd8. Latest consensus state: current_term: 1 leader_uuid: "e68faef5ba744349a655384e3cc32fd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e68faef5ba744349a655384e3cc32fd8" member_type: VOTER } }
I20260709 13:09:47.821859 26498 sys_catalog.cc:455] T 00000000000000000000000000000000 P e68faef5ba744349a655384e3cc32fd8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e68faef5ba744349a655384e3cc32fd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e68faef5ba744349a655384e3cc32fd8" member_type: VOTER } }
I20260709 13:09:47.822636 26499 sys_catalog.cc:458] T 00000000000000000000000000000000 P e68faef5ba744349a655384e3cc32fd8 [sys.catalog]: This master's current role is: LEADER
I20260709 13:09:47.822728 26498 sys_catalog.cc:458] T 00000000000000000000000000000000 P e68faef5ba744349a655384e3cc32fd8 [sys.catalog]: This master's current role is: LEADER
I20260709 13:09:47.827761 26502 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:09:47.835620 26502 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:09:47.837050 14748 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:09:47.844547 26502 catalog_manager.cc:1383] Generated new cluster ID: ad27b0d6f38349e68629199b027d169a
I20260709 13:09:47.844826 26502 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:09:47.863658 26502 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:09:47.864902 26502 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:09:47.874778 26502 catalog_manager.cc:6070] T 00000000000000000000000000000000 P e68faef5ba744349a655384e3cc32fd8: Generated new TSK 0
I20260709 13:09:47.875321 26502 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:09:47.903848 14748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:47.910044 26515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:47.911047 26516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:47.913908 26518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:47.915107 14748 server_base.cc:1061] running on GCE node
I20260709 13:09:47.915957 14748 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:47.916141 14748 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:47.916313 14748 hybrid_clock.cc:648] HybridClock initialized: now 1783602587916301 us; error 0 us; skew 500 ppm
I20260709 13:09:47.916780 14748 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:47.918922 14748 webserver.cc:533] Webserver started at http://127.14.103.1:35977/ using document root <none> and password file <none>
I20260709 13:09:47.919361 14748 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:47.919514 14748 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:47.919767 14748 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:47.920763 14748 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602247531521-14748-0/minicluster-data/ts-0-root/instance:
uuid: "dc9195961d41473e8f590b632eb998a0"
format_stamp: "Formatted at 2026-07-09 13:09:47 on dist-test-slave-fpp7"
I20260709 13:09:47.924979 14748 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:09:47.928136 26523 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:47.928835 14748 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:09:47.929124 14748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602247531521-14748-0/minicluster-data/ts-0-root
uuid: "dc9195961d41473e8f590b632eb998a0"
format_stamp: "Formatted at 2026-07-09 13:09:47 on dist-test-slave-fpp7"
I20260709 13:09:47.929379 14748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602247531521-14748-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602247531521-14748-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602247531521-14748-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:47.942292 14748 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:47.943356 14748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:47.944769 14748 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:09:47.946835 14748 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:09:47.947052 14748 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:47.947279 14748 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:09:47.947439 14748 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:47.985080 14748 rpc_server.cc:307] RPC server started. Bound to: 127.14.103.1:33787
I20260709 13:09:47.985189 26585 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.103.1:33787 every 8 connection(s)
I20260709 13:09:47.990037 14748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:47.996901 26590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:47.997743 26591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:47.999213 26593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:48.000159 14748 server_base.cc:1061] running on GCE node
I20260709 13:09:48.001952 14748 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:09:48.002108 26586 heartbeater.cc:344] Connected to a master server at 127.14.103.62:39907
W20260709 13:09:48.002199 14748 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:48.002617 14748 hybrid_clock.cc:648] HybridClock initialized: now 1783602588002598 us; error 0 us; skew 500 ppm
I20260709 13:09:48.002682 26586 heartbeater.cc:461] Registering TS with master...
I20260709 13:09:48.003216 14748 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:48.003497 26586 heartbeater.cc:507] Master 127.14.103.62:39907 requested a full tablet report, sending...
I20260709 13:09:48.005865 26459 ts_manager.cc:194] Registered new tserver with Master: dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787)
I20260709 13:09:48.005870 14748 webserver.cc:533] Webserver started at http://127.14.103.2:37311/ using document root <none> and password file <none>
I20260709 13:09:48.006512 14748 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:48.006729 14748 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:48.006985 14748 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:48.007776 26459 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55280
I20260709 13:09:48.008091 14748 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602247531521-14748-0/minicluster-data/ts-1-root/instance:
uuid: "68637fffcc4c4969ae0612be35aa58ec"
format_stamp: "Formatted at 2026-07-09 13:09:48 on dist-test-slave-fpp7"
I20260709 13:09:48.012703 14748 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:09:48.015813 26598 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:48.016496 14748 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:09:48.016773 14748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602247531521-14748-0/minicluster-data/ts-1-root
uuid: "68637fffcc4c4969ae0612be35aa58ec"
format_stamp: "Formatted at 2026-07-09 13:09:48 on dist-test-slave-fpp7"
I20260709 13:09:48.017042 14748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602247531521-14748-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602247531521-14748-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602247531521-14748-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:48.034835 14748 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:48.036103 14748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:48.037396 14748 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:09:48.039497 14748 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:09:48.039721 14748 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:48.039947 14748 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:09:48.040110 14748 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:48.076283 14748 rpc_server.cc:307] RPC server started. Bound to: 127.14.103.2:35285
I20260709 13:09:48.076385 26660 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.103.2:35285 every 8 connection(s)
I20260709 13:09:48.080948 14748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:48.088150 26664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:48.089715 26665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:48.093875 26667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:48.094805 26661 heartbeater.cc:344] Connected to a master server at 127.14.103.62:39907
I20260709 13:09:48.095269 26661 heartbeater.cc:461] Registering TS with master...
I20260709 13:09:48.095556 14748 server_base.cc:1061] running on GCE node
I20260709 13:09:48.097095 26661 heartbeater.cc:507] Master 127.14.103.62:39907 requested a full tablet report, sending...
I20260709 13:09:48.097396 14748 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:48.097740 14748 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:48.097949 14748 hybrid_clock.cc:648] HybridClock initialized: now 1783602588097932 us; error 0 us; skew 500 ppm
I20260709 13:09:48.098703 14748 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:48.099659 26459 ts_manager.cc:194] Registered new tserver with Master: 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285)
I20260709 13:09:48.101384 14748 webserver.cc:533] Webserver started at http://127.14.103.3:35685/ using document root <none> and password file <none>
I20260709 13:09:48.101437 26459 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55286
I20260709 13:09:48.102064 14748 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:48.102324 14748 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:48.102689 14748 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:48.103806 14748 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602247531521-14748-0/minicluster-data/ts-2-root/instance:
uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
format_stamp: "Formatted at 2026-07-09 13:09:48 on dist-test-slave-fpp7"
I20260709 13:09:48.107926 14748 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 13:09:48.111020 26672 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:48.111788 14748 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:09:48.112092 14748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602247531521-14748-0/minicluster-data/ts-2-root
uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
format_stamp: "Formatted at 2026-07-09 13:09:48 on dist-test-slave-fpp7"
I20260709 13:09:48.112367 14748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602247531521-14748-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602247531521-14748-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602247531521-14748-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:48.131398 14748 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:48.132447 14748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:48.133757 14748 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:09:48.135881 14748 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:09:48.136093 14748 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:48.136327 14748 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:09:48.136510 14748 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:48.173966 14748 rpc_server.cc:307] RPC server started. Bound to: 127.14.103.3:36339
I20260709 13:09:48.174096 26734 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.103.3:36339 every 8 connection(s)
I20260709 13:09:48.187250 26735 heartbeater.cc:344] Connected to a master server at 127.14.103.62:39907
I20260709 13:09:48.187750 26735 heartbeater.cc:461] Registering TS with master...
I20260709 13:09:48.188413 26735 heartbeater.cc:507] Master 127.14.103.62:39907 requested a full tablet report, sending...
I20260709 13:09:48.190111 26459 ts_manager.cc:194] Registered new tserver with Master: 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:48.191048 14748 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013340254s
I20260709 13:09:48.191447 26459 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55294
I20260709 13:09:48.213593 26459 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55304:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:09:48.216516 26459 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:09:48.286339 26551 tablet_service.cc:1511] Processing CreateTablet for tablet b16a06ac45ab44a186fbf820559732bb (DEFAULT_TABLE table=test-workload [id=e483fb6c700048d0b79b3b35954fa3e0]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:09:48.287779 26551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b16a06ac45ab44a186fbf820559732bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:48.292290 26549 tablet_service.cc:1511] Processing CreateTablet for tablet ef38f875bfdd4b669c39d02f0ea7472b (DEFAULT_TABLE table=test-workload [id=e483fb6c700048d0b79b3b35954fa3e0]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:09:48.292992 26626 tablet_service.cc:1511] Processing CreateTablet for tablet b16a06ac45ab44a186fbf820559732bb (DEFAULT_TABLE table=test-workload [id=e483fb6c700048d0b79b3b35954fa3e0]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:09:48.293505 26548 tablet_service.cc:1511] Processing CreateTablet for tablet 6c21616d95d04253a63e088134753cea (DEFAULT_TABLE table=test-workload [id=e483fb6c700048d0b79b3b35954fa3e0]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:09:48.294335 26626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b16a06ac45ab44a186fbf820559732bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:48.292290 26547 tablet_service.cc:1511] Processing CreateTablet for tablet 579b5cf4a4874b75a67f9cbf320addc2 (DEFAULT_TABLE table=test-workload [id=e483fb6c700048d0b79b3b35954fa3e0]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:09:48.293988 26549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef38f875bfdd4b669c39d02f0ea7472b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:48.295406 26623 tablet_service.cc:1511] Processing CreateTablet for tablet 6c21616d95d04253a63e088134753cea (DEFAULT_TABLE table=test-workload [id=e483fb6c700048d0b79b3b35954fa3e0]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:09:48.295876 26550 tablet_service.cc:1511] Processing CreateTablet for tablet 055a26fb2c8b41b4b6da92c7e20180e1 (DEFAULT_TABLE table=test-workload [id=e483fb6c700048d0b79b3b35954fa3e0]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:09:48.296209 26625 tablet_service.cc:1511] Processing CreateTablet for tablet 055a26fb2c8b41b4b6da92c7e20180e1 (DEFAULT_TABLE table=test-workload [id=e483fb6c700048d0b79b3b35954fa3e0]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:09:48.296768 26623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c21616d95d04253a63e088134753cea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:48.297225 26550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 055a26fb2c8b41b4b6da92c7e20180e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:48.297520 26625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 055a26fb2c8b41b4b6da92c7e20180e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:48.298689 26622 tablet_service.cc:1511] Processing CreateTablet for tablet 579b5cf4a4874b75a67f9cbf320addc2 (DEFAULT_TABLE table=test-workload [id=e483fb6c700048d0b79b3b35954fa3e0]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:09:48.294376 26624 tablet_service.cc:1511] Processing CreateTablet for tablet ef38f875bfdd4b669c39d02f0ea7472b (DEFAULT_TABLE table=test-workload [id=e483fb6c700048d0b79b3b35954fa3e0]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:09:48.299981 26622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 579b5cf4a4874b75a67f9cbf320addc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:48.300074 26547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 579b5cf4a4874b75a67f9cbf320addc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:48.300722 26624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef38f875bfdd4b669c39d02f0ea7472b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:48.302911 26548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c21616d95d04253a63e088134753cea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:48.338387 26546 tablet_service.cc:1511] Processing CreateTablet for tablet b8f3b339b6bd4573a9f7929c4d130af5 (DEFAULT_TABLE table=test-workload [id=e483fb6c700048d0b79b3b35954fa3e0]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:09:48.339998 26546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8f3b339b6bd4573a9f7929c4d130af5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:48.341025 26621 tablet_service.cc:1511] Processing CreateTablet for tablet b8f3b339b6bd4573a9f7929c4d130af5 (DEFAULT_TABLE table=test-workload [id=e483fb6c700048d0b79b3b35954fa3e0]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:09:48.342329 26621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8f3b339b6bd4573a9f7929c4d130af5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:48.354799 26750 tablet_bootstrap.cc:492] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec: Bootstrap starting.
I20260709 13:09:48.363409 26545 tablet_service.cc:1511] Processing CreateTablet for tablet 4af4eae89a594eb0ace69b0895e43372 (DEFAULT_TABLE table=test-workload [id=e483fb6c700048d0b79b3b35954fa3e0]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:09:48.364840 26545 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4af4eae89a594eb0ace69b0895e43372. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:48.372589 26699 tablet_service.cc:1511] Processing CreateTablet for tablet 055a26fb2c8b41b4b6da92c7e20180e1 (DEFAULT_TABLE table=test-workload [id=e483fb6c700048d0b79b3b35954fa3e0]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:09:48.372838 26698 tablet_service.cc:1511] Processing CreateTablet for tablet ef38f875bfdd4b669c39d02f0ea7472b (DEFAULT_TABLE table=test-workload [id=e483fb6c700048d0b79b3b35954fa3e0]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:09:48.373888 26700 tablet_service.cc:1511] Processing CreateTablet for tablet b16a06ac45ab44a186fbf820559732bb (DEFAULT_TABLE table=test-workload [id=e483fb6c700048d0b79b3b35954fa3e0]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:09:48.374622 26697 tablet_service.cc:1511] Processing CreateTablet for tablet 6c21616d95d04253a63e088134753cea (DEFAULT_TABLE table=test-workload [id=e483fb6c700048d0b79b3b35954fa3e0]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:09:48.376405 26696 tablet_service.cc:1511] Processing CreateTablet for tablet 579b5cf4a4874b75a67f9cbf320addc2 (DEFAULT_TABLE table=test-workload [id=e483fb6c700048d0b79b3b35954fa3e0]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:09:48.378109 26695 tablet_service.cc:1511] Processing CreateTablet for tablet b8f3b339b6bd4573a9f7929c4d130af5 (DEFAULT_TABLE table=test-workload [id=e483fb6c700048d0b79b3b35954fa3e0]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:09:48.379928 26626 tablet_service.cc:1511] Processing CreateTablet for tablet 25be33d2f6bf455996cd05210da81458 (DEFAULT_TABLE table=test-workload [id=e483fb6c700048d0b79b3b35954fa3e0]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:09:48.380602 26694 tablet_service.cc:1511] Processing CreateTablet for tablet 4af4eae89a594eb0ace69b0895e43372 (DEFAULT_TABLE table=test-workload [id=e483fb6c700048d0b79b3b35954fa3e0]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:09:48.381402 26626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25be33d2f6bf455996cd05210da81458. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:48.387179 26750 tablet_bootstrap.cc:654] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:48.388768 26699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 055a26fb2c8b41b4b6da92c7e20180e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:48.389389 26698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef38f875bfdd4b669c39d02f0ea7472b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:48.390305 26695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8f3b339b6bd4573a9f7929c4d130af5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:48.390908 26700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b16a06ac45ab44a186fbf820559732bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:48.391530 26694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4af4eae89a594eb0ace69b0895e43372. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:48.393267 26696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 579b5cf4a4874b75a67f9cbf320addc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:48.396461 26550 tablet_service.cc:1511] Processing CreateTablet for tablet 25be33d2f6bf455996cd05210da81458 (DEFAULT_TABLE table=test-workload [id=e483fb6c700048d0b79b3b35954fa3e0]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:09:48.397811 26550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25be33d2f6bf455996cd05210da81458. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:48.406294 26693 tablet_service.cc:1511] Processing CreateTablet for tablet 25be33d2f6bf455996cd05210da81458 (DEFAULT_TABLE table=test-workload [id=e483fb6c700048d0b79b3b35954fa3e0]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:09:48.408797 26697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c21616d95d04253a63e088134753cea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:48.411975 26693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25be33d2f6bf455996cd05210da81458. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:48.422551 26749 tablet_bootstrap.cc:492] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0: Bootstrap starting.
I20260709 13:09:48.429760 26622 tablet_service.cc:1511] Processing CreateTablet for tablet 4af4eae89a594eb0ace69b0895e43372 (DEFAULT_TABLE table=test-workload [id=e483fb6c700048d0b79b3b35954fa3e0]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:09:48.434991 26749 tablet_bootstrap.cc:654] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:48.436861 26750 tablet_bootstrap.cc:492] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec: No bootstrap required, opened a new log
I20260709 13:09:48.439486 26622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4af4eae89a594eb0ace69b0895e43372. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:48.446019 26750 ts_tablet_manager.cc:1403] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec: Time spent bootstrapping tablet: real 0.091s	user 0.011s	sys 0.023s
I20260709 13:09:48.447388 26545 tablet_service.cc:1511] Processing CreateTablet for tablet d8410badcf77497fbb604cf8c185db83 (DEFAULT_TABLE table=test-workload [id=e483fb6c700048d0b79b3b35954fa3e0]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:09:48.447587 26626 tablet_service.cc:1511] Processing CreateTablet for tablet d8410badcf77497fbb604cf8c185db83 (DEFAULT_TABLE table=test-workload [id=e483fb6c700048d0b79b3b35954fa3e0]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:09:48.448673 26545 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8410badcf77497fbb604cf8c185db83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:48.448946 26626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8410badcf77497fbb604cf8c185db83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:48.448959 26750 raft_consensus.cc:359] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.453109 26750 raft_consensus.cc:385] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:48.454130 26750 raft_consensus.cc:740] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Initialized, Role: FOLLOWER
I20260709 13:09:48.455556 26698 tablet_service.cc:1511] Processing CreateTablet for tablet d8410badcf77497fbb604cf8c185db83 (DEFAULT_TABLE table=test-workload [id=e483fb6c700048d0b79b3b35954fa3e0]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:09:48.456236 26750 consensus_queue.cc:260] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.468772 26698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8410badcf77497fbb604cf8c185db83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:48.470896 26749 tablet_bootstrap.cc:492] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0: No bootstrap required, opened a new log
I20260709 13:09:48.471545 26749 ts_tablet_manager.cc:1403] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0: Time spent bootstrapping tablet: real 0.049s	user 0.008s	sys 0.007s
I20260709 13:09:48.474462 26749 raft_consensus.cc:359] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.475136 26749 raft_consensus.cc:385] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:48.475423 26749 raft_consensus.cc:740] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9195961d41473e8f590b632eb998a0, State: Initialized, Role: FOLLOWER
I20260709 13:09:48.476118 26749 consensus_queue.cc:260] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.480029 26661 heartbeater.cc:499] Master 127.14.103.62:39907 was elected leader, sending a full tablet report...
I20260709 13:09:48.480304 26750 ts_tablet_manager.cc:1434] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.002s
I20260709 13:09:48.481251 26750 tablet_bootstrap.cc:492] T b8f3b339b6bd4573a9f7929c4d130af5 P 68637fffcc4c4969ae0612be35aa58ec: Bootstrap starting.
I20260709 13:09:48.492133 26750 tablet_bootstrap.cc:654] T b8f3b339b6bd4573a9f7929c4d130af5 P 68637fffcc4c4969ae0612be35aa58ec: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:48.495029 26752 tablet_bootstrap.cc:492] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78: Bootstrap starting.
I20260709 13:09:48.505522 26749 ts_tablet_manager.cc:1434] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.007s
I20260709 13:09:48.506254 26586 heartbeater.cc:499] Master 127.14.103.62:39907 was elected leader, sending a full tablet report...
I20260709 13:09:48.506551 26749 tablet_bootstrap.cc:492] T ef38f875bfdd4b669c39d02f0ea7472b P dc9195961d41473e8f590b632eb998a0: Bootstrap starting.
I20260709 13:09:48.512171 26749 tablet_bootstrap.cc:654] T ef38f875bfdd4b669c39d02f0ea7472b P dc9195961d41473e8f590b632eb998a0: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:48.516147 26750 tablet_bootstrap.cc:492] T b8f3b339b6bd4573a9f7929c4d130af5 P 68637fffcc4c4969ae0612be35aa58ec: No bootstrap required, opened a new log
I20260709 13:09:48.516811 26750 ts_tablet_manager.cc:1403] T b8f3b339b6bd4573a9f7929c4d130af5 P 68637fffcc4c4969ae0612be35aa58ec: Time spent bootstrapping tablet: real 0.036s	user 0.014s	sys 0.006s
I20260709 13:09:48.520478 26750 raft_consensus.cc:359] T b8f3b339b6bd4573a9f7929c4d130af5 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.521492 26750 raft_consensus.cc:385] T b8f3b339b6bd4573a9f7929c4d130af5 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:48.521863 26750 raft_consensus.cc:740] T b8f3b339b6bd4573a9f7929c4d130af5 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Initialized, Role: FOLLOWER
I20260709 13:09:48.522814 26750 consensus_queue.cc:260] T b8f3b339b6bd4573a9f7929c4d130af5 P 68637fffcc4c4969ae0612be35aa58ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.523897 26752 tablet_bootstrap.cc:654] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:48.526139 26750 ts_tablet_manager.cc:1434] T b8f3b339b6bd4573a9f7929c4d130af5 P 68637fffcc4c4969ae0612be35aa58ec: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:09:48.527575 26750 tablet_bootstrap.cc:492] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec: Bootstrap starting.
I20260709 13:09:48.535285 26750 tablet_bootstrap.cc:654] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:48.537696 26749 tablet_bootstrap.cc:492] T ef38f875bfdd4b669c39d02f0ea7472b P dc9195961d41473e8f590b632eb998a0: No bootstrap required, opened a new log
I20260709 13:09:48.538113 26749 ts_tablet_manager.cc:1403] T ef38f875bfdd4b669c39d02f0ea7472b P dc9195961d41473e8f590b632eb998a0: Time spent bootstrapping tablet: real 0.032s	user 0.012s	sys 0.017s
I20260709 13:09:48.538275 26752 tablet_bootstrap.cc:492] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78: No bootstrap required, opened a new log
I20260709 13:09:48.538918 26752 ts_tablet_manager.cc:1403] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78: Time spent bootstrapping tablet: real 0.044s	user 0.009s	sys 0.003s
I20260709 13:09:48.540351 26749 raft_consensus.cc:359] T ef38f875bfdd4b669c39d02f0ea7472b P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.540946 26749 raft_consensus.cc:385] T ef38f875bfdd4b669c39d02f0ea7472b P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:48.541186 26749 raft_consensus.cc:740] T ef38f875bfdd4b669c39d02f0ea7472b P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9195961d41473e8f590b632eb998a0, State: Initialized, Role: FOLLOWER
I20260709 13:09:48.541841 26749 consensus_queue.cc:260] T ef38f875bfdd4b669c39d02f0ea7472b P dc9195961d41473e8f590b632eb998a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.542598 26752 raft_consensus.cc:359] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.543761 26749 ts_tablet_manager.cc:1434] T ef38f875bfdd4b669c39d02f0ea7472b P dc9195961d41473e8f590b632eb998a0: Time spent starting tablet: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:09:48.543766 26752 raft_consensus.cc:385] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:48.544481 26752 raft_consensus.cc:740] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a00d5d20d5b47b99a1220a0e40a4a78, State: Initialized, Role: FOLLOWER
I20260709 13:09:48.544993 26749 tablet_bootstrap.cc:492] T 579b5cf4a4874b75a67f9cbf320addc2 P dc9195961d41473e8f590b632eb998a0: Bootstrap starting.
I20260709 13:09:48.545441 26752 consensus_queue.cc:260] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.546545 26750 tablet_bootstrap.cc:492] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec: No bootstrap required, opened a new log
I20260709 13:09:48.547219 26750 ts_tablet_manager.cc:1403] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.007s
I20260709 13:09:48.550855 26750 raft_consensus.cc:359] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.551800 26750 raft_consensus.cc:385] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:48.552173 26750 raft_consensus.cc:740] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Initialized, Role: FOLLOWER
I20260709 13:09:48.553077 26750 consensus_queue.cc:260] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.553877 26749 tablet_bootstrap.cc:654] T 579b5cf4a4874b75a67f9cbf320addc2 P dc9195961d41473e8f590b632eb998a0: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:48.555083 26735 heartbeater.cc:499] Master 127.14.103.62:39907 was elected leader, sending a full tablet report...
I20260709 13:09:48.555583 26752 ts_tablet_manager.cc:1434] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.008s
I20260709 13:09:48.556614 26752 tablet_bootstrap.cc:492] T ef38f875bfdd4b669c39d02f0ea7472b P 0a00d5d20d5b47b99a1220a0e40a4a78: Bootstrap starting.
I20260709 13:09:48.558324 26750 ts_tablet_manager.cc:1434] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.004s
I20260709 13:09:48.559351 26750 tablet_bootstrap.cc:492] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec: Bootstrap starting.
I20260709 13:09:48.562247 26752 tablet_bootstrap.cc:654] T ef38f875bfdd4b669c39d02f0ea7472b P 0a00d5d20d5b47b99a1220a0e40a4a78: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:48.565011 26750 tablet_bootstrap.cc:654] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:48.568612 26749 tablet_bootstrap.cc:492] T 579b5cf4a4874b75a67f9cbf320addc2 P dc9195961d41473e8f590b632eb998a0: No bootstrap required, opened a new log
I20260709 13:09:48.569185 26749 ts_tablet_manager.cc:1403] T 579b5cf4a4874b75a67f9cbf320addc2 P dc9195961d41473e8f590b632eb998a0: Time spent bootstrapping tablet: real 0.024s	user 0.019s	sys 0.001s
I20260709 13:09:48.572510 26749 raft_consensus.cc:359] T 579b5cf4a4874b75a67f9cbf320addc2 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.573256 26749 raft_consensus.cc:385] T 579b5cf4a4874b75a67f9cbf320addc2 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:48.573586 26749 raft_consensus.cc:740] T 579b5cf4a4874b75a67f9cbf320addc2 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9195961d41473e8f590b632eb998a0, State: Initialized, Role: FOLLOWER
I20260709 13:09:48.574401 26749 consensus_queue.cc:260] T 579b5cf4a4874b75a67f9cbf320addc2 P dc9195961d41473e8f590b632eb998a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.575166 26752 tablet_bootstrap.cc:492] T ef38f875bfdd4b669c39d02f0ea7472b P 0a00d5d20d5b47b99a1220a0e40a4a78: No bootstrap required, opened a new log
I20260709 13:09:48.577160 26752 ts_tablet_manager.cc:1403] T ef38f875bfdd4b669c39d02f0ea7472b P 0a00d5d20d5b47b99a1220a0e40a4a78: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.004s
I20260709 13:09:48.578914 26749 ts_tablet_manager.cc:1434] T 579b5cf4a4874b75a67f9cbf320addc2 P dc9195961d41473e8f590b632eb998a0: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:09:48.580052 26749 tablet_bootstrap.cc:492] T 6c21616d95d04253a63e088134753cea P dc9195961d41473e8f590b632eb998a0: Bootstrap starting.
I20260709 13:09:48.581139 26752 raft_consensus.cc:359] T ef38f875bfdd4b669c39d02f0ea7472b P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.585791 26752 raft_consensus.cc:385] T ef38f875bfdd4b669c39d02f0ea7472b P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:48.586481 26749 tablet_bootstrap.cc:654] T 6c21616d95d04253a63e088134753cea P dc9195961d41473e8f590b632eb998a0: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:48.586809 26752 raft_consensus.cc:740] T ef38f875bfdd4b669c39d02f0ea7472b P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a00d5d20d5b47b99a1220a0e40a4a78, State: Initialized, Role: FOLLOWER
I20260709 13:09:48.582154 26760 raft_consensus.cc:493] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.589267 26760 raft_consensus.cc:515] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.589627 26752 consensus_queue.cc:260] T ef38f875bfdd4b669c39d02f0ea7472b P 0a00d5d20d5b47b99a1220a0e40a4a78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.594267 26750 tablet_bootstrap.cc:492] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec: No bootstrap required, opened a new log
I20260709 13:09:48.594823 26752 ts_tablet_manager.cc:1434] T ef38f875bfdd4b669c39d02f0ea7472b P 0a00d5d20d5b47b99a1220a0e40a4a78: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 13:09:48.594823 26750 ts_tablet_manager.cc:1403] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec: Time spent bootstrapping tablet: real 0.036s	user 0.016s	sys 0.016s
I20260709 13:09:48.596027 26752 tablet_bootstrap.cc:492] T b16a06ac45ab44a186fbf820559732bb P 0a00d5d20d5b47b99a1220a0e40a4a78: Bootstrap starting.
I20260709 13:09:48.598064 26750 raft_consensus.cc:359] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.598794 26750 raft_consensus.cc:385] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:48.599119 26750 raft_consensus.cc:740] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Initialized, Role: FOLLOWER
I20260709 13:09:48.599942 26750 consensus_queue.cc:260] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.602341 26750 ts_tablet_manager.cc:1434] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 13:09:48.602401 26752 tablet_bootstrap.cc:654] T b16a06ac45ab44a186fbf820559732bb P 0a00d5d20d5b47b99a1220a0e40a4a78: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:48.603427 26750 tablet_bootstrap.cc:492] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec: Bootstrap starting.
I20260709 13:09:48.611101 26750 tablet_bootstrap.cc:654] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:48.618671 26561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f3b339b6bd4573a9f7929c4d130af5" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0" is_pre_election: true
W20260709 13:09:48.624102 26675 leader_election.cc:343] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:48.624547 26750 tablet_bootstrap.cc:492] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec: No bootstrap required, opened a new log
I20260709 13:09:48.625324 26750 ts_tablet_manager.cc:1403] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.007s
I20260709 13:09:48.629310 26750 raft_consensus.cc:359] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.630330 26750 raft_consensus.cc:385] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:48.630738 26750 raft_consensus.cc:740] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Initialized, Role: FOLLOWER
I20260709 13:09:48.631844 26750 consensus_queue.cc:260] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.635285 26750 ts_tablet_manager.cc:1434] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:09:48.636806 26750 tablet_bootstrap.cc:492] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec: Bootstrap starting.
I20260709 13:09:48.640803 26755 raft_consensus.cc:493] T ef38f875bfdd4b669c39d02f0ea7472b P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.641340 26755 raft_consensus.cc:515] T ef38f875bfdd4b669c39d02f0ea7472b P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.643877 26755 leader_election.cc:290] T ef38f875bfdd4b669c39d02f0ea7472b P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:48.644996 26750 tablet_bootstrap.cc:654] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:48.656347 26760 leader_election.cc:290] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285)
I20260709 13:09:48.657835 26750 tablet_bootstrap.cc:492] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec: No bootstrap required, opened a new log
I20260709 13:09:48.663635 26750 ts_tablet_manager.cc:1403] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec: Time spent bootstrapping tablet: real 0.027s	user 0.021s	sys 0.000s
I20260709 13:09:48.667332 26750 raft_consensus.cc:359] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.672523 26750 raft_consensus.cc:385] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:48.673369 26750 raft_consensus.cc:740] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Initialized, Role: FOLLOWER
I20260709 13:09:48.674714 26636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f3b339b6bd4573a9f7929c4d130af5" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68637fffcc4c4969ae0612be35aa58ec" is_pre_election: true
I20260709 13:09:48.674605 26750 consensus_queue.cc:260] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.675441 26636 raft_consensus.cc:2468] T b8f3b339b6bd4573a9f7929c4d130af5 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in term 0.
I20260709 13:09:48.676977 26675 leader_election.cc:304] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78, 68637fffcc4c4969ae0612be35aa58ec; no voters: dc9195961d41473e8f590b632eb998a0
I20260709 13:09:48.677480 26754 raft_consensus.cc:493] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.678519 26760 raft_consensus.cc:2804] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:48.678208 26754 raft_consensus.cc:515] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.680371 26760 raft_consensus.cc:493] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:48.680871 26760 raft_consensus.cc:3060] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:48.685160 26754 leader_election.cc:290] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:48.676957 26750 ts_tablet_manager.cc:1434] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:09:48.689229 26750 tablet_bootstrap.cc:492] T 25be33d2f6bf455996cd05210da81458 P 68637fffcc4c4969ae0612be35aa58ec: Bootstrap starting.
I20260709 13:09:48.692706 26760 raft_consensus.cc:515] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.702342 26561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f3b339b6bd4573a9f7929c4d130af5" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0"
I20260709 13:09:48.703176 26754 raft_consensus.cc:493] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:48.703877 26675 leader_election.cc:343] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:48.703745 26754 raft_consensus.cc:515] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.705257 26760 leader_election.cc:290] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 1 election: Requested vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285)
I20260709 13:09:48.705680 26754 leader_election.cc:290] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:48.708184 26750 tablet_bootstrap.cc:654] T 25be33d2f6bf455996cd05210da81458 P 68637fffcc4c4969ae0612be35aa58ec: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:48.733492 26754 raft_consensus.cc:493] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.734086 26754 raft_consensus.cc:515] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.735060 26750 tablet_bootstrap.cc:492] T 25be33d2f6bf455996cd05210da81458 P 68637fffcc4c4969ae0612be35aa58ec: No bootstrap required, opened a new log
I20260709 13:09:48.735371 26749 tablet_bootstrap.cc:492] T 6c21616d95d04253a63e088134753cea P dc9195961d41473e8f590b632eb998a0: No bootstrap required, opened a new log
I20260709 13:09:48.735569 26750 ts_tablet_manager.cc:1403] T 25be33d2f6bf455996cd05210da81458 P 68637fffcc4c4969ae0612be35aa58ec: Time spent bootstrapping tablet: real 0.047s	user 0.009s	sys 0.020s
I20260709 13:09:48.736109 26754 leader_election.cc:290] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:48.736148 26749 ts_tablet_manager.cc:1403] T 6c21616d95d04253a63e088134753cea P dc9195961d41473e8f590b632eb998a0: Time spent bootstrapping tablet: real 0.156s	user 0.048s	sys 0.043s
I20260709 13:09:48.737109 26636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f3b339b6bd4573a9f7929c4d130af5" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68637fffcc4c4969ae0612be35aa58ec"
I20260709 13:09:48.737694 26636 raft_consensus.cc:3060] T b8f3b339b6bd4573a9f7929c4d130af5 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:48.738740 26750 raft_consensus.cc:359] T 25be33d2f6bf455996cd05210da81458 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.739452 26750 raft_consensus.cc:385] T 25be33d2f6bf455996cd05210da81458 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:48.739739 26750 raft_consensus.cc:740] T 25be33d2f6bf455996cd05210da81458 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Initialized, Role: FOLLOWER
I20260709 13:09:48.740306 26749 raft_consensus.cc:359] T 6c21616d95d04253a63e088134753cea P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.741259 26749 raft_consensus.cc:385] T 6c21616d95d04253a63e088134753cea P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:48.741670 26752 tablet_bootstrap.cc:492] T b16a06ac45ab44a186fbf820559732bb P 0a00d5d20d5b47b99a1220a0e40a4a78: No bootstrap required, opened a new log
I20260709 13:09:48.741715 26749 raft_consensus.cc:740] T 6c21616d95d04253a63e088134753cea P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9195961d41473e8f590b632eb998a0, State: Initialized, Role: FOLLOWER
I20260709 13:09:48.742322 26752 ts_tablet_manager.cc:1403] T b16a06ac45ab44a186fbf820559732bb P 0a00d5d20d5b47b99a1220a0e40a4a78: Time spent bootstrapping tablet: real 0.147s	user 0.039s	sys 0.082s
I20260709 13:09:48.743261 26749 consensus_queue.cc:260] T 6c21616d95d04253a63e088134753cea P dc9195961d41473e8f590b632eb998a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.746340 26749 ts_tablet_manager.cc:1434] T 6c21616d95d04253a63e088134753cea P dc9195961d41473e8f590b632eb998a0: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:09:48.747706 26749 tablet_bootstrap.cc:492] T b16a06ac45ab44a186fbf820559732bb P dc9195961d41473e8f590b632eb998a0: Bootstrap starting.
I20260709 13:09:48.748190 26752 raft_consensus.cc:359] T b16a06ac45ab44a186fbf820559732bb P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.748930 26752 raft_consensus.cc:385] T b16a06ac45ab44a186fbf820559732bb P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:48.749220 26752 raft_consensus.cc:740] T b16a06ac45ab44a186fbf820559732bb P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a00d5d20d5b47b99a1220a0e40a4a78, State: Initialized, Role: FOLLOWER
I20260709 13:09:48.750845 26750 consensus_queue.cc:260] T 25be33d2f6bf455996cd05210da81458 P 68637fffcc4c4969ae0612be35aa58ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.751482 26755 raft_consensus.cc:493] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.752053 26755 raft_consensus.cc:515] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.753300 26750 ts_tablet_manager.cc:1434] T 25be33d2f6bf455996cd05210da81458 P 68637fffcc4c4969ae0612be35aa58ec: Time spent starting tablet: real 0.017s	user 0.003s	sys 0.002s
I20260709 13:09:48.754182 26755 leader_election.cc:290] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:48.754029 26752 consensus_queue.cc:260] T b16a06ac45ab44a186fbf820559732bb P 0a00d5d20d5b47b99a1220a0e40a4a78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.758630 26752 ts_tablet_manager.cc:1434] T b16a06ac45ab44a186fbf820559732bb P 0a00d5d20d5b47b99a1220a0e40a4a78: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.002s
I20260709 13:09:48.759694 26752 tablet_bootstrap.cc:492] T 4af4eae89a594eb0ace69b0895e43372 P 0a00d5d20d5b47b99a1220a0e40a4a78: Bootstrap starting.
I20260709 13:09:48.765731 26752 tablet_bootstrap.cc:654] T 4af4eae89a594eb0ace69b0895e43372 P 0a00d5d20d5b47b99a1220a0e40a4a78: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:48.755788 26749 tablet_bootstrap.cc:654] T b16a06ac45ab44a186fbf820559732bb P dc9195961d41473e8f590b632eb998a0: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:48.768884 26776 raft_consensus.cc:493] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.769450 26776 raft_consensus.cc:515] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.771157 26750 tablet_bootstrap.cc:492] T d8410badcf77497fbb604cf8c185db83 P 68637fffcc4c4969ae0612be35aa58ec: Bootstrap starting.
I20260709 13:09:48.775030 26710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055a26fb2c8b41b4b6da92c7e20180e1" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" is_pre_election: true
I20260709 13:09:48.776564 26709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b16a06ac45ab44a186fbf820559732bb" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" is_pre_election: true
I20260709 13:09:48.777231 26709 raft_consensus.cc:2468] T b16a06ac45ab44a186fbf820559732bb P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 0.
I20260709 13:09:48.777735 26749 tablet_bootstrap.cc:492] T b16a06ac45ab44a186fbf820559732bb P dc9195961d41473e8f590b632eb998a0: No bootstrap required, opened a new log
I20260709 13:09:48.778354 26749 ts_tablet_manager.cc:1403] T b16a06ac45ab44a186fbf820559732bb P dc9195961d41473e8f590b632eb998a0: Time spent bootstrapping tablet: real 0.031s	user 0.010s	sys 0.009s
I20260709 13:09:48.778854 26750 tablet_bootstrap.cc:654] T d8410badcf77497fbb604cf8c185db83 P 68637fffcc4c4969ae0612be35aa58ec: Neither blocks nor log segments found. Creating new log.
W20260709 13:09:48.779008 26601 leader_election.cc:343] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:48.782857 26601 leader_election.cc:304] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78, 68637fffcc4c4969ae0612be35aa58ec; no voters: 
I20260709 13:09:48.780905 26561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055a26fb2c8b41b4b6da92c7e20180e1" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0" is_pre_election: true
I20260709 13:09:48.783605 26561 raft_consensus.cc:2468] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 0.
I20260709 13:09:48.781441 26559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef38f875bfdd4b669c39d02f0ea7472b" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0" is_pre_election: true
I20260709 13:09:48.783622 26749 raft_consensus.cc:359] T b16a06ac45ab44a186fbf820559732bb P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.784667 26559 raft_consensus.cc:2468] T ef38f875bfdd4b669c39d02f0ea7472b P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 0.
I20260709 13:09:48.784782 26749 raft_consensus.cc:385] T b16a06ac45ab44a186fbf820559732bb P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:48.785070 26776 leader_election.cc:290] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:48.785255 26749 raft_consensus.cc:740] T b16a06ac45ab44a186fbf820559732bb P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9195961d41473e8f590b632eb998a0, State: Initialized, Role: FOLLOWER
I20260709 13:09:48.782006 26709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef38f875bfdd4b669c39d02f0ea7472b" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" is_pre_election: true
I20260709 13:09:48.786435 26709 raft_consensus.cc:2468] T ef38f875bfdd4b669c39d02f0ea7472b P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 0.
I20260709 13:09:48.786181 26749 consensus_queue.cc:260] T b16a06ac45ab44a186fbf820559732bb P dc9195961d41473e8f590b632eb998a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.780678 26560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b16a06ac45ab44a186fbf820559732bb" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0" is_pre_election: true
I20260709 13:09:48.787428 26560 raft_consensus.cc:2468] T b16a06ac45ab44a186fbf820559732bb P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 0.
I20260709 13:09:48.790997 26560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579b5cf4a4874b75a67f9cbf320addc2" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0" is_pre_election: true
I20260709 13:09:48.791640 26560 raft_consensus.cc:2468] T 579b5cf4a4874b75a67f9cbf320addc2 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 0.
I20260709 13:09:48.792186 26755 raft_consensus.cc:493] T 579b5cf4a4874b75a67f9cbf320addc2 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.793310 26709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579b5cf4a4874b75a67f9cbf320addc2" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" is_pre_election: true
I20260709 13:09:48.793373 26755 raft_consensus.cc:515] T 579b5cf4a4874b75a67f9cbf320addc2 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.796613 26776 raft_consensus.cc:2804] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:48.797070 26776 raft_consensus.cc:493] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:48.797252 26755 leader_election.cc:290] T 579b5cf4a4874b75a67f9cbf320addc2 P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:48.797498 26776 raft_consensus.cc:3060] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:48.790339 26601 leader_election.cc:304] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 68637fffcc4c4969ae0612be35aa58ec, dc9195961d41473e8f590b632eb998a0; no voters: 0a00d5d20d5b47b99a1220a0e40a4a78
I20260709 13:09:48.799429 26635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef38f875bfdd4b669c39d02f0ea7472b" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68637fffcc4c4969ae0612be35aa58ec" is_pre_election: true
I20260709 13:09:48.800173 26634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055a26fb2c8b41b4b6da92c7e20180e1" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68637fffcc4c4969ae0612be35aa58ec" is_pre_election: true
I20260709 13:09:48.800622 26635 raft_consensus.cc:2468] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc9195961d41473e8f590b632eb998a0 in term 0.
I20260709 13:09:48.800861 26634 raft_consensus.cc:2468] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc9195961d41473e8f590b632eb998a0 in term 0.
I20260709 13:09:48.801860 26633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579b5cf4a4874b75a67f9cbf320addc2" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68637fffcc4c4969ae0612be35aa58ec" is_pre_election: true
I20260709 13:09:48.802568 26633 raft_consensus.cc:2468] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc9195961d41473e8f590b632eb998a0 in term 0.
I20260709 13:09:48.803107 26601 leader_election.cc:304] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68637fffcc4c4969ae0612be35aa58ec, dc9195961d41473e8f590b632eb998a0; no voters: 
I20260709 13:09:48.804003 26754 raft_consensus.cc:2804] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:48.804479 26754 raft_consensus.cc:493] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:48.804875 26754 raft_consensus.cc:3060] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:48.807400 26776 raft_consensus.cc:515] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.808085 26749 ts_tablet_manager.cc:1434] T b16a06ac45ab44a186fbf820559732bb P dc9195961d41473e8f590b632eb998a0: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.002s
I20260709 13:09:48.812647 26526 leader_election.cc:304] T ef38f875bfdd4b669c39d02f0ea7472b P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68637fffcc4c4969ae0612be35aa58ec, dc9195961d41473e8f590b632eb998a0; no voters: 
I20260709 13:09:48.813416 26708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579b5cf4a4874b75a67f9cbf320addc2" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" is_pre_election: true
I20260709 13:09:48.814127 26755 raft_consensus.cc:2804] T ef38f875bfdd4b669c39d02f0ea7472b P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:48.814594 26755 raft_consensus.cc:493] T ef38f875bfdd4b669c39d02f0ea7472b P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:48.814921 26755 raft_consensus.cc:3060] T ef38f875bfdd4b669c39d02f0ea7472b P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:48.809504 26776 leader_election.cc:290] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 election: Requested vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:48.812057 26709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef38f875bfdd4b669c39d02f0ea7472b" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" is_pre_election: true
I20260709 13:09:48.815996 26776 raft_consensus.cc:2804] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:48.816387 26776 raft_consensus.cc:493] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:48.816359 26709 raft_consensus.cc:2468] T ef38f875bfdd4b669c39d02f0ea7472b P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc9195961d41473e8f590b632eb998a0 in term 0.
I20260709 13:09:48.816896 26776 raft_consensus.cc:3060] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:48.811308 26754 raft_consensus.cc:515] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.797498 26636 raft_consensus.cc:2468] T b8f3b339b6bd4573a9f7929c4d130af5 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in term 1.
I20260709 13:09:48.815315 26710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055a26fb2c8b41b4b6da92c7e20180e1" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" is_pre_election: true
I20260709 13:09:48.820066 26754 leader_election.cc:290] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 election: Requested vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:48.822527 26749 tablet_bootstrap.cc:492] T b8f3b339b6bd4573a9f7929c4d130af5 P dc9195961d41473e8f590b632eb998a0: Bootstrap starting.
I20260709 13:09:48.823007 26776 raft_consensus.cc:515] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.824882 26776 leader_election.cc:290] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 election: Requested vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:48.813722 26526 leader_election.cc:304] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68637fffcc4c4969ae0612be35aa58ec, dc9195961d41473e8f590b632eb998a0; no voters: 
I20260709 13:09:48.826182 26526 leader_election.cc:304] T 579b5cf4a4874b75a67f9cbf320addc2 P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68637fffcc4c4969ae0612be35aa58ec, dc9195961d41473e8f590b632eb998a0; no voters: 
I20260709 13:09:48.827338 26780 raft_consensus.cc:2804] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:48.828261 26780 raft_consensus.cc:493] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:48.829071 26780 raft_consensus.cc:3060] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:48.829604 26601 leader_election.cc:304] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68637fffcc4c4969ae0612be35aa58ec, dc9195961d41473e8f590b632eb998a0; no voters: 
I20260709 13:09:48.830209 26755 raft_consensus.cc:515] T ef38f875bfdd4b669c39d02f0ea7472b P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.832001 26781 raft_consensus.cc:2804] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:48.832432 26781 raft_consensus.cc:493] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:48.832749 26781 raft_consensus.cc:3060] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:48.832813 26755 leader_election.cc:290] T ef38f875bfdd4b669c39d02f0ea7472b P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 1 election: Requested vote from peers 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:48.833621 26755 raft_consensus.cc:2804] T 579b5cf4a4874b75a67f9cbf320addc2 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:48.834036 26755 raft_consensus.cc:493] T 579b5cf4a4874b75a67f9cbf320addc2 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:48.834420 26755 raft_consensus.cc:3060] T 579b5cf4a4874b75a67f9cbf320addc2 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:09:48.832018 26507 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:09:48.831459 26601 leader_election.cc:343] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:48.837172 26675 leader_election.cc:304] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78, 68637fffcc4c4969ae0612be35aa58ec; no voters: dc9195961d41473e8f590b632eb998a0
I20260709 13:09:48.837577 26560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b16a06ac45ab44a186fbf820559732bb" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0"
I20260709 13:09:48.838130 26560 raft_consensus.cc:3060] T b16a06ac45ab44a186fbf820559732bb P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:48.838258 26760 raft_consensus.cc:2804] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:48.838791 26760 raft_consensus.cc:697] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 LEADER]: Becoming Leader. State: Replica: 0a00d5d20d5b47b99a1220a0e40a4a78, State: Running, Role: LEADER
I20260709 13:09:48.839633 26760 consensus_queue.cc:237] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.840003 26710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b16a06ac45ab44a186fbf820559732bb" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
I20260709 13:09:48.840565 26749 tablet_bootstrap.cc:654] T b8f3b339b6bd4573a9f7929c4d130af5 P dc9195961d41473e8f590b632eb998a0: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:48.840963 26559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055a26fb2c8b41b4b6da92c7e20180e1" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0"
I20260709 13:09:48.841883 26561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef38f875bfdd4b669c39d02f0ea7472b" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0"
I20260709 13:09:48.841749 26781 raft_consensus.cc:515] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.842658 26561 raft_consensus.cc:2393] T ef38f875bfdd4b669c39d02f0ea7472b P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 68637fffcc4c4969ae0612be35aa58ec in current term 1: Already voted for candidate dc9195961d41473e8f590b632eb998a0 in this term.
I20260709 13:09:48.843920 26781 leader_election.cc:290] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 election: Requested vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:48.845732 26752 tablet_bootstrap.cc:492] T 4af4eae89a594eb0ace69b0895e43372 P 0a00d5d20d5b47b99a1220a0e40a4a78: No bootstrap required, opened a new log
I20260709 13:09:48.846263 26752 ts_tablet_manager.cc:1403] T 4af4eae89a594eb0ace69b0895e43372 P 0a00d5d20d5b47b99a1220a0e40a4a78: Time spent bootstrapping tablet: real 0.087s	user 0.021s	sys 0.013s
I20260709 13:09:48.847589 26709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055a26fb2c8b41b4b6da92c7e20180e1" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
I20260709 13:09:48.848783 26708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef38f875bfdd4b669c39d02f0ea7472b" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
I20260709 13:09:48.849375 26708 raft_consensus.cc:3060] T ef38f875bfdd4b669c39d02f0ea7472b P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:48.849192 26752 raft_consensus.cc:359] T 4af4eae89a594eb0ace69b0895e43372 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.849666 26750 tablet_bootstrap.cc:492] T d8410badcf77497fbb604cf8c185db83 P 68637fffcc4c4969ae0612be35aa58ec: No bootstrap required, opened a new log
I20260709 13:09:48.849913 26752 raft_consensus.cc:385] T 4af4eae89a594eb0ace69b0895e43372 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:48.850145 26750 ts_tablet_manager.cc:1403] T d8410badcf77497fbb604cf8c185db83 P 68637fffcc4c4969ae0612be35aa58ec: Time spent bootstrapping tablet: real 0.079s	user 0.017s	sys 0.018s
I20260709 13:09:48.850258 26752 raft_consensus.cc:740] T 4af4eae89a594eb0ace69b0895e43372 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a00d5d20d5b47b99a1220a0e40a4a78, State: Initialized, Role: FOLLOWER
I20260709 13:09:48.851169 26709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef38f875bfdd4b669c39d02f0ea7472b" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
I20260709 13:09:48.851255 26752 consensus_queue.cc:260] T 4af4eae89a594eb0ace69b0895e43372 P 0a00d5d20d5b47b99a1220a0e40a4a78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.852154 26707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579b5cf4a4874b75a67f9cbf320addc2" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
I20260709 13:09:48.852774 26750 raft_consensus.cc:359] T d8410badcf77497fbb604cf8c185db83 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.853468 26750 raft_consensus.cc:385] T d8410badcf77497fbb604cf8c185db83 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:48.853704 26750 raft_consensus.cc:740] T d8410badcf77497fbb604cf8c185db83 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Initialized, Role: FOLLOWER
I20260709 13:09:48.850179 26636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef38f875bfdd4b669c39d02f0ea7472b" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68637fffcc4c4969ae0612be35aa58ec"
I20260709 13:09:48.854962 26636 raft_consensus.cc:2393] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate dc9195961d41473e8f590b632eb998a0 in current term 1: Already voted for candidate 68637fffcc4c4969ae0612be35aa58ec in this term.
I20260709 13:09:48.855415 26780 raft_consensus.cc:515] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.840574 26710 raft_consensus.cc:3060] T b16a06ac45ab44a186fbf820559732bb P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:48.856653 26559 raft_consensus.cc:2393] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 68637fffcc4c4969ae0612be35aa58ec in current term 1: Already voted for candidate dc9195961d41473e8f590b632eb998a0 in this term.
I20260709 13:09:48.856523 26750 consensus_queue.cc:260] T d8410badcf77497fbb604cf8c185db83 P 68637fffcc4c4969ae0612be35aa58ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.853957 26752 ts_tablet_manager.cc:1434] T 4af4eae89a594eb0ace69b0895e43372 P 0a00d5d20d5b47b99a1220a0e40a4a78: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:09:48.858157 26752 tablet_bootstrap.cc:492] T 055a26fb2c8b41b4b6da92c7e20180e1 P 0a00d5d20d5b47b99a1220a0e40a4a78: Bootstrap starting.
I20260709 13:09:48.858680 26750 ts_tablet_manager.cc:1434] T d8410badcf77497fbb604cf8c185db83 P 68637fffcc4c4969ae0612be35aa58ec: Time spent starting tablet: real 0.008s	user 0.000s	sys 0.004s
I20260709 13:09:48.860569 26561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579b5cf4a4874b75a67f9cbf320addc2" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0"
I20260709 13:09:48.861047 26780 leader_election.cc:290] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 1 election: Requested vote from peers 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:48.862329 26750 tablet_bootstrap.cc:492] T 4af4eae89a594eb0ace69b0895e43372 P 68637fffcc4c4969ae0612be35aa58ec: Bootstrap starting.
I20260709 13:09:48.863981 26752 tablet_bootstrap.cc:654] T 055a26fb2c8b41b4b6da92c7e20180e1 P 0a00d5d20d5b47b99a1220a0e40a4a78: Neither blocks nor log segments found. Creating new log.
W20260709 13:09:48.852782 26526 leader_election.cc:343] T 579b5cf4a4874b75a67f9cbf320addc2 P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:48.873200 26526 leader_election.cc:343] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:48.873584 26457 catalog_manager.cc:5697] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3). New cstate: current_term: 1 leader_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:48.875574 26710 raft_consensus.cc:2468] T b16a06ac45ab44a186fbf820559732bb P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 1.
I20260709 13:09:48.876063 26707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055a26fb2c8b41b4b6da92c7e20180e1" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
I20260709 13:09:48.871655 26708 raft_consensus.cc:2468] T ef38f875bfdd4b669c39d02f0ea7472b P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 1.
W20260709 13:09:48.867200 26601 leader_election.cc:343] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:48.877179 26601 leader_election.cc:343] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:48.877555 26526 leader_election.cc:304] T ef38f875bfdd4b669c39d02f0ea7472b P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dc9195961d41473e8f590b632eb998a0; no voters: 0a00d5d20d5b47b99a1220a0e40a4a78, 68637fffcc4c4969ae0612be35aa58ec
I20260709 13:09:48.879458 26780 raft_consensus.cc:2749] T ef38f875bfdd4b669c39d02f0ea7472b P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:09:48.880367 26749 tablet_bootstrap.cc:492] T b8f3b339b6bd4573a9f7929c4d130af5 P dc9195961d41473e8f590b632eb998a0: No bootstrap required, opened a new log
W20260709 13:09:48.880692 26526 leader_election.cc:343] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:48.876636 26750 tablet_bootstrap.cc:654] T 4af4eae89a594eb0ace69b0895e43372 P 68637fffcc4c4969ae0612be35aa58ec: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:48.875259 26636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055a26fb2c8b41b4b6da92c7e20180e1" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68637fffcc4c4969ae0612be35aa58ec"
I20260709 13:09:48.868914 26752 tablet_bootstrap.cc:492] T 055a26fb2c8b41b4b6da92c7e20180e1 P 0a00d5d20d5b47b99a1220a0e40a4a78: No bootstrap required, opened a new log
I20260709 13:09:48.877959 26601 leader_election.cc:304] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78, 68637fffcc4c4969ae0612be35aa58ec; no voters: 
I20260709 13:09:48.881721 26636 raft_consensus.cc:2393] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate dc9195961d41473e8f590b632eb998a0 in current term 1: Already voted for candidate 68637fffcc4c4969ae0612be35aa58ec in this term.
I20260709 13:09:48.882288 26781 raft_consensus.cc:2804] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:48.882773 26601 leader_election.cc:304] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 68637fffcc4c4969ae0612be35aa58ec; no voters: 0a00d5d20d5b47b99a1220a0e40a4a78, dc9195961d41473e8f590b632eb998a0
I20260709 13:09:48.882783 26781 raft_consensus.cc:697] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Becoming Leader. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Running, Role: LEADER
I20260709 13:09:48.883667 26776 raft_consensus.cc:2749] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:09:48.883572 26755 raft_consensus.cc:515] T 579b5cf4a4874b75a67f9cbf320addc2 P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.883936 26781 consensus_queue.cc:237] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.884946 26561 raft_consensus.cc:2393] T 579b5cf4a4874b75a67f9cbf320addc2 P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 68637fffcc4c4969ae0612be35aa58ec in current term 1: Already voted for candidate dc9195961d41473e8f590b632eb998a0 in this term.
I20260709 13:09:48.885078 26601 leader_election.cc:304] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78, 68637fffcc4c4969ae0612be35aa58ec; no voters: dc9195961d41473e8f590b632eb998a0
I20260709 13:09:48.881738 26752 ts_tablet_manager.cc:1403] T 055a26fb2c8b41b4b6da92c7e20180e1 P 0a00d5d20d5b47b99a1220a0e40a4a78: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.000s
I20260709 13:09:48.887502 26526 leader_election.cc:304] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dc9195961d41473e8f590b632eb998a0; no voters: 0a00d5d20d5b47b99a1220a0e40a4a78, 68637fffcc4c4969ae0612be35aa58ec
I20260709 13:09:48.888680 26780 raft_consensus.cc:2749] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:09:48.889115 26752 raft_consensus.cc:359] T 055a26fb2c8b41b4b6da92c7e20180e1 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.889940 26752 raft_consensus.cc:385] T 055a26fb2c8b41b4b6da92c7e20180e1 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:48.890226 26752 raft_consensus.cc:740] T 055a26fb2c8b41b4b6da92c7e20180e1 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a00d5d20d5b47b99a1220a0e40a4a78, State: Initialized, Role: FOLLOWER
I20260709 13:09:48.891037 26776 raft_consensus.cc:2804] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:48.891170 26754 raft_consensus.cc:493] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.891651 26754 raft_consensus.cc:515] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.892843 26636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579b5cf4a4874b75a67f9cbf320addc2" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68637fffcc4c4969ae0612be35aa58ec"
I20260709 13:09:48.893934 26636 raft_consensus.cc:2393] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate dc9195961d41473e8f590b632eb998a0 in current term 1: Already voted for candidate 68637fffcc4c4969ae0612be35aa58ec in this term.
I20260709 13:09:48.894150 26754 leader_election.cc:290] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:48.867954 26560 raft_consensus.cc:2468] T b16a06ac45ab44a186fbf820559732bb P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 1.
I20260709 13:09:48.881050 26749 ts_tablet_manager.cc:1403] T b8f3b339b6bd4573a9f7929c4d130af5 P dc9195961d41473e8f590b632eb998a0: Time spent bootstrapping tablet: real 0.059s	user 0.014s	sys 0.004s
I20260709 13:09:48.895725 26561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c21616d95d04253a63e088134753cea" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0" is_pre_election: true
I20260709 13:09:48.896513 26561 raft_consensus.cc:2468] T 6c21616d95d04253a63e088134753cea P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 0.
I20260709 13:09:48.897287 26601 leader_election.cc:304] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 68637fffcc4c4969ae0612be35aa58ec; no voters: 0a00d5d20d5b47b99a1220a0e40a4a78, dc9195961d41473e8f590b632eb998a0
I20260709 13:09:48.895860 26710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c21616d95d04253a63e088134753cea" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" is_pre_election: true
I20260709 13:09:48.899830 26754 raft_consensus.cc:2749] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:09:48.899091 26749 raft_consensus.cc:359] T b8f3b339b6bd4573a9f7929c4d130af5 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.900723 26755 leader_election.cc:290] T 579b5cf4a4874b75a67f9cbf320addc2 P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 1 election: Requested vote from peers 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:48.903563 26752 consensus_queue.cc:260] T 055a26fb2c8b41b4b6da92c7e20180e1 P 0a00d5d20d5b47b99a1220a0e40a4a78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.900880 26749 raft_consensus.cc:385] T b8f3b339b6bd4573a9f7929c4d130af5 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:48.901142 26601 leader_election.cc:304] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68637fffcc4c4969ae0612be35aa58ec, dc9195961d41473e8f590b632eb998a0; no voters: 
I20260709 13:09:48.905022 26749 raft_consensus.cc:740] T b8f3b339b6bd4573a9f7929c4d130af5 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9195961d41473e8f590b632eb998a0, State: Initialized, Role: FOLLOWER
I20260709 13:09:48.902571 26710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579b5cf4a4874b75a67f9cbf320addc2" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
I20260709 13:09:48.906358 26750 tablet_bootstrap.cc:492] T 4af4eae89a594eb0ace69b0895e43372 P 68637fffcc4c4969ae0612be35aa58ec: No bootstrap required, opened a new log
I20260709 13:09:48.909610 26750 ts_tablet_manager.cc:1403] T 4af4eae89a594eb0ace69b0895e43372 P 68637fffcc4c4969ae0612be35aa58ec: Time spent bootstrapping tablet: real 0.047s	user 0.021s	sys 0.009s
I20260709 13:09:48.909754 26749 consensus_queue.cc:260] T b8f3b339b6bd4573a9f7929c4d130af5 P dc9195961d41473e8f590b632eb998a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
W20260709 13:09:48.910173 26526 leader_election.cc:343] T 579b5cf4a4874b75a67f9cbf320addc2 P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:48.910853 26526 leader_election.cc:304] T 579b5cf4a4874b75a67f9cbf320addc2 P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dc9195961d41473e8f590b632eb998a0; no voters: 0a00d5d20d5b47b99a1220a0e40a4a78, 68637fffcc4c4969ae0612be35aa58ec
I20260709 13:09:48.911540 26755 raft_consensus.cc:2749] T 579b5cf4a4874b75a67f9cbf320addc2 P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:09:48.912489 26750 raft_consensus.cc:359] T 4af4eae89a594eb0ace69b0895e43372 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.912705 26754 raft_consensus.cc:2804] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:48.913203 26776 raft_consensus.cc:697] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Becoming Leader. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Running, Role: LEADER
I20260709 13:09:48.913826 26754 raft_consensus.cc:493] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:48.915169 26749 ts_tablet_manager.cc:1434] T b8f3b339b6bd4573a9f7929c4d130af5 P dc9195961d41473e8f590b632eb998a0: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.004s
I20260709 13:09:48.915580 26750 raft_consensus.cc:385] T 4af4eae89a594eb0ace69b0895e43372 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:48.915059 26776 consensus_queue.cc:237] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
W20260709 13:09:48.912622 26601 leader_election.cc:343] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:48.917081 26749 tablet_bootstrap.cc:492] T 4af4eae89a594eb0ace69b0895e43372 P dc9195961d41473e8f590b632eb998a0: Bootstrap starting.
I20260709 13:09:48.915973 26750 raft_consensus.cc:740] T 4af4eae89a594eb0ace69b0895e43372 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Initialized, Role: FOLLOWER
I20260709 13:09:48.915992 26754 raft_consensus.cc:3060] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:48.919590 26752 ts_tablet_manager.cc:1434] T 055a26fb2c8b41b4b6da92c7e20180e1 P 0a00d5d20d5b47b99a1220a0e40a4a78: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.000s
I20260709 13:09:48.920917 26752 tablet_bootstrap.cc:492] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78: Bootstrap starting.
I20260709 13:09:48.923763 26749 tablet_bootstrap.cc:654] T 4af4eae89a594eb0ace69b0895e43372 P dc9195961d41473e8f590b632eb998a0: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:48.924573 26750 consensus_queue.cc:260] T 4af4eae89a594eb0ace69b0895e43372 P 68637fffcc4c4969ae0612be35aa58ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.928189 26750 ts_tablet_manager.cc:1434] T 4af4eae89a594eb0ace69b0895e43372 P 68637fffcc4c4969ae0612be35aa58ec: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 13:09:48.928927 26754 raft_consensus.cc:515] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.934255 26754 leader_election.cc:290] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 election: Requested vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:48.935879 26561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c21616d95d04253a63e088134753cea" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0"
I20260709 13:09:48.937593 26752 tablet_bootstrap.cc:654] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:48.936223 26710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c21616d95d04253a63e088134753cea" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
I20260709 13:09:48.937546 26561 raft_consensus.cc:3060] T 6c21616d95d04253a63e088134753cea P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:48.935222 26458 catalog_manager.cc:5697] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2). New cstate: current_term: 1 leader_uuid: "68637fffcc4c4969ae0612be35aa58ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } health_report { overall_health: UNKNOWN } } }
W20260709 13:09:48.938915 26601 leader_election.cc:343] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:48.941475 26749 tablet_bootstrap.cc:492] T 4af4eae89a594eb0ace69b0895e43372 P dc9195961d41473e8f590b632eb998a0: No bootstrap required, opened a new log
I20260709 13:09:48.942188 26749 ts_tablet_manager.cc:1403] T 4af4eae89a594eb0ace69b0895e43372 P dc9195961d41473e8f590b632eb998a0: Time spent bootstrapping tablet: real 0.025s	user 0.009s	sys 0.007s
I20260709 13:09:48.945941 26749 raft_consensus.cc:359] T 4af4eae89a594eb0ace69b0895e43372 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.946915 26749 raft_consensus.cc:385] T 4af4eae89a594eb0ace69b0895e43372 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:48.947312 26749 raft_consensus.cc:740] T 4af4eae89a594eb0ace69b0895e43372 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9195961d41473e8f590b632eb998a0, State: Initialized, Role: FOLLOWER
I20260709 13:09:48.947561 26561 raft_consensus.cc:2468] T 6c21616d95d04253a63e088134753cea P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 1.
I20260709 13:09:48.948350 26749 consensus_queue.cc:260] T 4af4eae89a594eb0ace69b0895e43372 P dc9195961d41473e8f590b632eb998a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.950479 26601 leader_election.cc:304] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 68637fffcc4c4969ae0612be35aa58ec, dc9195961d41473e8f590b632eb998a0; no voters: 0a00d5d20d5b47b99a1220a0e40a4a78
I20260709 13:09:48.951278 26749 ts_tablet_manager.cc:1434] T 4af4eae89a594eb0ace69b0895e43372 P dc9195961d41473e8f590b632eb998a0: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:09:48.951696 26754 raft_consensus.cc:2804] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:48.952471 26754 raft_consensus.cc:697] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Becoming Leader. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Running, Role: LEADER
I20260709 13:09:48.952714 26749 tablet_bootstrap.cc:492] T 25be33d2f6bf455996cd05210da81458 P dc9195961d41473e8f590b632eb998a0: Bootstrap starting.
I20260709 13:09:48.955181 26754 consensus_queue.cc:237] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.961619 26749 tablet_bootstrap.cc:654] T 25be33d2f6bf455996cd05210da81458 P dc9195961d41473e8f590b632eb998a0: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:48.966509 26752 tablet_bootstrap.cc:492] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78: No bootstrap required, opened a new log
I20260709 13:09:48.967216 26752 ts_tablet_manager.cc:1403] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78: Time spent bootstrapping tablet: real 0.047s	user 0.015s	sys 0.018s
I20260709 13:09:48.971040 26752 raft_consensus.cc:359] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.972510 26752 raft_consensus.cc:385] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:48.972754 26752 raft_consensus.cc:740] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a00d5d20d5b47b99a1220a0e40a4a78, State: Initialized, Role: FOLLOWER
I20260709 13:09:48.972795 26749 tablet_bootstrap.cc:492] T 25be33d2f6bf455996cd05210da81458 P dc9195961d41473e8f590b632eb998a0: No bootstrap required, opened a new log
I20260709 13:09:48.973347 26749 ts_tablet_manager.cc:1403] T 25be33d2f6bf455996cd05210da81458 P dc9195961d41473e8f590b632eb998a0: Time spent bootstrapping tablet: real 0.021s	user 0.018s	sys 0.002s
I20260709 13:09:48.973488 26752 consensus_queue.cc:260] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.975750 26752 ts_tablet_manager.cc:1434] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:09:48.975736 26749 raft_consensus.cc:359] T 25be33d2f6bf455996cd05210da81458 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.976606 26749 raft_consensus.cc:385] T 25be33d2f6bf455996cd05210da81458 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:48.976902 26749 raft_consensus.cc:740] T 25be33d2f6bf455996cd05210da81458 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9195961d41473e8f590b632eb998a0, State: Initialized, Role: FOLLOWER
I20260709 13:09:48.976994 26752 tablet_bootstrap.cc:492] T 6c21616d95d04253a63e088134753cea P 0a00d5d20d5b47b99a1220a0e40a4a78: Bootstrap starting.
I20260709 13:09:48.978312 26754 raft_consensus.cc:493] T 25be33d2f6bf455996cd05210da81458 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.977583 26458 catalog_manager.cc:5697] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2). New cstate: current_term: 1 leader_uuid: "68637fffcc4c4969ae0612be35aa58ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:48.978787 26754 raft_consensus.cc:515] T 25be33d2f6bf455996cd05210da81458 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.980758 26754 leader_election.cc:290] T 25be33d2f6bf455996cd05210da81458 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:48.980180 26458 catalog_manager.cc:5697] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2). New cstate: current_term: 1 leader_uuid: "68637fffcc4c4969ae0612be35aa58ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:48.981247 26754 raft_consensus.cc:493] T d8410badcf77497fbb604cf8c185db83 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.981421 26561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be33d2f6bf455996cd05210da81458" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0" is_pre_election: true
I20260709 13:09:48.981674 26754 raft_consensus.cc:515] T d8410badcf77497fbb604cf8c185db83 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.983649 26754 leader_election.cc:290] T d8410badcf77497fbb604cf8c185db83 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:48.983796 26752 tablet_bootstrap.cc:654] T 6c21616d95d04253a63e088134753cea P 0a00d5d20d5b47b99a1220a0e40a4a78: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:48.977610 26749 consensus_queue.cc:260] T 25be33d2f6bf455996cd05210da81458 P dc9195961d41473e8f590b632eb998a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:48.986140 26749 ts_tablet_manager.cc:1434] T 25be33d2f6bf455996cd05210da81458 P dc9195961d41473e8f590b632eb998a0: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.002s
I20260709 13:09:48.987054 26749 tablet_bootstrap.cc:492] T d8410badcf77497fbb604cf8c185db83 P dc9195961d41473e8f590b632eb998a0: Bootstrap starting.
I20260709 13:09:48.987663 26560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8410badcf77497fbb604cf8c185db83" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0" is_pre_election: true
I20260709 13:09:48.991780 26749 tablet_bootstrap.cc:654] T d8410badcf77497fbb604cf8c185db83 P dc9195961d41473e8f590b632eb998a0: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:48.999617 26561 raft_consensus.cc:2468] T 25be33d2f6bf455996cd05210da81458 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 0.
W20260709 13:09:49.000427 26601 leader_election.cc:343] T d8410badcf77497fbb604cf8c185db83 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:49.000931 26749 tablet_bootstrap.cc:492] T d8410badcf77497fbb604cf8c185db83 P dc9195961d41473e8f590b632eb998a0: No bootstrap required, opened a new log
I20260709 13:09:49.001385 26749 ts_tablet_manager.cc:1403] T d8410badcf77497fbb604cf8c185db83 P dc9195961d41473e8f590b632eb998a0: Time spent bootstrapping tablet: real 0.015s	user 0.012s	sys 0.000s
I20260709 13:09:49.002039 26708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8410badcf77497fbb604cf8c185db83" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" is_pre_election: true
W20260709 13:09:49.003895 26601 leader_election.cc:343] T d8410badcf77497fbb604cf8c185db83 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:49.003954 26749 raft_consensus.cc:359] T d8410badcf77497fbb604cf8c185db83 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:49.004408 26601 leader_election.cc:304] T d8410badcf77497fbb604cf8c185db83 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 68637fffcc4c4969ae0612be35aa58ec; no voters: 0a00d5d20d5b47b99a1220a0e40a4a78, dc9195961d41473e8f590b632eb998a0
I20260709 13:09:49.004726 26749 raft_consensus.cc:385] T d8410badcf77497fbb604cf8c185db83 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:49.005168 26754 raft_consensus.cc:2749] T d8410badcf77497fbb604cf8c185db83 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:49.005623 26601 leader_election.cc:304] T 25be33d2f6bf455996cd05210da81458 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68637fffcc4c4969ae0612be35aa58ec, dc9195961d41473e8f590b632eb998a0; no voters: 
I20260709 13:09:49.004998 26749 raft_consensus.cc:740] T d8410badcf77497fbb604cf8c185db83 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9195961d41473e8f590b632eb998a0, State: Initialized, Role: FOLLOWER
I20260709 13:09:49.006376 26754 raft_consensus.cc:2804] T 25be33d2f6bf455996cd05210da81458 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:48.983779 26710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be33d2f6bf455996cd05210da81458" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" is_pre_election: true
I20260709 13:09:49.006872 26754 raft_consensus.cc:493] T 25be33d2f6bf455996cd05210da81458 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:49.007216 26754 raft_consensus.cc:3060] T 25be33d2f6bf455996cd05210da81458 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:49.008279 26760 raft_consensus.cc:493] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:49.008807 26760 raft_consensus.cc:515] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:49.010851 26760 leader_election.cc:290] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285)
I20260709 13:09:49.011261 26752 tablet_bootstrap.cc:492] T 6c21616d95d04253a63e088134753cea P 0a00d5d20d5b47b99a1220a0e40a4a78: No bootstrap required, opened a new log
I20260709 13:09:49.011775 26752 ts_tablet_manager.cc:1403] T 6c21616d95d04253a63e088134753cea P 0a00d5d20d5b47b99a1220a0e40a4a78: Time spent bootstrapping tablet: real 0.035s	user 0.026s	sys 0.006s
W20260709 13:09:49.013049 26601 leader_election.cc:343] T 25be33d2f6bf455996cd05210da81458 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:49.013154 26636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579b5cf4a4874b75a67f9cbf320addc2" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68637fffcc4c4969ae0612be35aa58ec" is_pre_election: true
I20260709 13:09:49.014045 26636 raft_consensus.cc:2393] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in current term 1: Already voted for candidate 68637fffcc4c4969ae0612be35aa58ec in this term.
I20260709 13:09:49.008054 26749 consensus_queue.cc:260] T d8410badcf77497fbb604cf8c185db83 P dc9195961d41473e8f590b632eb998a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:49.015615 26754 raft_consensus.cc:515] T 25be33d2f6bf455996cd05210da81458 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:49.016208 26749 ts_tablet_manager.cc:1434] T d8410badcf77497fbb604cf8c185db83 P dc9195961d41473e8f590b632eb998a0: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:09:49.016987 26752 raft_consensus.cc:359] T 6c21616d95d04253a63e088134753cea P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:49.017755 26752 raft_consensus.cc:385] T 6c21616d95d04253a63e088134753cea P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:49.018050 26752 raft_consensus.cc:740] T 6c21616d95d04253a63e088134753cea P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a00d5d20d5b47b99a1220a0e40a4a78, State: Initialized, Role: FOLLOWER
I20260709 13:09:49.015920 26561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579b5cf4a4874b75a67f9cbf320addc2" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0" is_pre_election: true
I20260709 13:09:49.018702 26560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be33d2f6bf455996cd05210da81458" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0"
I20260709 13:09:49.019040 26561 raft_consensus.cc:2393] T 579b5cf4a4874b75a67f9cbf320addc2 P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in current term 1: Already voted for candidate dc9195961d41473e8f590b632eb998a0 in this term.
I20260709 13:09:49.019321 26560 raft_consensus.cc:3060] T 25be33d2f6bf455996cd05210da81458 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:49.018893 26752 consensus_queue.cc:260] T 6c21616d95d04253a63e088134753cea P 0a00d5d20d5b47b99a1220a0e40a4a78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:49.020349 26675 leader_election.cc:304] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78; no voters: 68637fffcc4c4969ae0612be35aa58ec, dc9195961d41473e8f590b632eb998a0
I20260709 13:09:49.020735 26754 leader_election.cc:290] T 25be33d2f6bf455996cd05210da81458 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 election: Requested vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:49.021371 26752 ts_tablet_manager.cc:1434] T 6c21616d95d04253a63e088134753cea P 0a00d5d20d5b47b99a1220a0e40a4a78: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:09:49.021843 26764 raft_consensus.cc:3060] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:49.022401 26752 tablet_bootstrap.cc:492] T 25be33d2f6bf455996cd05210da81458 P 0a00d5d20d5b47b99a1220a0e40a4a78: Bootstrap starting.
I20260709 13:09:49.024963 26710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be33d2f6bf455996cd05210da81458" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
W20260709 13:09:49.026429 26601 leader_election.cc:343] T 25be33d2f6bf455996cd05210da81458 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:49.026587 26560 raft_consensus.cc:2468] T 25be33d2f6bf455996cd05210da81458 P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 1.
I20260709 13:09:49.027881 26601 leader_election.cc:304] T 25be33d2f6bf455996cd05210da81458 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 68637fffcc4c4969ae0612be35aa58ec, dc9195961d41473e8f590b632eb998a0; no voters: 0a00d5d20d5b47b99a1220a0e40a4a78
I20260709 13:09:49.028342 26752 tablet_bootstrap.cc:654] T 25be33d2f6bf455996cd05210da81458 P 0a00d5d20d5b47b99a1220a0e40a4a78: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:49.029121 26754 raft_consensus.cc:2804] T 25be33d2f6bf455996cd05210da81458 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:49.029264 26764 raft_consensus.cc:2749] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:49.029902 26754 raft_consensus.cc:697] T 25be33d2f6bf455996cd05210da81458 P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Becoming Leader. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Running, Role: LEADER
I20260709 13:09:49.030854 26754 consensus_queue.cc:237] T 25be33d2f6bf455996cd05210da81458 P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:49.032979 26764 raft_consensus.cc:493] T 4af4eae89a594eb0ace69b0895e43372 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:49.033499 26752 tablet_bootstrap.cc:492] T 25be33d2f6bf455996cd05210da81458 P 0a00d5d20d5b47b99a1220a0e40a4a78: No bootstrap required, opened a new log
I20260709 13:09:49.033463 26764 raft_consensus.cc:515] T 4af4eae89a594eb0ace69b0895e43372 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:49.035724 26764 leader_election.cc:290] T 4af4eae89a594eb0ace69b0895e43372 P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285)
I20260709 13:09:49.034035 26752 ts_tablet_manager.cc:1403] T 25be33d2f6bf455996cd05210da81458 P 0a00d5d20d5b47b99a1220a0e40a4a78: Time spent bootstrapping tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 13:09:49.038589 26752 raft_consensus.cc:359] T 25be33d2f6bf455996cd05210da81458 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:49.039204 26752 raft_consensus.cc:385] T 25be33d2f6bf455996cd05210da81458 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:49.039446 26752 raft_consensus.cc:740] T 25be33d2f6bf455996cd05210da81458 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a00d5d20d5b47b99a1220a0e40a4a78, State: Initialized, Role: FOLLOWER
I20260709 13:09:49.040154 26752 consensus_queue.cc:260] T 25be33d2f6bf455996cd05210da81458 P 0a00d5d20d5b47b99a1220a0e40a4a78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:49.041013 26636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af4eae89a594eb0ace69b0895e43372" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68637fffcc4c4969ae0612be35aa58ec" is_pre_election: true
I20260709 13:09:49.041774 26636 raft_consensus.cc:2468] T 4af4eae89a594eb0ace69b0895e43372 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in term 0.
I20260709 13:09:49.042483 26752 ts_tablet_manager.cc:1434] T 25be33d2f6bf455996cd05210da81458 P 0a00d5d20d5b47b99a1220a0e40a4a78: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:09:49.043404 26752 tablet_bootstrap.cc:492] T d8410badcf77497fbb604cf8c185db83 P 0a00d5d20d5b47b99a1220a0e40a4a78: Bootstrap starting.
I20260709 13:09:49.043541 26675 leader_election.cc:304] T 4af4eae89a594eb0ace69b0895e43372 P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78, 68637fffcc4c4969ae0612be35aa58ec; no voters: 
I20260709 13:09:49.044505 26764 raft_consensus.cc:2804] T 4af4eae89a594eb0ace69b0895e43372 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:49.044852 26764 raft_consensus.cc:493] T 4af4eae89a594eb0ace69b0895e43372 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:49.045178 26764 raft_consensus.cc:3060] T 4af4eae89a594eb0ace69b0895e43372 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:49.045104 26560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af4eae89a594eb0ace69b0895e43372" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0" is_pre_election: true
I20260709 13:09:49.045843 26560 raft_consensus.cc:2468] T 4af4eae89a594eb0ace69b0895e43372 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in term 0.
I20260709 13:09:49.047605 26456 catalog_manager.cc:5697] T 25be33d2f6bf455996cd05210da81458 P 68637fffcc4c4969ae0612be35aa58ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2). New cstate: current_term: 1 leader_uuid: "68637fffcc4c4969ae0612be35aa58ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:49.048600 26752 tablet_bootstrap.cc:654] T d8410badcf77497fbb604cf8c185db83 P 0a00d5d20d5b47b99a1220a0e40a4a78: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:49.051601 26764 raft_consensus.cc:515] T 4af4eae89a594eb0ace69b0895e43372 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:49.052177 26760 raft_consensus.cc:493] T 055a26fb2c8b41b4b6da92c7e20180e1 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:49.052685 26760 raft_consensus.cc:515] T 055a26fb2c8b41b4b6da92c7e20180e1 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:49.053825 26752 tablet_bootstrap.cc:492] T d8410badcf77497fbb604cf8c185db83 P 0a00d5d20d5b47b99a1220a0e40a4a78: No bootstrap required, opened a new log
I20260709 13:09:49.054370 26752 ts_tablet_manager.cc:1403] T d8410badcf77497fbb604cf8c185db83 P 0a00d5d20d5b47b99a1220a0e40a4a78: Time spent bootstrapping tablet: real 0.011s	user 0.009s	sys 0.001s
I20260709 13:09:49.054457 26560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af4eae89a594eb0ace69b0895e43372" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0"
I20260709 13:09:49.054675 26760 leader_election.cc:290] T 055a26fb2c8b41b4b6da92c7e20180e1 P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285)
I20260709 13:09:49.055100 26560 raft_consensus.cc:3060] T 4af4eae89a594eb0ace69b0895e43372 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:49.056645 26561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055a26fb2c8b41b4b6da92c7e20180e1" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0" is_pre_election: true
I20260709 13:09:49.057422 26561 raft_consensus.cc:2393] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in current term 1: Already voted for candidate dc9195961d41473e8f590b632eb998a0 in this term.
I20260709 13:09:49.057310 26752 raft_consensus.cc:359] T d8410badcf77497fbb604cf8c185db83 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:49.058184 26752 raft_consensus.cc:385] T d8410badcf77497fbb604cf8c185db83 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:49.058511 26752 raft_consensus.cc:740] T d8410badcf77497fbb604cf8c185db83 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a00d5d20d5b47b99a1220a0e40a4a78, State: Initialized, Role: FOLLOWER
I20260709 13:09:49.058672 26636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055a26fb2c8b41b4b6da92c7e20180e1" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68637fffcc4c4969ae0612be35aa58ec" is_pre_election: true
I20260709 13:09:49.059427 26636 raft_consensus.cc:2393] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in current term 1: Already voted for candidate 68637fffcc4c4969ae0612be35aa58ec in this term.
I20260709 13:09:49.059234 26752 consensus_queue.cc:260] T d8410badcf77497fbb604cf8c185db83 P 0a00d5d20d5b47b99a1220a0e40a4a78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:49.060702 26675 leader_election.cc:304] T 055a26fb2c8b41b4b6da92c7e20180e1 P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78; no voters: 68637fffcc4c4969ae0612be35aa58ec, dc9195961d41473e8f590b632eb998a0
I20260709 13:09:49.061282 26752 ts_tablet_manager.cc:1434] T d8410badcf77497fbb604cf8c185db83 P 0a00d5d20d5b47b99a1220a0e40a4a78: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:09:49.061713 26760 raft_consensus.cc:3060] T 055a26fb2c8b41b4b6da92c7e20180e1 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:49.062496 26764 leader_election.cc:290] T 4af4eae89a594eb0ace69b0895e43372 P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 1 election: Requested vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285)
I20260709 13:09:49.064996 26636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af4eae89a594eb0ace69b0895e43372" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68637fffcc4c4969ae0612be35aa58ec"
I20260709 13:09:49.065598 26636 raft_consensus.cc:3060] T 4af4eae89a594eb0ace69b0895e43372 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:49.066818 26560 raft_consensus.cc:2468] T 4af4eae89a594eb0ace69b0895e43372 P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in term 1.
I20260709 13:09:49.068029 26675 leader_election.cc:304] T 4af4eae89a594eb0ace69b0895e43372 P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78, dc9195961d41473e8f590b632eb998a0; no voters: 
I20260709 13:09:49.068467 26760 raft_consensus.cc:2749] T 055a26fb2c8b41b4b6da92c7e20180e1 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:49.069036 26764 raft_consensus.cc:2804] T 4af4eae89a594eb0ace69b0895e43372 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:49.069573 26764 raft_consensus.cc:697] T 4af4eae89a594eb0ace69b0895e43372 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 LEADER]: Becoming Leader. State: Replica: 0a00d5d20d5b47b99a1220a0e40a4a78, State: Running, Role: LEADER
I20260709 13:09:49.070396 26764 consensus_queue.cc:237] T 4af4eae89a594eb0ace69b0895e43372 P 0a00d5d20d5b47b99a1220a0e40a4a78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:49.072860 26636 raft_consensus.cc:2468] T 4af4eae89a594eb0ace69b0895e43372 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in term 1.
I20260709 13:09:49.078263 26458 catalog_manager.cc:5697] T 4af4eae89a594eb0ace69b0895e43372 P 0a00d5d20d5b47b99a1220a0e40a4a78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3). New cstate: current_term: 1 leader_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:49.172318 26764 raft_consensus.cc:493] T 6c21616d95d04253a63e088134753cea P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:49.172894 26764 raft_consensus.cc:515] T 6c21616d95d04253a63e088134753cea P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:49.174736 26764 leader_election.cc:290] T 6c21616d95d04253a63e088134753cea P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285)
I20260709 13:09:49.175420 26560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c21616d95d04253a63e088134753cea" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0" is_pre_election: true
I20260709 13:09:49.175889 26636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c21616d95d04253a63e088134753cea" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68637fffcc4c4969ae0612be35aa58ec" is_pre_election: true
I20260709 13:09:49.176374 26560 raft_consensus.cc:2393] T 6c21616d95d04253a63e088134753cea P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in current term 1: Already voted for candidate 68637fffcc4c4969ae0612be35aa58ec in this term.
I20260709 13:09:49.177502 26675 leader_election.cc:304] T 6c21616d95d04253a63e088134753cea P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78; no voters: 68637fffcc4c4969ae0612be35aa58ec, dc9195961d41473e8f590b632eb998a0
I20260709 13:09:49.178148 26764 raft_consensus.cc:3060] T 6c21616d95d04253a63e088134753cea P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:49.183331 26764 raft_consensus.cc:2749] T 6c21616d95d04253a63e088134753cea P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:49.185603 26764 raft_consensus.cc:493] T 25be33d2f6bf455996cd05210da81458 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:49.186028 26764 raft_consensus.cc:515] T 25be33d2f6bf455996cd05210da81458 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:49.187963 26764 leader_election.cc:290] T 25be33d2f6bf455996cd05210da81458 P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285)
I20260709 13:09:49.188671 26560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be33d2f6bf455996cd05210da81458" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0" is_pre_election: true
I20260709 13:09:49.188968 26636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25be33d2f6bf455996cd05210da81458" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68637fffcc4c4969ae0612be35aa58ec" is_pre_election: true
I20260709 13:09:49.189329 26560 raft_consensus.cc:2393] T 25be33d2f6bf455996cd05210da81458 P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in current term 1: Already voted for candidate 68637fffcc4c4969ae0612be35aa58ec in this term.
I20260709 13:09:49.190404 26675 leader_election.cc:304] T 25be33d2f6bf455996cd05210da81458 P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78; no voters: 68637fffcc4c4969ae0612be35aa58ec, dc9195961d41473e8f590b632eb998a0
I20260709 13:09:49.190958 26764 raft_consensus.cc:3060] T 25be33d2f6bf455996cd05210da81458 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:49.195458 26764 raft_consensus.cc:2749] T 25be33d2f6bf455996cd05210da81458 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:49.230530 26755 raft_consensus.cc:493] T b8f3b339b6bd4573a9f7929c4d130af5 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:49.231067 26755 raft_consensus.cc:515] T b8f3b339b6bd4573a9f7929c4d130af5 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:49.233089 26755 leader_election.cc:290] T b8f3b339b6bd4573a9f7929c4d130af5 P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:49.233920 26636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f3b339b6bd4573a9f7929c4d130af5" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68637fffcc4c4969ae0612be35aa58ec" is_pre_election: true
I20260709 13:09:49.234076 26710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f3b339b6bd4573a9f7929c4d130af5" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" is_pre_election: true
I20260709 13:09:49.234748 26636 raft_consensus.cc:2393] T b8f3b339b6bd4573a9f7929c4d130af5 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dc9195961d41473e8f590b632eb998a0 in current term 1: Already voted for candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in this term.
I20260709 13:09:49.235615 26526 leader_election.cc:304] T b8f3b339b6bd4573a9f7929c4d130af5 P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dc9195961d41473e8f590b632eb998a0; no voters: 0a00d5d20d5b47b99a1220a0e40a4a78, 68637fffcc4c4969ae0612be35aa58ec
I20260709 13:09:49.236265 26755 raft_consensus.cc:3060] T b8f3b339b6bd4573a9f7929c4d130af5 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:49.241201 26755 raft_consensus.cc:2749] T b8f3b339b6bd4573a9f7929c4d130af5 P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:49.270711 26755 raft_consensus.cc:493] T d8410badcf77497fbb604cf8c185db83 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:49.271198 26755 raft_consensus.cc:515] T d8410badcf77497fbb604cf8c185db83 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:49.273005 26755 leader_election.cc:290] T d8410badcf77497fbb604cf8c185db83 P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:49.273811 26636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8410badcf77497fbb604cf8c185db83" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68637fffcc4c4969ae0612be35aa58ec" is_pre_election: true
I20260709 13:09:49.273972 26710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8410badcf77497fbb604cf8c185db83" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" is_pre_election: true
I20260709 13:09:49.274413 26636 raft_consensus.cc:2468] T d8410badcf77497fbb604cf8c185db83 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc9195961d41473e8f590b632eb998a0 in term 0.
I20260709 13:09:49.274636 26710 raft_consensus.cc:2468] T d8410badcf77497fbb604cf8c185db83 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc9195961d41473e8f590b632eb998a0 in term 0.
I20260709 13:09:49.275509 26526 leader_election.cc:304] T d8410badcf77497fbb604cf8c185db83 P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68637fffcc4c4969ae0612be35aa58ec, dc9195961d41473e8f590b632eb998a0; no voters: 
I20260709 13:09:49.276185 26755 raft_consensus.cc:2804] T d8410badcf77497fbb604cf8c185db83 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:49.276537 26755 raft_consensus.cc:493] T d8410badcf77497fbb604cf8c185db83 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:49.276810 26755 raft_consensus.cc:3060] T d8410badcf77497fbb604cf8c185db83 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:49.281497 26755 raft_consensus.cc:515] T d8410badcf77497fbb604cf8c185db83 P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:49.282948 26755 leader_election.cc:290] T d8410badcf77497fbb604cf8c185db83 P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 1 election: Requested vote from peers 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:49.283710 26636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8410badcf77497fbb604cf8c185db83" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68637fffcc4c4969ae0612be35aa58ec"
I20260709 13:09:49.283950 26710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8410badcf77497fbb604cf8c185db83" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
I20260709 13:09:49.284370 26636 raft_consensus.cc:3060] T d8410badcf77497fbb604cf8c185db83 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:49.284524 26710 raft_consensus.cc:3060] T d8410badcf77497fbb604cf8c185db83 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:49.290397 26754 consensus_queue.cc:1048] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:49.291188 26636 raft_consensus.cc:2468] T d8410badcf77497fbb604cf8c185db83 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9195961d41473e8f590b632eb998a0 in term 1.
I20260709 13:09:49.291188 26710 raft_consensus.cc:2468] T d8410badcf77497fbb604cf8c185db83 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9195961d41473e8f590b632eb998a0 in term 1.
I20260709 13:09:49.292465 26526 leader_election.cc:304] T d8410badcf77497fbb604cf8c185db83 P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68637fffcc4c4969ae0612be35aa58ec, dc9195961d41473e8f590b632eb998a0; no voters: 
I20260709 13:09:49.293618 26755 raft_consensus.cc:2804] T d8410badcf77497fbb604cf8c185db83 P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:49.294063 26755 raft_consensus.cc:697] T d8410badcf77497fbb604cf8c185db83 P dc9195961d41473e8f590b632eb998a0 [term 1 LEADER]: Becoming Leader. State: Replica: dc9195961d41473e8f590b632eb998a0, State: Running, Role: LEADER
I20260709 13:09:49.294984 26755 consensus_queue.cc:237] T d8410badcf77497fbb604cf8c185db83 P dc9195961d41473e8f590b632eb998a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:49.308354 26458 catalog_manager.cc:5697] T d8410badcf77497fbb604cf8c185db83 P dc9195961d41473e8f590b632eb998a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc9195961d41473e8f590b632eb998a0 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "dc9195961d41473e8f590b632eb998a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:49.309599 26754 consensus_queue.cc:1048] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:49.331605 26764 consensus_queue.cc:1048] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:49.335438 26754 consensus_queue.cc:1048] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:49.343355 26760 consensus_queue.cc:1048] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:49.363059 26754 consensus_queue.cc:1048] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:49.413921 26754 consensus_queue.cc:1048] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:49.431726 26776 consensus_queue.cc:1048] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:49.448510 26789 consensus_queue.cc:1048] T 25be33d2f6bf455996cd05210da81458 P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:49.462466 26789 consensus_queue.cc:1048] T 25be33d2f6bf455996cd05210da81458 P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:49.584270 26760 consensus_queue.cc:1048] T 4af4eae89a594eb0ace69b0895e43372 P 0a00d5d20d5b47b99a1220a0e40a4a78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:49.603296 26760 consensus_queue.cc:1048] T 4af4eae89a594eb0ace69b0895e43372 P 0a00d5d20d5b47b99a1220a0e40a4a78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:49.812553 26755 consensus_queue.cc:1048] T d8410badcf77497fbb604cf8c185db83 P dc9195961d41473e8f590b632eb998a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:49.827551 26822 consensus_queue.cc:1048] T d8410badcf77497fbb604cf8c185db83 P dc9195961d41473e8f590b632eb998a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:09:49.838583 26507 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:50.625083 26789 raft_consensus.cc:493] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:50.625787 26789 raft_consensus.cc:515] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:50.628098 26789 leader_election.cc:290] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:50.629343 26561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055a26fb2c8b41b4b6da92c7e20180e1" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0" is_pre_election: true
I20260709 13:09:50.629827 26708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055a26fb2c8b41b4b6da92c7e20180e1" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" is_pre_election: true
I20260709 13:09:50.630326 26561 raft_consensus.cc:2468] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 1.
I20260709 13:09:50.630693 26708 raft_consensus.cc:2468] T 055a26fb2c8b41b4b6da92c7e20180e1 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 1.
I20260709 13:09:50.631942 26601 leader_election.cc:304] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68637fffcc4c4969ae0612be35aa58ec, dc9195961d41473e8f590b632eb998a0; no voters: 
I20260709 13:09:50.632615 26789 raft_consensus.cc:2804] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:50.632965 26789 raft_consensus.cc:493] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:50.633268 26789 raft_consensus.cc:3060] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:50.640333 26789 raft_consensus.cc:515] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:50.641963 26789 leader_election.cc:290] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 2 election: Requested vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:50.642680 26561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055a26fb2c8b41b4b6da92c7e20180e1" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0"
I20260709 13:09:50.642901 26708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055a26fb2c8b41b4b6da92c7e20180e1" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
I20260709 13:09:50.643224 26561 raft_consensus.cc:3060] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:50.643548 26708 raft_consensus.cc:3060] T 055a26fb2c8b41b4b6da92c7e20180e1 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:50.649750 26708 raft_consensus.cc:2468] T 055a26fb2c8b41b4b6da92c7e20180e1 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 2.
I20260709 13:09:50.649870 26561 raft_consensus.cc:2468] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 2.
I20260709 13:09:50.650753 26601 leader_election.cc:304] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78, 68637fffcc4c4969ae0612be35aa58ec; no voters: 
I20260709 13:09:50.651372 26789 raft_consensus.cc:2804] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:50.652212 26789 raft_consensus.cc:697] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [term 2 LEADER]: Becoming Leader. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Running, Role: LEADER
I20260709 13:09:50.653210 26789 consensus_queue.cc:237] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:50.660354 26457 catalog_manager.cc:5697] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec reported cstate change: term changed from 0 to 2, leader changed from <none> to 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2). New cstate: current_term: 2 leader_uuid: "68637fffcc4c4969ae0612be35aa58ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:50.717347 26819 raft_consensus.cc:493] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:50.718075 26819 raft_consensus.cc:515] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:50.720031 26819 leader_election.cc:290] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285)
I20260709 13:09:50.720829 26561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579b5cf4a4874b75a67f9cbf320addc2" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0" is_pre_election: true
I20260709 13:09:50.721180 26636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579b5cf4a4874b75a67f9cbf320addc2" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68637fffcc4c4969ae0612be35aa58ec" is_pre_election: true
I20260709 13:09:50.721570 26561 raft_consensus.cc:2468] T 579b5cf4a4874b75a67f9cbf320addc2 P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in term 1.
I20260709 13:09:50.721903 26636 raft_consensus.cc:2468] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in term 1.
I20260709 13:09:50.722764 26675 leader_election.cc:304] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78, dc9195961d41473e8f590b632eb998a0; no voters: 
I20260709 13:09:50.723546 26819 raft_consensus.cc:2804] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:50.723855 26819 raft_consensus.cc:493] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:50.724123 26819 raft_consensus.cc:3060] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:50.730664 26819 raft_consensus.cc:515] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:50.732612 26819 leader_election.cc:290] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 2 election: Requested vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285)
I20260709 13:09:50.733464 26561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579b5cf4a4874b75a67f9cbf320addc2" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0"
I20260709 13:09:50.733664 26636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579b5cf4a4874b75a67f9cbf320addc2" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68637fffcc4c4969ae0612be35aa58ec"
I20260709 13:09:50.734014 26561 raft_consensus.cc:3060] T 579b5cf4a4874b75a67f9cbf320addc2 P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:50.734263 26636 raft_consensus.cc:3060] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:50.739530 26561 raft_consensus.cc:2468] T 579b5cf4a4874b75a67f9cbf320addc2 P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in term 2.
I20260709 13:09:50.740499 26675 leader_election.cc:304] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78, dc9195961d41473e8f590b632eb998a0; no voters: 
I20260709 13:09:50.740561 26636 raft_consensus.cc:2468] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in term 2.
I20260709 13:09:50.741182 26819 raft_consensus.cc:2804] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:50.742270 26819 raft_consensus.cc:697] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 LEADER]: Becoming Leader. State: Replica: 0a00d5d20d5b47b99a1220a0e40a4a78, State: Running, Role: LEADER
I20260709 13:09:50.743131 26819 consensus_queue.cc:237] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:50.749748 26457 catalog_manager.cc:5697] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 reported cstate change: term changed from 0 to 2, leader changed from <none> to 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3). New cstate: current_term: 2 leader_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:50.968052 26636 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b16a06ac45ab44a186fbf820559732bb"
dest_uuid: "68637fffcc4c4969ae0612be35aa58ec"
mode: GRACEFUL
new_leader_uuid: "dc9195961d41473e8f590b632eb998a0"
 from {username='slave'} at 127.0.0.1:45340
I20260709 13:09:50.968638 26636 raft_consensus.cc:606] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Received request to transfer leadership to TS dc9195961d41473e8f590b632eb998a0
I20260709 13:09:50.973680 26636 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "055a26fb2c8b41b4b6da92c7e20180e1"
dest_uuid: "68637fffcc4c4969ae0612be35aa58ec"
mode: GRACEFUL
new_leader_uuid: "dc9195961d41473e8f590b632eb998a0"
 from {username='slave'} at 127.0.0.1:45340
I20260709 13:09:50.974161 26636 raft_consensus.cc:606] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [term 2 LEADER]: Received request to transfer leadership to TS dc9195961d41473e8f590b632eb998a0
I20260709 13:09:50.978539 26636 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef38f875bfdd4b669c39d02f0ea7472b"
dest_uuid: "68637fffcc4c4969ae0612be35aa58ec"
mode: GRACEFUL
new_leader_uuid: "dc9195961d41473e8f590b632eb998a0"
 from {username='slave'} at 127.0.0.1:45340
I20260709 13:09:50.979049 26636 raft_consensus.cc:606] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Received request to transfer leadership to TS dc9195961d41473e8f590b632eb998a0
I20260709 13:09:50.983335 26636 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6c21616d95d04253a63e088134753cea"
dest_uuid: "68637fffcc4c4969ae0612be35aa58ec"
mode: GRACEFUL
new_leader_uuid: "dc9195961d41473e8f590b632eb998a0"
 from {username='slave'} at 127.0.0.1:45340
I20260709 13:09:50.983814 26636 raft_consensus.cc:606] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Received request to transfer leadership to TS dc9195961d41473e8f590b632eb998a0
I20260709 13:09:50.996361 26708 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "579b5cf4a4874b75a67f9cbf320addc2"
dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
mode: GRACEFUL
new_leader_uuid: "68637fffcc4c4969ae0612be35aa58ec"
 from {username='slave'} at 127.0.0.1:57210
I20260709 13:09:50.996901 26708 raft_consensus.cc:606] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 LEADER]: Received request to transfer leadership to TS 68637fffcc4c4969ae0612be35aa58ec
I20260709 13:09:51.001899 26708 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b8f3b339b6bd4573a9f7929c4d130af5"
dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
mode: GRACEFUL
new_leader_uuid: "68637fffcc4c4969ae0612be35aa58ec"
 from {username='slave'} at 127.0.0.1:57210
I20260709 13:09:51.002365 26708 raft_consensus.cc:606] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 LEADER]: Received request to transfer leadership to TS 68637fffcc4c4969ae0612be35aa58ec
I20260709 13:09:51.007531 26708 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4af4eae89a594eb0ace69b0895e43372"
dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
mode: GRACEFUL
new_leader_uuid: "68637fffcc4c4969ae0612be35aa58ec"
 from {username='slave'} at 127.0.0.1:57210
I20260709 13:09:51.007999 26708 raft_consensus.cc:606] T 4af4eae89a594eb0ace69b0895e43372 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 LEADER]: Received request to transfer leadership to TS 68637fffcc4c4969ae0612be35aa58ec
I20260709 13:09:51.020972 26832 raft_consensus.cc:993] T 4af4eae89a594eb0ace69b0895e43372 P 0a00d5d20d5b47b99a1220a0e40a4a78: : Instructing follower 68637fffcc4c4969ae0612be35aa58ec to start an election
I20260709 13:09:51.021441 26819 raft_consensus.cc:1081] T 4af4eae89a594eb0ace69b0895e43372 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 LEADER]: Signalling peer 68637fffcc4c4969ae0612be35aa58ec to start an election
I20260709 13:09:51.023054 26636 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4af4eae89a594eb0ace69b0895e43372"
dest_uuid: "68637fffcc4c4969ae0612be35aa58ec"
 from {username='slave'} at 127.0.0.1:45328
I20260709 13:09:51.023772 26636 raft_consensus.cc:493] T 4af4eae89a594eb0ace69b0895e43372 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:51.024116 26636 raft_consensus.cc:3060] T 4af4eae89a594eb0ace69b0895e43372 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:51.025190 26561 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8410badcf77497fbb604cf8c185db83"
dest_uuid: "dc9195961d41473e8f590b632eb998a0"
mode: GRACEFUL
new_leader_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
 from {username='slave'} at 127.0.0.1:58516
I20260709 13:09:51.025722 26561 raft_consensus.cc:606] T d8410badcf77497fbb604cf8c185db83 P dc9195961d41473e8f590b632eb998a0 [term 1 LEADER]: Received request to transfer leadership to TS 0a00d5d20d5b47b99a1220a0e40a4a78
I20260709 13:09:51.029780 26636 raft_consensus.cc:515] T 4af4eae89a594eb0ace69b0895e43372 P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:51.032367 26561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af4eae89a594eb0ace69b0895e43372" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dc9195961d41473e8f590b632eb998a0"
I20260709 13:09:51.032793 26636 leader_election.cc:290] T 4af4eae89a594eb0ace69b0895e43372 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 2 election: Requested vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:51.032996 26561 raft_consensus.cc:3060] T 4af4eae89a594eb0ace69b0895e43372 P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:51.033214 26708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af4eae89a594eb0ace69b0895e43372" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
I20260709 13:09:51.033900 26708 raft_consensus.cc:3055] T 4af4eae89a594eb0ace69b0895e43372 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:51.034221 26708 raft_consensus.cc:740] T 4af4eae89a594eb0ace69b0895e43372 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a00d5d20d5b47b99a1220a0e40a4a78, State: Running, Role: LEADER
I20260709 13:09:51.035127 26708 consensus_queue.cc:260] T 4af4eae89a594eb0ace69b0895e43372 P 0a00d5d20d5b47b99a1220a0e40a4a78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:51.036118 26708 raft_consensus.cc:3060] T 4af4eae89a594eb0ace69b0895e43372 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:51.039440 26561 raft_consensus.cc:2468] T 4af4eae89a594eb0ace69b0895e43372 P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 2.
I20260709 13:09:51.040431 26601 leader_election.cc:304] T 4af4eae89a594eb0ace69b0895e43372 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68637fffcc4c4969ae0612be35aa58ec, dc9195961d41473e8f590b632eb998a0; no voters: 
I20260709 13:09:51.040961 26708 raft_consensus.cc:2468] T 4af4eae89a594eb0ace69b0895e43372 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 2.
I20260709 13:09:51.041113 26789 raft_consensus.cc:2804] T 4af4eae89a594eb0ace69b0895e43372 P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:51.041651 26789 raft_consensus.cc:697] T 4af4eae89a594eb0ace69b0895e43372 P 68637fffcc4c4969ae0612be35aa58ec [term 2 LEADER]: Becoming Leader. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Running, Role: LEADER
I20260709 13:09:51.042438 26789 consensus_queue.cc:237] T 4af4eae89a594eb0ace69b0895e43372 P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:51.049118 26457 catalog_manager.cc:5697] T 4af4eae89a594eb0ace69b0895e43372 P 68637fffcc4c4969ae0612be35aa58ec reported cstate change: term changed from 1 to 2, leader changed from 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3) to 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2). New cstate: current_term: 2 leader_uuid: "68637fffcc4c4969ae0612be35aa58ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:51.214897 26561 raft_consensus.cc:1275] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Refusing update from remote peer 68637fffcc4c4969ae0612be35aa58ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:09:51.216207 26789 consensus_queue.cc:1048] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:51.223091 26826 raft_consensus.cc:993] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec: : Instructing follower dc9195961d41473e8f590b632eb998a0 to start an election
I20260709 13:09:51.223826 26789 raft_consensus.cc:1081] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [term 2 LEADER]: Signalling peer dc9195961d41473e8f590b632eb998a0 to start an election
I20260709 13:09:51.226776 26560 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "055a26fb2c8b41b4b6da92c7e20180e1"
dest_uuid: "dc9195961d41473e8f590b632eb998a0"
 from {username='slave'} at 127.0.0.1:58510
I20260709 13:09:51.228401 26560 raft_consensus.cc:493] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:51.228776 26560 raft_consensus.cc:3060] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:51.231374 26708 raft_consensus.cc:1275] T 055a26fb2c8b41b4b6da92c7e20180e1 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Refusing update from remote peer 68637fffcc4c4969ae0612be35aa58ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:09:51.232810 26840 consensus_queue.cc:1048] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:51.239439 26560 raft_consensus.cc:515] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:51.242062 26560 leader_election.cc:290] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 3 election: Requested vote from peers 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:51.243256 26636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055a26fb2c8b41b4b6da92c7e20180e1" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "68637fffcc4c4969ae0612be35aa58ec"
I20260709 13:09:51.243479 26708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055a26fb2c8b41b4b6da92c7e20180e1" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
I20260709 13:09:51.243935 26636 raft_consensus.cc:3055] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:51.244033 26708 raft_consensus.cc:3060] T 055a26fb2c8b41b4b6da92c7e20180e1 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:51.244359 26636 raft_consensus.cc:740] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Running, Role: LEADER
I20260709 13:09:51.245234 26636 consensus_queue.cc:260] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:51.246573 26636 raft_consensus.cc:3060] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:51.251614 26708 raft_consensus.cc:2468] T 055a26fb2c8b41b4b6da92c7e20180e1 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9195961d41473e8f590b632eb998a0 in term 3.
I20260709 13:09:51.254252 26526 leader_election.cc:304] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78, dc9195961d41473e8f590b632eb998a0; no voters: 
I20260709 13:09:51.254735 26636 raft_consensus.cc:2468] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9195961d41473e8f590b632eb998a0 in term 3.
I20260709 13:09:51.255155 26825 raft_consensus.cc:2804] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:51.255511 26633 raft_consensus.cc:1275] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Refusing update from remote peer 0a00d5d20d5b47b99a1220a0e40a4a78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:09:51.257359 26825 raft_consensus.cc:697] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0 [term 3 LEADER]: Becoming Leader. State: Replica: dc9195961d41473e8f590b632eb998a0, State: Running, Role: LEADER
I20260709 13:09:51.257293 26819 consensus_queue.cc:1048] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:51.258512 26825 consensus_queue.cc:237] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:51.264920 26832 raft_consensus.cc:993] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78: : Instructing follower 68637fffcc4c4969ae0612be35aa58ec to start an election
I20260709 13:09:51.265697 26819 raft_consensus.cc:1081] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 LEADER]: Signalling peer 68637fffcc4c4969ae0612be35aa58ec to start an election
I20260709 13:09:51.267976 26633 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "579b5cf4a4874b75a67f9cbf320addc2"
dest_uuid: "68637fffcc4c4969ae0612be35aa58ec"
 from {username='slave'} at 127.0.0.1:45328
I20260709 13:09:51.269317 26633 raft_consensus.cc:493] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:51.269726 26633 raft_consensus.cc:3060] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:51.271320 26456 catalog_manager.cc:5697] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0 reported cstate change: term changed from 2 to 3, leader changed from 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2) to dc9195961d41473e8f590b632eb998a0 (127.14.103.1). New cstate: current_term: 3 leader_uuid: "dc9195961d41473e8f590b632eb998a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:51.279393 26633 raft_consensus.cc:515] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:51.280444 26560 raft_consensus.cc:1275] T 579b5cf4a4874b75a67f9cbf320addc2 P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Refusing update from remote peer 0a00d5d20d5b47b99a1220a0e40a4a78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:09:51.282492 26633 leader_election.cc:290] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 3 election: Requested vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:51.283649 26708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579b5cf4a4874b75a67f9cbf320addc2" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
I20260709 13:09:51.284276 26708 raft_consensus.cc:3055] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:51.284569 26708 raft_consensus.cc:740] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0a00d5d20d5b47b99a1220a0e40a4a78, State: Running, Role: LEADER
I20260709 13:09:51.285353 26708 consensus_queue.cc:260] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:51.285856 26560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579b5cf4a4874b75a67f9cbf320addc2" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "dc9195961d41473e8f590b632eb998a0"
I20260709 13:09:51.286477 26560 raft_consensus.cc:3060] T 579b5cf4a4874b75a67f9cbf320addc2 P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:51.286496 26708 raft_consensus.cc:3060] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:09:51.286584 26819 consensus_queue.cc:1175] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "dc9195961d41473e8f590b632eb998a0" responder_term: 2 status { last_received { term: 0 index: 0 } last_committed_idx: 0 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:09:51.293475 26560 raft_consensus.cc:2468] T 579b5cf4a4874b75a67f9cbf320addc2 P dc9195961d41473e8f590b632eb998a0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 3.
I20260709 13:09:51.293464 26708 raft_consensus.cc:2468] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 3.
I20260709 13:09:51.294611 26601 leader_election.cc:304] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78, 68637fffcc4c4969ae0612be35aa58ec; no voters: 
I20260709 13:09:51.295311 26840 raft_consensus.cc:2804] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:51.295933 26840 raft_consensus.cc:697] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [term 3 LEADER]: Becoming Leader. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Running, Role: LEADER
I20260709 13:09:51.296782 26840 consensus_queue.cc:237] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:51.303179 26456 catalog_manager.cc:5697] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec reported cstate change: term changed from 2 to 3, leader changed from 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3) to 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2). New cstate: current_term: 3 leader_uuid: "68637fffcc4c4969ae0612be35aa58ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:51.309340 26826 raft_consensus.cc:993] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec: : Instructing follower dc9195961d41473e8f590b632eb998a0 to start an election
I20260709 13:09:51.309849 26840 raft_consensus.cc:1081] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Signalling peer dc9195961d41473e8f590b632eb998a0 to start an election
I20260709 13:09:51.311576 26560 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6c21616d95d04253a63e088134753cea"
dest_uuid: "dc9195961d41473e8f590b632eb998a0"
 from {username='slave'} at 127.0.0.1:58510
I20260709 13:09:51.312098 26560 raft_consensus.cc:493] T 6c21616d95d04253a63e088134753cea P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:51.312413 26560 raft_consensus.cc:3060] T 6c21616d95d04253a63e088134753cea P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:51.317355 26560 raft_consensus.cc:515] T 6c21616d95d04253a63e088134753cea P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:51.319001 26560 leader_election.cc:290] T 6c21616d95d04253a63e088134753cea P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 2 election: Requested vote from peers 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:51.319816 26633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c21616d95d04253a63e088134753cea" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "68637fffcc4c4969ae0612be35aa58ec"
I20260709 13:09:51.320117 26708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c21616d95d04253a63e088134753cea" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
I20260709 13:09:51.320439 26633 raft_consensus.cc:3055] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:51.320676 26708 raft_consensus.cc:3060] T 6c21616d95d04253a63e088134753cea P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:51.320822 26633 raft_consensus.cc:740] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Running, Role: LEADER
I20260709 13:09:51.321746 26633 consensus_queue.cc:260] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:51.322715 26633 raft_consensus.cc:3060] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:51.325927 26708 raft_consensus.cc:2468] T 6c21616d95d04253a63e088134753cea P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9195961d41473e8f590b632eb998a0 in term 2.
I20260709 13:09:51.326925 26526 leader_election.cc:304] T 6c21616d95d04253a63e088134753cea P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78, dc9195961d41473e8f590b632eb998a0; no voters: 
I20260709 13:09:51.327653 26825 raft_consensus.cc:2804] T 6c21616d95d04253a63e088134753cea P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:51.327744 26633 raft_consensus.cc:2468] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9195961d41473e8f590b632eb998a0 in term 2.
I20260709 13:09:51.328416 26825 raft_consensus.cc:697] T 6c21616d95d04253a63e088134753cea P dc9195961d41473e8f590b632eb998a0 [term 2 LEADER]: Becoming Leader. State: Replica: dc9195961d41473e8f590b632eb998a0, State: Running, Role: LEADER
I20260709 13:09:51.329444 26825 consensus_queue.cc:237] T 6c21616d95d04253a63e088134753cea P dc9195961d41473e8f590b632eb998a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:51.337054 26456 catalog_manager.cc:5697] T 6c21616d95d04253a63e088134753cea P dc9195961d41473e8f590b632eb998a0 reported cstate change: term changed from 1 to 2, leader changed from 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2) to dc9195961d41473e8f590b632eb998a0 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "dc9195961d41473e8f590b632eb998a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:51.352771 26845 raft_consensus.cc:993] T d8410badcf77497fbb604cf8c185db83 P dc9195961d41473e8f590b632eb998a0: : Instructing follower 0a00d5d20d5b47b99a1220a0e40a4a78 to start an election
I20260709 13:09:51.353248 26825 raft_consensus.cc:1081] T d8410badcf77497fbb604cf8c185db83 P dc9195961d41473e8f590b632eb998a0 [term 1 LEADER]: Signalling peer 0a00d5d20d5b47b99a1220a0e40a4a78 to start an election
I20260709 13:09:51.354565 26708 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d8410badcf77497fbb604cf8c185db83"
dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
 from {username='slave'} at 127.0.0.1:57200
I20260709 13:09:51.354969 26708 raft_consensus.cc:493] T d8410badcf77497fbb604cf8c185db83 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:51.355224 26708 raft_consensus.cc:3060] T d8410badcf77497fbb604cf8c185db83 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:51.360209 26708 raft_consensus.cc:515] T d8410badcf77497fbb604cf8c185db83 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:51.362160 26708 leader_election.cc:290] T d8410badcf77497fbb604cf8c185db83 P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 2 election: Requested vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285)
I20260709 13:09:51.363209 26561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8410badcf77497fbb604cf8c185db83" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dc9195961d41473e8f590b632eb998a0"
I20260709 13:09:51.363642 26633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8410badcf77497fbb604cf8c185db83" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "68637fffcc4c4969ae0612be35aa58ec"
I20260709 13:09:51.363821 26561 raft_consensus.cc:3055] T d8410badcf77497fbb604cf8c185db83 P dc9195961d41473e8f590b632eb998a0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:51.364197 26561 raft_consensus.cc:740] T d8410badcf77497fbb604cf8c185db83 P dc9195961d41473e8f590b632eb998a0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dc9195961d41473e8f590b632eb998a0, State: Running, Role: LEADER
I20260709 13:09:51.364226 26633 raft_consensus.cc:3060] T d8410badcf77497fbb604cf8c185db83 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:51.364979 26561 consensus_queue.cc:260] T d8410badcf77497fbb604cf8c185db83 P dc9195961d41473e8f590b632eb998a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:51.365906 26561 raft_consensus.cc:3060] T d8410badcf77497fbb604cf8c185db83 P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:51.369159 26633 raft_consensus.cc:2468] T d8410badcf77497fbb604cf8c185db83 P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in term 2.
I20260709 13:09:51.370150 26675 leader_election.cc:304] T d8410badcf77497fbb604cf8c185db83 P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78, 68637fffcc4c4969ae0612be35aa58ec; no voters: 
I20260709 13:09:51.370867 26819 raft_consensus.cc:2804] T d8410badcf77497fbb604cf8c185db83 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:51.371292 26819 raft_consensus.cc:697] T d8410badcf77497fbb604cf8c185db83 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 LEADER]: Becoming Leader. State: Replica: 0a00d5d20d5b47b99a1220a0e40a4a78, State: Running, Role: LEADER
I20260709 13:09:51.371619 26561 raft_consensus.cc:2468] T d8410badcf77497fbb604cf8c185db83 P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in term 2.
I20260709 13:09:51.372290 26819 consensus_queue.cc:237] T d8410badcf77497fbb604cf8c185db83 P 0a00d5d20d5b47b99a1220a0e40a4a78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:51.379035 26456 catalog_manager.cc:5697] T d8410badcf77497fbb604cf8c185db83 P 0a00d5d20d5b47b99a1220a0e40a4a78 reported cstate change: term changed from 1 to 2, leader changed from dc9195961d41473e8f590b632eb998a0 (127.14.103.1) to 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3). New cstate: current_term: 2 leader_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:51.397257 26826 raft_consensus.cc:993] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec: : Instructing follower dc9195961d41473e8f590b632eb998a0 to start an election
I20260709 13:09:51.397773 26840 raft_consensus.cc:1081] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Signalling peer dc9195961d41473e8f590b632eb998a0 to start an election
I20260709 13:09:51.399233 26561 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ef38f875bfdd4b669c39d02f0ea7472b"
dest_uuid: "dc9195961d41473e8f590b632eb998a0"
 from {username='slave'} at 127.0.0.1:58510
I20260709 13:09:51.399762 26561 raft_consensus.cc:493] T ef38f875bfdd4b669c39d02f0ea7472b P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:51.400048 26561 raft_consensus.cc:3060] T ef38f875bfdd4b669c39d02f0ea7472b P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:51.404919 26561 raft_consensus.cc:515] T ef38f875bfdd4b669c39d02f0ea7472b P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:51.406581 26561 leader_election.cc:290] T ef38f875bfdd4b669c39d02f0ea7472b P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 2 election: Requested vote from peers 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:51.407564 26708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef38f875bfdd4b669c39d02f0ea7472b" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
I20260709 13:09:51.407660 26633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef38f875bfdd4b669c39d02f0ea7472b" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "68637fffcc4c4969ae0612be35aa58ec"
I20260709 13:09:51.408247 26708 raft_consensus.cc:3060] T ef38f875bfdd4b669c39d02f0ea7472b P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:51.408357 26633 raft_consensus.cc:3055] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:51.409107 26633 raft_consensus.cc:740] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Running, Role: LEADER
I20260709 13:09:51.409961 26633 consensus_queue.cc:260] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:51.411259 26633 raft_consensus.cc:3060] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:51.413518 26708 raft_consensus.cc:2468] T ef38f875bfdd4b669c39d02f0ea7472b P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9195961d41473e8f590b632eb998a0 in term 2.
I20260709 13:09:51.414532 26526 leader_election.cc:304] T ef38f875bfdd4b669c39d02f0ea7472b P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78, dc9195961d41473e8f590b632eb998a0; no voters: 
I20260709 13:09:51.415282 26825 raft_consensus.cc:2804] T ef38f875bfdd4b669c39d02f0ea7472b P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:51.415724 26825 raft_consensus.cc:697] T ef38f875bfdd4b669c39d02f0ea7472b P dc9195961d41473e8f590b632eb998a0 [term 2 LEADER]: Becoming Leader. State: Replica: dc9195961d41473e8f590b632eb998a0, State: Running, Role: LEADER
I20260709 13:09:51.416585 26633 raft_consensus.cc:2468] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9195961d41473e8f590b632eb998a0 in term 2.
I20260709 13:09:51.416590 26825 consensus_queue.cc:237] T ef38f875bfdd4b669c39d02f0ea7472b P dc9195961d41473e8f590b632eb998a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:51.423817 26456 catalog_manager.cc:5697] T ef38f875bfdd4b669c39d02f0ea7472b P dc9195961d41473e8f590b632eb998a0 reported cstate change: term changed from 1 to 2, leader changed from 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2) to dc9195961d41473e8f590b632eb998a0 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "dc9195961d41473e8f590b632eb998a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:51.426131 26832 raft_consensus.cc:993] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78: : Instructing follower 68637fffcc4c4969ae0612be35aa58ec to start an election
I20260709 13:09:51.426676 26819 raft_consensus.cc:1081] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 LEADER]: Signalling peer 68637fffcc4c4969ae0612be35aa58ec to start an election
I20260709 13:09:51.428260 26633 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b8f3b339b6bd4573a9f7929c4d130af5"
dest_uuid: "68637fffcc4c4969ae0612be35aa58ec"
 from {username='slave'} at 127.0.0.1:45328
I20260709 13:09:51.428785 26633 raft_consensus.cc:493] T b8f3b339b6bd4573a9f7929c4d130af5 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:51.429102 26633 raft_consensus.cc:3060] T b8f3b339b6bd4573a9f7929c4d130af5 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:51.430449 26561 raft_consensus.cc:1275] T 4af4eae89a594eb0ace69b0895e43372 P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Refusing update from remote peer 68637fffcc4c4969ae0612be35aa58ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:51.431825 26840 consensus_queue.cc:1048] T 4af4eae89a594eb0ace69b0895e43372 P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:51.434846 26633 raft_consensus.cc:515] T b8f3b339b6bd4573a9f7929c4d130af5 P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:51.437039 26633 leader_election.cc:290] T b8f3b339b6bd4573a9f7929c4d130af5 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 2 election: Requested vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:51.438799 26561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f3b339b6bd4573a9f7929c4d130af5" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dc9195961d41473e8f590b632eb998a0"
I20260709 13:09:51.441439 26561 raft_consensus.cc:3060] T b8f3b339b6bd4573a9f7929c4d130af5 P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:51.448179 26710 raft_consensus.cc:1275] T 4af4eae89a594eb0ace69b0895e43372 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Refusing update from remote peer 68637fffcc4c4969ae0612be35aa58ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:51.449638 26561 raft_consensus.cc:2468] T b8f3b339b6bd4573a9f7929c4d130af5 P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 2.
I20260709 13:09:51.450606 26840 consensus_queue.cc:1048] T 4af4eae89a594eb0ace69b0895e43372 P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:51.456106 26601 leader_election.cc:304] T b8f3b339b6bd4573a9f7929c4d130af5 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68637fffcc4c4969ae0612be35aa58ec, dc9195961d41473e8f590b632eb998a0; no voters: 
I20260709 13:09:51.455009 26708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f3b339b6bd4573a9f7929c4d130af5" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
I20260709 13:09:51.457126 26852 raft_consensus.cc:2804] T b8f3b339b6bd4573a9f7929c4d130af5 P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:51.457469 26708 raft_consensus.cc:3055] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:51.457669 26852 raft_consensus.cc:697] T b8f3b339b6bd4573a9f7929c4d130af5 P 68637fffcc4c4969ae0612be35aa58ec [term 2 LEADER]: Becoming Leader. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Running, Role: LEADER
I20260709 13:09:51.457847 26708 raft_consensus.cc:740] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a00d5d20d5b47b99a1220a0e40a4a78, State: Running, Role: LEADER
I20260709 13:09:51.458858 26852 consensus_queue.cc:237] T b8f3b339b6bd4573a9f7929c4d130af5 P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:51.458949 26708 consensus_queue.cc:260] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:51.460378 26708 raft_consensus.cc:3060] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:51.468245 26458 catalog_manager.cc:5697] T b8f3b339b6bd4573a9f7929c4d130af5 P 68637fffcc4c4969ae0612be35aa58ec reported cstate change: term changed from 1 to 2, leader changed from 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3) to 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2). New cstate: current_term: 2 leader_uuid: "68637fffcc4c4969ae0612be35aa58ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:51.470845 26708 raft_consensus.cc:2468] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 2.
I20260709 13:09:51.500957 26840 raft_consensus.cc:993] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec: : Instructing follower dc9195961d41473e8f590b632eb998a0 to start an election
I20260709 13:09:51.501411 26852 raft_consensus.cc:1081] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Signalling peer dc9195961d41473e8f590b632eb998a0 to start an election
I20260709 13:09:51.502913 26561 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b16a06ac45ab44a186fbf820559732bb"
dest_uuid: "dc9195961d41473e8f590b632eb998a0"
 from {username='slave'} at 127.0.0.1:58510
I20260709 13:09:51.503388 26561 raft_consensus.cc:493] T b16a06ac45ab44a186fbf820559732bb P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:51.503674 26561 raft_consensus.cc:3060] T b16a06ac45ab44a186fbf820559732bb P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:51.508772 26561 raft_consensus.cc:515] T b16a06ac45ab44a186fbf820559732bb P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:51.510344 26561 leader_election.cc:290] T b16a06ac45ab44a186fbf820559732bb P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 2 election: Requested vote from peers 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:51.511238 26633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b16a06ac45ab44a186fbf820559732bb" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "68637fffcc4c4969ae0612be35aa58ec"
I20260709 13:09:51.511554 26708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b16a06ac45ab44a186fbf820559732bb" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
I20260709 13:09:51.511919 26633 raft_consensus.cc:3055] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:51.512147 26708 raft_consensus.cc:3060] T b16a06ac45ab44a186fbf820559732bb P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:51.512323 26633 raft_consensus.cc:740] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Running, Role: LEADER
I20260709 13:09:51.513279 26633 consensus_queue.cc:260] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:51.514604 26633 raft_consensus.cc:3060] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:51.519443 26708 raft_consensus.cc:2468] T b16a06ac45ab44a186fbf820559732bb P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9195961d41473e8f590b632eb998a0 in term 2.
I20260709 13:09:51.520671 26526 leader_election.cc:304] T b16a06ac45ab44a186fbf820559732bb P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78, dc9195961d41473e8f590b632eb998a0; no voters: 
I20260709 13:09:51.521584 26825 raft_consensus.cc:2804] T b16a06ac45ab44a186fbf820559732bb P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:51.522002 26825 raft_consensus.cc:697] T b16a06ac45ab44a186fbf820559732bb P dc9195961d41473e8f590b632eb998a0 [term 2 LEADER]: Becoming Leader. State: Replica: dc9195961d41473e8f590b632eb998a0, State: Running, Role: LEADER
I20260709 13:09:51.522222 26633 raft_consensus.cc:2468] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9195961d41473e8f590b632eb998a0 in term 2.
I20260709 13:09:51.522998 26825 consensus_queue.cc:237] T b16a06ac45ab44a186fbf820559732bb P dc9195961d41473e8f590b632eb998a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:51.531277 26458 catalog_manager.cc:5697] T b16a06ac45ab44a186fbf820559732bb P dc9195961d41473e8f590b632eb998a0 reported cstate change: term changed from 1 to 2, leader changed from 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2) to dc9195961d41473e8f590b632eb998a0 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "dc9195961d41473e8f590b632eb998a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:51.640420 26633 raft_consensus.cc:1275] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [term 3 FOLLOWER]: Refusing update from remote peer dc9195961d41473e8f590b632eb998a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:09:51.641700 26825 consensus_queue.cc:1048] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:51.649089 26708 raft_consensus.cc:1275] T 055a26fb2c8b41b4b6da92c7e20180e1 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 3 FOLLOWER]: Refusing update from remote peer dc9195961d41473e8f590b632eb998a0: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:09:51.650964 26825 consensus_queue.cc:1048] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:51.712570 26633 raft_consensus.cc:1275] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Refusing update from remote peer dc9195961d41473e8f590b632eb998a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:51.713784 26825 consensus_queue.cc:1048] T 6c21616d95d04253a63e088134753cea P dc9195961d41473e8f590b632eb998a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:51.722525 26708 raft_consensus.cc:1275] T 6c21616d95d04253a63e088134753cea P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Refusing update from remote peer dc9195961d41473e8f590b632eb998a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:51.724367 26856 consensus_queue.cc:1048] T 6c21616d95d04253a63e088134753cea P dc9195961d41473e8f590b632eb998a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:51.762187 26708 raft_consensus.cc:1275] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 3 FOLLOWER]: Refusing update from remote peer 68637fffcc4c4969ae0612be35aa58ec: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:09:51.763597 26852 consensus_queue.cc:1048] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:51.770731 26561 raft_consensus.cc:1275] T 579b5cf4a4874b75a67f9cbf320addc2 P dc9195961d41473e8f590b632eb998a0 [term 3 FOLLOWER]: Refusing update from remote peer 68637fffcc4c4969ae0612be35aa58ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:09:51.772581 26840 consensus_queue.cc:1048] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:51.849784 26708 raft_consensus.cc:1275] T ef38f875bfdd4b669c39d02f0ea7472b P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Refusing update from remote peer dc9195961d41473e8f590b632eb998a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:51.851032 26825 consensus_queue.cc:1048] T ef38f875bfdd4b669c39d02f0ea7472b P dc9195961d41473e8f590b632eb998a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:51.860559 26633 raft_consensus.cc:1275] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Refusing update from remote peer dc9195961d41473e8f590b632eb998a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:51.861326 26636 raft_consensus.cc:1275] T d8410badcf77497fbb604cf8c185db83 P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Refusing update from remote peer 0a00d5d20d5b47b99a1220a0e40a4a78: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:51.863276 26825 consensus_queue.cc:1048] T ef38f875bfdd4b669c39d02f0ea7472b P dc9195961d41473e8f590b632eb998a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:51.863293 26819 consensus_queue.cc:1048] T d8410badcf77497fbb604cf8c185db83 P 0a00d5d20d5b47b99a1220a0e40a4a78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:51.868628 26561 raft_consensus.cc:1275] T d8410badcf77497fbb604cf8c185db83 P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Refusing update from remote peer 0a00d5d20d5b47b99a1220a0e40a4a78: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:51.870291 26819 consensus_queue.cc:1048] T d8410badcf77497fbb604cf8c185db83 P 0a00d5d20d5b47b99a1220a0e40a4a78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:51.909368 26708 raft_consensus.cc:1275] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Refusing update from remote peer 68637fffcc4c4969ae0612be35aa58ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:51.910645 26840 consensus_queue.cc:1048] T b8f3b339b6bd4573a9f7929c4d130af5 P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:51.920121 26561 raft_consensus.cc:1275] T b8f3b339b6bd4573a9f7929c4d130af5 P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Refusing update from remote peer 68637fffcc4c4969ae0612be35aa58ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:51.921787 26852 consensus_queue.cc:1048] T b8f3b339b6bd4573a9f7929c4d130af5 P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:51.931468 26708 raft_consensus.cc:1275] T b16a06ac45ab44a186fbf820559732bb P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Refusing update from remote peer dc9195961d41473e8f590b632eb998a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:51.932927 26825 consensus_queue.cc:1048] T b16a06ac45ab44a186fbf820559732bb P dc9195961d41473e8f590b632eb998a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:51.943186 26633 raft_consensus.cc:1275] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Refusing update from remote peer dc9195961d41473e8f590b632eb998a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:51.944965 26856 consensus_queue.cc:1048] T b16a06ac45ab44a186fbf820559732bb P dc9195961d41473e8f590b632eb998a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:54.053170 26457 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55310:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:09:54.055485 26457 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:09:54.102643 26545 tablet_service.cc:1511] Processing CreateTablet for tablet 707de517a3aa4064baa586aa4b8190e8 (DEFAULT_TABLE table=soft_deleted_table [id=3c9aff1183bc43ef8c693879ccb7b87a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:09:54.103904 26622 tablet_service.cc:1511] Processing CreateTablet for tablet 707de517a3aa4064baa586aa4b8190e8 (DEFAULT_TABLE table=soft_deleted_table [id=3c9aff1183bc43ef8c693879ccb7b87a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:09:54.104382 26545 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 707de517a3aa4064baa586aa4b8190e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.105636 26622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 707de517a3aa4064baa586aa4b8190e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.106249 26698 tablet_service.cc:1511] Processing CreateTablet for tablet 707de517a3aa4064baa586aa4b8190e8 (DEFAULT_TABLE table=soft_deleted_table [id=3c9aff1183bc43ef8c693879ccb7b87a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:09:54.107802 26698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 707de517a3aa4064baa586aa4b8190e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.110597 26626 tablet_service.cc:1511] Processing CreateTablet for tablet ebb142b13c4440339f42938e6c1cd02d (DEFAULT_TABLE table=soft_deleted_table [id=3c9aff1183bc43ef8c693879ccb7b87a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:09:54.110769 26550 tablet_service.cc:1511] Processing CreateTablet for tablet ebb142b13c4440339f42938e6c1cd02d (DEFAULT_TABLE table=soft_deleted_table [id=3c9aff1183bc43ef8c693879ccb7b87a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:09:54.112020 26626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebb142b13c4440339f42938e6c1cd02d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.112211 26550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebb142b13c4440339f42938e6c1cd02d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.112506 26693 tablet_service.cc:1511] Processing CreateTablet for tablet ebb142b13c4440339f42938e6c1cd02d (DEFAULT_TABLE table=soft_deleted_table [id=3c9aff1183bc43ef8c693879ccb7b87a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:09:54.113888 26693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebb142b13c4440339f42938e6c1cd02d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.114784 26546 tablet_service.cc:1511] Processing CreateTablet for tablet 164a705fca2541d5a5be47dc3d77c40f (DEFAULT_TABLE table=soft_deleted_table [id=3c9aff1183bc43ef8c693879ccb7b87a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:09:54.116175 26546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 164a705fca2541d5a5be47dc3d77c40f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.129180 26623 tablet_service.cc:1511] Processing CreateTablet for tablet 164a705fca2541d5a5be47dc3d77c40f (DEFAULT_TABLE table=soft_deleted_table [id=3c9aff1183bc43ef8c693879ccb7b87a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:09:54.130623 26623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 164a705fca2541d5a5be47dc3d77c40f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.132895 26697 tablet_service.cc:1511] Processing CreateTablet for tablet 164a705fca2541d5a5be47dc3d77c40f (DEFAULT_TABLE table=soft_deleted_table [id=3c9aff1183bc43ef8c693879ccb7b87a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:09:54.134265 26697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 164a705fca2541d5a5be47dc3d77c40f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.134088 26548 tablet_service.cc:1511] Processing CreateTablet for tablet 8f4b44b3bd734567b03eb09289c769f0 (DEFAULT_TABLE table=soft_deleted_table [id=3c9aff1183bc43ef8c693879ccb7b87a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:09:54.135416 26548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f4b44b3bd734567b03eb09289c769f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.147819 26624 tablet_service.cc:1511] Processing CreateTablet for tablet 8f4b44b3bd734567b03eb09289c769f0 (DEFAULT_TABLE table=soft_deleted_table [id=3c9aff1183bc43ef8c693879ccb7b87a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:09:54.149757 26624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f4b44b3bd734567b03eb09289c769f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.165673 26696 tablet_service.cc:1511] Processing CreateTablet for tablet 8f4b44b3bd734567b03eb09289c769f0 (DEFAULT_TABLE table=soft_deleted_table [id=3c9aff1183bc43ef8c693879ccb7b87a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:09:54.166322 26551 tablet_service.cc:1511] Processing CreateTablet for tablet c5e7fa4752c243c1bd88bedc411e5719 (DEFAULT_TABLE table=soft_deleted_table [id=3c9aff1183bc43ef8c693879ccb7b87a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:09:54.167161 26696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f4b44b3bd734567b03eb09289c769f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.179352 26695 tablet_service.cc:1511] Processing CreateTablet for tablet f1b70e3455b04e46a5c1b08d44fd2a06 (DEFAULT_TABLE table=soft_deleted_table [id=3c9aff1183bc43ef8c693879ccb7b87a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:09:54.179661 26699 tablet_service.cc:1511] Processing CreateTablet for tablet 751c0153edc248998be14ca90435f08a (DEFAULT_TABLE table=soft_deleted_table [id=3c9aff1183bc43ef8c693879ccb7b87a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:09:54.180824 26695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1b70e3455b04e46a5c1b08d44fd2a06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.181473 26547 tablet_service.cc:1511] Processing CreateTablet for tablet 751c0153edc248998be14ca90435f08a (DEFAULT_TABLE table=soft_deleted_table [id=3c9aff1183bc43ef8c693879ccb7b87a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:09:54.182042 26694 tablet_service.cc:1511] Processing CreateTablet for tablet 015286f7e4be4234893d493fb008635e (DEFAULT_TABLE table=soft_deleted_table [id=3c9aff1183bc43ef8c693879ccb7b87a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:09:54.183362 26694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 015286f7e4be4234893d493fb008635e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.183781 26549 tablet_service.cc:1511] Processing CreateTablet for tablet 015286f7e4be4234893d493fb008635e (DEFAULT_TABLE table=soft_deleted_table [id=3c9aff1183bc43ef8c693879ccb7b87a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:09:54.167598 26551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5e7fa4752c243c1bd88bedc411e5719. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.185884 26699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 751c0153edc248998be14ca90435f08a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.188084 26625 tablet_service.cc:1511] Processing CreateTablet for tablet f1b70e3455b04e46a5c1b08d44fd2a06 (DEFAULT_TABLE table=soft_deleted_table [id=3c9aff1183bc43ef8c693879ccb7b87a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:09:54.189352 26625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1b70e3455b04e46a5c1b08d44fd2a06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.191815 26549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 015286f7e4be4234893d493fb008635e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.192499 26626 tablet_service.cc:1511] Processing CreateTablet for tablet c5e7fa4752c243c1bd88bedc411e5719 (DEFAULT_TABLE table=soft_deleted_table [id=3c9aff1183bc43ef8c693879ccb7b87a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:09:54.192812 26621 tablet_service.cc:1511] Processing CreateTablet for tablet 751c0153edc248998be14ca90435f08a (DEFAULT_TABLE table=soft_deleted_table [id=3c9aff1183bc43ef8c693879ccb7b87a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:09:54.193765 26626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5e7fa4752c243c1bd88bedc411e5719. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.196586 26547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 751c0153edc248998be14ca90435f08a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.198462 26621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 751c0153edc248998be14ca90435f08a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.209306 26546 tablet_service.cc:1511] Processing CreateTablet for tablet 8838204cfad3497aaec5d171bbbb38b3 (DEFAULT_TABLE table=soft_deleted_table [id=3c9aff1183bc43ef8c693879ccb7b87a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:09:54.210677 26546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8838204cfad3497aaec5d171bbbb38b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.212494 26882 tablet_bootstrap.cc:492] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec: Bootstrap starting.
I20260709 13:09:54.213135 26700 tablet_service.cc:1511] Processing CreateTablet for tablet 8838204cfad3497aaec5d171bbbb38b3 (DEFAULT_TABLE table=soft_deleted_table [id=3c9aff1183bc43ef8c693879ccb7b87a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:09:54.214468 26700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8838204cfad3497aaec5d171bbbb38b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.215108 26623 tablet_service.cc:1511] Processing CreateTablet for tablet 015286f7e4be4234893d493fb008635e (DEFAULT_TABLE table=soft_deleted_table [id=3c9aff1183bc43ef8c693879ccb7b87a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:09:54.216403 26623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 015286f7e4be4234893d493fb008635e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.237579 26883 tablet_bootstrap.cc:492] T 707de517a3aa4064baa586aa4b8190e8 P dc9195961d41473e8f590b632eb998a0: Bootstrap starting.
I20260709 13:09:54.237785 26550 tablet_service.cc:1511] Processing CreateTablet for tablet f1b70e3455b04e46a5c1b08d44fd2a06 (DEFAULT_TABLE table=soft_deleted_table [id=3c9aff1183bc43ef8c693879ccb7b87a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:09:54.239290 26622 tablet_service.cc:1511] Processing CreateTablet for tablet 8838204cfad3497aaec5d171bbbb38b3 (DEFAULT_TABLE table=soft_deleted_table [id=3c9aff1183bc43ef8c693879ccb7b87a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:09:54.240494 26550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1b70e3455b04e46a5c1b08d44fd2a06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.237998 26698 tablet_service.cc:1511] Processing CreateTablet for tablet c5e7fa4752c243c1bd88bedc411e5719 (DEFAULT_TABLE table=soft_deleted_table [id=3c9aff1183bc43ef8c693879ccb7b87a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:09:54.244513 26698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5e7fa4752c243c1bd88bedc411e5719. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.258441 26882 tablet_bootstrap.cc:654] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.269958 26622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8838204cfad3497aaec5d171bbbb38b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:54.270607 26884 tablet_bootstrap.cc:492] T ebb142b13c4440339f42938e6c1cd02d P 0a00d5d20d5b47b99a1220a0e40a4a78: Bootstrap starting.
I20260709 13:09:54.287737 26884 tablet_bootstrap.cc:654] T ebb142b13c4440339f42938e6c1cd02d P 0a00d5d20d5b47b99a1220a0e40a4a78: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.289919 26883 tablet_bootstrap.cc:654] T 707de517a3aa4064baa586aa4b8190e8 P dc9195961d41473e8f590b632eb998a0: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.304728 26884 tablet_bootstrap.cc:492] T ebb142b13c4440339f42938e6c1cd02d P 0a00d5d20d5b47b99a1220a0e40a4a78: No bootstrap required, opened a new log
I20260709 13:09:54.305311 26884 ts_tablet_manager.cc:1403] T ebb142b13c4440339f42938e6c1cd02d P 0a00d5d20d5b47b99a1220a0e40a4a78: Time spent bootstrapping tablet: real 0.035s	user 0.015s	sys 0.003s
I20260709 13:09:54.308450 26884 raft_consensus.cc:359] T ebb142b13c4440339f42938e6c1cd02d P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.309144 26884 raft_consensus.cc:385] T ebb142b13c4440339f42938e6c1cd02d P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.311815 26884 raft_consensus.cc:740] T ebb142b13c4440339f42938e6c1cd02d P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a00d5d20d5b47b99a1220a0e40a4a78, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.311923 26882 tablet_bootstrap.cc:492] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec: No bootstrap required, opened a new log
I20260709 13:09:54.312584 26882 ts_tablet_manager.cc:1403] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec: Time spent bootstrapping tablet: real 0.100s	user 0.015s	sys 0.007s
I20260709 13:09:54.312810 26883 tablet_bootstrap.cc:492] T 707de517a3aa4064baa586aa4b8190e8 P dc9195961d41473e8f590b632eb998a0: No bootstrap required, opened a new log
I20260709 13:09:54.312695 26884 consensus_queue.cc:260] T ebb142b13c4440339f42938e6c1cd02d P 0a00d5d20d5b47b99a1220a0e40a4a78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.313428 26883 ts_tablet_manager.cc:1403] T 707de517a3aa4064baa586aa4b8190e8 P dc9195961d41473e8f590b632eb998a0: Time spent bootstrapping tablet: real 0.076s	user 0.012s	sys 0.000s
I20260709 13:09:54.315222 26882 raft_consensus.cc:359] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.315608 26884 ts_tablet_manager.cc:1434] T ebb142b13c4440339f42938e6c1cd02d P 0a00d5d20d5b47b99a1220a0e40a4a78: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.000s
I20260709 13:09:54.315861 26882 raft_consensus.cc:385] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.316200 26882 raft_consensus.cc:740] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.316728 26884 tablet_bootstrap.cc:492] T 707de517a3aa4064baa586aa4b8190e8 P 0a00d5d20d5b47b99a1220a0e40a4a78: Bootstrap starting.
I20260709 13:09:54.316975 26882 consensus_queue.cc:260] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.319448 26883 raft_consensus.cc:359] T 707de517a3aa4064baa586aa4b8190e8 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.320259 26883 raft_consensus.cc:385] T 707de517a3aa4064baa586aa4b8190e8 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.320657 26883 raft_consensus.cc:740] T 707de517a3aa4064baa586aa4b8190e8 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9195961d41473e8f590b632eb998a0, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.321470 26883 consensus_queue.cc:260] T 707de517a3aa4064baa586aa4b8190e8 P dc9195961d41473e8f590b632eb998a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.323889 26884 tablet_bootstrap.cc:654] T 707de517a3aa4064baa586aa4b8190e8 P 0a00d5d20d5b47b99a1220a0e40a4a78: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.324301 26883 ts_tablet_manager.cc:1434] T 707de517a3aa4064baa586aa4b8190e8 P dc9195961d41473e8f590b632eb998a0: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 13:09:54.324971 26882 ts_tablet_manager.cc:1434] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.002s
I20260709 13:09:54.326574 26883 tablet_bootstrap.cc:492] T ebb142b13c4440339f42938e6c1cd02d P dc9195961d41473e8f590b632eb998a0: Bootstrap starting.
I20260709 13:09:54.334923 26883 tablet_bootstrap.cc:654] T ebb142b13c4440339f42938e6c1cd02d P dc9195961d41473e8f590b632eb998a0: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.338409 26852 raft_consensus.cc:493] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.339082 26852 raft_consensus.cc:515] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.341760 26852 leader_election.cc:290] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:54.347283 26882 tablet_bootstrap.cc:492] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec: Bootstrap starting.
I20260709 13:09:54.347460 26561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb142b13c4440339f42938e6c1cd02d" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0" is_pre_election: true
I20260709 13:09:54.347692 26710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb142b13c4440339f42938e6c1cd02d" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" is_pre_election: true
I20260709 13:09:54.351305 26710 raft_consensus.cc:2468] T ebb142b13c4440339f42938e6c1cd02d P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 0.
I20260709 13:09:54.352548 26601 leader_election.cc:304] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78, 68637fffcc4c4969ae0612be35aa58ec; no voters: 
W20260709 13:09:54.353448 26601 leader_election.cc:343] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:54.356101 26852 raft_consensus.cc:2804] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:54.356715 26852 raft_consensus.cc:493] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:54.357143 26852 raft_consensus.cc:3060] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:54.357549 26882 tablet_bootstrap.cc:654] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.364411 26884 tablet_bootstrap.cc:492] T 707de517a3aa4064baa586aa4b8190e8 P 0a00d5d20d5b47b99a1220a0e40a4a78: No bootstrap required, opened a new log
I20260709 13:09:54.364902 26884 ts_tablet_manager.cc:1403] T 707de517a3aa4064baa586aa4b8190e8 P 0a00d5d20d5b47b99a1220a0e40a4a78: Time spent bootstrapping tablet: real 0.048s	user 0.024s	sys 0.023s
I20260709 13:09:54.367851 26884 raft_consensus.cc:359] T 707de517a3aa4064baa586aa4b8190e8 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.368343 26882 tablet_bootstrap.cc:492] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec: No bootstrap required, opened a new log
I20260709 13:09:54.368566 26884 raft_consensus.cc:385] T 707de517a3aa4064baa586aa4b8190e8 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.368889 26884 raft_consensus.cc:740] T 707de517a3aa4064baa586aa4b8190e8 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a00d5d20d5b47b99a1220a0e40a4a78, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.368889 26882 ts_tablet_manager.cc:1403] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec: Time spent bootstrapping tablet: real 0.043s	user 0.015s	sys 0.001s
I20260709 13:09:54.369608 26884 consensus_queue.cc:260] T 707de517a3aa4064baa586aa4b8190e8 P 0a00d5d20d5b47b99a1220a0e40a4a78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.372112 26884 ts_tablet_manager.cc:1434] T 707de517a3aa4064baa586aa4b8190e8 P 0a00d5d20d5b47b99a1220a0e40a4a78: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:09:54.372931 26882 raft_consensus.cc:359] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.373674 26882 raft_consensus.cc:385] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.373958 26882 raft_consensus.cc:740] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.374544 26884 tablet_bootstrap.cc:492] T 164a705fca2541d5a5be47dc3d77c40f P 0a00d5d20d5b47b99a1220a0e40a4a78: Bootstrap starting.
I20260709 13:09:54.374680 26882 consensus_queue.cc:260] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.377064 26882 ts_tablet_manager.cc:1434] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:09:54.378050 26882 tablet_bootstrap.cc:492] T 707de517a3aa4064baa586aa4b8190e8 P 68637fffcc4c4969ae0612be35aa58ec: Bootstrap starting.
I20260709 13:09:54.382879 26884 tablet_bootstrap.cc:654] T 164a705fca2541d5a5be47dc3d77c40f P 0a00d5d20d5b47b99a1220a0e40a4a78: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.384531 26882 tablet_bootstrap.cc:654] T 707de517a3aa4064baa586aa4b8190e8 P 68637fffcc4c4969ae0612be35aa58ec: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.395071 26884 tablet_bootstrap.cc:492] T 164a705fca2541d5a5be47dc3d77c40f P 0a00d5d20d5b47b99a1220a0e40a4a78: No bootstrap required, opened a new log
I20260709 13:09:54.395573 26884 ts_tablet_manager.cc:1403] T 164a705fca2541d5a5be47dc3d77c40f P 0a00d5d20d5b47b99a1220a0e40a4a78: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.008s
I20260709 13:09:54.398550 26884 raft_consensus.cc:359] T 164a705fca2541d5a5be47dc3d77c40f P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.399317 26884 raft_consensus.cc:385] T 164a705fca2541d5a5be47dc3d77c40f P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.399616 26884 raft_consensus.cc:740] T 164a705fca2541d5a5be47dc3d77c40f P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a00d5d20d5b47b99a1220a0e40a4a78, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.400315 26884 consensus_queue.cc:260] T 164a705fca2541d5a5be47dc3d77c40f P 0a00d5d20d5b47b99a1220a0e40a4a78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.400774 26882 tablet_bootstrap.cc:492] T 707de517a3aa4064baa586aa4b8190e8 P 68637fffcc4c4969ae0612be35aa58ec: No bootstrap required, opened a new log
I20260709 13:09:54.401304 26882 ts_tablet_manager.cc:1403] T 707de517a3aa4064baa586aa4b8190e8 P 68637fffcc4c4969ae0612be35aa58ec: Time spent bootstrapping tablet: real 0.023s	user 0.005s	sys 0.013s
I20260709 13:09:54.402698 26884 ts_tablet_manager.cc:1434] T 164a705fca2541d5a5be47dc3d77c40f P 0a00d5d20d5b47b99a1220a0e40a4a78: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:09:54.403784 26884 tablet_bootstrap.cc:492] T 751c0153edc248998be14ca90435f08a P 0a00d5d20d5b47b99a1220a0e40a4a78: Bootstrap starting.
I20260709 13:09:54.404259 26882 raft_consensus.cc:359] T 707de517a3aa4064baa586aa4b8190e8 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.405005 26882 raft_consensus.cc:385] T 707de517a3aa4064baa586aa4b8190e8 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.405285 26882 raft_consensus.cc:740] T 707de517a3aa4064baa586aa4b8190e8 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.406013 26882 consensus_queue.cc:260] T 707de517a3aa4064baa586aa4b8190e8 P 68637fffcc4c4969ae0612be35aa58ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.408252 26882 ts_tablet_manager.cc:1434] T 707de517a3aa4064baa586aa4b8190e8 P 68637fffcc4c4969ae0612be35aa58ec: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.007s
I20260709 13:09:54.409209 26882 tablet_bootstrap.cc:492] T 751c0153edc248998be14ca90435f08a P 68637fffcc4c4969ae0612be35aa58ec: Bootstrap starting.
I20260709 13:09:54.414099 26884 tablet_bootstrap.cc:654] T 751c0153edc248998be14ca90435f08a P 0a00d5d20d5b47b99a1220a0e40a4a78: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.415885 26870 raft_consensus.cc:493] T 707de517a3aa4064baa586aa4b8190e8 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.416556 26870 raft_consensus.cc:515] T 707de517a3aa4064baa586aa4b8190e8 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.422281 26882 tablet_bootstrap.cc:654] T 751c0153edc248998be14ca90435f08a P 68637fffcc4c4969ae0612be35aa58ec: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.419386 26870 leader_election.cc:290] T 707de517a3aa4064baa586aa4b8190e8 P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:54.442127 26852 raft_consensus.cc:515] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
W20260709 13:09:54.460032 26526 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:35285, user_credentials={real_user=slave}} blocked reactor thread for 44960.9us
I20260709 13:09:54.461161 26882 tablet_bootstrap.cc:492] T 751c0153edc248998be14ca90435f08a P 68637fffcc4c4969ae0612be35aa58ec: No bootstrap required, opened a new log
I20260709 13:09:54.461895 26882 ts_tablet_manager.cc:1403] T 751c0153edc248998be14ca90435f08a P 68637fffcc4c4969ae0612be35aa58ec: Time spent bootstrapping tablet: real 0.053s	user 0.020s	sys 0.016s
I20260709 13:09:54.461988 26636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "707de517a3aa4064baa586aa4b8190e8" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68637fffcc4c4969ae0612be35aa58ec" is_pre_election: true
I20260709 13:09:54.462953 26636 raft_consensus.cc:2468] T 707de517a3aa4064baa586aa4b8190e8 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc9195961d41473e8f590b632eb998a0 in term 0.
I20260709 13:09:54.465674 26710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "707de517a3aa4064baa586aa4b8190e8" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" is_pre_election: true
I20260709 13:09:54.466518 26710 raft_consensus.cc:2468] T 707de517a3aa4064baa586aa4b8190e8 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc9195961d41473e8f590b632eb998a0 in term 0.
I20260709 13:09:54.466174 26882 raft_consensus.cc:359] T 751c0153edc248998be14ca90435f08a P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.467252 26882 raft_consensus.cc:385] T 751c0153edc248998be14ca90435f08a P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.467648 26882 raft_consensus.cc:740] T 751c0153edc248998be14ca90435f08a P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.468207 26891 raft_consensus.cc:493] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.468840 26882 consensus_queue.cc:260] T 751c0153edc248998be14ca90435f08a P 68637fffcc4c4969ae0612be35aa58ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.468879 26891 raft_consensus.cc:515] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.473134 26561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "164a705fca2541d5a5be47dc3d77c40f" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0" is_pre_election: true
W20260709 13:09:54.474860 26601 leader_election.cc:343] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:54.477270 26710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "164a705fca2541d5a5be47dc3d77c40f" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" is_pre_election: true
I20260709 13:09:54.477900 26710 raft_consensus.cc:2468] T 164a705fca2541d5a5be47dc3d77c40f P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 0.
I20260709 13:09:54.479256 26601 leader_election.cc:304] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78, 68637fffcc4c4969ae0612be35aa58ec; no voters: dc9195961d41473e8f590b632eb998a0
I20260709 13:09:54.479887 26882 ts_tablet_manager.cc:1434] T 751c0153edc248998be14ca90435f08a P 68637fffcc4c4969ae0612be35aa58ec: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.011s
I20260709 13:09:54.480793 26891 leader_election.cc:290] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:54.481305 26899 raft_consensus.cc:2804] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:54.482035 26899 raft_consensus.cc:493] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:54.482899 26899 raft_consensus.cc:3060] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:54.484367 26526 leader_election.cc:304] T 707de517a3aa4064baa586aa4b8190e8 P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78, dc9195961d41473e8f590b632eb998a0; no voters: 
I20260709 13:09:54.485204 26870 raft_consensus.cc:2804] T 707de517a3aa4064baa586aa4b8190e8 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:54.485239 26882 tablet_bootstrap.cc:492] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec: Bootstrap starting.
I20260709 13:09:54.485643 26870 raft_consensus.cc:493] T 707de517a3aa4064baa586aa4b8190e8 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:54.486064 26870 raft_consensus.cc:3060] T 707de517a3aa4064baa586aa4b8190e8 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:54.488333 26561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb142b13c4440339f42938e6c1cd02d" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0"
W20260709 13:09:54.489835 26601 leader_election.cc:343] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:54.505534 26710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb142b13c4440339f42938e6c1cd02d" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
I20260709 13:09:54.505805 26884 tablet_bootstrap.cc:492] T 751c0153edc248998be14ca90435f08a P 0a00d5d20d5b47b99a1220a0e40a4a78: No bootstrap required, opened a new log
I20260709 13:09:54.506160 26710 raft_consensus.cc:3060] T ebb142b13c4440339f42938e6c1cd02d P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:54.506417 26884 ts_tablet_manager.cc:1403] T 751c0153edc248998be14ca90435f08a P 0a00d5d20d5b47b99a1220a0e40a4a78: Time spent bootstrapping tablet: real 0.103s	user 0.041s	sys 0.055s
I20260709 13:09:54.509613 26884 raft_consensus.cc:359] T 751c0153edc248998be14ca90435f08a P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.510377 26884 raft_consensus.cc:385] T 751c0153edc248998be14ca90435f08a P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.510667 26884 raft_consensus.cc:740] T 751c0153edc248998be14ca90435f08a P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a00d5d20d5b47b99a1220a0e40a4a78, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.511399 26884 consensus_queue.cc:260] T 751c0153edc248998be14ca90435f08a P 0a00d5d20d5b47b99a1220a0e40a4a78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.513137 26884 ts_tablet_manager.cc:1434] T 751c0153edc248998be14ca90435f08a P 0a00d5d20d5b47b99a1220a0e40a4a78: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:09:54.513417 26882 tablet_bootstrap.cc:654] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.514134 26884 tablet_bootstrap.cc:492] T 8f4b44b3bd734567b03eb09289c769f0 P 0a00d5d20d5b47b99a1220a0e40a4a78: Bootstrap starting.
I20260709 13:09:54.518882 26870 raft_consensus.cc:515] T 707de517a3aa4064baa586aa4b8190e8 P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.521085 26884 tablet_bootstrap.cc:654] T 8f4b44b3bd734567b03eb09289c769f0 P 0a00d5d20d5b47b99a1220a0e40a4a78: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.522707 26899 raft_consensus.cc:515] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.524078 26870 leader_election.cc:290] T 707de517a3aa4064baa586aa4b8190e8 P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 1 election: Requested vote from peers 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:54.526252 26636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "707de517a3aa4064baa586aa4b8190e8" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68637fffcc4c4969ae0612be35aa58ec"
I20260709 13:09:54.526840 26636 raft_consensus.cc:3060] T 707de517a3aa4064baa586aa4b8190e8 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:54.527024 26899 leader_election.cc:290] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 election: Requested vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:54.527920 26708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "707de517a3aa4064baa586aa4b8190e8" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
I20260709 13:09:54.528491 26708 raft_consensus.cc:3060] T 707de517a3aa4064baa586aa4b8190e8 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:54.530139 26882 tablet_bootstrap.cc:492] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec: No bootstrap required, opened a new log
I20260709 13:09:54.530736 26882 ts_tablet_manager.cc:1403] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec: Time spent bootstrapping tablet: real 0.046s	user 0.009s	sys 0.008s
I20260709 13:09:54.530824 26884 tablet_bootstrap.cc:492] T 8f4b44b3bd734567b03eb09289c769f0 P 0a00d5d20d5b47b99a1220a0e40a4a78: No bootstrap required, opened a new log
I20260709 13:09:54.531335 26884 ts_tablet_manager.cc:1403] T 8f4b44b3bd734567b03eb09289c769f0 P 0a00d5d20d5b47b99a1220a0e40a4a78: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.004s
I20260709 13:09:54.532328 26561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "164a705fca2541d5a5be47dc3d77c40f" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0"
I20260709 13:09:54.533495 26707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "164a705fca2541d5a5be47dc3d77c40f" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
I20260709 13:09:54.533915 26882 raft_consensus.cc:359] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.534220 26884 raft_consensus.cc:359] T 8f4b44b3bd734567b03eb09289c769f0 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
W20260709 13:09:54.534664 26601 leader_election.cc:343] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:54.534827 26882 raft_consensus.cc:385] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.534906 26884 raft_consensus.cc:385] T 8f4b44b3bd734567b03eb09289c769f0 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.535328 26882 raft_consensus.cc:740] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.535370 26884 raft_consensus.cc:740] T 8f4b44b3bd734567b03eb09289c769f0 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a00d5d20d5b47b99a1220a0e40a4a78, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.536233 26884 consensus_queue.cc:260] T 8f4b44b3bd734567b03eb09289c769f0 P 0a00d5d20d5b47b99a1220a0e40a4a78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.536233 26882 consensus_queue.cc:260] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.538630 26884 ts_tablet_manager.cc:1434] T 8f4b44b3bd734567b03eb09289c769f0 P 0a00d5d20d5b47b99a1220a0e40a4a78: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:09:54.539597 26884 tablet_bootstrap.cc:492] T f1b70e3455b04e46a5c1b08d44fd2a06 P 0a00d5d20d5b47b99a1220a0e40a4a78: Bootstrap starting.
I20260709 13:09:54.541884 26708 raft_consensus.cc:2468] T 707de517a3aa4064baa586aa4b8190e8 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9195961d41473e8f590b632eb998a0 in term 1.
I20260709 13:09:54.533989 26707 raft_consensus.cc:3060] T 164a705fca2541d5a5be47dc3d77c40f P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:54.543079 26526 leader_election.cc:304] T 707de517a3aa4064baa586aa4b8190e8 P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78, dc9195961d41473e8f590b632eb998a0; no voters: 
I20260709 13:09:54.543974 26870 raft_consensus.cc:2804] T 707de517a3aa4064baa586aa4b8190e8 P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:54.544438 26870 raft_consensus.cc:697] T 707de517a3aa4064baa586aa4b8190e8 P dc9195961d41473e8f590b632eb998a0 [term 1 LEADER]: Becoming Leader. State: Replica: dc9195961d41473e8f590b632eb998a0, State: Running, Role: LEADER
I20260709 13:09:54.545248 26870 consensus_queue.cc:237] T 707de517a3aa4064baa586aa4b8190e8 P dc9195961d41473e8f590b632eb998a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.546016 26884 tablet_bootstrap.cc:654] T f1b70e3455b04e46a5c1b08d44fd2a06 P 0a00d5d20d5b47b99a1220a0e40a4a78: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.545713 26882 ts_tablet_manager.cc:1434] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec: Time spent starting tablet: real 0.015s	user 0.003s	sys 0.003s
I20260709 13:09:54.550673 26707 raft_consensus.cc:2468] T 164a705fca2541d5a5be47dc3d77c40f P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 1.
I20260709 13:09:54.554670 26636 raft_consensus.cc:2468] T 707de517a3aa4064baa586aa4b8190e8 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9195961d41473e8f590b632eb998a0 in term 1.
I20260709 13:09:54.555718 26710 raft_consensus.cc:2468] T ebb142b13c4440339f42938e6c1cd02d P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 1.
I20260709 13:09:54.557822 26884 tablet_bootstrap.cc:492] T f1b70e3455b04e46a5c1b08d44fd2a06 P 0a00d5d20d5b47b99a1220a0e40a4a78: No bootstrap required, opened a new log
I20260709 13:09:54.557945 26882 tablet_bootstrap.cc:492] T c5e7fa4752c243c1bd88bedc411e5719 P 68637fffcc4c4969ae0612be35aa58ec: Bootstrap starting.
I20260709 13:09:54.556157 26852 leader_election.cc:290] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 election: Requested vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:54.558470 26884 ts_tablet_manager.cc:1403] T f1b70e3455b04e46a5c1b08d44fd2a06 P 0a00d5d20d5b47b99a1220a0e40a4a78: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.008s
I20260709 13:09:54.559855 26601 leader_election.cc:304] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78, 68637fffcc4c4969ae0612be35aa58ec; no voters: dc9195961d41473e8f590b632eb998a0
I20260709 13:09:54.560132 26883 tablet_bootstrap.cc:492] T ebb142b13c4440339f42938e6c1cd02d P dc9195961d41473e8f590b632eb998a0: No bootstrap required, opened a new log
I20260709 13:09:54.560607 26883 ts_tablet_manager.cc:1403] T ebb142b13c4440339f42938e6c1cd02d P dc9195961d41473e8f590b632eb998a0: Time spent bootstrapping tablet: real 0.234s	user 0.042s	sys 0.083s
I20260709 13:09:54.560958 26601 leader_election.cc:304] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78, 68637fffcc4c4969ae0612be35aa58ec; no voters: dc9195961d41473e8f590b632eb998a0
I20260709 13:09:54.561596 26884 raft_consensus.cc:359] T f1b70e3455b04e46a5c1b08d44fd2a06 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.562389 26884 raft_consensus.cc:385] T f1b70e3455b04e46a5c1b08d44fd2a06 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.562669 26884 raft_consensus.cc:740] T f1b70e3455b04e46a5c1b08d44fd2a06 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a00d5d20d5b47b99a1220a0e40a4a78, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.563788 26852 raft_consensus.cc:2804] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:54.563396 26884 consensus_queue.cc:260] T f1b70e3455b04e46a5c1b08d44fd2a06 P 0a00d5d20d5b47b99a1220a0e40a4a78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.564272 26852 raft_consensus.cc:697] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Becoming Leader. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Running, Role: LEADER
I20260709 13:09:54.564986 26882 tablet_bootstrap.cc:654] T c5e7fa4752c243c1bd88bedc411e5719 P 68637fffcc4c4969ae0612be35aa58ec: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.565096 26852 consensus_queue.cc:237] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.565934 26884 ts_tablet_manager.cc:1434] T f1b70e3455b04e46a5c1b08d44fd2a06 P 0a00d5d20d5b47b99a1220a0e40a4a78: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:09:54.566993 26884 tablet_bootstrap.cc:492] T 8838204cfad3497aaec5d171bbbb38b3 P 0a00d5d20d5b47b99a1220a0e40a4a78: Bootstrap starting.
I20260709 13:09:54.568120 26899 raft_consensus.cc:2804] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:54.563493 26883 raft_consensus.cc:359] T ebb142b13c4440339f42938e6c1cd02d P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.570282 26883 raft_consensus.cc:385] T ebb142b13c4440339f42938e6c1cd02d P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.570637 26883 raft_consensus.cc:740] T ebb142b13c4440339f42938e6c1cd02d P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9195961d41473e8f590b632eb998a0, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.571420 26883 consensus_queue.cc:260] T ebb142b13c4440339f42938e6c1cd02d P dc9195961d41473e8f590b632eb998a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.573870 26883 ts_tablet_manager.cc:1434] T ebb142b13c4440339f42938e6c1cd02d P dc9195961d41473e8f590b632eb998a0: Time spent starting tablet: real 0.013s	user 0.001s	sys 0.006s
I20260709 13:09:54.569900 26899 raft_consensus.cc:697] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Becoming Leader. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Running, Role: LEADER
I20260709 13:09:54.577405 26899 consensus_queue.cc:237] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.574220 26898 raft_consensus.cc:493] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.579968 26898 raft_consensus.cc:515] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.581794 26898 leader_election.cc:290] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:54.575012 26884 tablet_bootstrap.cc:654] T 8838204cfad3497aaec5d171bbbb38b3 P 0a00d5d20d5b47b99a1220a0e40a4a78: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.582449 26561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1b70e3455b04e46a5c1b08d44fd2a06" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0" is_pre_election: true
I20260709 13:09:54.582820 26707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1b70e3455b04e46a5c1b08d44fd2a06" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" is_pre_election: true
I20260709 13:09:54.583421 26707 raft_consensus.cc:2468] T f1b70e3455b04e46a5c1b08d44fd2a06 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 0.
W20260709 13:09:54.583956 26601 leader_election.cc:343] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:54.584709 26601 leader_election.cc:304] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78, 68637fffcc4c4969ae0612be35aa58ec; no voters: dc9195961d41473e8f590b632eb998a0
I20260709 13:09:54.576510 26882 tablet_bootstrap.cc:492] T c5e7fa4752c243c1bd88bedc411e5719 P 68637fffcc4c4969ae0612be35aa58ec: No bootstrap required, opened a new log
I20260709 13:09:54.576597 26883 tablet_bootstrap.cc:492] T 164a705fca2541d5a5be47dc3d77c40f P dc9195961d41473e8f590b632eb998a0: Bootstrap starting.
I20260709 13:09:54.586347 26882 ts_tablet_manager.cc:1403] T c5e7fa4752c243c1bd88bedc411e5719 P 68637fffcc4c4969ae0612be35aa58ec: Time spent bootstrapping tablet: real 0.029s	user 0.014s	sys 0.004s
I20260709 13:09:54.591423 26884 tablet_bootstrap.cc:492] T 8838204cfad3497aaec5d171bbbb38b3 P 0a00d5d20d5b47b99a1220a0e40a4a78: No bootstrap required, opened a new log
I20260709 13:09:54.593672 26899 raft_consensus.cc:2804] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:54.591100 26456 catalog_manager.cc:5697] T 707de517a3aa4064baa586aa4b8190e8 P dc9195961d41473e8f590b632eb998a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc9195961d41473e8f590b632eb998a0 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "dc9195961d41473e8f590b632eb998a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:54.600991 26884 ts_tablet_manager.cc:1403] T 8838204cfad3497aaec5d171bbbb38b3 P 0a00d5d20d5b47b99a1220a0e40a4a78: Time spent bootstrapping tablet: real 0.034s	user 0.012s	sys 0.008s
I20260709 13:09:54.601318 26899 raft_consensus.cc:493] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:54.601732 26899 raft_consensus.cc:3060] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:54.601020 26457 catalog_manager.cc:5697] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2). New cstate: current_term: 1 leader_uuid: "68637fffcc4c4969ae0612be35aa58ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:54.602797 26457 catalog_manager.cc:5697] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2). New cstate: current_term: 1 leader_uuid: "68637fffcc4c4969ae0612be35aa58ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:54.604223 26884 raft_consensus.cc:359] T 8838204cfad3497aaec5d171bbbb38b3 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.604956 26884 raft_consensus.cc:385] T 8838204cfad3497aaec5d171bbbb38b3 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.605229 26884 raft_consensus.cc:740] T 8838204cfad3497aaec5d171bbbb38b3 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a00d5d20d5b47b99a1220a0e40a4a78, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.605859 26882 raft_consensus.cc:359] T c5e7fa4752c243c1bd88bedc411e5719 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.605978 26884 consensus_queue.cc:260] T 8838204cfad3497aaec5d171bbbb38b3 P 0a00d5d20d5b47b99a1220a0e40a4a78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.606647 26882 raft_consensus.cc:385] T c5e7fa4752c243c1bd88bedc411e5719 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.606962 26882 raft_consensus.cc:740] T c5e7fa4752c243c1bd88bedc411e5719 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.608180 26899 raft_consensus.cc:515] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.608847 26884 ts_tablet_manager.cc:1434] T 8838204cfad3497aaec5d171bbbb38b3 P 0a00d5d20d5b47b99a1220a0e40a4a78: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:09:54.609892 26884 tablet_bootstrap.cc:492] T c5e7fa4752c243c1bd88bedc411e5719 P 0a00d5d20d5b47b99a1220a0e40a4a78: Bootstrap starting.
I20260709 13:09:54.610682 26561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1b70e3455b04e46a5c1b08d44fd2a06" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0"
W20260709 13:09:54.611922 26601 leader_election.cc:343] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:54.612335 26899 leader_election.cc:290] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 election: Requested vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:54.613406 26710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1b70e3455b04e46a5c1b08d44fd2a06" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
I20260709 13:09:54.613354 26882 consensus_queue.cc:260] T c5e7fa4752c243c1bd88bedc411e5719 P 68637fffcc4c4969ae0612be35aa58ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.614243 26710 raft_consensus.cc:3060] T f1b70e3455b04e46a5c1b08d44fd2a06 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:54.616706 26884 tablet_bootstrap.cc:654] T c5e7fa4752c243c1bd88bedc411e5719 P 0a00d5d20d5b47b99a1220a0e40a4a78: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.611544 26883 tablet_bootstrap.cc:654] T 164a705fca2541d5a5be47dc3d77c40f P dc9195961d41473e8f590b632eb998a0: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.622538 26710 raft_consensus.cc:2468] T f1b70e3455b04e46a5c1b08d44fd2a06 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 1.
I20260709 13:09:54.623996 26601 leader_election.cc:304] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78, 68637fffcc4c4969ae0612be35aa58ec; no voters: dc9195961d41473e8f590b632eb998a0
I20260709 13:09:54.625034 26882 ts_tablet_manager.cc:1434] T c5e7fa4752c243c1bd88bedc411e5719 P 68637fffcc4c4969ae0612be35aa58ec: Time spent starting tablet: real 0.024s	user 0.002s	sys 0.006s
I20260709 13:09:54.625259 26899 raft_consensus.cc:2804] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:54.625990 26899 raft_consensus.cc:697] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Becoming Leader. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Running, Role: LEADER
I20260709 13:09:54.626356 26882 tablet_bootstrap.cc:492] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec: Bootstrap starting.
I20260709 13:09:54.626788 26899 consensus_queue.cc:237] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.629287 26884 tablet_bootstrap.cc:492] T c5e7fa4752c243c1bd88bedc411e5719 P 0a00d5d20d5b47b99a1220a0e40a4a78: No bootstrap required, opened a new log
I20260709 13:09:54.629804 26884 ts_tablet_manager.cc:1403] T c5e7fa4752c243c1bd88bedc411e5719 P 0a00d5d20d5b47b99a1220a0e40a4a78: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.005s
I20260709 13:09:54.632522 26884 raft_consensus.cc:359] T c5e7fa4752c243c1bd88bedc411e5719 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.633127 26884 raft_consensus.cc:385] T c5e7fa4752c243c1bd88bedc411e5719 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.633383 26884 raft_consensus.cc:740] T c5e7fa4752c243c1bd88bedc411e5719 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a00d5d20d5b47b99a1220a0e40a4a78, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.634028 26884 consensus_queue.cc:260] T c5e7fa4752c243c1bd88bedc411e5719 P 0a00d5d20d5b47b99a1220a0e40a4a78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.636003 26884 ts_tablet_manager.cc:1434] T c5e7fa4752c243c1bd88bedc411e5719 P 0a00d5d20d5b47b99a1220a0e40a4a78: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:09:54.637176 26884 tablet_bootstrap.cc:492] T 015286f7e4be4234893d493fb008635e P 0a00d5d20d5b47b99a1220a0e40a4a78: Bootstrap starting.
I20260709 13:09:54.643818 26884 tablet_bootstrap.cc:654] T 015286f7e4be4234893d493fb008635e P 0a00d5d20d5b47b99a1220a0e40a4a78: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.643625 26882 tablet_bootstrap.cc:654] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.651497 26898 raft_consensus.cc:493] T 751c0153edc248998be14ca90435f08a P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.652062 26898 raft_consensus.cc:515] T 751c0153edc248998be14ca90435f08a P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.654248 26898 leader_election.cc:290] T 751c0153edc248998be14ca90435f08a P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:54.655032 26882 tablet_bootstrap.cc:492] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec: No bootstrap required, opened a new log
I20260709 13:09:54.655764 26882 ts_tablet_manager.cc:1403] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec: Time spent bootstrapping tablet: real 0.030s	user 0.010s	sys 0.004s
I20260709 13:09:54.655887 26561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "751c0153edc248998be14ca90435f08a" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0" is_pre_election: true
W20260709 13:09:54.657856 26601 leader_election.cc:343] T 751c0153edc248998be14ca90435f08a P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:54.659500 26710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "751c0153edc248998be14ca90435f08a" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" is_pre_election: true
I20260709 13:09:54.660413 26710 raft_consensus.cc:2468] T 751c0153edc248998be14ca90435f08a P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 0.
I20260709 13:09:54.660876 26882 raft_consensus.cc:359] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.661592 26882 raft_consensus.cc:385] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.661875 26882 raft_consensus.cc:740] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.662106 26884 tablet_bootstrap.cc:492] T 015286f7e4be4234893d493fb008635e P 0a00d5d20d5b47b99a1220a0e40a4a78: No bootstrap required, opened a new log
I20260709 13:09:54.662075 26601 leader_election.cc:304] T 751c0153edc248998be14ca90435f08a P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78, 68637fffcc4c4969ae0612be35aa58ec; no voters: dc9195961d41473e8f590b632eb998a0
I20260709 13:09:54.662815 26884 ts_tablet_manager.cc:1403] T 015286f7e4be4234893d493fb008635e P 0a00d5d20d5b47b99a1220a0e40a4a78: Time spent bootstrapping tablet: real 0.026s	user 0.008s	sys 0.016s
I20260709 13:09:54.662598 26882 consensus_queue.cc:260] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.663160 26898 raft_consensus.cc:2804] T 751c0153edc248998be14ca90435f08a P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:54.663614 26898 raft_consensus.cc:493] T 751c0153edc248998be14ca90435f08a P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:54.663970 26898 raft_consensus.cc:3060] T 751c0153edc248998be14ca90435f08a P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:54.665603 26884 raft_consensus.cc:359] T 015286f7e4be4234893d493fb008635e P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.666293 26884 raft_consensus.cc:385] T 015286f7e4be4234893d493fb008635e P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.666551 26884 raft_consensus.cc:740] T 015286f7e4be4234893d493fb008635e P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a00d5d20d5b47b99a1220a0e40a4a78, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.666690 26883 tablet_bootstrap.cc:492] T 164a705fca2541d5a5be47dc3d77c40f P dc9195961d41473e8f590b632eb998a0: No bootstrap required, opened a new log
I20260709 13:09:54.667207 26883 ts_tablet_manager.cc:1403] T 164a705fca2541d5a5be47dc3d77c40f P dc9195961d41473e8f590b632eb998a0: Time spent bootstrapping tablet: real 0.091s	user 0.013s	sys 0.006s
I20260709 13:09:54.667258 26884 consensus_queue.cc:260] T 015286f7e4be4234893d493fb008635e P 0a00d5d20d5b47b99a1220a0e40a4a78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.669497 26884 ts_tablet_manager.cc:1434] T 015286f7e4be4234893d493fb008635e P 0a00d5d20d5b47b99a1220a0e40a4a78: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:09:54.670207 26883 raft_consensus.cc:359] T 164a705fca2541d5a5be47dc3d77c40f P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.670730 26883 raft_consensus.cc:385] T 164a705fca2541d5a5be47dc3d77c40f P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.670902 26883 raft_consensus.cc:740] T 164a705fca2541d5a5be47dc3d77c40f P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9195961d41473e8f590b632eb998a0, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.664973 26882 ts_tablet_manager.cc:1434] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:09:54.671450 26883 consensus_queue.cc:260] T 164a705fca2541d5a5be47dc3d77c40f P dc9195961d41473e8f590b632eb998a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.671856 26898 raft_consensus.cc:515] T 751c0153edc248998be14ca90435f08a P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.672650 26882 tablet_bootstrap.cc:492] T 8838204cfad3497aaec5d171bbbb38b3 P 68637fffcc4c4969ae0612be35aa58ec: Bootstrap starting.
I20260709 13:09:54.673938 26883 ts_tablet_manager.cc:1434] T 164a705fca2541d5a5be47dc3d77c40f P dc9195961d41473e8f590b632eb998a0: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:09:54.674697 26561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "751c0153edc248998be14ca90435f08a" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0"
I20260709 13:09:54.674943 26883 tablet_bootstrap.cc:492] T 8838204cfad3497aaec5d171bbbb38b3 P dc9195961d41473e8f590b632eb998a0: Bootstrap starting.
W20260709 13:09:54.676445 26601 leader_election.cc:343] T 751c0153edc248998be14ca90435f08a P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:54.679296 26710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "751c0153edc248998be14ca90435f08a" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
I20260709 13:09:54.679581 26882 tablet_bootstrap.cc:654] T 8838204cfad3497aaec5d171bbbb38b3 P 68637fffcc4c4969ae0612be35aa58ec: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.679826 26710 raft_consensus.cc:3060] T 751c0153edc248998be14ca90435f08a P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:54.681084 26898 leader_election.cc:290] T 751c0153edc248998be14ca90435f08a P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 election: Requested vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:54.685926 26883 tablet_bootstrap.cc:654] T 8838204cfad3497aaec5d171bbbb38b3 P dc9195961d41473e8f590b632eb998a0: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.689504 26710 raft_consensus.cc:2468] T 751c0153edc248998be14ca90435f08a P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 1.
I20260709 13:09:54.690649 26601 leader_election.cc:304] T 751c0153edc248998be14ca90435f08a P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78, 68637fffcc4c4969ae0612be35aa58ec; no voters: dc9195961d41473e8f590b632eb998a0
I20260709 13:09:54.691654 26898 raft_consensus.cc:2804] T 751c0153edc248998be14ca90435f08a P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:54.692078 26898 raft_consensus.cc:697] T 751c0153edc248998be14ca90435f08a P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Becoming Leader. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Running, Role: LEADER
I20260709 13:09:54.692778 26898 consensus_queue.cc:237] T 751c0153edc248998be14ca90435f08a P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.692802 26882 tablet_bootstrap.cc:492] T 8838204cfad3497aaec5d171bbbb38b3 P 68637fffcc4c4969ae0612be35aa58ec: No bootstrap required, opened a new log
I20260709 13:09:54.692829 26883 tablet_bootstrap.cc:492] T 8838204cfad3497aaec5d171bbbb38b3 P dc9195961d41473e8f590b632eb998a0: No bootstrap required, opened a new log
I20260709 13:09:54.693653 26882 ts_tablet_manager.cc:1403] T 8838204cfad3497aaec5d171bbbb38b3 P 68637fffcc4c4969ae0612be35aa58ec: Time spent bootstrapping tablet: real 0.021s	user 0.019s	sys 0.001s
I20260709 13:09:54.693709 26883 ts_tablet_manager.cc:1403] T 8838204cfad3497aaec5d171bbbb38b3 P dc9195961d41473e8f590b632eb998a0: Time spent bootstrapping tablet: real 0.019s	user 0.015s	sys 0.000s
I20260709 13:09:54.696688 26882 raft_consensus.cc:359] T 8838204cfad3497aaec5d171bbbb38b3 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.696959 26883 raft_consensus.cc:359] T 8838204cfad3497aaec5d171bbbb38b3 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.697382 26882 raft_consensus.cc:385] T 8838204cfad3497aaec5d171bbbb38b3 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.697652 26883 raft_consensus.cc:385] T 8838204cfad3497aaec5d171bbbb38b3 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.697746 26882 raft_consensus.cc:740] T 8838204cfad3497aaec5d171bbbb38b3 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.699085 26882 consensus_queue.cc:260] T 8838204cfad3497aaec5d171bbbb38b3 P 68637fffcc4c4969ae0612be35aa58ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.698081 26883 raft_consensus.cc:740] T 8838204cfad3497aaec5d171bbbb38b3 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9195961d41473e8f590b632eb998a0, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.701269 26882 ts_tablet_manager.cc:1434] T 8838204cfad3497aaec5d171bbbb38b3 P 68637fffcc4c4969ae0612be35aa58ec: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:09:54.702157 26882 tablet_bootstrap.cc:492] T 015286f7e4be4234893d493fb008635e P 68637fffcc4c4969ae0612be35aa58ec: Bootstrap starting.
I20260709 13:09:54.705964 26883 consensus_queue.cc:260] T 8838204cfad3497aaec5d171bbbb38b3 P dc9195961d41473e8f590b632eb998a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.707978 26882 tablet_bootstrap.cc:654] T 015286f7e4be4234893d493fb008635e P 68637fffcc4c4969ae0612be35aa58ec: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.708662 26883 ts_tablet_manager.cc:1434] T 8838204cfad3497aaec5d171bbbb38b3 P dc9195961d41473e8f590b632eb998a0: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:09:54.709748 26883 tablet_bootstrap.cc:492] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0: Bootstrap starting.
I20260709 13:09:54.715129 26882 tablet_bootstrap.cc:492] T 015286f7e4be4234893d493fb008635e P 68637fffcc4c4969ae0612be35aa58ec: No bootstrap required, opened a new log
I20260709 13:09:54.714936 26459 catalog_manager.cc:5697] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2). New cstate: current_term: 1 leader_uuid: "68637fffcc4c4969ae0612be35aa58ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:54.715636 26882 ts_tablet_manager.cc:1403] T 015286f7e4be4234893d493fb008635e P 68637fffcc4c4969ae0612be35aa58ec: Time spent bootstrapping tablet: real 0.014s	user 0.008s	sys 0.004s
I20260709 13:09:54.717664 26883 tablet_bootstrap.cc:654] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.717350 26459 catalog_manager.cc:5697] T 751c0153edc248998be14ca90435f08a P 68637fffcc4c4969ae0612be35aa58ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2). New cstate: current_term: 1 leader_uuid: "68637fffcc4c4969ae0612be35aa58ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:54.718048 26882 raft_consensus.cc:359] T 015286f7e4be4234893d493fb008635e P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.718771 26882 raft_consensus.cc:385] T 015286f7e4be4234893d493fb008635e P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.719048 26882 raft_consensus.cc:740] T 015286f7e4be4234893d493fb008635e P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.719655 26882 consensus_queue.cc:260] T 015286f7e4be4234893d493fb008635e P 68637fffcc4c4969ae0612be35aa58ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.721693 26882 ts_tablet_manager.cc:1434] T 015286f7e4be4234893d493fb008635e P 68637fffcc4c4969ae0612be35aa58ec: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.003s
I20260709 13:09:54.723155 26883 tablet_bootstrap.cc:492] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0: No bootstrap required, opened a new log
I20260709 13:09:54.723598 26883 ts_tablet_manager.cc:1403] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0: Time spent bootstrapping tablet: real 0.014s	user 0.008s	sys 0.003s
I20260709 13:09:54.726354 26883 raft_consensus.cc:359] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.726904 26883 raft_consensus.cc:385] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.727142 26883 raft_consensus.cc:740] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9195961d41473e8f590b632eb998a0, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.727766 26883 consensus_queue.cc:260] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.729806 26883 ts_tablet_manager.cc:1434] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:09:54.730907 26883 tablet_bootstrap.cc:492] T 751c0153edc248998be14ca90435f08a P dc9195961d41473e8f590b632eb998a0: Bootstrap starting.
I20260709 13:09:54.736737 26883 tablet_bootstrap.cc:654] T 751c0153edc248998be14ca90435f08a P dc9195961d41473e8f590b632eb998a0: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.741065 26883 tablet_bootstrap.cc:492] T 751c0153edc248998be14ca90435f08a P dc9195961d41473e8f590b632eb998a0: No bootstrap required, opened a new log
I20260709 13:09:54.741477 26883 ts_tablet_manager.cc:1403] T 751c0153edc248998be14ca90435f08a P dc9195961d41473e8f590b632eb998a0: Time spent bootstrapping tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:09:54.743494 26883 raft_consensus.cc:359] T 751c0153edc248998be14ca90435f08a P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.744061 26883 raft_consensus.cc:385] T 751c0153edc248998be14ca90435f08a P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.744289 26883 raft_consensus.cc:740] T 751c0153edc248998be14ca90435f08a P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9195961d41473e8f590b632eb998a0, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.744817 26883 consensus_queue.cc:260] T 751c0153edc248998be14ca90435f08a P dc9195961d41473e8f590b632eb998a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.746542 26883 ts_tablet_manager.cc:1434] T 751c0153edc248998be14ca90435f08a P dc9195961d41473e8f590b632eb998a0: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:09:54.747274 26883 tablet_bootstrap.cc:492] T 8f4b44b3bd734567b03eb09289c769f0 P dc9195961d41473e8f590b632eb998a0: Bootstrap starting.
I20260709 13:09:54.752429 26883 tablet_bootstrap.cc:654] T 8f4b44b3bd734567b03eb09289c769f0 P dc9195961d41473e8f590b632eb998a0: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.756721 26883 tablet_bootstrap.cc:492] T 8f4b44b3bd734567b03eb09289c769f0 P dc9195961d41473e8f590b632eb998a0: No bootstrap required, opened a new log
I20260709 13:09:54.756819 26864 raft_consensus.cc:493] T 8838204cfad3497aaec5d171bbbb38b3 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.757193 26883 ts_tablet_manager.cc:1403] T 8f4b44b3bd734567b03eb09289c769f0 P dc9195961d41473e8f590b632eb998a0: Time spent bootstrapping tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:09:54.757329 26864 raft_consensus.cc:515] T 8838204cfad3497aaec5d171bbbb38b3 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.759619 26864 leader_election.cc:290] T 8838204cfad3497aaec5d171bbbb38b3 P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285)
I20260709 13:09:54.760154 26561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8838204cfad3497aaec5d171bbbb38b3" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0" is_pre_election: true
I20260709 13:09:54.760533 26636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8838204cfad3497aaec5d171bbbb38b3" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68637fffcc4c4969ae0612be35aa58ec" is_pre_election: true
I20260709 13:09:54.760667 26561 raft_consensus.cc:2468] T 8838204cfad3497aaec5d171bbbb38b3 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in term 0.
I20260709 13:09:54.760663 26883 raft_consensus.cc:359] T 8f4b44b3bd734567b03eb09289c769f0 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.761209 26636 raft_consensus.cc:2468] T 8838204cfad3497aaec5d171bbbb38b3 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in term 0.
I20260709 13:09:54.761533 26883 raft_consensus.cc:385] T 8f4b44b3bd734567b03eb09289c769f0 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.762097 26883 raft_consensus.cc:740] T 8f4b44b3bd734567b03eb09289c769f0 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9195961d41473e8f590b632eb998a0, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.762001 26675 leader_election.cc:304] T 8838204cfad3497aaec5d171bbbb38b3 P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78, dc9195961d41473e8f590b632eb998a0; no voters: 
I20260709 13:09:54.762797 26864 raft_consensus.cc:2804] T 8838204cfad3497aaec5d171bbbb38b3 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:54.762804 26883 consensus_queue.cc:260] T 8f4b44b3bd734567b03eb09289c769f0 P dc9195961d41473e8f590b632eb998a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.763307 26864 raft_consensus.cc:493] T 8838204cfad3497aaec5d171bbbb38b3 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:54.763695 26864 raft_consensus.cc:3060] T 8838204cfad3497aaec5d171bbbb38b3 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:54.765098 26883 ts_tablet_manager.cc:1434] T 8f4b44b3bd734567b03eb09289c769f0 P dc9195961d41473e8f590b632eb998a0: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:09:54.766099 26883 tablet_bootstrap.cc:492] T 015286f7e4be4234893d493fb008635e P dc9195961d41473e8f590b632eb998a0: Bootstrap starting.
I20260709 13:09:54.772470 26883 tablet_bootstrap.cc:654] T 015286f7e4be4234893d493fb008635e P dc9195961d41473e8f590b632eb998a0: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.772840 26864 raft_consensus.cc:515] T 8838204cfad3497aaec5d171bbbb38b3 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.775658 26561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8838204cfad3497aaec5d171bbbb38b3" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0"
I20260709 13:09:54.776266 26561 raft_consensus.cc:3060] T 8838204cfad3497aaec5d171bbbb38b3 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:54.776398 26864 leader_election.cc:290] T 8838204cfad3497aaec5d171bbbb38b3 P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 1 election: Requested vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285)
I20260709 13:09:54.777706 26883 tablet_bootstrap.cc:492] T 015286f7e4be4234893d493fb008635e P dc9195961d41473e8f590b632eb998a0: No bootstrap required, opened a new log
I20260709 13:09:54.778187 26883 ts_tablet_manager.cc:1403] T 015286f7e4be4234893d493fb008635e P dc9195961d41473e8f590b632eb998a0: Time spent bootstrapping tablet: real 0.012s	user 0.010s	sys 0.000s
I20260709 13:09:54.778769 26636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8838204cfad3497aaec5d171bbbb38b3" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68637fffcc4c4969ae0612be35aa58ec"
I20260709 13:09:54.779392 26636 raft_consensus.cc:3060] T 8838204cfad3497aaec5d171bbbb38b3 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:54.781108 26883 raft_consensus.cc:359] T 015286f7e4be4234893d493fb008635e P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.781767 26883 raft_consensus.cc:385] T 015286f7e4be4234893d493fb008635e P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.782025 26883 raft_consensus.cc:740] T 015286f7e4be4234893d493fb008635e P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9195961d41473e8f590b632eb998a0, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.782919 26561 raft_consensus.cc:2468] T 8838204cfad3497aaec5d171bbbb38b3 P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in term 1.
I20260709 13:09:54.782675 26883 consensus_queue.cc:260] T 015286f7e4be4234893d493fb008635e P dc9195961d41473e8f590b632eb998a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.784091 26675 leader_election.cc:304] T 8838204cfad3497aaec5d171bbbb38b3 P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78, dc9195961d41473e8f590b632eb998a0; no voters: 
I20260709 13:09:54.784830 26883 ts_tablet_manager.cc:1434] T 015286f7e4be4234893d493fb008635e P dc9195961d41473e8f590b632eb998a0: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:09:54.784905 26636 raft_consensus.cc:2468] T 8838204cfad3497aaec5d171bbbb38b3 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in term 1.
I20260709 13:09:54.785251 26864 raft_consensus.cc:2804] T 8838204cfad3497aaec5d171bbbb38b3 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:54.786334 26883 tablet_bootstrap.cc:492] T f1b70e3455b04e46a5c1b08d44fd2a06 P dc9195961d41473e8f590b632eb998a0: Bootstrap starting.
I20260709 13:09:54.786727 26864 raft_consensus.cc:697] T 8838204cfad3497aaec5d171bbbb38b3 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 LEADER]: Becoming Leader. State: Replica: 0a00d5d20d5b47b99a1220a0e40a4a78, State: Running, Role: LEADER
I20260709 13:09:54.787644 26864 consensus_queue.cc:237] T 8838204cfad3497aaec5d171bbbb38b3 P 0a00d5d20d5b47b99a1220a0e40a4a78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.793006 26883 tablet_bootstrap.cc:654] T f1b70e3455b04e46a5c1b08d44fd2a06 P dc9195961d41473e8f590b632eb998a0: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:54.797786 26883 tablet_bootstrap.cc:492] T f1b70e3455b04e46a5c1b08d44fd2a06 P dc9195961d41473e8f590b632eb998a0: No bootstrap required, opened a new log
I20260709 13:09:54.798223 26883 ts_tablet_manager.cc:1403] T f1b70e3455b04e46a5c1b08d44fd2a06 P dc9195961d41473e8f590b632eb998a0: Time spent bootstrapping tablet: real 0.012s	user 0.010s	sys 0.000s
I20260709 13:09:54.797415 26456 catalog_manager.cc:5697] T 8838204cfad3497aaec5d171bbbb38b3 P 0a00d5d20d5b47b99a1220a0e40a4a78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3). New cstate: current_term: 1 leader_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:54.800185 26883 raft_consensus.cc:359] T f1b70e3455b04e46a5c1b08d44fd2a06 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.800668 26883 raft_consensus.cc:385] T f1b70e3455b04e46a5c1b08d44fd2a06 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:54.800876 26883 raft_consensus.cc:740] T f1b70e3455b04e46a5c1b08d44fd2a06 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9195961d41473e8f590b632eb998a0, State: Initialized, Role: FOLLOWER
I20260709 13:09:54.801433 26883 consensus_queue.cc:260] T f1b70e3455b04e46a5c1b08d44fd2a06 P dc9195961d41473e8f590b632eb998a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.803335 26883 ts_tablet_manager.cc:1434] T f1b70e3455b04e46a5c1b08d44fd2a06 P dc9195961d41473e8f590b632eb998a0: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20260709 13:09:54.807719 26864 raft_consensus.cc:493] T 015286f7e4be4234893d493fb008635e P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.808266 26864 raft_consensus.cc:515] T 015286f7e4be4234893d493fb008635e P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.810298 26864 leader_election.cc:290] T 015286f7e4be4234893d493fb008635e P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285)
I20260709 13:09:54.811378 26636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "015286f7e4be4234893d493fb008635e" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68637fffcc4c4969ae0612be35aa58ec" is_pre_election: true
I20260709 13:09:54.811406 26561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "015286f7e4be4234893d493fb008635e" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0" is_pre_election: true
I20260709 13:09:54.812268 26561 raft_consensus.cc:2468] T 015286f7e4be4234893d493fb008635e P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in term 0.
I20260709 13:09:54.812194 26636 raft_consensus.cc:2468] T 015286f7e4be4234893d493fb008635e P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in term 0.
I20260709 13:09:54.813470 26675 leader_election.cc:304] T 015286f7e4be4234893d493fb008635e P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78, 68637fffcc4c4969ae0612be35aa58ec; no voters: 
I20260709 13:09:54.814239 26864 raft_consensus.cc:2804] T 015286f7e4be4234893d493fb008635e P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:54.814617 26864 raft_consensus.cc:493] T 015286f7e4be4234893d493fb008635e P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:54.814945 26864 raft_consensus.cc:3060] T 015286f7e4be4234893d493fb008635e P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:54.820036 26864 raft_consensus.cc:515] T 015286f7e4be4234893d493fb008635e P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.821833 26864 leader_election.cc:290] T 015286f7e4be4234893d493fb008635e P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 1 election: Requested vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285)
I20260709 13:09:54.822443 26870 raft_consensus.cc:493] T 164a705fca2541d5a5be47dc3d77c40f P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.822714 26561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "015286f7e4be4234893d493fb008635e" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0"
I20260709 13:09:54.823318 26561 raft_consensus.cc:3060] T 015286f7e4be4234893d493fb008635e P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:54.822981 26870 raft_consensus.cc:515] T 164a705fca2541d5a5be47dc3d77c40f P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.822930 26636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "015286f7e4be4234893d493fb008635e" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68637fffcc4c4969ae0612be35aa58ec"
I20260709 13:09:54.824201 26636 raft_consensus.cc:3060] T 015286f7e4be4234893d493fb008635e P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:54.825165 26870 leader_election.cc:290] T 164a705fca2541d5a5be47dc3d77c40f P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:54.826231 26633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "164a705fca2541d5a5be47dc3d77c40f" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68637fffcc4c4969ae0612be35aa58ec" is_pre_election: true
I20260709 13:09:54.826637 26710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "164a705fca2541d5a5be47dc3d77c40f" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" is_pre_election: true
I20260709 13:09:54.827409 26710 raft_consensus.cc:2393] T 164a705fca2541d5a5be47dc3d77c40f P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dc9195961d41473e8f590b632eb998a0 in current term 1: Already voted for candidate 68637fffcc4c4969ae0612be35aa58ec in this term.
I20260709 13:09:54.828835 26526 leader_election.cc:304] T 164a705fca2541d5a5be47dc3d77c40f P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dc9195961d41473e8f590b632eb998a0; no voters: 0a00d5d20d5b47b99a1220a0e40a4a78, 68637fffcc4c4969ae0612be35aa58ec
I20260709 13:09:54.829630 26870 raft_consensus.cc:3060] T 164a705fca2541d5a5be47dc3d77c40f P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:54.829648 26561 raft_consensus.cc:2468] T 015286f7e4be4234893d493fb008635e P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in term 1.
I20260709 13:09:54.830767 26675 leader_election.cc:304] T 015286f7e4be4234893d493fb008635e P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78, dc9195961d41473e8f590b632eb998a0; no voters: 
I20260709 13:09:54.831573 26636 raft_consensus.cc:2468] T 015286f7e4be4234893d493fb008635e P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in term 1.
I20260709 13:09:54.834116 26864 raft_consensus.cc:2804] T 015286f7e4be4234893d493fb008635e P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:54.834147 26894 raft_consensus.cc:493] T f1b70e3455b04e46a5c1b08d44fd2a06 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.834628 26864 raft_consensus.cc:697] T 015286f7e4be4234893d493fb008635e P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 LEADER]: Becoming Leader. State: Replica: 0a00d5d20d5b47b99a1220a0e40a4a78, State: Running, Role: LEADER
I20260709 13:09:54.834779 26894 raft_consensus.cc:515] T f1b70e3455b04e46a5c1b08d44fd2a06 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.835640 26870 raft_consensus.cc:2749] T 164a705fca2541d5a5be47dc3d77c40f P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:54.835453 26864 consensus_queue.cc:237] T 015286f7e4be4234893d493fb008635e P 0a00d5d20d5b47b99a1220a0e40a4a78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.837065 26894 leader_election.cc:290] T f1b70e3455b04e46a5c1b08d44fd2a06 P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:54.839056 26924 raft_consensus.cc:493] T 8f4b44b3bd734567b03eb09289c769f0 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.839596 26924 raft_consensus.cc:515] T 8f4b44b3bd734567b03eb09289c769f0 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.840389 26898 raft_consensus.cc:493] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.840880 26898 raft_consensus.cc:515] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.841454 26710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1b70e3455b04e46a5c1b08d44fd2a06" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" is_pre_election: true
I20260709 13:09:54.842017 26924 leader_election.cc:290] T 8f4b44b3bd734567b03eb09289c769f0 P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285)
I20260709 13:09:54.842216 26710 raft_consensus.cc:2393] T f1b70e3455b04e46a5c1b08d44fd2a06 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dc9195961d41473e8f590b632eb998a0 in current term 1: Already voted for candidate 68637fffcc4c4969ae0612be35aa58ec in this term.
I20260709 13:09:54.842805 26898 leader_election.cc:290] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:54.843003 26561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4b44b3bd734567b03eb09289c769f0" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0" is_pre_election: true
I20260709 13:09:54.843652 26561 raft_consensus.cc:2468] T 8f4b44b3bd734567b03eb09289c769f0 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in term 0.
I20260709 13:09:54.843931 26636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1b70e3455b04e46a5c1b08d44fd2a06" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68637fffcc4c4969ae0612be35aa58ec" is_pre_election: true
I20260709 13:09:54.845116 26633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4b44b3bd734567b03eb09289c769f0" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68637fffcc4c4969ae0612be35aa58ec" is_pre_election: true
I20260709 13:09:54.845322 26526 leader_election.cc:304] T f1b70e3455b04e46a5c1b08d44fd2a06 P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dc9195961d41473e8f590b632eb998a0; no voters: 0a00d5d20d5b47b99a1220a0e40a4a78, 68637fffcc4c4969ae0612be35aa58ec
I20260709 13:09:54.845640 26633 raft_consensus.cc:2468] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in term 0.
I20260709 13:09:54.846146 26894 raft_consensus.cc:3060] T f1b70e3455b04e46a5c1b08d44fd2a06 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:54.846886 26561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4b44b3bd734567b03eb09289c769f0" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0" is_pre_election: true
I20260709 13:09:54.847458 26561 raft_consensus.cc:2468] T 8f4b44b3bd734567b03eb09289c769f0 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 0.
I20260709 13:09:54.847034 26459 catalog_manager.cc:5697] T 015286f7e4be4234893d493fb008635e P 0a00d5d20d5b47b99a1220a0e40a4a78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3). New cstate: current_term: 1 leader_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:54.847715 26710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4b44b3bd734567b03eb09289c769f0" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" is_pre_election: true
I20260709 13:09:54.848331 26710 raft_consensus.cc:2468] T 8f4b44b3bd734567b03eb09289c769f0 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 0.
I20260709 13:09:54.849332 26601 leader_election.cc:304] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78, 68637fffcc4c4969ae0612be35aa58ec; no voters: 
I20260709 13:09:54.850018 26898 raft_consensus.cc:2804] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:54.850006 26675 leader_election.cc:304] T 8f4b44b3bd734567b03eb09289c769f0 P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78, dc9195961d41473e8f590b632eb998a0; no voters: 
I20260709 13:09:54.850612 26898 raft_consensus.cc:493] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:54.850979 26898 raft_consensus.cc:3060] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:54.851549 26924 raft_consensus.cc:2804] T 8f4b44b3bd734567b03eb09289c769f0 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:54.851969 26924 raft_consensus.cc:493] T 8f4b44b3bd734567b03eb09289c769f0 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:54.852284 26924 raft_consensus.cc:3060] T 8f4b44b3bd734567b03eb09289c769f0 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:54.853334 26894 raft_consensus.cc:2749] T f1b70e3455b04e46a5c1b08d44fd2a06 P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:54.857209 26898 raft_consensus.cc:515] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.859624 26898 leader_election.cc:290] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 election: Requested vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:54.859537 26924 raft_consensus.cc:515] T 8f4b44b3bd734567b03eb09289c769f0 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.860154 26561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4b44b3bd734567b03eb09289c769f0" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0"
I20260709 13:09:54.860730 26561 raft_consensus.cc:3060] T 8f4b44b3bd734567b03eb09289c769f0 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:54.861140 26710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4b44b3bd734567b03eb09289c769f0" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
I20260709 13:09:54.861992 26899 raft_consensus.cc:493] T c5e7fa4752c243c1bd88bedc411e5719 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.862397 26560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4b44b3bd734567b03eb09289c769f0" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0"
I20260709 13:09:54.862592 26924 leader_election.cc:290] T 8f4b44b3bd734567b03eb09289c769f0 P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 1 election: Requested vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285)
I20260709 13:09:54.862504 26899 raft_consensus.cc:515] T c5e7fa4752c243c1bd88bedc411e5719 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.863758 26710 raft_consensus.cc:2393] T 8f4b44b3bd734567b03eb09289c769f0 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 68637fffcc4c4969ae0612be35aa58ec in current term 1: Already voted for candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in this term.
I20260709 13:09:54.863696 26894 raft_consensus.cc:493] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.864518 26894 raft_consensus.cc:515] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.868045 26636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5e7fa4752c243c1bd88bedc411e5719" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68637fffcc4c4969ae0612be35aa58ec" is_pre_election: true
I20260709 13:09:54.868842 26636 raft_consensus.cc:2468] T c5e7fa4752c243c1bd88bedc411e5719 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc9195961d41473e8f590b632eb998a0 in term 0.
I20260709 13:09:54.870257 26526 leader_election.cc:304] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68637fffcc4c4969ae0612be35aa58ec, dc9195961d41473e8f590b632eb998a0; no voters: 
I20260709 13:09:54.873112 26710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5e7fa4752c243c1bd88bedc411e5719" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" is_pre_election: true
I20260709 13:09:54.873891 26710 raft_consensus.cc:2468] T c5e7fa4752c243c1bd88bedc411e5719 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc9195961d41473e8f590b632eb998a0 in term 0.
I20260709 13:09:54.873910 26633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4b44b3bd734567b03eb09289c769f0" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68637fffcc4c4969ae0612be35aa58ec"
I20260709 13:09:54.874868 26633 raft_consensus.cc:2393] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in current term 1: Already voted for candidate 68637fffcc4c4969ae0612be35aa58ec in this term.
I20260709 13:09:54.875281 26894 leader_election.cc:290] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:54.875998 26894 raft_consensus.cc:2804] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:54.876271 26894 raft_consensus.cc:493] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:54.876567 26894 raft_consensus.cc:3060] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:54.878046 26561 raft_consensus.cc:2468] T 8f4b44b3bd734567b03eb09289c769f0 P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 1.
I20260709 13:09:54.879176 26899 leader_election.cc:290] T c5e7fa4752c243c1bd88bedc411e5719 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:54.880439 26675 leader_election.cc:304] T 8f4b44b3bd734567b03eb09289c769f0 P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78; no voters: 68637fffcc4c4969ae0612be35aa58ec, dc9195961d41473e8f590b632eb998a0
I20260709 13:09:54.880664 26601 leader_election.cc:304] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 68637fffcc4c4969ae0612be35aa58ec, dc9195961d41473e8f590b632eb998a0; no voters: 0a00d5d20d5b47b99a1220a0e40a4a78
I20260709 13:09:54.881471 26561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5e7fa4752c243c1bd88bedc411e5719" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9195961d41473e8f590b632eb998a0" is_pre_election: true
I20260709 13:09:54.881604 26924 raft_consensus.cc:2749] T 8f4b44b3bd734567b03eb09289c769f0 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:09:54.881717 26899 raft_consensus.cc:2804] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:54.881801 26894 raft_consensus.cc:515] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.882536 26899 raft_consensus.cc:697] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Becoming Leader. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Running, Role: LEADER
I20260709 13:09:54.882754 26710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5e7fa4752c243c1bd88bedc411e5719" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" is_pre_election: true
I20260709 13:09:54.883620 26710 raft_consensus.cc:2468] T c5e7fa4752c243c1bd88bedc411e5719 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 0.
I20260709 13:09:54.883890 26899 consensus_queue.cc:237] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.884647 26894 leader_election.cc:290] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 1 election: Requested vote from peers 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:54.885386 26561 raft_consensus.cc:2393] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 68637fffcc4c4969ae0612be35aa58ec in current term 1: Already voted for candidate dc9195961d41473e8f590b632eb998a0 in this term.
I20260709 13:09:54.885764 26710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5e7fa4752c243c1bd88bedc411e5719" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
I20260709 13:09:54.885824 26633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5e7fa4752c243c1bd88bedc411e5719" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68637fffcc4c4969ae0612be35aa58ec"
I20260709 13:09:54.886569 26710 raft_consensus.cc:3060] T c5e7fa4752c243c1bd88bedc411e5719 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:54.885974 26601 leader_election.cc:304] T c5e7fa4752c243c1bd88bedc411e5719 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78, 68637fffcc4c4969ae0612be35aa58ec; no voters: 
I20260709 13:09:54.886986 26633 raft_consensus.cc:3060] T c5e7fa4752c243c1bd88bedc411e5719 P 68637fffcc4c4969ae0612be35aa58ec [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:54.894953 26710 raft_consensus.cc:2468] T c5e7fa4752c243c1bd88bedc411e5719 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9195961d41473e8f590b632eb998a0 in term 1.
I20260709 13:09:54.895032 26633 raft_consensus.cc:2468] T c5e7fa4752c243c1bd88bedc411e5719 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9195961d41473e8f590b632eb998a0 in term 1.
I20260709 13:09:54.895854 26898 raft_consensus.cc:2764] T c5e7fa4752c243c1bd88bedc411e5719 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:09:54.896395 26526 leader_election.cc:304] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68637fffcc4c4969ae0612be35aa58ec, dc9195961d41473e8f590b632eb998a0; no voters: 
I20260709 13:09:54.897274 26870 raft_consensus.cc:2804] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:54.895774 26459 catalog_manager.cc:5697] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2). New cstate: current_term: 1 leader_uuid: "68637fffcc4c4969ae0612be35aa58ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:54.898619 26870 raft_consensus.cc:697] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0 [term 1 LEADER]: Becoming Leader. State: Replica: dc9195961d41473e8f590b632eb998a0, State: Running, Role: LEADER
I20260709 13:09:54.899590 26870 consensus_queue.cc:237] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.902418 26894 raft_consensus.cc:493] T ebb142b13c4440339f42938e6c1cd02d P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.902920 26894 raft_consensus.cc:515] T ebb142b13c4440339f42938e6c1cd02d P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:54.904963 26894 leader_election.cc:290] T ebb142b13c4440339f42938e6c1cd02d P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
W20260709 13:09:54.906924 26507 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:54.906764 26633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb142b13c4440339f42938e6c1cd02d" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68637fffcc4c4969ae0612be35aa58ec" is_pre_election: true
I20260709 13:09:54.907624 26707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb142b13c4440339f42938e6c1cd02d" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" is_pre_election: true
I20260709 13:09:54.908593 26707 raft_consensus.cc:2393] T ebb142b13c4440339f42938e6c1cd02d P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dc9195961d41473e8f590b632eb998a0 in current term 1: Already voted for candidate 68637fffcc4c4969ae0612be35aa58ec in this term.
I20260709 13:09:54.909824 26526 leader_election.cc:304] T ebb142b13c4440339f42938e6c1cd02d P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dc9195961d41473e8f590b632eb998a0; no voters: 0a00d5d20d5b47b99a1220a0e40a4a78, 68637fffcc4c4969ae0612be35aa58ec
I20260709 13:09:54.910887 26894 raft_consensus.cc:3060] T ebb142b13c4440339f42938e6c1cd02d P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:54.913767 26459 catalog_manager.cc:5697] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc9195961d41473e8f590b632eb998a0 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "dc9195961d41473e8f590b632eb998a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:54.916199 26894 raft_consensus.cc:2749] T ebb142b13c4440339f42938e6c1cd02d P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:54.950161 26894 consensus_queue.cc:1048] T 707de517a3aa4064baa586aa4b8190e8 P dc9195961d41473e8f590b632eb998a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:54.961356 26870 consensus_queue.cc:1048] T 707de517a3aa4064baa586aa4b8190e8 P dc9195961d41473e8f590b632eb998a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:54.983119 26898 consensus_queue.cc:1048] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:54.993523 26898 consensus_queue.cc:1048] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:55.018210 26636 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ebb142b13c4440339f42938e6c1cd02d"
dest_uuid: "68637fffcc4c4969ae0612be35aa58ec"
mode: GRACEFUL
new_leader_uuid: "dc9195961d41473e8f590b632eb998a0"
 from {username='slave'} at 127.0.0.1:45340
I20260709 13:09:55.018833 26636 raft_consensus.cc:606] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Received request to transfer leadership to TS dc9195961d41473e8f590b632eb998a0
I20260709 13:09:55.023898 26636 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "164a705fca2541d5a5be47dc3d77c40f"
dest_uuid: "68637fffcc4c4969ae0612be35aa58ec"
mode: GRACEFUL
new_leader_uuid: "dc9195961d41473e8f590b632eb998a0"
 from {username='slave'} at 127.0.0.1:45340
I20260709 13:09:55.024403 26636 raft_consensus.cc:606] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Received request to transfer leadership to TS dc9195961d41473e8f590b632eb998a0
I20260709 13:09:55.025696 26852 consensus_queue.cc:1048] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:55.029752 26636 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f4b44b3bd734567b03eb09289c769f0"
dest_uuid: "68637fffcc4c4969ae0612be35aa58ec"
mode: GRACEFUL
new_leader_uuid: "dc9195961d41473e8f590b632eb998a0"
 from {username='slave'} at 127.0.0.1:45340
I20260709 13:09:55.030458 26636 raft_consensus.cc:606] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Received request to transfer leadership to TS dc9195961d41473e8f590b632eb998a0
I20260709 13:09:55.037643 26560 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5e7fa4752c243c1bd88bedc411e5719"
dest_uuid: "dc9195961d41473e8f590b632eb998a0"
mode: GRACEFUL
new_leader_uuid: "68637fffcc4c4969ae0612be35aa58ec"
 from {username='slave'} at 127.0.0.1:58516
I20260709 13:09:55.038225 26560 raft_consensus.cc:606] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0 [term 1 LEADER]: Received request to transfer leadership to TS 68637fffcc4c4969ae0612be35aa58ec
I20260709 13:09:55.040520 26891 consensus_queue.cc:1048] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:55.051616 26709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "015286f7e4be4234893d493fb008635e"
dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
mode: GRACEFUL
new_leader_uuid: "68637fffcc4c4969ae0612be35aa58ec"
 from {username='slave'} at 127.0.0.1:57210
I20260709 13:09:55.052251 26709 raft_consensus.cc:606] T 015286f7e4be4234893d493fb008635e P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 LEADER]: Received request to transfer leadership to TS 68637fffcc4c4969ae0612be35aa58ec
I20260709 13:09:55.133622 26852 consensus_queue.cc:1048] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:55.140966 26891 raft_consensus.cc:993] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec: : Instructing follower dc9195961d41473e8f590b632eb998a0 to start an election
I20260709 13:09:55.141433 26899 raft_consensus.cc:1081] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Signalling peer dc9195961d41473e8f590b632eb998a0 to start an election
I20260709 13:09:55.143594 26561 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ebb142b13c4440339f42938e6c1cd02d"
dest_uuid: "dc9195961d41473e8f590b632eb998a0"
 from {username='slave'} at 127.0.0.1:58510
I20260709 13:09:55.145030 26561 raft_consensus.cc:493] T ebb142b13c4440339f42938e6c1cd02d P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:55.145385 26561 raft_consensus.cc:3060] T ebb142b13c4440339f42938e6c1cd02d P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:55.146409 26891 consensus_queue.cc:1048] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:55.151901 26561 raft_consensus.cc:515] T ebb142b13c4440339f42938e6c1cd02d P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:55.157518 26561 leader_election.cc:290] T ebb142b13c4440339f42938e6c1cd02d P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 2 election: Requested vote from peers 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:55.159022 26709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb142b13c4440339f42938e6c1cd02d" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
I20260709 13:09:55.158780 26636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb142b13c4440339f42938e6c1cd02d" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "68637fffcc4c4969ae0612be35aa58ec"
I20260709 13:09:55.159920 26636 raft_consensus.cc:3055] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:55.160215 26636 raft_consensus.cc:740] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Running, Role: LEADER
I20260709 13:09:55.161211 26709 raft_consensus.cc:3060] T ebb142b13c4440339f42938e6c1cd02d P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:55.160936 26636 consensus_queue.cc:260] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:55.162226 26636 raft_consensus.cc:3060] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:55.168447 26636 raft_consensus.cc:2468] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9195961d41473e8f590b632eb998a0 in term 2.
I20260709 13:09:55.169268 26891 consensus_queue.cc:1048] T 751c0153edc248998be14ca90435f08a P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:55.169401 26526 leader_election.cc:304] T ebb142b13c4440339f42938e6c1cd02d P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68637fffcc4c4969ae0612be35aa58ec, dc9195961d41473e8f590b632eb998a0; no voters: 
I20260709 13:09:55.169651 26924 consensus_queue.cc:1048] T 8838204cfad3497aaec5d171bbbb38b3 P 0a00d5d20d5b47b99a1220a0e40a4a78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:55.170485 26870 raft_consensus.cc:2804] T ebb142b13c4440339f42938e6c1cd02d P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:55.171013 26870 raft_consensus.cc:697] T ebb142b13c4440339f42938e6c1cd02d P dc9195961d41473e8f590b632eb998a0 [term 2 LEADER]: Becoming Leader. State: Replica: dc9195961d41473e8f590b632eb998a0, State: Running, Role: LEADER
I20260709 13:09:55.171931 26870 consensus_queue.cc:237] T ebb142b13c4440339f42938e6c1cd02d P dc9195961d41473e8f590b632eb998a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:55.175550 26709 raft_consensus.cc:2468] T ebb142b13c4440339f42938e6c1cd02d P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9195961d41473e8f590b632eb998a0 in term 2.
I20260709 13:09:55.178020 26870 raft_consensus.cc:493] T 751c0153edc248998be14ca90435f08a P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:55.178530 26870 raft_consensus.cc:515] T 751c0153edc248998be14ca90435f08a P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:55.181422 26633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "751c0153edc248998be14ca90435f08a" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68637fffcc4c4969ae0612be35aa58ec" is_pre_election: true
I20260709 13:09:55.180967 26457 catalog_manager.cc:5697] T ebb142b13c4440339f42938e6c1cd02d P dc9195961d41473e8f590b632eb998a0 reported cstate change: term changed from 1 to 2, leader changed from 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2) to dc9195961d41473e8f590b632eb998a0 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "dc9195961d41473e8f590b632eb998a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:55.186620 26561 raft_consensus.cc:3060] T 751c0153edc248998be14ca90435f08a P dc9195961d41473e8f590b632eb998a0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:55.188076 26709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "751c0153edc248998be14ca90435f08a" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" is_pre_election: true
I20260709 13:09:55.186869 26870 leader_election.cc:290] T 751c0153edc248998be14ca90435f08a P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:55.190171 26526 leader_election.cc:304] T 751c0153edc248998be14ca90435f08a P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dc9195961d41473e8f590b632eb998a0; no voters: 0a00d5d20d5b47b99a1220a0e40a4a78, 68637fffcc4c4969ae0612be35aa58ec
I20260709 13:09:55.194319 26897 raft_consensus.cc:2749] T 751c0153edc248998be14ca90435f08a P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:55.200675 26943 consensus_queue.cc:1048] T 8838204cfad3497aaec5d171bbbb38b3 P 0a00d5d20d5b47b99a1220a0e40a4a78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:55.201270 26852 consensus_queue.cc:1048] T 751c0153edc248998be14ca90435f08a P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:55.226334 26943 consensus_queue.cc:1048] T 015286f7e4be4234893d493fb008635e P 0a00d5d20d5b47b99a1220a0e40a4a78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:55.240185 26864 consensus_queue.cc:1048] T 015286f7e4be4234893d493fb008635e P 0a00d5d20d5b47b99a1220a0e40a4a78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:55.247133 26864 raft_consensus.cc:993] T 015286f7e4be4234893d493fb008635e P 0a00d5d20d5b47b99a1220a0e40a4a78: : Instructing follower 68637fffcc4c4969ae0612be35aa58ec to start an election
I20260709 13:09:55.247596 26864 raft_consensus.cc:1081] T 015286f7e4be4234893d493fb008635e P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 LEADER]: Signalling peer 68637fffcc4c4969ae0612be35aa58ec to start an election
I20260709 13:09:55.250733 26636 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "015286f7e4be4234893d493fb008635e"
dest_uuid: "68637fffcc4c4969ae0612be35aa58ec"
 from {username='slave'} at 127.0.0.1:45328
I20260709 13:09:55.251216 26636 raft_consensus.cc:493] T 015286f7e4be4234893d493fb008635e P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:55.251547 26636 raft_consensus.cc:3060] T 015286f7e4be4234893d493fb008635e P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:55.257301 26636 raft_consensus.cc:515] T 015286f7e4be4234893d493fb008635e P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:55.258831 26636 leader_election.cc:290] T 015286f7e4be4234893d493fb008635e P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 2 election: Requested vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:55.262449 26709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "015286f7e4be4234893d493fb008635e" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
I20260709 13:09:55.261979 26560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "015286f7e4be4234893d493fb008635e" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dc9195961d41473e8f590b632eb998a0"
I20260709 13:09:55.263628 26560 raft_consensus.cc:3060] T 015286f7e4be4234893d493fb008635e P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:55.263077 26709 raft_consensus.cc:3055] T 015286f7e4be4234893d493fb008635e P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:55.264412 26709 raft_consensus.cc:740] T 015286f7e4be4234893d493fb008635e P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a00d5d20d5b47b99a1220a0e40a4a78, State: Running, Role: LEADER
I20260709 13:09:55.265277 26709 consensus_queue.cc:260] T 015286f7e4be4234893d493fb008635e P 0a00d5d20d5b47b99a1220a0e40a4a78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:55.270675 26560 raft_consensus.cc:2468] T 015286f7e4be4234893d493fb008635e P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 2.
I20260709 13:09:55.271826 26709 raft_consensus.cc:3060] T 015286f7e4be4234893d493fb008635e P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:55.271656 26601 leader_election.cc:304] T 015286f7e4be4234893d493fb008635e P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68637fffcc4c4969ae0612be35aa58ec, dc9195961d41473e8f590b632eb998a0; no voters: 
I20260709 13:09:55.272714 26891 raft_consensus.cc:2804] T 015286f7e4be4234893d493fb008635e P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:55.273169 26891 raft_consensus.cc:697] T 015286f7e4be4234893d493fb008635e P 68637fffcc4c4969ae0612be35aa58ec [term 2 LEADER]: Becoming Leader. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Running, Role: LEADER
I20260709 13:09:55.273981 26891 consensus_queue.cc:237] T 015286f7e4be4234893d493fb008635e P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:55.278601 26709 raft_consensus.cc:2468] T 015286f7e4be4234893d493fb008635e P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 2.
I20260709 13:09:55.287061 26891 consensus_queue.cc:1048] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:55.286530 26457 catalog_manager.cc:5697] T 015286f7e4be4234893d493fb008635e P 68637fffcc4c4969ae0612be35aa58ec reported cstate change: term changed from 1 to 2, leader changed from 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3) to 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2). New cstate: current_term: 2 leader_uuid: "68637fffcc4c4969ae0612be35aa58ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:55.298790 26899 consensus_queue.cc:1048] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:55.307446 26852 raft_consensus.cc:993] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec: : Instructing follower dc9195961d41473e8f590b632eb998a0 to start an election
I20260709 13:09:55.307920 26899 raft_consensus.cc:1081] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Signalling peer dc9195961d41473e8f590b632eb998a0 to start an election
I20260709 13:09:55.309413 26560 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8f4b44b3bd734567b03eb09289c769f0"
dest_uuid: "dc9195961d41473e8f590b632eb998a0"
 from {username='slave'} at 127.0.0.1:58510
I20260709 13:09:55.309950 26560 raft_consensus.cc:493] T 8f4b44b3bd734567b03eb09289c769f0 P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:55.310307 26560 raft_consensus.cc:3060] T 8f4b44b3bd734567b03eb09289c769f0 P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:55.312990 26897 consensus_queue.cc:1048] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:55.316875 26560 raft_consensus.cc:515] T 8f4b44b3bd734567b03eb09289c769f0 P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:55.319103 26560 leader_election.cc:290] T 8f4b44b3bd734567b03eb09289c769f0 P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 2 election: Requested vote from peers 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:55.320212 26707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4b44b3bd734567b03eb09289c769f0" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
I20260709 13:09:55.320782 26707 raft_consensus.cc:3060] T 8f4b44b3bd734567b03eb09289c769f0 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:55.321967 26636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f4b44b3bd734567b03eb09289c769f0" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "68637fffcc4c4969ae0612be35aa58ec"
I20260709 13:09:55.322515 26636 raft_consensus.cc:3055] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:55.322789 26636 raft_consensus.cc:740] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Running, Role: LEADER
I20260709 13:09:55.323490 26636 consensus_queue.cc:260] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:55.324669 26636 raft_consensus.cc:3060] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:55.328143 26870 consensus_queue.cc:1048] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:55.328382 26707 raft_consensus.cc:2468] T 8f4b44b3bd734567b03eb09289c769f0 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9195961d41473e8f590b632eb998a0 in term 2.
I20260709 13:09:55.335610 26870 raft_consensus.cc:993] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0: : Instructing follower 68637fffcc4c4969ae0612be35aa58ec to start an election
I20260709 13:09:55.336107 26897 raft_consensus.cc:1081] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0 [term 1 LEADER]: Signalling peer 68637fffcc4c4969ae0612be35aa58ec to start an election
I20260709 13:09:55.337423 26526 leader_election.cc:304] T 8f4b44b3bd734567b03eb09289c769f0 P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78, dc9195961d41473e8f590b632eb998a0; no voters: 
I20260709 13:09:55.337438 26636 raft_consensus.cc:2468] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9195961d41473e8f590b632eb998a0 in term 2.
I20260709 13:09:55.337694 26633 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c5e7fa4752c243c1bd88bedc411e5719"
dest_uuid: "68637fffcc4c4969ae0612be35aa58ec"
 from {username='slave'} at 127.0.0.1:45338
I20260709 13:09:55.338383 26633 raft_consensus.cc:493] T c5e7fa4752c243c1bd88bedc411e5719 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:55.338762 26633 raft_consensus.cc:3060] T c5e7fa4752c243c1bd88bedc411e5719 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:55.339864 26897 raft_consensus.cc:2804] T 8f4b44b3bd734567b03eb09289c769f0 P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:55.340384 26897 raft_consensus.cc:697] T 8f4b44b3bd734567b03eb09289c769f0 P dc9195961d41473e8f590b632eb998a0 [term 2 LEADER]: Becoming Leader. State: Replica: dc9195961d41473e8f590b632eb998a0, State: Running, Role: LEADER
I20260709 13:09:55.341066 26897 consensus_queue.cc:237] T 8f4b44b3bd734567b03eb09289c769f0 P dc9195961d41473e8f590b632eb998a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:55.343978 26633 raft_consensus.cc:515] T c5e7fa4752c243c1bd88bedc411e5719 P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:55.346673 26560 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5e7fa4752c243c1bd88bedc411e5719" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dc9195961d41473e8f590b632eb998a0"
I20260709 13:09:55.347252 26633 leader_election.cc:290] T c5e7fa4752c243c1bd88bedc411e5719 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 2 election: Requested vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:55.347983 26560 raft_consensus.cc:3055] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:55.348376 26560 raft_consensus.cc:740] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dc9195961d41473e8f590b632eb998a0, State: Running, Role: LEADER
I20260709 13:09:55.349206 26560 consensus_queue.cc:260] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:55.350426 26560 raft_consensus.cc:3060] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:55.350607 26456 catalog_manager.cc:5697] T 8f4b44b3bd734567b03eb09289c769f0 P dc9195961d41473e8f590b632eb998a0 reported cstate change: term changed from 1 to 2, leader changed from 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2) to dc9195961d41473e8f590b632eb998a0 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "dc9195961d41473e8f590b632eb998a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:55.348398 26707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5e7fa4752c243c1bd88bedc411e5719" candidate_uuid: "68637fffcc4c4969ae0612be35aa58ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
I20260709 13:09:55.359498 26560 raft_consensus.cc:2468] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 2.
I20260709 13:09:55.360198 26707 raft_consensus.cc:3060] T c5e7fa4752c243c1bd88bedc411e5719 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:55.360798 26601 leader_election.cc:304] T c5e7fa4752c243c1bd88bedc411e5719 P 68637fffcc4c4969ae0612be35aa58ec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68637fffcc4c4969ae0612be35aa58ec, dc9195961d41473e8f590b632eb998a0; no voters: 
I20260709 13:09:55.361521 26899 raft_consensus.cc:2804] T c5e7fa4752c243c1bd88bedc411e5719 P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:55.361989 26899 raft_consensus.cc:697] T c5e7fa4752c243c1bd88bedc411e5719 P 68637fffcc4c4969ae0612be35aa58ec [term 2 LEADER]: Becoming Leader. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Running, Role: LEADER
I20260709 13:09:55.362761 26899 consensus_queue.cc:237] T c5e7fa4752c243c1bd88bedc411e5719 P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:55.367898 26707 raft_consensus.cc:2468] T c5e7fa4752c243c1bd88bedc411e5719 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68637fffcc4c4969ae0612be35aa58ec in term 2.
I20260709 13:09:55.370617 26456 catalog_manager.cc:5697] T c5e7fa4752c243c1bd88bedc411e5719 P 68637fffcc4c4969ae0612be35aa58ec reported cstate change: term changed from 1 to 2, leader changed from dc9195961d41473e8f590b632eb998a0 (127.14.103.1) to 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2). New cstate: current_term: 2 leader_uuid: "68637fffcc4c4969ae0612be35aa58ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:55.600875 26707 raft_consensus.cc:1275] T ebb142b13c4440339f42938e6c1cd02d P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Refusing update from remote peer dc9195961d41473e8f590b632eb998a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:55.602195 26899 raft_consensus.cc:993] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec: : Instructing follower dc9195961d41473e8f590b632eb998a0 to start an election
I20260709 13:09:55.602766 26898 raft_consensus.cc:1081] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Signalling peer dc9195961d41473e8f590b632eb998a0 to start an election
I20260709 13:09:55.602419 26897 consensus_queue.cc:1048] T ebb142b13c4440339f42938e6c1cd02d P dc9195961d41473e8f590b632eb998a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:55.604229 26560 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "164a705fca2541d5a5be47dc3d77c40f"
dest_uuid: "dc9195961d41473e8f590b632eb998a0"
 from {username='slave'} at 127.0.0.1:58510
I20260709 13:09:55.605386 26560 raft_consensus.cc:493] T 164a705fca2541d5a5be47dc3d77c40f P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:55.605752 26560 raft_consensus.cc:3060] T 164a705fca2541d5a5be47dc3d77c40f P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:55.610981 26633 raft_consensus.cc:1275] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Refusing update from remote peer dc9195961d41473e8f590b632eb998a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:55.614745 26560 raft_consensus.cc:515] T 164a705fca2541d5a5be47dc3d77c40f P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:55.615621 26870 consensus_queue.cc:1048] T ebb142b13c4440339f42938e6c1cd02d P dc9195961d41473e8f590b632eb998a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:55.616897 26560 leader_election.cc:290] T 164a705fca2541d5a5be47dc3d77c40f P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 2 election: Requested vote from peers 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285), 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3:36339)
I20260709 13:09:55.618039 26633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "164a705fca2541d5a5be47dc3d77c40f" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "68637fffcc4c4969ae0612be35aa58ec"
I20260709 13:09:55.618613 26633 raft_consensus.cc:3055] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:55.618957 26633 raft_consensus.cc:740] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Running, Role: LEADER
I20260709 13:09:55.619644 26633 consensus_queue.cc:260] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:55.620980 26633 raft_consensus.cc:3060] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:55.624073 26707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "164a705fca2541d5a5be47dc3d77c40f" candidate_uuid: "dc9195961d41473e8f590b632eb998a0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
I20260709 13:09:55.624667 26707 raft_consensus.cc:3060] T 164a705fca2541d5a5be47dc3d77c40f P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:55.628492 26633 raft_consensus.cc:2468] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9195961d41473e8f590b632eb998a0 in term 2.
I20260709 13:09:55.632058 26526 leader_election.cc:304] T 164a705fca2541d5a5be47dc3d77c40f P dc9195961d41473e8f590b632eb998a0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68637fffcc4c4969ae0612be35aa58ec, dc9195961d41473e8f590b632eb998a0; no voters: 
I20260709 13:09:55.632829 26897 raft_consensus.cc:2804] T 164a705fca2541d5a5be47dc3d77c40f P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:55.633273 26897 raft_consensus.cc:697] T 164a705fca2541d5a5be47dc3d77c40f P dc9195961d41473e8f590b632eb998a0 [term 2 LEADER]: Becoming Leader. State: Replica: dc9195961d41473e8f590b632eb998a0, State: Running, Role: LEADER
I20260709 13:09:55.634042 26897 consensus_queue.cc:237] T 164a705fca2541d5a5be47dc3d77c40f P dc9195961d41473e8f590b632eb998a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:55.637758 26707 raft_consensus.cc:2468] T 164a705fca2541d5a5be47dc3d77c40f P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9195961d41473e8f590b632eb998a0 in term 2.
I20260709 13:09:55.641108 26457 catalog_manager.cc:5697] T 164a705fca2541d5a5be47dc3d77c40f P dc9195961d41473e8f590b632eb998a0 reported cstate change: term changed from 1 to 2, leader changed from 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2) to dc9195961d41473e8f590b632eb998a0 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "dc9195961d41473e8f590b632eb998a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:55.720228 26707 raft_consensus.cc:1275] T 015286f7e4be4234893d493fb008635e P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Refusing update from remote peer 68637fffcc4c4969ae0612be35aa58ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:55.721588 26898 consensus_queue.cc:1048] T 015286f7e4be4234893d493fb008635e P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:55.730938 26560 raft_consensus.cc:1275] T 015286f7e4be4234893d493fb008635e P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Refusing update from remote peer 68637fffcc4c4969ae0612be35aa58ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:55.733585 26891 consensus_queue.cc:1048] T 015286f7e4be4234893d493fb008635e P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:55.742271 26707 raft_consensus.cc:1275] T c5e7fa4752c243c1bd88bedc411e5719 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Refusing update from remote peer 68637fffcc4c4969ae0612be35aa58ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:55.743343 26852 consensus_queue.cc:1048] T c5e7fa4752c243c1bd88bedc411e5719 P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:55.750695 26560 raft_consensus.cc:1275] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Refusing update from remote peer 68637fffcc4c4969ae0612be35aa58ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:55.752267 26899 consensus_queue.cc:1048] T c5e7fa4752c243c1bd88bedc411e5719 P 68637fffcc4c4969ae0612be35aa58ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:55.874480 26707 raft_consensus.cc:1275] T 8f4b44b3bd734567b03eb09289c769f0 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Refusing update from remote peer dc9195961d41473e8f590b632eb998a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:55.875814 26952 consensus_queue.cc:1048] T 8f4b44b3bd734567b03eb09289c769f0 P dc9195961d41473e8f590b632eb998a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:55.882764 26633 raft_consensus.cc:1275] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Refusing update from remote peer dc9195961d41473e8f590b632eb998a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:55.884910 26870 consensus_queue.cc:1048] T 8f4b44b3bd734567b03eb09289c769f0 P dc9195961d41473e8f590b632eb998a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:56.055832 26633 raft_consensus.cc:1275] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Refusing update from remote peer dc9195961d41473e8f590b632eb998a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:56.057078 26870 consensus_queue.cc:1048] T 164a705fca2541d5a5be47dc3d77c40f P dc9195961d41473e8f590b632eb998a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:56.064996 26707 raft_consensus.cc:1275] T 164a705fca2541d5a5be47dc3d77c40f P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Refusing update from remote peer dc9195961d41473e8f590b632eb998a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:56.068619 26952 consensus_queue.cc:1048] T 164a705fca2541d5a5be47dc3d77c40f P dc9195961d41473e8f590b632eb998a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:58.058449 26459 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55310:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:09:58.077998 26459 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55310:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:09:58.177788 14748 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:09:58.235935 26979 tablet.cc:1724] T 707de517a3aa4064baa586aa4b8190e8 P 68637fffcc4c4969ae0612be35aa58ec: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:09:58.261129 26662 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:707de517a3aa4064baa586aa4b8190e8)
I20260709 13:09:58.320255 26978 tablet.cc:1724] T 707de517a3aa4064baa586aa4b8190e8 P dc9195961d41473e8f590b632eb998a0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:58.377784 27005 tablet.cc:1724] T ebb142b13c4440339f42938e6c1cd02d P dc9195961d41473e8f590b632eb998a0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:09:58.413779 26587 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:707de517a3aa4064baa586aa4b8190e8)
W20260709 13:09:58.409313 26662 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:707de517a3aa4064baa586aa4b8190e8)
I20260709 13:09:58.455399 27004 tablet.cc:1724] T 164a705fca2541d5a5be47dc3d77c40f P dc9195961d41473e8f590b632eb998a0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:09:58.464059 26601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.3:36339, user_credentials={real_user=slave}} blocked reactor thread for 53671.4us
I20260709 13:09:58.483143 27019 tablet.cc:1724] T 8838204cfad3497aaec5d171bbbb38b3 P 0a00d5d20d5b47b99a1220a0e40a4a78: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:58.478782 27015 tablet.cc:1724] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:58.487140 27017 tablet.cc:1724] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:58.475497 27013 tablet.cc:1724] T 164a705fca2541d5a5be47dc3d77c40f P 0a00d5d20d5b47b99a1220a0e40a4a78: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:58.472417 27020 tablet.cc:1724] T ebb142b13c4440339f42938e6c1cd02d P 0a00d5d20d5b47b99a1220a0e40a4a78: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:09:58.531230 27014 log.cc:927] Time spent T c5e7fa4752c243c1bd88bedc411e5719 P 0a00d5d20d5b47b99a1220a0e40a4a78: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:09:58.546089 26979 tablet.cc:1724] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:58.572407 27024 tablet.cc:1724] T 8838204cfad3497aaec5d171bbbb38b3 P 68637fffcc4c4969ae0612be35aa58ec: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:58.572937 27022 tablet.cc:1724] T 707de517a3aa4064baa586aa4b8190e8 P 0a00d5d20d5b47b99a1220a0e40a4a78: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:58.575589 27017 tablet.cc:1724] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:58.579248 27004 tablet.cc:1724] T f1b70e3455b04e46a5c1b08d44fd2a06 P dc9195961d41473e8f590b632eb998a0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:58.597064 27021 tablet.cc:1724] T 8f4b44b3bd734567b03eb09289c769f0 P dc9195961d41473e8f590b632eb998a0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:58.616426 27013 tablet.cc:1724] T 8f4b44b3bd734567b03eb09289c769f0 P 0a00d5d20d5b47b99a1220a0e40a4a78: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:58.584905 26978 tablet.cc:1724] T 8838204cfad3497aaec5d171bbbb38b3 P dc9195961d41473e8f590b632eb998a0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:58.622337 26559 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "707de517a3aa4064baa586aa4b8190e8"
dest_uuid: "dc9195961d41473e8f590b632eb998a0"
mode: GRACEFUL
new_leader_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
 from {username='slave'} at 127.0.0.1:34822
I20260709 13:09:58.623152 26559 raft_consensus.cc:606] T 707de517a3aa4064baa586aa4b8190e8 P dc9195961d41473e8f590b632eb998a0 [term 1 LEADER]: Received request to transfer leadership to TS 0a00d5d20d5b47b99a1220a0e40a4a78
I20260709 13:09:58.624295 26459 catalog_manager.cc:5789] TS dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787) does not have the latest schema for tablet 8f4b44b3bd734567b03eb09289c769f0 (table soft_deleted_table [id=3c9aff1183bc43ef8c693879ccb7b87a]). Expected version 1 got 0
I20260709 13:09:58.631207 27020 tablet.cc:1724] T c5e7fa4752c243c1bd88bedc411e5719 P 0a00d5d20d5b47b99a1220a0e40a4a78: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:58.631515 27004 tablet.cc:1724] T 015286f7e4be4234893d493fb008635e P dc9195961d41473e8f590b632eb998a0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:58.626947 27015 tablet.cc:1724] T 015286f7e4be4234893d493fb008635e P 68637fffcc4c4969ae0612be35aa58ec: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:58.592777 27017 tablet.cc:1724] T c5e7fa4752c243c1bd88bedc411e5719 P 68637fffcc4c4969ae0612be35aa58ec: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:58.644004 26457 catalog_manager.cc:5789] TS 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285) does not have the latest schema for tablet 015286f7e4be4234893d493fb008635e (table soft_deleted_table [id=3c9aff1183bc43ef8c693879ccb7b87a]). Expected version 1 got 0
I20260709 13:09:58.645612 26457 catalog_manager.cc:5789] TS 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285) does not have the latest schema for tablet 751c0153edc248998be14ca90435f08a (table soft_deleted_table [id=3c9aff1183bc43ef8c693879ccb7b87a]). Expected version 1 got 0
I20260709 13:09:58.646425 26457 catalog_manager.cc:5789] TS 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285) does not have the latest schema for tablet c5e7fa4752c243c1bd88bedc411e5719 (table soft_deleted_table [id=3c9aff1183bc43ef8c693879ccb7b87a]). Expected version 1 got 0
I20260709 13:09:58.661087 27024 tablet.cc:1724] T 751c0153edc248998be14ca90435f08a P 68637fffcc4c4969ae0612be35aa58ec: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:58.666090 27005 tablet.cc:1724] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:58.674078 27020 tablet.cc:1724] T 751c0153edc248998be14ca90435f08a P 0a00d5d20d5b47b99a1220a0e40a4a78: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:58.668053 26978 tablet.cc:1724] T 751c0153edc248998be14ca90435f08a P dc9195961d41473e8f590b632eb998a0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:58.675607 27019 tablet.cc:1724] T 015286f7e4be4234893d493fb008635e P 0a00d5d20d5b47b99a1220a0e40a4a78: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:58.663532 27022 tablet.cc:1724] T f1b70e3455b04e46a5c1b08d44fd2a06 P 0a00d5d20d5b47b99a1220a0e40a4a78: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:58.706564 26632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f1b70e3455b04e46a5c1b08d44fd2a06"
dest_uuid: "68637fffcc4c4969ae0612be35aa58ec"
mode: GRACEFUL
new_leader_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
 from {username='slave'} at 127.0.0.1:58690
I20260709 13:09:58.707134 26632 raft_consensus.cc:606] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Received request to transfer leadership to TS 0a00d5d20d5b47b99a1220a0e40a4a78
I20260709 13:09:58.708756 14748 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 13:09:58.709466 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:58.723995 26632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "579b5cf4a4874b75a67f9cbf320addc2"
dest_uuid: "68637fffcc4c4969ae0612be35aa58ec"
mode: GRACEFUL
new_leader_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
 from {username='slave'} at 127.0.0.1:58690
I20260709 13:09:58.724632 26632 raft_consensus.cc:606] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [term 3 LEADER]: Received request to transfer leadership to TS 0a00d5d20d5b47b99a1220a0e40a4a78
I20260709 13:09:58.730525 26558 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b16a06ac45ab44a186fbf820559732bb"
dest_uuid: "dc9195961d41473e8f590b632eb998a0"
mode: GRACEFUL
new_leader_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
 from {username='slave'} at 127.0.0.1:34822
I20260709 13:09:58.731217 26558 raft_consensus.cc:606] T b16a06ac45ab44a186fbf820559732bb P dc9195961d41473e8f590b632eb998a0 [term 2 LEADER]: Received request to transfer leadership to TS 0a00d5d20d5b47b99a1220a0e40a4a78
I20260709 13:09:58.732780 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:09:58.733474 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:09:58.734321 26980 raft_consensus.cc:993] T b16a06ac45ab44a186fbf820559732bb P dc9195961d41473e8f590b632eb998a0: : Instructing follower 0a00d5d20d5b47b99a1220a0e40a4a78 to start an election
I20260709 13:09:58.735033 26897 raft_consensus.cc:1081] T b16a06ac45ab44a186fbf820559732bb P dc9195961d41473e8f590b632eb998a0 [term 2 LEADER]: Signalling peer 0a00d5d20d5b47b99a1220a0e40a4a78 to start an election
I20260709 13:09:58.734368 26459 catalog_manager.cc:6242] T 00000000000000000000000000000000 P e68faef5ba744349a655384e3cc32fd8: soft_deleted_table [id=3c9aff1183bc43ef8c693879ccb7b87a] alter complete (version 1)
I20260709 13:09:58.736848 26709 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b16a06ac45ab44a186fbf820559732bb"
dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
 from {username='slave'} at 127.0.0.1:57200
I20260709 13:09:58.738001 26709 raft_consensus.cc:493] T b16a06ac45ab44a186fbf820559732bb P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:58.738395 26709 raft_consensus.cc:3060] T b16a06ac45ab44a186fbf820559732bb P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:58.746487 26709 raft_consensus.cc:515] T b16a06ac45ab44a186fbf820559732bb P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:58.748404 26709 leader_election.cc:290] T b16a06ac45ab44a186fbf820559732bb P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 3 election: Requested vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285)
I20260709 13:09:58.749125 26558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b16a06ac45ab44a186fbf820559732bb" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "dc9195961d41473e8f590b632eb998a0"
I20260709 13:09:58.749413 26632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b16a06ac45ab44a186fbf820559732bb" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "68637fffcc4c4969ae0612be35aa58ec"
I20260709 13:09:58.749861 26558 raft_consensus.cc:3055] T b16a06ac45ab44a186fbf820559732bb P dc9195961d41473e8f590b632eb998a0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:58.750171 26558 raft_consensus.cc:740] T b16a06ac45ab44a186fbf820559732bb P dc9195961d41473e8f590b632eb998a0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: dc9195961d41473e8f590b632eb998a0, State: Running, Role: LEADER
I20260709 13:09:58.750224 26632 raft_consensus.cc:3060] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:58.750939 26558 consensus_queue.cc:260] T b16a06ac45ab44a186fbf820559732bb P dc9195961d41473e8f590b632eb998a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:58.751961 26558 raft_consensus.cc:3060] T b16a06ac45ab44a186fbf820559732bb P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:58.757181 26632 raft_consensus.cc:2468] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in term 3.
I20260709 13:09:58.758322 26675 leader_election.cc:304] T b16a06ac45ab44a186fbf820559732bb P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78, 68637fffcc4c4969ae0612be35aa58ec; no voters: 
I20260709 13:09:58.758603 26558 raft_consensus.cc:2468] T b16a06ac45ab44a186fbf820559732bb P dc9195961d41473e8f590b632eb998a0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in term 3.
I20260709 13:09:58.759186 26985 raft_consensus.cc:2804] T b16a06ac45ab44a186fbf820559732bb P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:58.759780 26985 raft_consensus.cc:697] T b16a06ac45ab44a186fbf820559732bb P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 3 LEADER]: Becoming Leader. State: Replica: 0a00d5d20d5b47b99a1220a0e40a4a78, State: Running, Role: LEADER
I20260709 13:09:58.760661 26985 consensus_queue.cc:237] T b16a06ac45ab44a186fbf820559732bb P 0a00d5d20d5b47b99a1220a0e40a4a78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:58.769616 26457 catalog_manager.cc:5697] T b16a06ac45ab44a186fbf820559732bb P 0a00d5d20d5b47b99a1220a0e40a4a78 reported cstate change: term changed from 2 to 3, leader changed from dc9195961d41473e8f590b632eb998a0 (127.14.103.1) to 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3). New cstate: current_term: 3 leader_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:58.991912 26897 raft_consensus.cc:993] T 707de517a3aa4064baa586aa4b8190e8 P dc9195961d41473e8f590b632eb998a0: : Instructing follower 0a00d5d20d5b47b99a1220a0e40a4a78 to start an election
I20260709 13:09:58.992441 26980 raft_consensus.cc:1081] T 707de517a3aa4064baa586aa4b8190e8 P dc9195961d41473e8f590b632eb998a0 [term 1 LEADER]: Signalling peer 0a00d5d20d5b47b99a1220a0e40a4a78 to start an election
I20260709 13:09:58.993219 27009 raft_consensus.cc:993] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec: : Instructing follower 0a00d5d20d5b47b99a1220a0e40a4a78 to start an election
I20260709 13:09:58.993881 27016 raft_consensus.cc:1081] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [term 3 LEADER]: Signalling peer 0a00d5d20d5b47b99a1220a0e40a4a78 to start an election
I20260709 13:09:58.994011 26709 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "707de517a3aa4064baa586aa4b8190e8"
dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
 from {username='slave'} at 127.0.0.1:57200
I20260709 13:09:58.994570 26709 raft_consensus.cc:493] T 707de517a3aa4064baa586aa4b8190e8 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:58.994874 26709 raft_consensus.cc:3060] T 707de517a3aa4064baa586aa4b8190e8 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:58.995482 26703 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "579b5cf4a4874b75a67f9cbf320addc2"
dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
 from {username='slave'} at 127.0.0.1:57192
I20260709 13:09:58.996052 26703 raft_consensus.cc:493] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:58.996369 26703 raft_consensus.cc:3060] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:59.000196 26709 raft_consensus.cc:515] T 707de517a3aa4064baa586aa4b8190e8 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:59.001449 26703 raft_consensus.cc:515] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:59.001997 26709 leader_election.cc:290] T 707de517a3aa4064baa586aa4b8190e8 P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 2 election: Requested vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285)
I20260709 13:09:59.002673 26558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "707de517a3aa4064baa586aa4b8190e8" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "dc9195961d41473e8f590b632eb998a0"
I20260709 13:09:59.003288 26558 raft_consensus.cc:3055] T 707de517a3aa4064baa586aa4b8190e8 P dc9195961d41473e8f590b632eb998a0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:59.003242 26632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "707de517a3aa4064baa586aa4b8190e8" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "68637fffcc4c4969ae0612be35aa58ec"
I20260709 13:09:59.003849 26558 raft_consensus.cc:740] T 707de517a3aa4064baa586aa4b8190e8 P dc9195961d41473e8f590b632eb998a0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dc9195961d41473e8f590b632eb998a0, State: Running, Role: LEADER
I20260709 13:09:59.003854 26703 leader_election.cc:290] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 4 election: Requested vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285)
I20260709 13:09:59.004869 26557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579b5cf4a4874b75a67f9cbf320addc2" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "dc9195961d41473e8f590b632eb998a0"
I20260709 13:09:59.011401 26557 raft_consensus.cc:3060] T 579b5cf4a4874b75a67f9cbf320addc2 P dc9195961d41473e8f590b632eb998a0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:59.011816 26558 consensus_queue.cc:260] T 707de517a3aa4064baa586aa4b8190e8 P dc9195961d41473e8f590b632eb998a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:59.012319 26632 raft_consensus.cc:3060] T 707de517a3aa4064baa586aa4b8190e8 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:59.013114 26558 raft_consensus.cc:3060] T 707de517a3aa4064baa586aa4b8190e8 P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:59.005136 26634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579b5cf4a4874b75a67f9cbf320addc2" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "68637fffcc4c4969ae0612be35aa58ec"
I20260709 13:09:59.013970 26634 raft_consensus.cc:3055] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:09:59.014361 26634 raft_consensus.cc:740] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Running, Role: LEADER
I20260709 13:09:59.015327 26634 consensus_queue.cc:260] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:59.016624 26634 raft_consensus.cc:3060] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:59.019595 26632 raft_consensus.cc:2468] T 707de517a3aa4064baa586aa4b8190e8 P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in term 2.
I20260709 13:09:59.022063 26675 leader_election.cc:304] T 707de517a3aa4064baa586aa4b8190e8 P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78, 68637fffcc4c4969ae0612be35aa58ec; no voters: 
I20260709 13:09:59.022893 26985 raft_consensus.cc:2804] T 707de517a3aa4064baa586aa4b8190e8 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:59.023423 26985 raft_consensus.cc:697] T 707de517a3aa4064baa586aa4b8190e8 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 LEADER]: Becoming Leader. State: Replica: 0a00d5d20d5b47b99a1220a0e40a4a78, State: Running, Role: LEADER
I20260709 13:09:59.024271 26985 consensus_queue.cc:237] T 707de517a3aa4064baa586aa4b8190e8 P 0a00d5d20d5b47b99a1220a0e40a4a78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:59.024940 26634 raft_consensus.cc:2468] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in term 4.
I20260709 13:09:59.025990 26675 leader_election.cc:304] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78, 68637fffcc4c4969ae0612be35aa58ec; no voters: 
I20260709 13:09:59.026878 26954 raft_consensus.cc:2804] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:09:59.028106 26954 raft_consensus.cc:697] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 4 LEADER]: Becoming Leader. State: Replica: 0a00d5d20d5b47b99a1220a0e40a4a78, State: Running, Role: LEADER
I20260709 13:09:59.028875 26557 raft_consensus.cc:2468] T 579b5cf4a4874b75a67f9cbf320addc2 P dc9195961d41473e8f590b632eb998a0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in term 4.
I20260709 13:09:59.028935 26954 consensus_queue.cc:237] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:59.026381 26558 raft_consensus.cc:2468] T 707de517a3aa4064baa586aa4b8190e8 P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in term 2.
I20260709 13:09:59.037796 26457 catalog_manager.cc:5697] T 707de517a3aa4064baa586aa4b8190e8 P 0a00d5d20d5b47b99a1220a0e40a4a78 reported cstate change: term changed from 1 to 2, leader changed from dc9195961d41473e8f590b632eb998a0 (127.14.103.1) to 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3). New cstate: current_term: 2 leader_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:59.039089 26457 catalog_manager.cc:5697] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 reported cstate change: term changed from 3 to 4, leader changed from 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2) to 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3). New cstate: current_term: 4 leader_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:59.054540 27009 raft_consensus.cc:993] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec: : Instructing follower 0a00d5d20d5b47b99a1220a0e40a4a78 to start an election
I20260709 13:09:59.055032 27016 raft_consensus.cc:1081] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Signalling peer 0a00d5d20d5b47b99a1220a0e40a4a78 to start an election
I20260709 13:09:59.056411 26709 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f1b70e3455b04e46a5c1b08d44fd2a06"
dest_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78"
 from {username='slave'} at 127.0.0.1:57192
I20260709 13:09:59.056957 26709 raft_consensus.cc:493] T f1b70e3455b04e46a5c1b08d44fd2a06 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:59.057279 26709 raft_consensus.cc:3060] T f1b70e3455b04e46a5c1b08d44fd2a06 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:59.062489 26709 raft_consensus.cc:515] T f1b70e3455b04e46a5c1b08d44fd2a06 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:59.064224 26709 leader_election.cc:290] T f1b70e3455b04e46a5c1b08d44fd2a06 P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 2 election: Requested vote from peers dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787), 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285)
I20260709 13:09:59.064953 26558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1b70e3455b04e46a5c1b08d44fd2a06" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "dc9195961d41473e8f590b632eb998a0"
I20260709 13:09:59.065214 26634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1b70e3455b04e46a5c1b08d44fd2a06" candidate_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "68637fffcc4c4969ae0612be35aa58ec"
I20260709 13:09:59.065541 26558 raft_consensus.cc:3060] T f1b70e3455b04e46a5c1b08d44fd2a06 P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:59.065838 26634 raft_consensus.cc:3055] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:59.066143 26634 raft_consensus.cc:740] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 68637fffcc4c4969ae0612be35aa58ec, State: Running, Role: LEADER
I20260709 13:09:59.066808 26634 consensus_queue.cc:260] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:59.067840 26634 raft_consensus.cc:3060] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:59.070781 26558 raft_consensus.cc:2468] T f1b70e3455b04e46a5c1b08d44fd2a06 P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in term 2.
I20260709 13:09:59.072099 26675 leader_election.cc:304] T f1b70e3455b04e46a5c1b08d44fd2a06 P 0a00d5d20d5b47b99a1220a0e40a4a78 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a00d5d20d5b47b99a1220a0e40a4a78, dc9195961d41473e8f590b632eb998a0; no voters: 
I20260709 13:09:59.072769 26954 raft_consensus.cc:2804] T f1b70e3455b04e46a5c1b08d44fd2a06 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:59.073194 26954 raft_consensus.cc:697] T f1b70e3455b04e46a5c1b08d44fd2a06 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 LEADER]: Becoming Leader. State: Replica: 0a00d5d20d5b47b99a1220a0e40a4a78, State: Running, Role: LEADER
I20260709 13:09:59.073468 26634 raft_consensus.cc:2468] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a00d5d20d5b47b99a1220a0e40a4a78 in term 2.
I20260709 13:09:59.073946 26954 consensus_queue.cc:237] T f1b70e3455b04e46a5c1b08d44fd2a06 P 0a00d5d20d5b47b99a1220a0e40a4a78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } }
I20260709 13:09:59.080811 26457 catalog_manager.cc:5697] T f1b70e3455b04e46a5c1b08d44fd2a06 P 0a00d5d20d5b47b99a1220a0e40a4a78 reported cstate change: term changed from 1 to 2, leader changed from 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2) to 0a00d5d20d5b47b99a1220a0e40a4a78 (127.14.103.3). New cstate: current_term: 2 leader_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a00d5d20d5b47b99a1220a0e40a4a78" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 36339 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:59.194605 26558 raft_consensus.cc:1275] T b16a06ac45ab44a186fbf820559732bb P dc9195961d41473e8f590b632eb998a0 [term 3 FOLLOWER]: Refusing update from remote peer 0a00d5d20d5b47b99a1220a0e40a4a78: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:59.195801 26954 consensus_queue.cc:1048] T b16a06ac45ab44a186fbf820559732bb P 0a00d5d20d5b47b99a1220a0e40a4a78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:59.204897 26634 raft_consensus.cc:1275] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec [term 3 FOLLOWER]: Refusing update from remote peer 0a00d5d20d5b47b99a1220a0e40a4a78: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:59.206302 26954 consensus_queue.cc:1048] T b16a06ac45ab44a186fbf820559732bb P 0a00d5d20d5b47b99a1220a0e40a4a78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:59.421335 26634 raft_consensus.cc:1275] T 707de517a3aa4064baa586aa4b8190e8 P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Refusing update from remote peer 0a00d5d20d5b47b99a1220a0e40a4a78: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:09:59.422660 26954 consensus_queue.cc:1048] T 707de517a3aa4064baa586aa4b8190e8 P 0a00d5d20d5b47b99a1220a0e40a4a78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:59.466169 26558 raft_consensus.cc:1275] T 707de517a3aa4064baa586aa4b8190e8 P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Refusing update from remote peer 0a00d5d20d5b47b99a1220a0e40a4a78: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:09:59.472707 26985 consensus_queue.cc:1048] T 707de517a3aa4064baa586aa4b8190e8 P 0a00d5d20d5b47b99a1220a0e40a4a78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:59.510995 26634 raft_consensus.cc:1275] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [term 4 FOLLOWER]: Refusing update from remote peer 0a00d5d20d5b47b99a1220a0e40a4a78: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:09:59.512748 26985 consensus_queue.cc:1048] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:59.513576 26558 raft_consensus.cc:1275] T f1b70e3455b04e46a5c1b08d44fd2a06 P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Refusing update from remote peer 0a00d5d20d5b47b99a1220a0e40a4a78: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:09:59.515837 26985 consensus_queue.cc:1048] T f1b70e3455b04e46a5c1b08d44fd2a06 P 0a00d5d20d5b47b99a1220a0e40a4a78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:59.529206 26558 raft_consensus.cc:1275] T 579b5cf4a4874b75a67f9cbf320addc2 P dc9195961d41473e8f590b632eb998a0 [term 4 FOLLOWER]: Refusing update from remote peer 0a00d5d20d5b47b99a1220a0e40a4a78: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:09:59.532723 26954 consensus_queue.cc:1048] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc9195961d41473e8f590b632eb998a0" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 33787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:59.533025 26635 raft_consensus.cc:1275] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Refusing update from remote peer 0a00d5d20d5b47b99a1220a0e40a4a78: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:09:59.535602 26954 consensus_queue.cc:1048] T f1b70e3455b04e46a5c1b08d44fd2a06 P 0a00d5d20d5b47b99a1220a0e40a4a78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68637fffcc4c4969ae0612be35aa58ec" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:59.792621 14748 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:09:59.797679 14748 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 13:09:59.798141 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:59.802592 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 13:09:59.802948 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:00.832284 14748 tablet_server.cc:179] TabletServer@127.14.103.1:0 shutting down...
W20260709 13:10:00.841907 26601 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 8 similar messages]
W20260709 13:10:00.852438 26601 consensus_peers.cc:597] T c5e7fa4752c243c1bd88bedc411e5719 P 68637fffcc4c4969ae0612be35aa58ec -> Peer dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787): Couldn't send request to peer dc9195961d41473e8f590b632eb998a0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:00.877439 14748 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:00.878520 14748 tablet_replica.cc:333] T f1b70e3455b04e46a5c1b08d44fd2a06 P dc9195961d41473e8f590b632eb998a0: stopping tablet replica
I20260709 13:10:00.879415 14748 raft_consensus.cc:2243] T f1b70e3455b04e46a5c1b08d44fd2a06 P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:00.880193 14748 raft_consensus.cc:2272] T f1b70e3455b04e46a5c1b08d44fd2a06 P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:00.882965 14748 tablet_replica.cc:333] T 015286f7e4be4234893d493fb008635e P dc9195961d41473e8f590b632eb998a0: stopping tablet replica
I20260709 13:10:00.883623 14748 raft_consensus.cc:2243] T 015286f7e4be4234893d493fb008635e P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:00.884239 14748 raft_consensus.cc:2272] T 015286f7e4be4234893d493fb008635e P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:00.887042 14748 tablet_replica.cc:333] T 751c0153edc248998be14ca90435f08a P dc9195961d41473e8f590b632eb998a0: stopping tablet replica
I20260709 13:10:00.887853 14748 raft_consensus.cc:2243] T 751c0153edc248998be14ca90435f08a P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:00.888401 14748 raft_consensus.cc:2272] T 751c0153edc248998be14ca90435f08a P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:00.891065 14748 tablet_replica.cc:333] T 8838204cfad3497aaec5d171bbbb38b3 P dc9195961d41473e8f590b632eb998a0: stopping tablet replica
I20260709 13:10:00.891723 14748 raft_consensus.cc:2243] T 8838204cfad3497aaec5d171bbbb38b3 P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:00.892364 14748 raft_consensus.cc:2272] T 8838204cfad3497aaec5d171bbbb38b3 P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:00.895064 14748 tablet_replica.cc:333] T 164a705fca2541d5a5be47dc3d77c40f P dc9195961d41473e8f590b632eb998a0: stopping tablet replica
I20260709 13:10:00.895953 14748 raft_consensus.cc:2243] T 164a705fca2541d5a5be47dc3d77c40f P dc9195961d41473e8f590b632eb998a0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:00.897176 14748 raft_consensus.cc:2272] T 164a705fca2541d5a5be47dc3d77c40f P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:00.900285 14748 tablet_replica.cc:333] T 8f4b44b3bd734567b03eb09289c769f0 P dc9195961d41473e8f590b632eb998a0: stopping tablet replica
I20260709 13:10:00.900853 14748 raft_consensus.cc:2243] T 8f4b44b3bd734567b03eb09289c769f0 P dc9195961d41473e8f590b632eb998a0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:00.901983 14748 raft_consensus.cc:2272] T 8f4b44b3bd734567b03eb09289c769f0 P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:00.904625 14748 tablet_replica.cc:333] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0: stopping tablet replica
I20260709 13:10:00.905259 14748 raft_consensus.cc:2243] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:00.905730 14748 raft_consensus.cc:2272] T c5e7fa4752c243c1bd88bedc411e5719 P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:00.907927 14748 tablet_replica.cc:333] T 579b5cf4a4874b75a67f9cbf320addc2 P dc9195961d41473e8f590b632eb998a0: stopping tablet replica
I20260709 13:10:00.908514 14748 raft_consensus.cc:2243] T 579b5cf4a4874b75a67f9cbf320addc2 P dc9195961d41473e8f590b632eb998a0 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:00.908943 14748 raft_consensus.cc:2272] T 579b5cf4a4874b75a67f9cbf320addc2 P dc9195961d41473e8f590b632eb998a0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:00.911106 14748 tablet_replica.cc:333] T d8410badcf77497fbb604cf8c185db83 P dc9195961d41473e8f590b632eb998a0: stopping tablet replica
I20260709 13:10:00.911602 14748 raft_consensus.cc:2243] T d8410badcf77497fbb604cf8c185db83 P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:00.912092 14748 raft_consensus.cc:2272] T d8410badcf77497fbb604cf8c185db83 P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:00.914418 14748 tablet_replica.cc:333] T b16a06ac45ab44a186fbf820559732bb P dc9195961d41473e8f590b632eb998a0: stopping tablet replica
I20260709 13:10:00.914974 14748 raft_consensus.cc:2243] T b16a06ac45ab44a186fbf820559732bb P dc9195961d41473e8f590b632eb998a0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:00.915419 14748 raft_consensus.cc:2272] T b16a06ac45ab44a186fbf820559732bb P dc9195961d41473e8f590b632eb998a0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:00.917590 14748 tablet_replica.cc:333] T 707de517a3aa4064baa586aa4b8190e8 P dc9195961d41473e8f590b632eb998a0: stopping tablet replica
I20260709 13:10:00.918085 14748 raft_consensus.cc:2243] T 707de517a3aa4064baa586aa4b8190e8 P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:00.918509 14748 raft_consensus.cc:2272] T 707de517a3aa4064baa586aa4b8190e8 P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:00.921657 14748 tablet_replica.cc:333] T 4af4eae89a594eb0ace69b0895e43372 P dc9195961d41473e8f590b632eb998a0: stopping tablet replica
I20260709 13:10:00.922458 14748 raft_consensus.cc:2243] T 4af4eae89a594eb0ace69b0895e43372 P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:00.923061 14748 raft_consensus.cc:2272] T 4af4eae89a594eb0ace69b0895e43372 P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:00.926154 14748 tablet_replica.cc:333] T 25be33d2f6bf455996cd05210da81458 P dc9195961d41473e8f590b632eb998a0: stopping tablet replica
I20260709 13:10:00.926818 14748 raft_consensus.cc:2243] T 25be33d2f6bf455996cd05210da81458 P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:00.927528 14748 raft_consensus.cc:2272] T 25be33d2f6bf455996cd05210da81458 P dc9195961d41473e8f590b632eb998a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:00.930716 14748 tablet_replica.cc:333] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0: stopping tablet replica
I20260709 13:10:00.931291 14748 raft_consensus.cc:2243] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:10:00.932241 14748 raft_consensus.cc:2272] T 055a26fb2c8b41b4b6da92c7e20180e1 P dc9195961d41473e8f590b632eb998a0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:00.934444 14748 tablet_replica.cc:333] T ebb142b13c4440339f42938e6c1cd02d P dc9195961d41473e8f590b632eb998a0: stopping tablet replica
I20260709 13:10:00.934937 14748 raft_consensus.cc:2243] T ebb142b13c4440339f42938e6c1cd02d P dc9195961d41473e8f590b632eb998a0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:00.935868 14748 raft_consensus.cc:2272] T ebb142b13c4440339f42938e6c1cd02d P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:00.938235 14748 tablet_replica.cc:333] T 6c21616d95d04253a63e088134753cea P dc9195961d41473e8f590b632eb998a0: stopping tablet replica
I20260709 13:10:00.938714 14748 raft_consensus.cc:2243] T 6c21616d95d04253a63e088134753cea P dc9195961d41473e8f590b632eb998a0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:00.939533 14748 raft_consensus.cc:2272] T 6c21616d95d04253a63e088134753cea P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:00.941776 14748 tablet_replica.cc:333] T b8f3b339b6bd4573a9f7929c4d130af5 P dc9195961d41473e8f590b632eb998a0: stopping tablet replica
I20260709 13:10:00.942253 14748 raft_consensus.cc:2243] T b8f3b339b6bd4573a9f7929c4d130af5 P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:00.942732 14748 raft_consensus.cc:2272] T b8f3b339b6bd4573a9f7929c4d130af5 P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:00.945133 14748 tablet_replica.cc:333] T ef38f875bfdd4b669c39d02f0ea7472b P dc9195961d41473e8f590b632eb998a0: stopping tablet replica
I20260709 13:10:00.945590 14748 raft_consensus.cc:2243] T ef38f875bfdd4b669c39d02f0ea7472b P dc9195961d41473e8f590b632eb998a0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:00.946401 14748 raft_consensus.cc:2272] T ef38f875bfdd4b669c39d02f0ea7472b P dc9195961d41473e8f590b632eb998a0 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:00.985996 26601 consensus_peers.cc:597] T b8f3b339b6bd4573a9f7929c4d130af5 P 68637fffcc4c4969ae0612be35aa58ec -> Peer dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787): Couldn't send request to peer dc9195961d41473e8f590b632eb998a0. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:33787: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:00.989604 14748 tablet_server.cc:196] TabletServer@127.14.103.1:0 shutdown complete.
W20260709 13:10:01.025792 26675 consensus_peers.cc:597] T 707de517a3aa4064baa586aa4b8190e8 P 0a00d5d20d5b47b99a1220a0e40a4a78 -> Peer dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787): Couldn't send request to peer dc9195961d41473e8f590b632eb998a0. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:33787: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:01.035621 26675 consensus_peers.cc:597] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 -> Peer dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787): Couldn't send request to peer dc9195961d41473e8f590b632eb998a0. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:33787: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:01.041617 14748 tablet_server.cc:179] TabletServer@127.14.103.2:0 shutting down...
W20260709 13:10:01.048628 26601 consensus_peers.cc:597] T 751c0153edc248998be14ca90435f08a P 68637fffcc4c4969ae0612be35aa58ec -> Peer dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787): Couldn't send request to peer dc9195961d41473e8f590b632eb998a0. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:33787: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:01.054668 26601 consensus_peers.cc:597] T 25be33d2f6bf455996cd05210da81458 P 68637fffcc4c4969ae0612be35aa58ec -> Peer dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787): Couldn't send request to peer dc9195961d41473e8f590b632eb998a0. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:33787: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:01.064477 26675 consensus_peers.cc:597] T d8410badcf77497fbb604cf8c185db83 P 0a00d5d20d5b47b99a1220a0e40a4a78 -> Peer dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787): Couldn't send request to peer dc9195961d41473e8f590b632eb998a0. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:33787: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:01.080622 26601 consensus_peers.cc:597] T 4af4eae89a594eb0ace69b0895e43372 P 68637fffcc4c4969ae0612be35aa58ec -> Peer dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787): Couldn't send request to peer dc9195961d41473e8f590b632eb998a0. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:33787: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:01.083020 14748 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:01.083891 14748 tablet_replica.cc:333] T 015286f7e4be4234893d493fb008635e P 68637fffcc4c4969ae0612be35aa58ec: stopping tablet replica
I20260709 13:10:01.084996 14748 raft_consensus.cc:2243] T 015286f7e4be4234893d493fb008635e P 68637fffcc4c4969ae0612be35aa58ec [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:01.085825 14748 raft_consensus.cc:2272] T 015286f7e4be4234893d493fb008635e P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.088101 14748 tablet_replica.cc:333] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec: stopping tablet replica
I20260709 13:10:01.088626 14748 raft_consensus.cc:2243] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.088987 14748 raft_consensus.cc:2272] T f1b70e3455b04e46a5c1b08d44fd2a06 P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.091564 14748 tablet_replica.cc:333] T 751c0153edc248998be14ca90435f08a P 68637fffcc4c4969ae0612be35aa58ec: stopping tablet replica
I20260709 13:10:01.092525 14748 raft_consensus.cc:2243] T 751c0153edc248998be14ca90435f08a P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:01.093663 14748 raft_consensus.cc:2272] T 751c0153edc248998be14ca90435f08a P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:01.094769 26675 consensus_peers.cc:597] T f1b70e3455b04e46a5c1b08d44fd2a06 P 0a00d5d20d5b47b99a1220a0e40a4a78 -> Peer dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787): Couldn't send request to peer dc9195961d41473e8f590b632eb998a0. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:33787: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:01.096347 14748 tablet_replica.cc:333] T 8838204cfad3497aaec5d171bbbb38b3 P 68637fffcc4c4969ae0612be35aa58ec: stopping tablet replica
I20260709 13:10:01.096939 14748 raft_consensus.cc:2243] T 8838204cfad3497aaec5d171bbbb38b3 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.097453 14748 raft_consensus.cc:2272] T 8838204cfad3497aaec5d171bbbb38b3 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.099612 14748 tablet_replica.cc:333] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec: stopping tablet replica
I20260709 13:10:01.100263 14748 raft_consensus.cc:2243] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.100805 14748 raft_consensus.cc:2272] T 164a705fca2541d5a5be47dc3d77c40f P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.102967 14748 tablet_replica.cc:333] T 4af4eae89a594eb0ace69b0895e43372 P 68637fffcc4c4969ae0612be35aa58ec: stopping tablet replica
I20260709 13:10:01.103510 14748 raft_consensus.cc:2243] T 4af4eae89a594eb0ace69b0895e43372 P 68637fffcc4c4969ae0612be35aa58ec [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:01.104329 14748 raft_consensus.cc:2272] T 4af4eae89a594eb0ace69b0895e43372 P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.106472 14748 tablet_replica.cc:333] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec: stopping tablet replica
I20260709 13:10:01.107259 14748 raft_consensus.cc:2243] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.107808 14748 raft_consensus.cc:2272] T 8f4b44b3bd734567b03eb09289c769f0 P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.110100 14748 tablet_replica.cc:333] T c5e7fa4752c243c1bd88bedc411e5719 P 68637fffcc4c4969ae0612be35aa58ec: stopping tablet replica
I20260709 13:10:01.110810 14748 raft_consensus.cc:2243] T c5e7fa4752c243c1bd88bedc411e5719 P 68637fffcc4c4969ae0612be35aa58ec [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:01.111636 14748 raft_consensus.cc:2272] T c5e7fa4752c243c1bd88bedc411e5719 P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.113595 14748 tablet_replica.cc:333] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec: stopping tablet replica
I20260709 13:10:01.114145 14748 raft_consensus.cc:2243] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.114563 14748 raft_consensus.cc:2272] T 579b5cf4a4874b75a67f9cbf320addc2 P 68637fffcc4c4969ae0612be35aa58ec [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.116681 14748 tablet_replica.cc:333] T d8410badcf77497fbb604cf8c185db83 P 68637fffcc4c4969ae0612be35aa58ec: stopping tablet replica
I20260709 13:10:01.117215 14748 raft_consensus.cc:2243] T d8410badcf77497fbb604cf8c185db83 P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.117703 14748 raft_consensus.cc:2272] T d8410badcf77497fbb604cf8c185db83 P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.119987 14748 tablet_replica.cc:333] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec: stopping tablet replica
I20260709 13:10:01.120913 14748 raft_consensus.cc:2243] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.121462 14748 raft_consensus.cc:2272] T ebb142b13c4440339f42938e6c1cd02d P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.123466 14748 tablet_replica.cc:333] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec: stopping tablet replica
I20260709 13:10:01.123991 14748 raft_consensus.cc:2243] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.124485 14748 raft_consensus.cc:2272] T 6c21616d95d04253a63e088134753cea P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.126541 14748 tablet_replica.cc:333] T 25be33d2f6bf455996cd05210da81458 P 68637fffcc4c4969ae0612be35aa58ec: stopping tablet replica
I20260709 13:10:01.126996 14748 raft_consensus.cc:2243] T 25be33d2f6bf455996cd05210da81458 P 68637fffcc4c4969ae0612be35aa58ec [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:01.127805 14748 raft_consensus.cc:2272] T 25be33d2f6bf455996cd05210da81458 P 68637fffcc4c4969ae0612be35aa58ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.129801 14748 tablet_replica.cc:333] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec: stopping tablet replica
I20260709 13:10:01.130295 14748 raft_consensus.cc:2243] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.130784 14748 raft_consensus.cc:2272] T 055a26fb2c8b41b4b6da92c7e20180e1 P 68637fffcc4c4969ae0612be35aa58ec [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.132936 14748 tablet_replica.cc:333] T 707de517a3aa4064baa586aa4b8190e8 P 68637fffcc4c4969ae0612be35aa58ec: stopping tablet replica
I20260709 13:10:01.133374 14748 raft_consensus.cc:2243] T 707de517a3aa4064baa586aa4b8190e8 P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.133832 14748 raft_consensus.cc:2272] T 707de517a3aa4064baa586aa4b8190e8 P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.136283 14748 tablet_replica.cc:333] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec: stopping tablet replica
I20260709 13:10:01.136976 14748 raft_consensus.cc:2243] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.137495 14748 raft_consensus.cc:2272] T b16a06ac45ab44a186fbf820559732bb P 68637fffcc4c4969ae0612be35aa58ec [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.139851 14748 tablet_replica.cc:333] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec: stopping tablet replica
I20260709 13:10:01.140518 14748 raft_consensus.cc:2243] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.140897 14748 raft_consensus.cc:2272] T ef38f875bfdd4b669c39d02f0ea7472b P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.142831 14748 tablet_replica.cc:333] T b8f3b339b6bd4573a9f7929c4d130af5 P 68637fffcc4c4969ae0612be35aa58ec: stopping tablet replica
I20260709 13:10:01.143230 14748 raft_consensus.cc:2243] T b8f3b339b6bd4573a9f7929c4d130af5 P 68637fffcc4c4969ae0612be35aa58ec [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:01.143970 14748 raft_consensus.cc:2272] T b8f3b339b6bd4573a9f7929c4d130af5 P 68637fffcc4c4969ae0612be35aa58ec [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:01.172143 26675 consensus_peers.cc:597] T b16a06ac45ab44a186fbf820559732bb P 0a00d5d20d5b47b99a1220a0e40a4a78 -> Peer dc9195961d41473e8f590b632eb998a0 (127.14.103.1:33787): Couldn't send request to peer dc9195961d41473e8f590b632eb998a0. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:33787: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:01.186214 14748 tablet_server.cc:196] TabletServer@127.14.103.2:0 shutdown complete.
W20260709 13:10:01.215364 26675 consensus_peers.cc:597] T 8838204cfad3497aaec5d171bbbb38b3 P 0a00d5d20d5b47b99a1220a0e40a4a78 -> Peer 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285): Couldn't send request to peer 68637fffcc4c4969ae0612be35aa58ec. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:35285: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:01.221149 26675 consensus_peers.cc:597] T b16a06ac45ab44a186fbf820559732bb P 0a00d5d20d5b47b99a1220a0e40a4a78 -> Peer 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285): Couldn't send request to peer 68637fffcc4c4969ae0612be35aa58ec. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:35285: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:01.229257 14748 tablet_server.cc:179] TabletServer@127.14.103.3:0 shutting down...
W20260709 13:10:01.244289 26675 consensus_peers.cc:597] T 707de517a3aa4064baa586aa4b8190e8 P 0a00d5d20d5b47b99a1220a0e40a4a78 -> Peer 68637fffcc4c4969ae0612be35aa58ec (127.14.103.2:35285): Couldn't send request to peer 68637fffcc4c4969ae0612be35aa58ec. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:35285: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:01.257925 14748 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:01.258657 14748 tablet_replica.cc:333] T 015286f7e4be4234893d493fb008635e P 0a00d5d20d5b47b99a1220a0e40a4a78: stopping tablet replica
I20260709 13:10:01.259438 14748 raft_consensus.cc:2243] T 015286f7e4be4234893d493fb008635e P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.260098 14748 raft_consensus.cc:2272] T 015286f7e4be4234893d493fb008635e P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.262403 14748 tablet_replica.cc:333] T f1b70e3455b04e46a5c1b08d44fd2a06 P 0a00d5d20d5b47b99a1220a0e40a4a78: stopping tablet replica
I20260709 13:10:01.262969 14748 raft_consensus.cc:2243] T f1b70e3455b04e46a5c1b08d44fd2a06 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:01.263767 14748 raft_consensus.cc:2272] T f1b70e3455b04e46a5c1b08d44fd2a06 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.265854 14748 tablet_replica.cc:333] T 751c0153edc248998be14ca90435f08a P 0a00d5d20d5b47b99a1220a0e40a4a78: stopping tablet replica
I20260709 13:10:01.266399 14748 raft_consensus.cc:2243] T 751c0153edc248998be14ca90435f08a P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.266851 14748 raft_consensus.cc:2272] T 751c0153edc248998be14ca90435f08a P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.268987 14748 tablet_replica.cc:333] T 8838204cfad3497aaec5d171bbbb38b3 P 0a00d5d20d5b47b99a1220a0e40a4a78: stopping tablet replica
I20260709 13:10:01.269505 14748 raft_consensus.cc:2243] T 8838204cfad3497aaec5d171bbbb38b3 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:01.270310 14748 raft_consensus.cc:2272] T 8838204cfad3497aaec5d171bbbb38b3 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.272370 14748 tablet_replica.cc:333] T 164a705fca2541d5a5be47dc3d77c40f P 0a00d5d20d5b47b99a1220a0e40a4a78: stopping tablet replica
I20260709 13:10:01.272899 14748 raft_consensus.cc:2243] T 164a705fca2541d5a5be47dc3d77c40f P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.273366 14748 raft_consensus.cc:2272] T 164a705fca2541d5a5be47dc3d77c40f P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.275329 14748 tablet_replica.cc:333] T c5e7fa4752c243c1bd88bedc411e5719 P 0a00d5d20d5b47b99a1220a0e40a4a78: stopping tablet replica
I20260709 13:10:01.275920 14748 raft_consensus.cc:2243] T c5e7fa4752c243c1bd88bedc411e5719 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.276399 14748 raft_consensus.cc:2272] T c5e7fa4752c243c1bd88bedc411e5719 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.278409 14748 tablet_replica.cc:333] T 8f4b44b3bd734567b03eb09289c769f0 P 0a00d5d20d5b47b99a1220a0e40a4a78: stopping tablet replica
I20260709 13:10:01.278929 14748 raft_consensus.cc:2243] T 8f4b44b3bd734567b03eb09289c769f0 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.279387 14748 raft_consensus.cc:2272] T 8f4b44b3bd734567b03eb09289c769f0 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.281343 14748 tablet_replica.cc:333] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78: stopping tablet replica
I20260709 13:10:01.281852 14748 raft_consensus.cc:2243] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:10:01.282578 14748 raft_consensus.cc:2272] T 579b5cf4a4874b75a67f9cbf320addc2 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.284708 14748 tablet_replica.cc:333] T d8410badcf77497fbb604cf8c185db83 P 0a00d5d20d5b47b99a1220a0e40a4a78: stopping tablet replica
I20260709 13:10:01.285231 14748 raft_consensus.cc:2243] T d8410badcf77497fbb604cf8c185db83 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:01.285985 14748 raft_consensus.cc:2272] T d8410badcf77497fbb604cf8c185db83 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.287989 14748 tablet_replica.cc:333] T ebb142b13c4440339f42938e6c1cd02d P 0a00d5d20d5b47b99a1220a0e40a4a78: stopping tablet replica
I20260709 13:10:01.288487 14748 raft_consensus.cc:2243] T ebb142b13c4440339f42938e6c1cd02d P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.288976 14748 raft_consensus.cc:2272] T ebb142b13c4440339f42938e6c1cd02d P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.290917 14748 tablet_replica.cc:333] T 6c21616d95d04253a63e088134753cea P 0a00d5d20d5b47b99a1220a0e40a4a78: stopping tablet replica
I20260709 13:10:01.291429 14748 raft_consensus.cc:2243] T 6c21616d95d04253a63e088134753cea P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.291990 14748 raft_consensus.cc:2272] T 6c21616d95d04253a63e088134753cea P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.293918 14748 tablet_replica.cc:333] T 25be33d2f6bf455996cd05210da81458 P 0a00d5d20d5b47b99a1220a0e40a4a78: stopping tablet replica
I20260709 13:10:01.294407 14748 raft_consensus.cc:2243] T 25be33d2f6bf455996cd05210da81458 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.294833 14748 raft_consensus.cc:2272] T 25be33d2f6bf455996cd05210da81458 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.296792 14748 tablet_replica.cc:333] T 055a26fb2c8b41b4b6da92c7e20180e1 P 0a00d5d20d5b47b99a1220a0e40a4a78: stopping tablet replica
I20260709 13:10:01.297295 14748 raft_consensus.cc:2243] T 055a26fb2c8b41b4b6da92c7e20180e1 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.297833 14748 raft_consensus.cc:2272] T 055a26fb2c8b41b4b6da92c7e20180e1 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.299988 14748 tablet_replica.cc:333] T b16a06ac45ab44a186fbf820559732bb P 0a00d5d20d5b47b99a1220a0e40a4a78: stopping tablet replica
I20260709 13:10:01.300460 14748 raft_consensus.cc:2243] T b16a06ac45ab44a186fbf820559732bb P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:10:01.301168 14748 raft_consensus.cc:2272] T b16a06ac45ab44a186fbf820559732bb P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.303267 14748 tablet_replica.cc:333] T 707de517a3aa4064baa586aa4b8190e8 P 0a00d5d20d5b47b99a1220a0e40a4a78: stopping tablet replica
I20260709 13:10:01.303727 14748 raft_consensus.cc:2243] T 707de517a3aa4064baa586aa4b8190e8 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:01.304445 14748 raft_consensus.cc:2272] T 707de517a3aa4064baa586aa4b8190e8 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.306288 14748 tablet_replica.cc:333] T 4af4eae89a594eb0ace69b0895e43372 P 0a00d5d20d5b47b99a1220a0e40a4a78: stopping tablet replica
I20260709 13:10:01.306717 14748 raft_consensus.cc:2243] T 4af4eae89a594eb0ace69b0895e43372 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.307093 14748 raft_consensus.cc:2272] T 4af4eae89a594eb0ace69b0895e43372 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.309000 14748 tablet_replica.cc:333] T ef38f875bfdd4b669c39d02f0ea7472b P 0a00d5d20d5b47b99a1220a0e40a4a78: stopping tablet replica
I20260709 13:10:01.309417 14748 raft_consensus.cc:2243] T ef38f875bfdd4b669c39d02f0ea7472b P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.309827 14748 raft_consensus.cc:2272] T ef38f875bfdd4b669c39d02f0ea7472b P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.311967 14748 tablet_replica.cc:333] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78: stopping tablet replica
I20260709 13:10:01.312381 14748 raft_consensus.cc:2243] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:01.312736 14748 raft_consensus.cc:2272] T b8f3b339b6bd4573a9f7929c4d130af5 P 0a00d5d20d5b47b99a1220a0e40a4a78 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.354413 14748 tablet_server.cc:196] TabletServer@127.14.103.3:0 shutdown complete.
I20260709 13:10:01.389422 14748 master.cc:562] Master@127.14.103.62:39907 shutting down...
I20260709 13:10:01.410537 14748 raft_consensus.cc:2243] T 00000000000000000000000000000000 P e68faef5ba744349a655384e3cc32fd8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:01.411156 14748 raft_consensus.cc:2272] T 00000000000000000000000000000000 P e68faef5ba744349a655384e3cc32fd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:01.411506 14748 tablet_replica.cc:333] T 00000000000000000000000000000000 P e68faef5ba744349a655384e3cc32fd8: stopping tablet replica
I20260709 13:10:01.432832 14748 master.cc:584] Master@127.14.103.62:39907 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (13780 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 13:10:01.485018 14748 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.103.62:44831
I20260709 13:10:01.486119 14748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:01.491843 27049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:01.491926 27050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:01.492482 27052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:01.493489 14748 server_base.cc:1061] running on GCE node
I20260709 13:10:01.494292 14748 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:01.494503 14748 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:01.494663 14748 hybrid_clock.cc:648] HybridClock initialized: now 1783602601494647 us; error 0 us; skew 500 ppm
I20260709 13:10:01.495155 14748 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:01.497566 14748 webserver.cc:533] Webserver started at http://127.14.103.62:43901/ using document root <none> and password file <none>
I20260709 13:10:01.498062 14748 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:01.498260 14748 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:01.498528 14748 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:01.499615 14748 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602247531521-14748-0/minicluster-data/master-0-root/instance:
uuid: "dcdd1fd6000d46a4b7bd49014ad3e41f"
format_stamp: "Formatted at 2026-07-09 13:10:01 on dist-test-slave-fpp7"
I20260709 13:10:01.503896 14748 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 13:10:01.506911 27057 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:01.507753 14748 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:10:01.507987 14748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602247531521-14748-0/minicluster-data/master-0-root
uuid: "dcdd1fd6000d46a4b7bd49014ad3e41f"
format_stamp: "Formatted at 2026-07-09 13:10:01 on dist-test-slave-fpp7"
I20260709 13:10:01.508203 14748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602247531521-14748-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602247531521-14748-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602247531521-14748-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:01.549185 14748 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:01.550297 14748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:01.586608 14748 rpc_server.cc:307] RPC server started. Bound to: 127.14.103.62:44831
I20260709 13:10:01.586716 27108 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.103.62:44831 every 8 connection(s)
I20260709 13:10:01.590555 27109 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:01.603482 27109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcdd1fd6000d46a4b7bd49014ad3e41f: Bootstrap starting.
I20260709 13:10:01.608337 27109 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dcdd1fd6000d46a4b7bd49014ad3e41f: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:01.612521 27109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcdd1fd6000d46a4b7bd49014ad3e41f: No bootstrap required, opened a new log
I20260709 13:10:01.614542 27109 raft_consensus.cc:359] T 00000000000000000000000000000000 P dcdd1fd6000d46a4b7bd49014ad3e41f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcdd1fd6000d46a4b7bd49014ad3e41f" member_type: VOTER }
I20260709 13:10:01.614897 27109 raft_consensus.cc:385] T 00000000000000000000000000000000 P dcdd1fd6000d46a4b7bd49014ad3e41f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:01.615078 27109 raft_consensus.cc:740] T 00000000000000000000000000000000 P dcdd1fd6000d46a4b7bd49014ad3e41f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcdd1fd6000d46a4b7bd49014ad3e41f, State: Initialized, Role: FOLLOWER
I20260709 13:10:01.615633 27109 consensus_queue.cc:260] T 00000000000000000000000000000000 P dcdd1fd6000d46a4b7bd49014ad3e41f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcdd1fd6000d46a4b7bd49014ad3e41f" member_type: VOTER }
I20260709 13:10:01.616103 27109 raft_consensus.cc:399] T 00000000000000000000000000000000 P dcdd1fd6000d46a4b7bd49014ad3e41f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:10:01.616312 27109 raft_consensus.cc:493] T 00000000000000000000000000000000 P dcdd1fd6000d46a4b7bd49014ad3e41f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:10:01.616520 27109 raft_consensus.cc:3060] T 00000000000000000000000000000000 P dcdd1fd6000d46a4b7bd49014ad3e41f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:01.620962 27109 raft_consensus.cc:515] T 00000000000000000000000000000000 P dcdd1fd6000d46a4b7bd49014ad3e41f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcdd1fd6000d46a4b7bd49014ad3e41f" member_type: VOTER }
I20260709 13:10:01.621404 27109 leader_election.cc:304] T 00000000000000000000000000000000 P dcdd1fd6000d46a4b7bd49014ad3e41f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dcdd1fd6000d46a4b7bd49014ad3e41f; no voters: 
I20260709 13:10:01.622401 27109 leader_election.cc:290] T 00000000000000000000000000000000 P dcdd1fd6000d46a4b7bd49014ad3e41f [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:10:01.622701 27112 raft_consensus.cc:2804] T 00000000000000000000000000000000 P dcdd1fd6000d46a4b7bd49014ad3e41f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:01.624083 27112 raft_consensus.cc:697] T 00000000000000000000000000000000 P dcdd1fd6000d46a4b7bd49014ad3e41f [term 1 LEADER]: Becoming Leader. State: Replica: dcdd1fd6000d46a4b7bd49014ad3e41f, State: Running, Role: LEADER
I20260709 13:10:01.624818 27112 consensus_queue.cc:237] T 00000000000000000000000000000000 P dcdd1fd6000d46a4b7bd49014ad3e41f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcdd1fd6000d46a4b7bd49014ad3e41f" member_type: VOTER }
I20260709 13:10:01.625367 27109 sys_catalog.cc:565] T 00000000000000000000000000000000 P dcdd1fd6000d46a4b7bd49014ad3e41f [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:10:01.629755 27113 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcdd1fd6000d46a4b7bd49014ad3e41f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dcdd1fd6000d46a4b7bd49014ad3e41f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcdd1fd6000d46a4b7bd49014ad3e41f" member_type: VOTER } }
I20260709 13:10:01.630069 27114 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcdd1fd6000d46a4b7bd49014ad3e41f [sys.catalog]: SysCatalogTable state changed. Reason: New leader dcdd1fd6000d46a4b7bd49014ad3e41f. Latest consensus state: current_term: 1 leader_uuid: "dcdd1fd6000d46a4b7bd49014ad3e41f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcdd1fd6000d46a4b7bd49014ad3e41f" member_type: VOTER } }
I20260709 13:10:01.630602 27113 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcdd1fd6000d46a4b7bd49014ad3e41f [sys.catalog]: This master's current role is: LEADER
I20260709 13:10:01.630848 27114 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcdd1fd6000d46a4b7bd49014ad3e41f [sys.catalog]: This master's current role is: LEADER
I20260709 13:10:01.633940 27119 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:10:01.640465 27119 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:10:01.641572 14748 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:10:01.649031 27119 catalog_manager.cc:1383] Generated new cluster ID: a12df21060d745ce8be2bea237f112d9
I20260709 13:10:01.649322 27119 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:10:01.664089 27119 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:10:01.665391 27119 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:10:01.680780 27119 catalog_manager.cc:6070] T 00000000000000000000000000000000 P dcdd1fd6000d46a4b7bd49014ad3e41f: Generated new TSK 0
I20260709 13:10:01.681371 27119 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:10:01.709668 14748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:01.715384 27130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:01.716472 27131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:01.718603 27133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:01.718767 14748 server_base.cc:1061] running on GCE node
I20260709 13:10:01.719765 14748 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:01.719977 14748 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:01.720137 14748 hybrid_clock.cc:648] HybridClock initialized: now 1783602601720120 us; error 0 us; skew 500 ppm
I20260709 13:10:01.720623 14748 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:01.722819 14748 webserver.cc:533] Webserver started at http://127.14.103.1:39203/ using document root <none> and password file <none>
I20260709 13:10:01.723428 14748 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:01.723645 14748 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:01.723929 14748 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:01.724931 14748 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602247531521-14748-0/minicluster-data/ts-0-root/instance:
uuid: "3e58e2a069aa48c59858cbadb8ab9394"
format_stamp: "Formatted at 2026-07-09 13:10:01 on dist-test-slave-fpp7"
I20260709 13:10:01.729074 14748 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.003s
I20260709 13:10:01.732198 27138 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:01.732919 14748 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:10:01.733153 14748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602247531521-14748-0/minicluster-data/ts-0-root
uuid: "3e58e2a069aa48c59858cbadb8ab9394"
format_stamp: "Formatted at 2026-07-09 13:10:01 on dist-test-slave-fpp7"
I20260709 13:10:01.733464 14748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602247531521-14748-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602247531521-14748-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602247531521-14748-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:01.756867 14748 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:01.757985 14748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:01.759299 14748 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:01.761536 14748 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:01.761737 14748 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:01.762001 14748 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:01.762174 14748 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:01.802047 14748 rpc_server.cc:307] RPC server started. Bound to: 127.14.103.1:38667
I20260709 13:10:01.802152 27200 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.103.1:38667 every 8 connection(s)
I20260709 13:10:01.807236 14748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:01.814631 27205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:01.816813 27206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:01.817608 27208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:01.819012 14748 server_base.cc:1061] running on GCE node
I20260709 13:10:01.820689 14748 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:10:01.820935 27201 heartbeater.cc:344] Connected to a master server at 127.14.103.62:44831
W20260709 13:10:01.820956 14748 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:01.821350 14748 hybrid_clock.cc:648] HybridClock initialized: now 1783602601821325 us; error 0 us; skew 500 ppm
I20260709 13:10:01.821394 27201 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:01.821965 14748 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:01.822201 27201 heartbeater.cc:507] Master 127.14.103.62:44831 requested a full tablet report, sending...
I20260709 13:10:01.824384 27074 ts_manager.cc:194] Registered new tserver with Master: 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667)
I20260709 13:10:01.824755 14748 webserver.cc:533] Webserver started at http://127.14.103.2:41341/ using document root <none> and password file <none>
I20260709 13:10:01.825278 14748 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:01.825460 14748 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:01.825728 14748 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:01.826304 27074 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55706
I20260709 13:10:01.827016 14748 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602247531521-14748-0/minicluster-data/ts-1-root/instance:
uuid: "df70ca0962e140ba92abead00b2fc477"
format_stamp: "Formatted at 2026-07-09 13:10:01 on dist-test-slave-fpp7"
I20260709 13:10:01.831364 14748 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:10:01.834964 27213 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:01.835788 14748 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:10:01.836071 14748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602247531521-14748-0/minicluster-data/ts-1-root
uuid: "df70ca0962e140ba92abead00b2fc477"
format_stamp: "Formatted at 2026-07-09 13:10:01 on dist-test-slave-fpp7"
I20260709 13:10:01.836370 14748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602247531521-14748-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602247531521-14748-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602247531521-14748-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:01.859030 14748 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:01.860299 14748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:01.861698 14748 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:01.863835 14748 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:01.864053 14748 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:01.864292 14748 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:01.864461 14748 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:01.903839 14748 rpc_server.cc:307] RPC server started. Bound to: 127.14.103.2:34461
I20260709 13:10:01.903934 27275 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.103.2:34461 every 8 connection(s)
I20260709 13:10:01.908504 14748 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:01.915851 27279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:01.918891 27280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:01.920735 27283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:01.921994 27276 heartbeater.cc:344] Connected to a master server at 127.14.103.62:44831
I20260709 13:10:01.922379 27276 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:01.923112 27276 heartbeater.cc:507] Master 127.14.103.62:44831 requested a full tablet report, sending...
I20260709 13:10:01.924480 14748 server_base.cc:1061] running on GCE node
I20260709 13:10:01.925360 27074 ts_manager.cc:194] Registered new tserver with Master: df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461)
I20260709 13:10:01.925606 14748 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:01.925894 14748 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:01.926090 14748 hybrid_clock.cc:648] HybridClock initialized: now 1783602601926072 us; error 0 us; skew 500 ppm
I20260709 13:10:01.926875 14748 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:01.927234 27074 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55722
I20260709 13:10:01.929632 14748 webserver.cc:533] Webserver started at http://127.14.103.3:46525/ using document root <none> and password file <none>
I20260709 13:10:01.930119 14748 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:01.930274 14748 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:01.930469 14748 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:01.931516 14748 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602247531521-14748-0/minicluster-data/ts-2-root/instance:
uuid: "16f9474cee3e43c89e5d9e42266cb250"
format_stamp: "Formatted at 2026-07-09 13:10:01 on dist-test-slave-fpp7"
I20260709 13:10:01.935782 14748 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 13:10:01.939182 27287 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:01.939989 14748 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260709 13:10:01.940281 14748 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602247531521-14748-0/minicluster-data/ts-2-root
uuid: "16f9474cee3e43c89e5d9e42266cb250"
format_stamp: "Formatted at 2026-07-09 13:10:01 on dist-test-slave-fpp7"
I20260709 13:10:01.940567 14748 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602247531521-14748-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602247531521-14748-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taske4yFvf/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602247531521-14748-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:01.961928 14748 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:01.963070 14748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:01.964509 14748 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:01.966569 14748 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:01.966789 14748 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:01.967028 14748 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:01.967196 14748 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:02.006151 14748 rpc_server.cc:307] RPC server started. Bound to: 127.14.103.3:33233
I20260709 13:10:02.006242 27349 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.103.3:33233 every 8 connection(s)
I20260709 13:10:02.019378 27350 heartbeater.cc:344] Connected to a master server at 127.14.103.62:44831
I20260709 13:10:02.019865 27350 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:02.020516 27350 heartbeater.cc:507] Master 127.14.103.62:44831 requested a full tablet report, sending...
I20260709 13:10:02.022351 27074 ts_manager.cc:194] Registered new tserver with Master: 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:02.023391 14748 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014251701s
I20260709 13:10:02.023608 27074 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55734
I20260709 13:10:02.048090 27074 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55744:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:10:02.050724 27074 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:10:02.115996 27166 tablet_service.cc:1511] Processing CreateTablet for tablet acb5efb1d0d2422ba071fc5bc49e6390 (DEFAULT_TABLE table=test-workload [id=467bab49e844435287e6833e7f61744f]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:10:02.117467 27166 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acb5efb1d0d2422ba071fc5bc49e6390. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:02.119061 27164 tablet_service.cc:1511] Processing CreateTablet for tablet 094982455ad24795a7cf35af53248051 (DEFAULT_TABLE table=test-workload [id=467bab49e844435287e6833e7f61744f]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:10:02.120537 27164 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 094982455ad24795a7cf35af53248051. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:02.120924 27165 tablet_service.cc:1511] Processing CreateTablet for tablet 713697e046ae4150a4047a4bf2c1180f (DEFAULT_TABLE table=test-workload [id=467bab49e844435287e6833e7f61744f]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:10:02.122248 27165 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 713697e046ae4150a4047a4bf2c1180f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:02.122174 27315 tablet_service.cc:1511] Processing CreateTablet for tablet acb5efb1d0d2422ba071fc5bc49e6390 (DEFAULT_TABLE table=test-workload [id=467bab49e844435287e6833e7f61744f]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:10:02.123566 27315 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acb5efb1d0d2422ba071fc5bc49e6390. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:02.126359 27314 tablet_service.cc:1511] Processing CreateTablet for tablet 713697e046ae4150a4047a4bf2c1180f (DEFAULT_TABLE table=test-workload [id=467bab49e844435287e6833e7f61744f]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:10:02.127735 27314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 713697e046ae4150a4047a4bf2c1180f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:02.140051 27313 tablet_service.cc:1511] Processing CreateTablet for tablet 094982455ad24795a7cf35af53248051 (DEFAULT_TABLE table=test-workload [id=467bab49e844435287e6833e7f61744f]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:10:02.141562 27313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 094982455ad24795a7cf35af53248051. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:02.153244 27166 tablet_service.cc:1511] Processing CreateTablet for tablet f49bdf47ae7b41369b04e42ba388c1f1 (DEFAULT_TABLE table=test-workload [id=467bab49e844435287e6833e7f61744f]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:10:02.158241 27166 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f49bdf47ae7b41369b04e42ba388c1f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:02.165403 27312 tablet_service.cc:1511] Processing CreateTablet for tablet f49bdf47ae7b41369b04e42ba388c1f1 (DEFAULT_TABLE table=test-workload [id=467bab49e844435287e6833e7f61744f]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:10:02.167186 27365 tablet_bootstrap.cc:492] T 094982455ad24795a7cf35af53248051 P 3e58e2a069aa48c59858cbadb8ab9394: Bootstrap starting.
I20260709 13:10:02.167666 27312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f49bdf47ae7b41369b04e42ba388c1f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:02.181835 27165 tablet_service.cc:1511] Processing CreateTablet for tablet 7a551aeaa560472abebcd6cc7d456faf (DEFAULT_TABLE table=test-workload [id=467bab49e844435287e6833e7f61744f]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:10:02.184798 27365 tablet_bootstrap.cc:654] T 094982455ad24795a7cf35af53248051 P 3e58e2a069aa48c59858cbadb8ab9394: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:02.188939 27240 tablet_service.cc:1511] Processing CreateTablet for tablet 713697e046ae4150a4047a4bf2c1180f (DEFAULT_TABLE table=test-workload [id=467bab49e844435287e6833e7f61744f]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:10:02.189802 27315 tablet_service.cc:1511] Processing CreateTablet for tablet 7a551aeaa560472abebcd6cc7d456faf (DEFAULT_TABLE table=test-workload [id=467bab49e844435287e6833e7f61744f]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:10:02.190320 27240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 713697e046ae4150a4047a4bf2c1180f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:02.191037 27315 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a551aeaa560472abebcd6cc7d456faf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:02.192842 27239 tablet_service.cc:1511] Processing CreateTablet for tablet 094982455ad24795a7cf35af53248051 (DEFAULT_TABLE table=test-workload [id=467bab49e844435287e6833e7f61744f]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:10:02.193041 27314 tablet_service.cc:1511] Processing CreateTablet for tablet 7d767accb08c4a38a74ae205bc54d739 (DEFAULT_TABLE table=test-workload [id=467bab49e844435287e6833e7f61744f]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:10:02.194077 27239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 094982455ad24795a7cf35af53248051. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:02.194763 27314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d767accb08c4a38a74ae205bc54d739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:02.196825 27238 tablet_service.cc:1511] Processing CreateTablet for tablet f49bdf47ae7b41369b04e42ba388c1f1 (DEFAULT_TABLE table=test-workload [id=467bab49e844435287e6833e7f61744f]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:10:02.197068 27166 tablet_service.cc:1511] Processing CreateTablet for tablet 7d767accb08c4a38a74ae205bc54d739 (DEFAULT_TABLE table=test-workload [id=467bab49e844435287e6833e7f61744f]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:10:02.198146 27238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f49bdf47ae7b41369b04e42ba388c1f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:02.194792 27165 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a551aeaa560472abebcd6cc7d456faf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:02.200835 27237 tablet_service.cc:1511] Processing CreateTablet for tablet 7d767accb08c4a38a74ae205bc54d739 (DEFAULT_TABLE table=test-workload [id=467bab49e844435287e6833e7f61744f]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:10:02.202075 27237 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d767accb08c4a38a74ae205bc54d739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:02.206354 27241 tablet_service.cc:1511] Processing CreateTablet for tablet acb5efb1d0d2422ba071fc5bc49e6390 (DEFAULT_TABLE table=test-workload [id=467bab49e844435287e6833e7f61744f]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:10:02.208630 27236 tablet_service.cc:1511] Processing CreateTablet for tablet 7a551aeaa560472abebcd6cc7d456faf (DEFAULT_TABLE table=test-workload [id=467bab49e844435287e6833e7f61744f]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:10:02.210333 27236 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a551aeaa560472abebcd6cc7d456faf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:02.212237 27241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acb5efb1d0d2422ba071fc5bc49e6390. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:02.220016 27166 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d767accb08c4a38a74ae205bc54d739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:02.236725 27164 tablet_service.cc:1511] Processing CreateTablet for tablet e184a0b144684ada99885c4811605036 (DEFAULT_TABLE table=test-workload [id=467bab49e844435287e6833e7f61744f]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:10:02.238598 27164 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e184a0b144684ada99885c4811605036. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:02.251932 27366 tablet_bootstrap.cc:492] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250: Bootstrap starting.
I20260709 13:10:02.254355 27235 tablet_service.cc:1511] Processing CreateTablet for tablet e184a0b144684ada99885c4811605036 (DEFAULT_TABLE table=test-workload [id=467bab49e844435287e6833e7f61744f]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:10:02.255697 27235 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e184a0b144684ada99885c4811605036. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:02.268970 27365 tablet_bootstrap.cc:492] T 094982455ad24795a7cf35af53248051 P 3e58e2a069aa48c59858cbadb8ab9394: No bootstrap required, opened a new log
I20260709 13:10:02.269541 27365 ts_tablet_manager.cc:1403] T 094982455ad24795a7cf35af53248051 P 3e58e2a069aa48c59858cbadb8ab9394: Time spent bootstrapping tablet: real 0.103s	user 0.015s	sys 0.029s
I20260709 13:10:02.272648 27365 raft_consensus.cc:359] T 094982455ad24795a7cf35af53248051 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.274981 27366 tablet_bootstrap.cc:654] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:02.276230 27313 tablet_service.cc:1511] Processing CreateTablet for tablet e184a0b144684ada99885c4811605036 (DEFAULT_TABLE table=test-workload [id=467bab49e844435287e6833e7f61744f]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:10:02.292272 27313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e184a0b144684ada99885c4811605036. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:02.294582 27366 tablet_bootstrap.cc:492] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250: No bootstrap required, opened a new log
I20260709 13:10:02.294827 27164 tablet_service.cc:1511] Processing CreateTablet for tablet a51060799a3243a881c471fc62688815 (DEFAULT_TABLE table=test-workload [id=467bab49e844435287e6833e7f61744f]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:10:02.299058 27164 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a51060799a3243a881c471fc62688815. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:02.300801 27366 ts_tablet_manager.cc:1403] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250: Time spent bootstrapping tablet: real 0.058s	user 0.000s	sys 0.013s
I20260709 13:10:02.303968 27366 raft_consensus.cc:359] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.297731 27312 tablet_service.cc:1511] Processing CreateTablet for tablet a51060799a3243a881c471fc62688815 (DEFAULT_TABLE table=test-workload [id=467bab49e844435287e6833e7f61744f]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:10:02.310473 27312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a51060799a3243a881c471fc62688815. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:02.311076 27314 tablet_service.cc:1511] Processing CreateTablet for tablet 74072bd16e8d49749ab1f70a27f71bea (DEFAULT_TABLE table=test-workload [id=467bab49e844435287e6833e7f61744f]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:10:02.312302 27314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74072bd16e8d49749ab1f70a27f71bea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:02.314287 27366 raft_consensus.cc:385] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:02.314680 27366 raft_consensus.cc:740] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16f9474cee3e43c89e5d9e42266cb250, State: Initialized, Role: FOLLOWER
I20260709 13:10:02.315447 27366 consensus_queue.cc:260] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.306941 27165 tablet_service.cc:1511] Processing CreateTablet for tablet 74072bd16e8d49749ab1f70a27f71bea (DEFAULT_TABLE table=test-workload [id=467bab49e844435287e6833e7f61744f]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:10:02.296550 27239 tablet_service.cc:1511] Processing CreateTablet for tablet a51060799a3243a881c471fc62688815 (DEFAULT_TABLE table=test-workload [id=467bab49e844435287e6833e7f61744f]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:10:02.322039 27239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a51060799a3243a881c471fc62688815. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:02.308831 27241 tablet_service.cc:1511] Processing CreateTablet for tablet 74072bd16e8d49749ab1f70a27f71bea (DEFAULT_TABLE table=test-workload [id=467bab49e844435287e6833e7f61744f]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:10:02.290453 27365 raft_consensus.cc:385] T 094982455ad24795a7cf35af53248051 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:02.325227 27350 heartbeater.cc:499] Master 127.14.103.62:44831 was elected leader, sending a full tablet report...
I20260709 13:10:02.325532 27165 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74072bd16e8d49749ab1f70a27f71bea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:02.324250 27365 raft_consensus.cc:740] T 094982455ad24795a7cf35af53248051 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e58e2a069aa48c59858cbadb8ab9394, State: Initialized, Role: FOLLOWER
I20260709 13:10:02.327694 27365 consensus_queue.cc:260] T 094982455ad24795a7cf35af53248051 P 3e58e2a069aa48c59858cbadb8ab9394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.325660 27366 ts_tablet_manager.cc:1434] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250: Time spent starting tablet: real 0.024s	user 0.013s	sys 0.000s
I20260709 13:10:02.329789 27366 tablet_bootstrap.cc:492] T 713697e046ae4150a4047a4bf2c1180f P 16f9474cee3e43c89e5d9e42266cb250: Bootstrap starting.
I20260709 13:10:02.332662 27241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74072bd16e8d49749ab1f70a27f71bea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:02.339871 27368 tablet_bootstrap.cc:492] T 094982455ad24795a7cf35af53248051 P df70ca0962e140ba92abead00b2fc477: Bootstrap starting.
I20260709 13:10:02.353709 27368 tablet_bootstrap.cc:654] T 094982455ad24795a7cf35af53248051 P df70ca0962e140ba92abead00b2fc477: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:02.355417 27366 tablet_bootstrap.cc:654] T 713697e046ae4150a4047a4bf2c1180f P 16f9474cee3e43c89e5d9e42266cb250: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:02.353911 27201 heartbeater.cc:499] Master 127.14.103.62:44831 was elected leader, sending a full tablet report...
I20260709 13:10:02.371788 27365 ts_tablet_manager.cc:1434] T 094982455ad24795a7cf35af53248051 P 3e58e2a069aa48c59858cbadb8ab9394: Time spent starting tablet: real 0.102s	user 0.007s	sys 0.019s
I20260709 13:10:02.371953 27366 tablet_bootstrap.cc:492] T 713697e046ae4150a4047a4bf2c1180f P 16f9474cee3e43c89e5d9e42266cb250: No bootstrap required, opened a new log
I20260709 13:10:02.372941 27366 ts_tablet_manager.cc:1403] T 713697e046ae4150a4047a4bf2c1180f P 16f9474cee3e43c89e5d9e42266cb250: Time spent bootstrapping tablet: real 0.043s	user 0.011s	sys 0.007s
I20260709 13:10:02.373654 27365 tablet_bootstrap.cc:492] T acb5efb1d0d2422ba071fc5bc49e6390 P 3e58e2a069aa48c59858cbadb8ab9394: Bootstrap starting.
I20260709 13:10:02.375763 27366 raft_consensus.cc:359] T 713697e046ae4150a4047a4bf2c1180f P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.378702 27368 tablet_bootstrap.cc:492] T 094982455ad24795a7cf35af53248051 P df70ca0962e140ba92abead00b2fc477: No bootstrap required, opened a new log
I20260709 13:10:02.379967 27366 raft_consensus.cc:385] T 713697e046ae4150a4047a4bf2c1180f P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:02.380389 27366 raft_consensus.cc:740] T 713697e046ae4150a4047a4bf2c1180f P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16f9474cee3e43c89e5d9e42266cb250, State: Initialized, Role: FOLLOWER
I20260709 13:10:02.380494 27368 ts_tablet_manager.cc:1403] T 094982455ad24795a7cf35af53248051 P df70ca0962e140ba92abead00b2fc477: Time spent bootstrapping tablet: real 0.056s	user 0.016s	sys 0.017s
I20260709 13:10:02.381245 27366 consensus_queue.cc:260] T 713697e046ae4150a4047a4bf2c1180f P 16f9474cee3e43c89e5d9e42266cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.383637 27368 raft_consensus.cc:359] T 094982455ad24795a7cf35af53248051 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.384449 27368 raft_consensus.cc:385] T 094982455ad24795a7cf35af53248051 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:02.384784 27368 raft_consensus.cc:740] T 094982455ad24795a7cf35af53248051 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df70ca0962e140ba92abead00b2fc477, State: Initialized, Role: FOLLOWER
I20260709 13:10:02.385653 27366 ts_tablet_manager.cc:1434] T 713697e046ae4150a4047a4bf2c1180f P 16f9474cee3e43c89e5d9e42266cb250: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:10:02.385895 27365 tablet_bootstrap.cc:654] T acb5efb1d0d2422ba071fc5bc49e6390 P 3e58e2a069aa48c59858cbadb8ab9394: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:02.385663 27368 consensus_queue.cc:260] T 094982455ad24795a7cf35af53248051 P df70ca0962e140ba92abead00b2fc477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.386821 27366 tablet_bootstrap.cc:492] T acb5efb1d0d2422ba071fc5bc49e6390 P 16f9474cee3e43c89e5d9e42266cb250: Bootstrap starting.
I20260709 13:10:02.395264 27366 tablet_bootstrap.cc:654] T acb5efb1d0d2422ba071fc5bc49e6390 P 16f9474cee3e43c89e5d9e42266cb250: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:02.395450 27276 heartbeater.cc:499] Master 127.14.103.62:44831 was elected leader, sending a full tablet report...
I20260709 13:10:02.395709 27368 ts_tablet_manager.cc:1434] T 094982455ad24795a7cf35af53248051 P df70ca0962e140ba92abead00b2fc477: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.008s
I20260709 13:10:02.397765 27368 tablet_bootstrap.cc:492] T 713697e046ae4150a4047a4bf2c1180f P df70ca0962e140ba92abead00b2fc477: Bootstrap starting.
I20260709 13:10:02.398806 27365 tablet_bootstrap.cc:492] T acb5efb1d0d2422ba071fc5bc49e6390 P 3e58e2a069aa48c59858cbadb8ab9394: No bootstrap required, opened a new log
I20260709 13:10:02.399498 27365 ts_tablet_manager.cc:1403] T acb5efb1d0d2422ba071fc5bc49e6390 P 3e58e2a069aa48c59858cbadb8ab9394: Time spent bootstrapping tablet: real 0.026s	user 0.005s	sys 0.013s
I20260709 13:10:02.403496 27365 raft_consensus.cc:359] T acb5efb1d0d2422ba071fc5bc49e6390 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } }
I20260709 13:10:02.404685 27365 raft_consensus.cc:385] T acb5efb1d0d2422ba071fc5bc49e6390 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:02.404281 27370 raft_consensus.cc:493] T 713697e046ae4150a4047a4bf2c1180f P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:02.405457 27368 tablet_bootstrap.cc:654] T 713697e046ae4150a4047a4bf2c1180f P df70ca0962e140ba92abead00b2fc477: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:02.405516 27365 raft_consensus.cc:740] T acb5efb1d0d2422ba071fc5bc49e6390 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e58e2a069aa48c59858cbadb8ab9394, State: Initialized, Role: FOLLOWER
I20260709 13:10:02.406716 27365 consensus_queue.cc:260] T acb5efb1d0d2422ba071fc5bc49e6390 P 3e58e2a069aa48c59858cbadb8ab9394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } }
I20260709 13:10:02.409564 27365 ts_tablet_manager.cc:1434] T acb5efb1d0d2422ba071fc5bc49e6390 P 3e58e2a069aa48c59858cbadb8ab9394: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:10:02.410522 27365 tablet_bootstrap.cc:492] T 713697e046ae4150a4047a4bf2c1180f P 3e58e2a069aa48c59858cbadb8ab9394: Bootstrap starting.
I20260709 13:10:02.411900 27366 tablet_bootstrap.cc:492] T acb5efb1d0d2422ba071fc5bc49e6390 P 16f9474cee3e43c89e5d9e42266cb250: No bootstrap required, opened a new log
I20260709 13:10:02.412416 27366 ts_tablet_manager.cc:1403] T acb5efb1d0d2422ba071fc5bc49e6390 P 16f9474cee3e43c89e5d9e42266cb250: Time spent bootstrapping tablet: real 0.026s	user 0.017s	sys 0.007s
I20260709 13:10:02.415130 27368 tablet_bootstrap.cc:492] T 713697e046ae4150a4047a4bf2c1180f P df70ca0962e140ba92abead00b2fc477: No bootstrap required, opened a new log
I20260709 13:10:02.415781 27368 ts_tablet_manager.cc:1403] T 713697e046ae4150a4047a4bf2c1180f P df70ca0962e140ba92abead00b2fc477: Time spent bootstrapping tablet: real 0.018s	user 0.011s	sys 0.003s
I20260709 13:10:02.415374 27366 raft_consensus.cc:359] T acb5efb1d0d2422ba071fc5bc49e6390 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } }
I20260709 13:10:02.416363 27366 raft_consensus.cc:385] T acb5efb1d0d2422ba071fc5bc49e6390 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:02.416734 27366 raft_consensus.cc:740] T acb5efb1d0d2422ba071fc5bc49e6390 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16f9474cee3e43c89e5d9e42266cb250, State: Initialized, Role: FOLLOWER
I20260709 13:10:02.417635 27366 consensus_queue.cc:260] T acb5efb1d0d2422ba071fc5bc49e6390 P 16f9474cee3e43c89e5d9e42266cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } }
I20260709 13:10:02.418130 27365 tablet_bootstrap.cc:654] T 713697e046ae4150a4047a4bf2c1180f P 3e58e2a069aa48c59858cbadb8ab9394: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:02.418874 27368 raft_consensus.cc:359] T 713697e046ae4150a4047a4bf2c1180f P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.419793 27368 raft_consensus.cc:385] T 713697e046ae4150a4047a4bf2c1180f P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:02.420280 27368 raft_consensus.cc:740] T 713697e046ae4150a4047a4bf2c1180f P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df70ca0962e140ba92abead00b2fc477, State: Initialized, Role: FOLLOWER
I20260709 13:10:02.419831 27370 raft_consensus.cc:515] T 713697e046ae4150a4047a4bf2c1180f P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.421257 27368 consensus_queue.cc:260] T 713697e046ae4150a4047a4bf2c1180f P df70ca0962e140ba92abead00b2fc477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.423614 27368 ts_tablet_manager.cc:1434] T 713697e046ae4150a4047a4bf2c1180f P df70ca0962e140ba92abead00b2fc477: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.004s
I20260709 13:10:02.424552 27368 tablet_bootstrap.cc:492] T f49bdf47ae7b41369b04e42ba388c1f1 P df70ca0962e140ba92abead00b2fc477: Bootstrap starting.
I20260709 13:10:02.427446 27370 leader_election.cc:290] T 713697e046ae4150a4047a4bf2c1180f P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461)
I20260709 13:10:02.428406 27366 ts_tablet_manager.cc:1434] T acb5efb1d0d2422ba071fc5bc49e6390 P 16f9474cee3e43c89e5d9e42266cb250: Time spent starting tablet: real 0.016s	user 0.010s	sys 0.003s
I20260709 13:10:02.431280 27368 tablet_bootstrap.cc:654] T f49bdf47ae7b41369b04e42ba388c1f1 P df70ca0962e140ba92abead00b2fc477: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:02.432502 27366 tablet_bootstrap.cc:492] T 7a551aeaa560472abebcd6cc7d456faf P 16f9474cee3e43c89e5d9e42266cb250: Bootstrap starting.
I20260709 13:10:02.443353 27365 tablet_bootstrap.cc:492] T 713697e046ae4150a4047a4bf2c1180f P 3e58e2a069aa48c59858cbadb8ab9394: No bootstrap required, opened a new log
I20260709 13:10:02.443928 27365 ts_tablet_manager.cc:1403] T 713697e046ae4150a4047a4bf2c1180f P 3e58e2a069aa48c59858cbadb8ab9394: Time spent bootstrapping tablet: real 0.034s	user 0.006s	sys 0.016s
I20260709 13:10:02.446955 27365 raft_consensus.cc:359] T 713697e046ae4150a4047a4bf2c1180f P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.447268 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "713697e046ae4150a4047a4bf2c1180f" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477" is_pre_election: true
I20260709 13:10:02.447878 27365 raft_consensus.cc:385] T 713697e046ae4150a4047a4bf2c1180f P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:02.448352 27365 raft_consensus.cc:740] T 713697e046ae4150a4047a4bf2c1180f P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e58e2a069aa48c59858cbadb8ab9394, State: Initialized, Role: FOLLOWER
I20260709 13:10:02.448290 27251 raft_consensus.cc:2468] T 713697e046ae4150a4047a4bf2c1180f P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 0.
I20260709 13:10:02.449386 27365 consensus_queue.cc:260] T 713697e046ae4150a4047a4bf2c1180f P 3e58e2a069aa48c59858cbadb8ab9394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.451756 27366 tablet_bootstrap.cc:654] T 7a551aeaa560472abebcd6cc7d456faf P 16f9474cee3e43c89e5d9e42266cb250: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:02.451474 27289 leader_election.cc:304] T 713697e046ae4150a4047a4bf2c1180f P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250, df70ca0962e140ba92abead00b2fc477; no voters: 
I20260709 13:10:02.454516 27365 ts_tablet_manager.cc:1434] T 713697e046ae4150a4047a4bf2c1180f P 3e58e2a069aa48c59858cbadb8ab9394: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:10:02.455574 27365 tablet_bootstrap.cc:492] T f49bdf47ae7b41369b04e42ba388c1f1 P 3e58e2a069aa48c59858cbadb8ab9394: Bootstrap starting.
I20260709 13:10:02.461853 27365 tablet_bootstrap.cc:654] T f49bdf47ae7b41369b04e42ba388c1f1 P 3e58e2a069aa48c59858cbadb8ab9394: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:02.461690 27176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "713697e046ae4150a4047a4bf2c1180f" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394" is_pre_election: true
I20260709 13:10:02.462383 27176 raft_consensus.cc:2468] T 713697e046ae4150a4047a4bf2c1180f P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 0.
I20260709 13:10:02.463342 27370 raft_consensus.cc:2804] T 713697e046ae4150a4047a4bf2c1180f P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:02.463881 27370 raft_consensus.cc:493] T 713697e046ae4150a4047a4bf2c1180f P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:02.464322 27370 raft_consensus.cc:3060] T 713697e046ae4150a4047a4bf2c1180f P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:02.478899 27365 tablet_bootstrap.cc:492] T f49bdf47ae7b41369b04e42ba388c1f1 P 3e58e2a069aa48c59858cbadb8ab9394: No bootstrap required, opened a new log
I20260709 13:10:02.479787 27365 ts_tablet_manager.cc:1403] T f49bdf47ae7b41369b04e42ba388c1f1 P 3e58e2a069aa48c59858cbadb8ab9394: Time spent bootstrapping tablet: real 0.024s	user 0.015s	sys 0.005s
I20260709 13:10:02.489982 27370 raft_consensus.cc:515] T 713697e046ae4150a4047a4bf2c1180f P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.493590 27366 tablet_bootstrap.cc:492] T 7a551aeaa560472abebcd6cc7d456faf P 16f9474cee3e43c89e5d9e42266cb250: No bootstrap required, opened a new log
I20260709 13:10:02.494196 27366 ts_tablet_manager.cc:1403] T 7a551aeaa560472abebcd6cc7d456faf P 16f9474cee3e43c89e5d9e42266cb250: Time spent bootstrapping tablet: real 0.062s	user 0.028s	sys 0.013s
I20260709 13:10:02.494338 27176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "713697e046ae4150a4047a4bf2c1180f" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394"
I20260709 13:10:02.494959 27176 raft_consensus.cc:3060] T 713697e046ae4150a4047a4bf2c1180f P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:02.494511 27365 raft_consensus.cc:359] T f49bdf47ae7b41369b04e42ba388c1f1 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.495592 27365 raft_consensus.cc:385] T f49bdf47ae7b41369b04e42ba388c1f1 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:02.496006 27365 raft_consensus.cc:740] T f49bdf47ae7b41369b04e42ba388c1f1 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e58e2a069aa48c59858cbadb8ab9394, State: Initialized, Role: FOLLOWER
I20260709 13:10:02.497617 27366 raft_consensus.cc:359] T 7a551aeaa560472abebcd6cc7d456faf P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.497013 27365 consensus_queue.cc:260] T f49bdf47ae7b41369b04e42ba388c1f1 P 3e58e2a069aa48c59858cbadb8ab9394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.498442 27366 raft_consensus.cc:385] T 7a551aeaa560472abebcd6cc7d456faf P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:02.498894 27366 raft_consensus.cc:740] T 7a551aeaa560472abebcd6cc7d456faf P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16f9474cee3e43c89e5d9e42266cb250, State: Initialized, Role: FOLLOWER
I20260709 13:10:02.499795 27366 consensus_queue.cc:260] T 7a551aeaa560472abebcd6cc7d456faf P 16f9474cee3e43c89e5d9e42266cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.503057 27365 ts_tablet_manager.cc:1434] T f49bdf47ae7b41369b04e42ba388c1f1 P 3e58e2a069aa48c59858cbadb8ab9394: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.002s
I20260709 13:10:02.504491 27365 tablet_bootstrap.cc:492] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394: Bootstrap starting.
I20260709 13:10:02.505095 27176 raft_consensus.cc:2468] T 713697e046ae4150a4047a4bf2c1180f P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 1.
I20260709 13:10:02.506364 27290 leader_election.cc:304] T 713697e046ae4150a4047a4bf2c1180f P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250, 3e58e2a069aa48c59858cbadb8ab9394; no voters: 
I20260709 13:10:02.507611 27378 raft_consensus.cc:2804] T 713697e046ae4150a4047a4bf2c1180f P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:02.507993 27366 ts_tablet_manager.cc:1434] T 7a551aeaa560472abebcd6cc7d456faf P 16f9474cee3e43c89e5d9e42266cb250: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:10:02.509131 27366 tablet_bootstrap.cc:492] T 7d767accb08c4a38a74ae205bc54d739 P 16f9474cee3e43c89e5d9e42266cb250: Bootstrap starting.
I20260709 13:10:02.513265 27378 raft_consensus.cc:697] T 713697e046ae4150a4047a4bf2c1180f P 16f9474cee3e43c89e5d9e42266cb250 [term 1 LEADER]: Becoming Leader. State: Replica: 16f9474cee3e43c89e5d9e42266cb250, State: Running, Role: LEADER
I20260709 13:10:02.514494 27378 consensus_queue.cc:237] T 713697e046ae4150a4047a4bf2c1180f P 16f9474cee3e43c89e5d9e42266cb250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.513209 27365 tablet_bootstrap.cc:654] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:02.517614 27368 tablet_bootstrap.cc:492] T f49bdf47ae7b41369b04e42ba388c1f1 P df70ca0962e140ba92abead00b2fc477: No bootstrap required, opened a new log
I20260709 13:10:02.518177 27368 ts_tablet_manager.cc:1403] T f49bdf47ae7b41369b04e42ba388c1f1 P df70ca0962e140ba92abead00b2fc477: Time spent bootstrapping tablet: real 0.094s	user 0.050s	sys 0.038s
I20260709 13:10:02.519862 27366 tablet_bootstrap.cc:654] T 7d767accb08c4a38a74ae205bc54d739 P 16f9474cee3e43c89e5d9e42266cb250: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:02.521242 27368 raft_consensus.cc:359] T f49bdf47ae7b41369b04e42ba388c1f1 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.522035 27368 raft_consensus.cc:385] T f49bdf47ae7b41369b04e42ba388c1f1 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:02.522315 27368 raft_consensus.cc:740] T f49bdf47ae7b41369b04e42ba388c1f1 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df70ca0962e140ba92abead00b2fc477, State: Initialized, Role: FOLLOWER
I20260709 13:10:02.523036 27368 consensus_queue.cc:260] T f49bdf47ae7b41369b04e42ba388c1f1 P df70ca0962e140ba92abead00b2fc477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.524585 27370 leader_election.cc:290] T 713697e046ae4150a4047a4bf2c1180f P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 election: Requested vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461)
I20260709 13:10:02.526558 27368 ts_tablet_manager.cc:1434] T f49bdf47ae7b41369b04e42ba388c1f1 P df70ca0962e140ba92abead00b2fc477: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:10:02.527755 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "713697e046ae4150a4047a4bf2c1180f" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477"
I20260709 13:10:02.527981 27368 tablet_bootstrap.cc:492] T e184a0b144684ada99885c4811605036 P df70ca0962e140ba92abead00b2fc477: Bootstrap starting.
I20260709 13:10:02.528241 27251 raft_consensus.cc:3060] T 713697e046ae4150a4047a4bf2c1180f P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:02.535140 27368 tablet_bootstrap.cc:654] T e184a0b144684ada99885c4811605036 P df70ca0962e140ba92abead00b2fc477: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:02.538920 27251 raft_consensus.cc:2468] T 713697e046ae4150a4047a4bf2c1180f P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 1.
I20260709 13:10:02.544126 27368 tablet_bootstrap.cc:492] T e184a0b144684ada99885c4811605036 P df70ca0962e140ba92abead00b2fc477: No bootstrap required, opened a new log
I20260709 13:10:02.544715 27368 ts_tablet_manager.cc:1403] T e184a0b144684ada99885c4811605036 P df70ca0962e140ba92abead00b2fc477: Time spent bootstrapping tablet: real 0.017s	user 0.016s	sys 0.000s
I20260709 13:10:02.547511 27368 raft_consensus.cc:359] T e184a0b144684ada99885c4811605036 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.548112 27368 raft_consensus.cc:385] T e184a0b144684ada99885c4811605036 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:02.548408 27368 raft_consensus.cc:740] T e184a0b144684ada99885c4811605036 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df70ca0962e140ba92abead00b2fc477, State: Initialized, Role: FOLLOWER
I20260709 13:10:02.549141 27368 consensus_queue.cc:260] T e184a0b144684ada99885c4811605036 P df70ca0962e140ba92abead00b2fc477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.550043 27365 tablet_bootstrap.cc:492] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394: No bootstrap required, opened a new log
I20260709 13:10:02.550359 27366 tablet_bootstrap.cc:492] T 7d767accb08c4a38a74ae205bc54d739 P 16f9474cee3e43c89e5d9e42266cb250: No bootstrap required, opened a new log
I20260709 13:10:02.551016 27365 ts_tablet_manager.cc:1403] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394: Time spent bootstrapping tablet: real 0.047s	user 0.012s	sys 0.015s
I20260709 13:10:02.551162 27366 ts_tablet_manager.cc:1403] T 7d767accb08c4a38a74ae205bc54d739 P 16f9474cee3e43c89e5d9e42266cb250: Time spent bootstrapping tablet: real 0.042s	user 0.009s	sys 0.013s
I20260709 13:10:02.551541 27368 ts_tablet_manager.cc:1434] T e184a0b144684ada99885c4811605036 P df70ca0962e140ba92abead00b2fc477: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:10:02.552601 27368 tablet_bootstrap.cc:492] T 7a551aeaa560472abebcd6cc7d456faf P df70ca0962e140ba92abead00b2fc477: Bootstrap starting.
I20260709 13:10:02.554003 27365 raft_consensus.cc:359] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.554801 27365 raft_consensus.cc:385] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:02.549433 27072 catalog_manager.cc:5697] T 713697e046ae4150a4047a4bf2c1180f P 16f9474cee3e43c89e5d9e42266cb250 reported cstate change: term changed from 0 to 1, leader changed from <none> to 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3). New cstate: current_term: 1 leader_uuid: "16f9474cee3e43c89e5d9e42266cb250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:02.555171 27365 raft_consensus.cc:740] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e58e2a069aa48c59858cbadb8ab9394, State: Initialized, Role: FOLLOWER
I20260709 13:10:02.554425 27366 raft_consensus.cc:359] T 7d767accb08c4a38a74ae205bc54d739 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.555846 27366 raft_consensus.cc:385] T 7d767accb08c4a38a74ae205bc54d739 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:02.558591 27365 consensus_queue.cc:260] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.561185 27365 ts_tablet_manager.cc:1434] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 13:10:02.562212 27365 tablet_bootstrap.cc:492] T 7a551aeaa560472abebcd6cc7d456faf P 3e58e2a069aa48c59858cbadb8ab9394: Bootstrap starting.
I20260709 13:10:02.556195 27366 raft_consensus.cc:740] T 7d767accb08c4a38a74ae205bc54d739 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16f9474cee3e43c89e5d9e42266cb250, State: Initialized, Role: FOLLOWER
I20260709 13:10:02.562974 27366 consensus_queue.cc:260] T 7d767accb08c4a38a74ae205bc54d739 P 16f9474cee3e43c89e5d9e42266cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.564893 27368 tablet_bootstrap.cc:654] T 7a551aeaa560472abebcd6cc7d456faf P df70ca0962e140ba92abead00b2fc477: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:02.565363 27366 ts_tablet_manager.cc:1434] T 7d767accb08c4a38a74ae205bc54d739 P 16f9474cee3e43c89e5d9e42266cb250: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:10:02.566450 27366 tablet_bootstrap.cc:492] T f49bdf47ae7b41369b04e42ba388c1f1 P 16f9474cee3e43c89e5d9e42266cb250: Bootstrap starting.
I20260709 13:10:02.571444 27374 raft_consensus.cc:493] T f49bdf47ae7b41369b04e42ba388c1f1 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:02.571990 27374 raft_consensus.cc:515] T f49bdf47ae7b41369b04e42ba388c1f1 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.574494 27368 tablet_bootstrap.cc:492] T 7a551aeaa560472abebcd6cc7d456faf P df70ca0962e140ba92abead00b2fc477: No bootstrap required, opened a new log
I20260709 13:10:02.575124 27368 ts_tablet_manager.cc:1403] T 7a551aeaa560472abebcd6cc7d456faf P df70ca0962e140ba92abead00b2fc477: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.007s
I20260709 13:10:02.577003 27374 leader_election.cc:290] T f49bdf47ae7b41369b04e42ba388c1f1 P df70ca0962e140ba92abead00b2fc477 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:02.577661 27365 tablet_bootstrap.cc:654] T 7a551aeaa560472abebcd6cc7d456faf P 3e58e2a069aa48c59858cbadb8ab9394: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:02.579140 27371 raft_consensus.cc:493] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:02.580374 27368 raft_consensus.cc:359] T 7a551aeaa560472abebcd6cc7d456faf P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.580530 27371 raft_consensus.cc:515] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.581137 27368 raft_consensus.cc:385] T 7a551aeaa560472abebcd6cc7d456faf P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:02.581506 27368 raft_consensus.cc:740] T 7a551aeaa560472abebcd6cc7d456faf P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df70ca0962e140ba92abead00b2fc477, State: Initialized, Role: FOLLOWER
I20260709 13:10:02.583207 27371 leader_election.cc:290] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:02.583788 27371 raft_consensus.cc:493] T f49bdf47ae7b41369b04e42ba388c1f1 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:02.584189 27371 raft_consensus.cc:515] T f49bdf47ae7b41369b04e42ba388c1f1 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.584093 27368 consensus_queue.cc:260] T 7a551aeaa560472abebcd6cc7d456faf P df70ca0962e140ba92abead00b2fc477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.586253 27371 leader_election.cc:290] T f49bdf47ae7b41369b04e42ba388c1f1 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:02.586860 27368 ts_tablet_manager.cc:1434] T 7a551aeaa560472abebcd6cc7d456faf P df70ca0962e140ba92abead00b2fc477: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.003s
I20260709 13:10:02.588008 27368 tablet_bootstrap.cc:492] T acb5efb1d0d2422ba071fc5bc49e6390 P df70ca0962e140ba92abead00b2fc477: Bootstrap starting.
I20260709 13:10:02.580052 27366 tablet_bootstrap.cc:654] T f49bdf47ae7b41369b04e42ba388c1f1 P 16f9474cee3e43c89e5d9e42266cb250: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:02.607944 27366 tablet_bootstrap.cc:492] T f49bdf47ae7b41369b04e42ba388c1f1 P 16f9474cee3e43c89e5d9e42266cb250: No bootstrap required, opened a new log
I20260709 13:10:02.608536 27366 ts_tablet_manager.cc:1403] T f49bdf47ae7b41369b04e42ba388c1f1 P 16f9474cee3e43c89e5d9e42266cb250: Time spent bootstrapping tablet: real 0.042s	user 0.011s	sys 0.002s
I20260709 13:10:02.609095 27368 tablet_bootstrap.cc:654] T acb5efb1d0d2422ba071fc5bc49e6390 P df70ca0962e140ba92abead00b2fc477: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:02.611270 27176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f49bdf47ae7b41369b04e42ba388c1f1" candidate_uuid: "df70ca0962e140ba92abead00b2fc477" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394" is_pre_election: true
I20260709 13:10:02.611984 27176 raft_consensus.cc:2468] T f49bdf47ae7b41369b04e42ba388c1f1 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df70ca0962e140ba92abead00b2fc477 in term 0.
I20260709 13:10:02.611719 27366 raft_consensus.cc:359] T f49bdf47ae7b41369b04e42ba388c1f1 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.613734 27216 leader_election.cc:304] T f49bdf47ae7b41369b04e42ba388c1f1 P df70ca0962e140ba92abead00b2fc477 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e58e2a069aa48c59858cbadb8ab9394, df70ca0962e140ba92abead00b2fc477; no voters: 
I20260709 13:10:02.614616 27374 raft_consensus.cc:2804] T f49bdf47ae7b41369b04e42ba388c1f1 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:02.614977 27374 raft_consensus.cc:493] T f49bdf47ae7b41369b04e42ba388c1f1 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:02.615171 27365 tablet_bootstrap.cc:492] T 7a551aeaa560472abebcd6cc7d456faf P 3e58e2a069aa48c59858cbadb8ab9394: No bootstrap required, opened a new log
I20260709 13:10:02.615338 27374 raft_consensus.cc:3060] T f49bdf47ae7b41369b04e42ba388c1f1 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:02.615371 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f49bdf47ae7b41369b04e42ba388c1f1" candidate_uuid: "df70ca0962e140ba92abead00b2fc477" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16f9474cee3e43c89e5d9e42266cb250" is_pre_election: true
I20260709 13:10:02.616180 27365 ts_tablet_manager.cc:1403] T 7a551aeaa560472abebcd6cc7d456faf P 3e58e2a069aa48c59858cbadb8ab9394: Time spent bootstrapping tablet: real 0.054s	user 0.011s	sys 0.005s
I20260709 13:10:02.612551 27366 raft_consensus.cc:385] T f49bdf47ae7b41369b04e42ba388c1f1 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:02.616868 27366 raft_consensus.cc:740] T f49bdf47ae7b41369b04e42ba388c1f1 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16f9474cee3e43c89e5d9e42266cb250, State: Initialized, Role: FOLLOWER
I20260709 13:10:02.617745 27366 consensus_queue.cc:260] T f49bdf47ae7b41369b04e42ba388c1f1 P 16f9474cee3e43c89e5d9e42266cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.618662 27325 raft_consensus.cc:2468] T f49bdf47ae7b41369b04e42ba388c1f1 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df70ca0962e140ba92abead00b2fc477 in term 0.
I20260709 13:10:02.619714 27365 raft_consensus.cc:359] T 7a551aeaa560472abebcd6cc7d456faf P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.620652 27365 raft_consensus.cc:385] T 7a551aeaa560472abebcd6cc7d456faf P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:02.620985 27365 raft_consensus.cc:740] T 7a551aeaa560472abebcd6cc7d456faf P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e58e2a069aa48c59858cbadb8ab9394, State: Initialized, Role: FOLLOWER
I20260709 13:10:02.622431 27378 raft_consensus.cc:493] T 7a551aeaa560472abebcd6cc7d456faf P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:02.622943 27378 raft_consensus.cc:515] T 7a551aeaa560472abebcd6cc7d456faf P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.622061 27365 consensus_queue.cc:260] T 7a551aeaa560472abebcd6cc7d456faf P 3e58e2a069aa48c59858cbadb8ab9394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.624881 27378 leader_election.cc:290] T 7a551aeaa560472abebcd6cc7d456faf P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461)
I20260709 13:10:02.626281 27365 ts_tablet_manager.cc:1434] T 7a551aeaa560472abebcd6cc7d456faf P 3e58e2a069aa48c59858cbadb8ab9394: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 13:10:02.626452 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a551aeaa560472abebcd6cc7d456faf" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477" is_pre_election: true
I20260709 13:10:02.627012 27371 raft_consensus.cc:493] T 7a551aeaa560472abebcd6cc7d456faf P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:02.627080 27251 raft_consensus.cc:2468] T 7a551aeaa560472abebcd6cc7d456faf P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 0.
I20260709 13:10:02.627645 27371 raft_consensus.cc:515] T 7a551aeaa560472abebcd6cc7d456faf P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.629062 27289 leader_election.cc:304] T 7a551aeaa560472abebcd6cc7d456faf P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250, df70ca0962e140ba92abead00b2fc477; no voters: 
I20260709 13:10:02.629873 27371 leader_election.cc:290] T 7a551aeaa560472abebcd6cc7d456faf P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:02.629999 27370 raft_consensus.cc:2804] T 7a551aeaa560472abebcd6cc7d456faf P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:02.630481 27370 raft_consensus.cc:493] T 7a551aeaa560472abebcd6cc7d456faf P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:02.630822 27370 raft_consensus.cc:3060] T 7a551aeaa560472abebcd6cc7d456faf P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:02.631162 27366 ts_tablet_manager.cc:1434] T f49bdf47ae7b41369b04e42ba388c1f1 P 16f9474cee3e43c89e5d9e42266cb250: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260709 13:10:02.632279 27366 tablet_bootstrap.cc:492] T a51060799a3243a881c471fc62688815 P 16f9474cee3e43c89e5d9e42266cb250: Bootstrap starting.
I20260709 13:10:02.635366 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e184a0b144684ada99885c4811605036" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477" is_pre_election: true
W20260709 13:10:02.637137 27122 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:02.637663 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e184a0b144684ada99885c4811605036" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16f9474cee3e43c89e5d9e42266cb250" is_pre_election: true
I20260709 13:10:02.638983 27366 tablet_bootstrap.cc:654] T a51060799a3243a881c471fc62688815 P 16f9474cee3e43c89e5d9e42266cb250: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:02.639039 27324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f49bdf47ae7b41369b04e42ba388c1f1" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16f9474cee3e43c89e5d9e42266cb250" is_pre_election: true
I20260709 13:10:02.639878 27324 raft_consensus.cc:2468] T f49bdf47ae7b41369b04e42ba388c1f1 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e58e2a069aa48c59858cbadb8ab9394 in term 0.
I20260709 13:10:02.640954 27323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a551aeaa560472abebcd6cc7d456faf" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16f9474cee3e43c89e5d9e42266cb250" is_pre_election: true
W20260709 13:10:02.642567 27142 leader_election.cc:343] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:02.642797 27176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a551aeaa560472abebcd6cc7d456faf" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394" is_pre_election: true
I20260709 13:10:02.643374 27142 leader_election.cc:304] T f49bdf47ae7b41369b04e42ba388c1f1 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250, 3e58e2a069aa48c59858cbadb8ab9394; no voters: 
I20260709 13:10:02.643711 27176 raft_consensus.cc:2468] T 7a551aeaa560472abebcd6cc7d456faf P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 0.
I20260709 13:10:02.645141 27250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f49bdf47ae7b41369b04e42ba388c1f1" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477" is_pre_election: true
I20260709 13:10:02.645571 27374 raft_consensus.cc:515] T f49bdf47ae7b41369b04e42ba388c1f1 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.636044 27251 raft_consensus.cc:2468] T e184a0b144684ada99885c4811605036 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e58e2a069aa48c59858cbadb8ab9394 in term 0.
I20260709 13:10:02.647316 27250 raft_consensus.cc:2393] T f49bdf47ae7b41369b04e42ba388c1f1 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e58e2a069aa48c59858cbadb8ab9394 in current term 1: Already voted for candidate df70ca0962e140ba92abead00b2fc477 in this term.
I20260709 13:10:02.649019 27374 leader_election.cc:290] T f49bdf47ae7b41369b04e42ba388c1f1 P df70ca0962e140ba92abead00b2fc477 [CANDIDATE]: Term 1 election: Requested vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:02.649876 27371 raft_consensus.cc:493] T acb5efb1d0d2422ba071fc5bc49e6390 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:02.649966 27249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a551aeaa560472abebcd6cc7d456faf" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477" is_pre_election: true
I20260709 13:10:02.650449 27368 tablet_bootstrap.cc:492] T acb5efb1d0d2422ba071fc5bc49e6390 P df70ca0962e140ba92abead00b2fc477: No bootstrap required, opened a new log
I20260709 13:10:02.650771 27378 raft_consensus.cc:493] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:02.648934 27324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f49bdf47ae7b41369b04e42ba388c1f1" candidate_uuid: "df70ca0962e140ba92abead00b2fc477" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16f9474cee3e43c89e5d9e42266cb250"
I20260709 13:10:02.650993 27368 ts_tablet_manager.cc:1403] T acb5efb1d0d2422ba071fc5bc49e6390 P df70ca0962e140ba92abead00b2fc477: Time spent bootstrapping tablet: real 0.063s	user 0.010s	sys 0.002s
I20260709 13:10:02.651409 27324 raft_consensus.cc:3060] T f49bdf47ae7b41369b04e42ba388c1f1 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:02.651336 27378 raft_consensus.cc:515] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.650720 27249 raft_consensus.cc:2468] T 7a551aeaa560472abebcd6cc7d456faf P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e58e2a069aa48c59858cbadb8ab9394 in term 0.
I20260709 13:10:02.654201 27176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "094982455ad24795a7cf35af53248051" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394" is_pre_election: true
I20260709 13:10:02.650458 27371 raft_consensus.cc:515] T acb5efb1d0d2422ba071fc5bc49e6390 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } }
I20260709 13:10:02.655236 27176 raft_consensus.cc:2468] T 094982455ad24795a7cf35af53248051 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 0.
I20260709 13:10:02.656090 27384 raft_consensus.cc:493] T acb5efb1d0d2422ba071fc5bc49e6390 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:02.656589 27384 raft_consensus.cc:515] T acb5efb1d0d2422ba071fc5bc49e6390 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } }
I20260709 13:10:02.658174 27366 tablet_bootstrap.cc:492] T a51060799a3243a881c471fc62688815 P 16f9474cee3e43c89e5d9e42266cb250: No bootstrap required, opened a new log
I20260709 13:10:02.654198 27368 raft_consensus.cc:359] T acb5efb1d0d2422ba071fc5bc49e6390 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } }
I20260709 13:10:02.658732 27366 ts_tablet_manager.cc:1403] T a51060799a3243a881c471fc62688815 P 16f9474cee3e43c89e5d9e42266cb250: Time spent bootstrapping tablet: real 0.027s	user 0.002s	sys 0.016s
I20260709 13:10:02.658864 27368 raft_consensus.cc:385] T acb5efb1d0d2422ba071fc5bc49e6390 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:02.658900 27384 leader_election.cc:290] T acb5efb1d0d2422ba071fc5bc49e6390 P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461)
I20260709 13:10:02.659304 27368 raft_consensus.cc:740] T acb5efb1d0d2422ba071fc5bc49e6390 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df70ca0962e140ba92abead00b2fc477, State: Initialized, Role: FOLLOWER
I20260709 13:10:02.659979 27371 leader_election.cc:290] T acb5efb1d0d2422ba071fc5bc49e6390 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233), df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461)
I20260709 13:10:02.660528 27371 raft_consensus.cc:2804] T f49bdf47ae7b41369b04e42ba388c1f1 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:02.660164 27368 consensus_queue.cc:260] T acb5efb1d0d2422ba071fc5bc49e6390 P df70ca0962e140ba92abead00b2fc477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } }
I20260709 13:10:02.660873 27371 raft_consensus.cc:493] T f49bdf47ae7b41369b04e42ba388c1f1 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:02.661039 27176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f49bdf47ae7b41369b04e42ba388c1f1" candidate_uuid: "df70ca0962e140ba92abead00b2fc477" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394"
I20260709 13:10:02.662019 27368 ts_tablet_manager.cc:1434] T acb5efb1d0d2422ba071fc5bc49e6390 P df70ca0962e140ba92abead00b2fc477: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:10:02.661244 27371 raft_consensus.cc:3060] T f49bdf47ae7b41369b04e42ba388c1f1 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:02.662544 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb5efb1d0d2422ba071fc5bc49e6390" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16f9474cee3e43c89e5d9e42266cb250" is_pre_election: true
I20260709 13:10:02.663228 27325 raft_consensus.cc:2468] T acb5efb1d0d2422ba071fc5bc49e6390 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e58e2a069aa48c59858cbadb8ab9394 in term 0.
I20260709 13:10:02.663946 27249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb5efb1d0d2422ba071fc5bc49e6390" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477" is_pre_election: true
I20260709 13:10:02.664464 27142 leader_election.cc:304] T acb5efb1d0d2422ba071fc5bc49e6390 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250, 3e58e2a069aa48c59858cbadb8ab9394; no voters: 
I20260709 13:10:02.662499 27392 raft_consensus.cc:493] T 094982455ad24795a7cf35af53248051 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:02.665764 27368 tablet_bootstrap.cc:492] T 7d767accb08c4a38a74ae205bc54d739 P df70ca0962e140ba92abead00b2fc477: Bootstrap starting.
I20260709 13:10:02.664551 27249 raft_consensus.cc:2468] T acb5efb1d0d2422ba071fc5bc49e6390 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 0.
I20260709 13:10:02.665839 27392 raft_consensus.cc:515] T 094982455ad24795a7cf35af53248051 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.667203 27289 leader_election.cc:304] T acb5efb1d0d2422ba071fc5bc49e6390 P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250, df70ca0962e140ba92abead00b2fc477; no voters: 
I20260709 13:10:02.656795 27290 leader_election.cc:304] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250, 3e58e2a069aa48c59858cbadb8ab9394; no voters: 
I20260709 13:10:02.668745 27384 raft_consensus.cc:2804] T acb5efb1d0d2422ba071fc5bc49e6390 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:02.669412 27384 raft_consensus.cc:493] T acb5efb1d0d2422ba071fc5bc49e6390 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:02.670506 27384 raft_consensus.cc:3060] T acb5efb1d0d2422ba071fc5bc49e6390 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:02.671205 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "094982455ad24795a7cf35af53248051" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16f9474cee3e43c89e5d9e42266cb250" is_pre_election: true
I20260709 13:10:02.670747 27366 raft_consensus.cc:359] T a51060799a3243a881c471fc62688815 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.657753 27370 raft_consensus.cc:515] T 7a551aeaa560472abebcd6cc7d456faf P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.673291 27175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb5efb1d0d2422ba071fc5bc49e6390" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394" is_pre_election: true
I20260709 13:10:02.673988 27175 raft_consensus.cc:2468] T acb5efb1d0d2422ba071fc5bc49e6390 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 0.
I20260709 13:10:02.674124 27370 leader_election.cc:290] T 7a551aeaa560472abebcd6cc7d456faf P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 election: Requested vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461)
I20260709 13:10:02.674822 27366 raft_consensus.cc:385] T a51060799a3243a881c471fc62688815 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:02.674955 27323 raft_consensus.cc:2393] T 7a551aeaa560472abebcd6cc7d456faf P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e58e2a069aa48c59858cbadb8ab9394 in current term 1: Already voted for candidate 16f9474cee3e43c89e5d9e42266cb250 in this term.
I20260709 13:10:02.675181 27249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a551aeaa560472abebcd6cc7d456faf" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477"
I20260709 13:10:02.677091 27175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a551aeaa560472abebcd6cc7d456faf" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394"
I20260709 13:10:02.677810 27175 raft_consensus.cc:3060] T 7a551aeaa560472abebcd6cc7d456faf P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:02.677851 27249 raft_consensus.cc:3060] T 7a551aeaa560472abebcd6cc7d456faf P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:02.679102 27392 leader_election.cc:290] T 094982455ad24795a7cf35af53248051 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:02.679723 27392 raft_consensus.cc:2804] T acb5efb1d0d2422ba071fc5bc49e6390 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:02.680102 27392 raft_consensus.cc:493] T acb5efb1d0d2422ba071fc5bc49e6390 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:02.680553 27392 raft_consensus.cc:3060] T acb5efb1d0d2422ba071fc5bc49e6390 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:02.684726 27374 raft_consensus.cc:493] T 094982455ad24795a7cf35af53248051 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:02.687108 27374 raft_consensus.cc:515] T 094982455ad24795a7cf35af53248051 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.672212 27325 raft_consensus.cc:2468] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e58e2a069aa48c59858cbadb8ab9394 in term 0.
I20260709 13:10:02.691856 27142 leader_election.cc:304] T 094982455ad24795a7cf35af53248051 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250, 3e58e2a069aa48c59858cbadb8ab9394; no voters: 
I20260709 13:10:02.687337 27324 raft_consensus.cc:2468] T f49bdf47ae7b41369b04e42ba388c1f1 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df70ca0962e140ba92abead00b2fc477 in term 1.
I20260709 13:10:02.692740 27370 raft_consensus.cc:2804] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:02.693161 27400 raft_consensus.cc:2804] T 094982455ad24795a7cf35af53248051 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:02.693578 27217 leader_election.cc:304] T f49bdf47ae7b41369b04e42ba388c1f1 P df70ca0962e140ba92abead00b2fc477 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250, df70ca0962e140ba92abead00b2fc477; no voters: 
I20260709 13:10:02.693964 27324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "094982455ad24795a7cf35af53248051" candidate_uuid: "df70ca0962e140ba92abead00b2fc477" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16f9474cee3e43c89e5d9e42266cb250" is_pre_election: true
I20260709 13:10:02.694669 27324 raft_consensus.cc:2468] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df70ca0962e140ba92abead00b2fc477 in term 0.
I20260709 13:10:02.695127 27370 raft_consensus.cc:493] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:02.695526 27370 raft_consensus.cc:3060] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:02.696303 27140 leader_election.cc:304] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e58e2a069aa48c59858cbadb8ab9394, df70ca0962e140ba92abead00b2fc477; no voters: 16f9474cee3e43c89e5d9e42266cb250
I20260709 13:10:02.689234 27250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "094982455ad24795a7cf35af53248051" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477" is_pre_election: true
I20260709 13:10:02.696897 27174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "094982455ad24795a7cf35af53248051" candidate_uuid: "df70ca0962e140ba92abead00b2fc477" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394" is_pre_election: true
I20260709 13:10:02.697506 27175 raft_consensus.cc:2468] T 7a551aeaa560472abebcd6cc7d456faf P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 1.
I20260709 13:10:02.697814 27250 raft_consensus.cc:2468] T 094982455ad24795a7cf35af53248051 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 0.
I20260709 13:10:02.698405 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb5efb1d0d2422ba071fc5bc49e6390" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477" is_pre_election: true
I20260709 13:10:02.698719 27290 leader_election.cc:304] T 7a551aeaa560472abebcd6cc7d456faf P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250, 3e58e2a069aa48c59858cbadb8ab9394; no voters: 
I20260709 13:10:02.699046 27251 raft_consensus.cc:2468] T acb5efb1d0d2422ba071fc5bc49e6390 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e58e2a069aa48c59858cbadb8ab9394 in term 0.
I20260709 13:10:02.627369 27365 tablet_bootstrap.cc:492] T 7d767accb08c4a38a74ae205bc54d739 P 3e58e2a069aa48c59858cbadb8ab9394: Bootstrap starting.
I20260709 13:10:02.699975 27401 raft_consensus.cc:2804] T 7a551aeaa560472abebcd6cc7d456faf P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:02.701282 27378 leader_election.cc:290] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461)
I20260709 13:10:02.693608 27400 raft_consensus.cc:493] T 094982455ad24795a7cf35af53248051 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:02.704157 27400 raft_consensus.cc:3060] T 094982455ad24795a7cf35af53248051 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:02.706475 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "094982455ad24795a7cf35af53248051" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477" is_pre_election: true
I20260709 13:10:02.707165 27251 raft_consensus.cc:2468] T 094982455ad24795a7cf35af53248051 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e58e2a069aa48c59858cbadb8ab9394 in term 0.
I20260709 13:10:02.707456 27365 tablet_bootstrap.cc:654] T 7d767accb08c4a38a74ae205bc54d739 P 3e58e2a069aa48c59858cbadb8ab9394: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:02.687752 27371 raft_consensus.cc:515] T f49bdf47ae7b41369b04e42ba388c1f1 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.708487 27368 tablet_bootstrap.cc:654] T 7d767accb08c4a38a74ae205bc54d739 P df70ca0962e140ba92abead00b2fc477: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:02.692231 27374 leader_election.cc:290] T 094982455ad24795a7cf35af53248051 P df70ca0962e140ba92abead00b2fc477 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:02.675222 27366 raft_consensus.cc:740] T a51060799a3243a881c471fc62688815 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16f9474cee3e43c89e5d9e42266cb250, State: Initialized, Role: FOLLOWER
I20260709 13:10:02.711594 27374 raft_consensus.cc:2804] T f49bdf47ae7b41369b04e42ba388c1f1 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:02.712142 27374 raft_consensus.cc:697] T f49bdf47ae7b41369b04e42ba388c1f1 P df70ca0962e140ba92abead00b2fc477 [term 1 LEADER]: Becoming Leader. State: Replica: df70ca0962e140ba92abead00b2fc477, State: Running, Role: LEADER
I20260709 13:10:02.712913 27366 consensus_queue.cc:260] T a51060799a3243a881c471fc62688815 P 16f9474cee3e43c89e5d9e42266cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.713477 27370 raft_consensus.cc:515] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.713222 27374 consensus_queue.cc:237] T f49bdf47ae7b41369b04e42ba388c1f1 P df70ca0962e140ba92abead00b2fc477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.716449 27175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "094982455ad24795a7cf35af53248051" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394"
I20260709 13:10:02.719180 27403 raft_consensus.cc:2804] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:02.722121 27403 raft_consensus.cc:493] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:02.722546 27403 raft_consensus.cc:3060] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:02.724776 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "094982455ad24795a7cf35af53248051" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477"
I20260709 13:10:02.725411 27251 raft_consensus.cc:3060] T 094982455ad24795a7cf35af53248051 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:02.720798 27392 raft_consensus.cc:515] T acb5efb1d0d2422ba071fc5bc49e6390 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } }
I20260709 13:10:02.723521 27370 leader_election.cc:290] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 election: Requested vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461)
I20260709 13:10:02.725457 27176 raft_consensus.cc:2393] T f49bdf47ae7b41369b04e42ba388c1f1 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate df70ca0962e140ba92abead00b2fc477 in current term 1: Already voted for candidate 3e58e2a069aa48c59858cbadb8ab9394 in this term.
I20260709 13:10:02.729442 27324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f49bdf47ae7b41369b04e42ba388c1f1" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16f9474cee3e43c89e5d9e42266cb250"
I20260709 13:10:02.733260 27324 raft_consensus.cc:2393] T f49bdf47ae7b41369b04e42ba388c1f1 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3e58e2a069aa48c59858cbadb8ab9394 in current term 1: Already voted for candidate df70ca0962e140ba92abead00b2fc477 in this term.
I20260709 13:10:02.730021 27392 leader_election.cc:290] T acb5efb1d0d2422ba071fc5bc49e6390 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 election: Requested vote from peers 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233), df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461)
I20260709 13:10:02.731470 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb5efb1d0d2422ba071fc5bc49e6390" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16f9474cee3e43c89e5d9e42266cb250"
I20260709 13:10:02.731642 27400 raft_consensus.cc:515] T 094982455ad24795a7cf35af53248051 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.722590 27401 raft_consensus.cc:697] T 7a551aeaa560472abebcd6cc7d456faf P 16f9474cee3e43c89e5d9e42266cb250 [term 1 LEADER]: Becoming Leader. State: Replica: 16f9474cee3e43c89e5d9e42266cb250, State: Running, Role: LEADER
I20260709 13:10:02.735808 27174 raft_consensus.cc:2393] T 094982455ad24795a7cf35af53248051 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate df70ca0962e140ba92abead00b2fc477 in current term 1: Already voted for candidate 3e58e2a069aa48c59858cbadb8ab9394 in this term.
I20260709 13:10:02.736536 27401 consensus_queue.cc:237] T 7a551aeaa560472abebcd6cc7d456faf P 16f9474cee3e43c89e5d9e42266cb250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.739886 27400 leader_election.cc:290] T 094982455ad24795a7cf35af53248051 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 election: Requested vote from peers df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:02.739791 27324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "094982455ad24795a7cf35af53248051" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16f9474cee3e43c89e5d9e42266cb250"
I20260709 13:10:02.742938 27324 raft_consensus.cc:2393] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3e58e2a069aa48c59858cbadb8ab9394 in current term 1: Already voted for candidate 16f9474cee3e43c89e5d9e42266cb250 in this term.
I20260709 13:10:02.740168 27251 raft_consensus.cc:2468] T 094982455ad24795a7cf35af53248051 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 1.
I20260709 13:10:02.722640 27140 leader_election.cc:304] T 7a551aeaa560472abebcd6cc7d456faf P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e58e2a069aa48c59858cbadb8ab9394, df70ca0962e140ba92abead00b2fc477; no voters: 16f9474cee3e43c89e5d9e42266cb250
I20260709 13:10:02.739709 27371 leader_election.cc:290] T f49bdf47ae7b41369b04e42ba388c1f1 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 election: Requested vote from peers df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:02.745314 27365 tablet_bootstrap.cc:492] T 7d767accb08c4a38a74ae205bc54d739 P 3e58e2a069aa48c59858cbadb8ab9394: No bootstrap required, opened a new log
I20260709 13:10:02.743649 27403 raft_consensus.cc:515] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.745183 27289 leader_election.cc:304] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250, df70ca0962e140ba92abead00b2fc477; no voters: 3e58e2a069aa48c59858cbadb8ab9394
I20260709 13:10:02.746078 27365 ts_tablet_manager.cc:1403] T 7d767accb08c4a38a74ae205bc54d739 P 3e58e2a069aa48c59858cbadb8ab9394: Time spent bootstrapping tablet: real 0.119s	user 0.024s	sys 0.018s
I20260709 13:10:02.746977 27370 raft_consensus.cc:2804] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:02.715242 27366 ts_tablet_manager.cc:1434] T a51060799a3243a881c471fc62688815 P 16f9474cee3e43c89e5d9e42266cb250: Time spent starting tablet: real 0.047s	user 0.004s	sys 0.003s
I20260709 13:10:02.747531 27370 raft_consensus.cc:697] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 LEADER]: Becoming Leader. State: Replica: 16f9474cee3e43c89e5d9e42266cb250, State: Running, Role: LEADER
I20260709 13:10:02.745335 27371 raft_consensus.cc:2764] T 7a551aeaa560472abebcd6cc7d456faf P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:10:02.748257 27366 tablet_bootstrap.cc:492] T 74072bd16e8d49749ab1f70a27f71bea P 16f9474cee3e43c89e5d9e42266cb250: Bootstrap starting.
I20260709 13:10:02.748405 27370 consensus_queue.cc:237] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.749847 27250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb5efb1d0d2422ba071fc5bc49e6390" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477"
I20260709 13:10:02.750007 27365 raft_consensus.cc:359] T 7d767accb08c4a38a74ae205bc54d739 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.750821 27365 raft_consensus.cc:385] T 7d767accb08c4a38a74ae205bc54d739 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:02.750432 27250 raft_consensus.cc:3060] T acb5efb1d0d2422ba071fc5bc49e6390 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:02.751235 27365 raft_consensus.cc:740] T 7d767accb08c4a38a74ae205bc54d739 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e58e2a069aa48c59858cbadb8ab9394, State: Initialized, Role: FOLLOWER
I20260709 13:10:02.751163 27248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "094982455ad24795a7cf35af53248051" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477"
I20260709 13:10:02.752169 27248 raft_consensus.cc:2393] T 094982455ad24795a7cf35af53248051 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3e58e2a069aa48c59858cbadb8ab9394 in current term 1: Already voted for candidate 16f9474cee3e43c89e5d9e42266cb250 in this term.
I20260709 13:10:02.752281 27365 consensus_queue.cc:260] T 7d767accb08c4a38a74ae205bc54d739 P 3e58e2a069aa48c59858cbadb8ab9394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.755074 27366 tablet_bootstrap.cc:654] T 74072bd16e8d49749ab1f70a27f71bea P 16f9474cee3e43c89e5d9e42266cb250: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:02.721131 27249 raft_consensus.cc:2468] T 7a551aeaa560472abebcd6cc7d456faf P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 1.
W20260709 13:10:02.742000 27217 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.103.3:33233, user_credentials={real_user=slave}} blocked reactor thread for 48512.7us
I20260709 13:10:02.743646 27384 raft_consensus.cc:515] T acb5efb1d0d2422ba071fc5bc49e6390 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } }
I20260709 13:10:02.750142 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f49bdf47ae7b41369b04e42ba388c1f1" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477"
I20260709 13:10:02.767113 27403 leader_election.cc:290] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 election: Requested vote from peers df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:02.767911 27217 leader_election.cc:304] T 094982455ad24795a7cf35af53248051 P df70ca0962e140ba92abead00b2fc477 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250, df70ca0962e140ba92abead00b2fc477; no voters: 3e58e2a069aa48c59858cbadb8ab9394
I20260709 13:10:02.768244 27324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e184a0b144684ada99885c4811605036" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16f9474cee3e43c89e5d9e42266cb250"
I20260709 13:10:02.771839 27140 leader_election.cc:304] T 094982455ad24795a7cf35af53248051 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e58e2a069aa48c59858cbadb8ab9394; no voters: 16f9474cee3e43c89e5d9e42266cb250, df70ca0962e140ba92abead00b2fc477
I20260709 13:10:02.772787 27403 raft_consensus.cc:2749] T 094982455ad24795a7cf35af53248051 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:10:02.773190 27140 leader_election.cc:304] T f49bdf47ae7b41369b04e42ba388c1f1 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e58e2a069aa48c59858cbadb8ab9394; no voters: 16f9474cee3e43c89e5d9e42266cb250, df70ca0962e140ba92abead00b2fc477
I20260709 13:10:02.772912 27074 catalog_manager.cc:5697] T 7a551aeaa560472abebcd6cc7d456faf P 16f9474cee3e43c89e5d9e42266cb250 reported cstate change: term changed from 0 to 1, leader changed from <none> to 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3). New cstate: current_term: 1 leader_uuid: "16f9474cee3e43c89e5d9e42266cb250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:02.774075 27403 raft_consensus.cc:2749] T f49bdf47ae7b41369b04e42ba388c1f1 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:10:02.775421 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e184a0b144684ada99885c4811605036" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477"
I20260709 13:10:02.775995 27325 raft_consensus.cc:2393] T acb5efb1d0d2422ba071fc5bc49e6390 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3e58e2a069aa48c59858cbadb8ab9394 in current term 1: Already voted for candidate 16f9474cee3e43c89e5d9e42266cb250 in this term.
I20260709 13:10:02.776098 27251 raft_consensus.cc:3060] T e184a0b144684ada99885c4811605036 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:02.770728 27374 raft_consensus.cc:2764] T 094982455ad24795a7cf35af53248051 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
W20260709 13:10:02.778236 27142 leader_election.cc:343] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:02.781978 27250 raft_consensus.cc:2468] T acb5efb1d0d2422ba071fc5bc49e6390 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e58e2a069aa48c59858cbadb8ab9394 in term 1.
I20260709 13:10:02.783385 27140 leader_election.cc:304] T acb5efb1d0d2422ba071fc5bc49e6390 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e58e2a069aa48c59858cbadb8ab9394, df70ca0962e140ba92abead00b2fc477; no voters: 16f9474cee3e43c89e5d9e42266cb250
I20260709 13:10:02.759629 27072 catalog_manager.cc:5697] T f49bdf47ae7b41369b04e42ba388c1f1 P df70ca0962e140ba92abead00b2fc477 reported cstate change: term changed from 0 to 1, leader changed from <none> to df70ca0962e140ba92abead00b2fc477 (127.14.103.2). New cstate: current_term: 1 leader_uuid: "df70ca0962e140ba92abead00b2fc477" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:02.785323 27251 raft_consensus.cc:2468] T e184a0b144684ada99885c4811605036 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e58e2a069aa48c59858cbadb8ab9394 in term 1.
I20260709 13:10:02.785426 27403 raft_consensus.cc:2804] T acb5efb1d0d2422ba071fc5bc49e6390 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:02.786194 27403 raft_consensus.cc:697] T acb5efb1d0d2422ba071fc5bc49e6390 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 LEADER]: Becoming Leader. State: Replica: 3e58e2a069aa48c59858cbadb8ab9394, State: Running, Role: LEADER
I20260709 13:10:02.787372 27140 leader_election.cc:304] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e58e2a069aa48c59858cbadb8ab9394, df70ca0962e140ba92abead00b2fc477; no voters: 16f9474cee3e43c89e5d9e42266cb250
I20260709 13:10:02.787276 27403 consensus_queue.cc:237] T acb5efb1d0d2422ba071fc5bc49e6390 P 3e58e2a069aa48c59858cbadb8ab9394 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } }
I20260709 13:10:02.788285 27371 raft_consensus.cc:2804] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:02.788803 27371 raft_consensus.cc:697] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 LEADER]: Becoming Leader. State: Replica: 3e58e2a069aa48c59858cbadb8ab9394, State: Running, Role: LEADER
I20260709 13:10:02.789575 27371 consensus_queue.cc:237] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.797809 27384 leader_election.cc:290] T acb5efb1d0d2422ba071fc5bc49e6390 P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 election: Requested vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461)
I20260709 13:10:02.798874 27174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb5efb1d0d2422ba071fc5bc49e6390" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394"
I20260709 13:10:02.799926 27365 ts_tablet_manager.cc:1434] T 7d767accb08c4a38a74ae205bc54d739 P 3e58e2a069aa48c59858cbadb8ab9394: Time spent starting tablet: real 0.053s	user 0.004s	sys 0.004s
I20260709 13:10:02.809075 27366 tablet_bootstrap.cc:492] T 74072bd16e8d49749ab1f70a27f71bea P 16f9474cee3e43c89e5d9e42266cb250: No bootstrap required, opened a new log
I20260709 13:10:02.809520 27365 tablet_bootstrap.cc:492] T a51060799a3243a881c471fc62688815 P 3e58e2a069aa48c59858cbadb8ab9394: Bootstrap starting.
I20260709 13:10:02.801551 27368 tablet_bootstrap.cc:492] T 7d767accb08c4a38a74ae205bc54d739 P df70ca0962e140ba92abead00b2fc477: No bootstrap required, opened a new log
I20260709 13:10:02.812410 27368 ts_tablet_manager.cc:1403] T 7d767accb08c4a38a74ae205bc54d739 P df70ca0962e140ba92abead00b2fc477: Time spent bootstrapping tablet: real 0.147s	user 0.018s	sys 0.014s
I20260709 13:10:02.812287 27074 catalog_manager.cc:5697] T acb5efb1d0d2422ba071fc5bc49e6390 P 3e58e2a069aa48c59858cbadb8ab9394 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "3e58e2a069aa48c59858cbadb8ab9394" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:02.802345 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb5efb1d0d2422ba071fc5bc49e6390" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477"
I20260709 13:10:02.806488 27370 raft_consensus.cc:493] T a51060799a3243a881c471fc62688815 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:02.816658 27365 tablet_bootstrap.cc:654] T a51060799a3243a881c471fc62688815 P 3e58e2a069aa48c59858cbadb8ab9394: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:02.817082 27370 raft_consensus.cc:515] T a51060799a3243a881c471fc62688815 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.816682 27368 raft_consensus.cc:359] T 7d767accb08c4a38a74ae205bc54d739 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.819851 27368 raft_consensus.cc:385] T 7d767accb08c4a38a74ae205bc54d739 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:02.820178 27368 raft_consensus.cc:740] T 7d767accb08c4a38a74ae205bc54d739 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df70ca0962e140ba92abead00b2fc477, State: Initialized, Role: FOLLOWER
I20260709 13:10:02.821416 27250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51060799a3243a881c471fc62688815" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477" is_pre_election: true
I20260709 13:10:02.821748 27251 raft_consensus.cc:2393] T acb5efb1d0d2422ba071fc5bc49e6390 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 16f9474cee3e43c89e5d9e42266cb250 in current term 1: Already voted for candidate 3e58e2a069aa48c59858cbadb8ab9394 in this term.
I20260709 13:10:02.809593 27366 ts_tablet_manager.cc:1403] T 74072bd16e8d49749ab1f70a27f71bea P 16f9474cee3e43c89e5d9e42266cb250: Time spent bootstrapping tablet: real 0.062s	user 0.020s	sys 0.020s
W20260709 13:10:02.823207 27289 leader_election.cc:343] T a51060799a3243a881c471fc62688815 P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:02.821225 27370 leader_election.cc:290] T a51060799a3243a881c471fc62688815 P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461)
I20260709 13:10:02.823951 27174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51060799a3243a881c471fc62688815" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394" is_pre_election: true
I20260709 13:10:02.824280 27289 leader_election.cc:304] T acb5efb1d0d2422ba071fc5bc49e6390 P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250; no voters: 3e58e2a069aa48c59858cbadb8ab9394, df70ca0962e140ba92abead00b2fc477
W20260709 13:10:02.825706 27290 leader_election.cc:343] T a51060799a3243a881c471fc62688815 P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:02.826045 27370 raft_consensus.cc:2749] T acb5efb1d0d2422ba071fc5bc49e6390 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:10:02.826334 27290 leader_election.cc:304] T a51060799a3243a881c471fc62688815 P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250; no voters: 3e58e2a069aa48c59858cbadb8ab9394, df70ca0962e140ba92abead00b2fc477
I20260709 13:10:02.828224 27370 raft_consensus.cc:2749] T a51060799a3243a881c471fc62688815 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:02.832432 27366 raft_consensus.cc:359] T 74072bd16e8d49749ab1f70a27f71bea P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.833194 27366 raft_consensus.cc:385] T 74072bd16e8d49749ab1f70a27f71bea P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:02.832543 27072 catalog_manager.cc:5697] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250 reported cstate change: term changed from 0 to 1, leader changed from <none> to 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3). New cstate: current_term: 1 leader_uuid: "16f9474cee3e43c89e5d9e42266cb250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:02.821043 27368 consensus_queue.cc:260] T 7d767accb08c4a38a74ae205bc54d739 P df70ca0962e140ba92abead00b2fc477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.833519 27366 raft_consensus.cc:740] T 74072bd16e8d49749ab1f70a27f71bea P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16f9474cee3e43c89e5d9e42266cb250, State: Initialized, Role: FOLLOWER
I20260709 13:10:02.834527 27366 consensus_queue.cc:260] T 74072bd16e8d49749ab1f70a27f71bea P 16f9474cee3e43c89e5d9e42266cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.837062 27366 ts_tablet_manager.cc:1434] T 74072bd16e8d49749ab1f70a27f71bea P 16f9474cee3e43c89e5d9e42266cb250: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:10:02.838028 27366 tablet_bootstrap.cc:492] T e184a0b144684ada99885c4811605036 P 16f9474cee3e43c89e5d9e42266cb250: Bootstrap starting.
I20260709 13:10:02.844280 27366 tablet_bootstrap.cc:654] T e184a0b144684ada99885c4811605036 P 16f9474cee3e43c89e5d9e42266cb250: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:02.845017 27365 tablet_bootstrap.cc:492] T a51060799a3243a881c471fc62688815 P 3e58e2a069aa48c59858cbadb8ab9394: No bootstrap required, opened a new log
I20260709 13:10:02.845532 27365 ts_tablet_manager.cc:1403] T a51060799a3243a881c471fc62688815 P 3e58e2a069aa48c59858cbadb8ab9394: Time spent bootstrapping tablet: real 0.036s	user 0.012s	sys 0.016s
I20260709 13:10:02.849834 27365 raft_consensus.cc:359] T a51060799a3243a881c471fc62688815 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.850569 27365 raft_consensus.cc:385] T a51060799a3243a881c471fc62688815 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:02.850898 27365 raft_consensus.cc:740] T a51060799a3243a881c471fc62688815 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e58e2a069aa48c59858cbadb8ab9394, State: Initialized, Role: FOLLOWER
I20260709 13:10:02.851778 27368 ts_tablet_manager.cc:1434] T 7d767accb08c4a38a74ae205bc54d739 P df70ca0962e140ba92abead00b2fc477: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.000s
I20260709 13:10:02.851759 27365 consensus_queue.cc:260] T a51060799a3243a881c471fc62688815 P 3e58e2a069aa48c59858cbadb8ab9394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.852984 27368 tablet_bootstrap.cc:492] T 74072bd16e8d49749ab1f70a27f71bea P df70ca0962e140ba92abead00b2fc477: Bootstrap starting.
I20260709 13:10:02.854074 27365 ts_tablet_manager.cc:1434] T a51060799a3243a881c471fc62688815 P 3e58e2a069aa48c59858cbadb8ab9394: Time spent starting tablet: real 0.008s	user 0.001s	sys 0.004s
I20260709 13:10:02.855026 27365 tablet_bootstrap.cc:492] T 74072bd16e8d49749ab1f70a27f71bea P 3e58e2a069aa48c59858cbadb8ab9394: Bootstrap starting.
I20260709 13:10:02.853747 27070 catalog_manager.cc:5697] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "3e58e2a069aa48c59858cbadb8ab9394" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:02.860271 27368 tablet_bootstrap.cc:654] T 74072bd16e8d49749ab1f70a27f71bea P df70ca0962e140ba92abead00b2fc477: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:02.861497 27365 tablet_bootstrap.cc:654] T 74072bd16e8d49749ab1f70a27f71bea P 3e58e2a069aa48c59858cbadb8ab9394: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:02.867410 27366 tablet_bootstrap.cc:492] T e184a0b144684ada99885c4811605036 P 16f9474cee3e43c89e5d9e42266cb250: No bootstrap required, opened a new log
I20260709 13:10:02.867930 27366 ts_tablet_manager.cc:1403] T e184a0b144684ada99885c4811605036 P 16f9474cee3e43c89e5d9e42266cb250: Time spent bootstrapping tablet: real 0.030s	user 0.014s	sys 0.014s
I20260709 13:10:02.870484 27366 raft_consensus.cc:359] T e184a0b144684ada99885c4811605036 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.871227 27366 raft_consensus.cc:385] T e184a0b144684ada99885c4811605036 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:02.871505 27366 raft_consensus.cc:740] T e184a0b144684ada99885c4811605036 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16f9474cee3e43c89e5d9e42266cb250, State: Initialized, Role: FOLLOWER
I20260709 13:10:02.872313 27366 consensus_queue.cc:260] T e184a0b144684ada99885c4811605036 P 16f9474cee3e43c89e5d9e42266cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.874565 27366 ts_tablet_manager.cc:1434] T e184a0b144684ada99885c4811605036 P 16f9474cee3e43c89e5d9e42266cb250: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:10:02.878916 27365 tablet_bootstrap.cc:492] T 74072bd16e8d49749ab1f70a27f71bea P 3e58e2a069aa48c59858cbadb8ab9394: No bootstrap required, opened a new log
I20260709 13:10:02.879446 27365 ts_tablet_manager.cc:1403] T 74072bd16e8d49749ab1f70a27f71bea P 3e58e2a069aa48c59858cbadb8ab9394: Time spent bootstrapping tablet: real 0.025s	user 0.010s	sys 0.010s
I20260709 13:10:02.882036 27365 raft_consensus.cc:359] T 74072bd16e8d49749ab1f70a27f71bea P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.882697 27365 raft_consensus.cc:385] T 74072bd16e8d49749ab1f70a27f71bea P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:02.882966 27365 raft_consensus.cc:740] T 74072bd16e8d49749ab1f70a27f71bea P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e58e2a069aa48c59858cbadb8ab9394, State: Initialized, Role: FOLLOWER
I20260709 13:10:02.883567 27365 consensus_queue.cc:260] T 74072bd16e8d49749ab1f70a27f71bea P 3e58e2a069aa48c59858cbadb8ab9394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.885450 27365 ts_tablet_manager.cc:1434] T 74072bd16e8d49749ab1f70a27f71bea P 3e58e2a069aa48c59858cbadb8ab9394: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:10:02.889250 27368 tablet_bootstrap.cc:492] T 74072bd16e8d49749ab1f70a27f71bea P df70ca0962e140ba92abead00b2fc477: No bootstrap required, opened a new log
I20260709 13:10:02.889799 27368 ts_tablet_manager.cc:1403] T 74072bd16e8d49749ab1f70a27f71bea P df70ca0962e140ba92abead00b2fc477: Time spent bootstrapping tablet: real 0.037s	user 0.009s	sys 0.006s
I20260709 13:10:02.891894 27368 raft_consensus.cc:359] T 74072bd16e8d49749ab1f70a27f71bea P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.892467 27368 raft_consensus.cc:385] T 74072bd16e8d49749ab1f70a27f71bea P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:02.892768 27368 raft_consensus.cc:740] T 74072bd16e8d49749ab1f70a27f71bea P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df70ca0962e140ba92abead00b2fc477, State: Initialized, Role: FOLLOWER
I20260709 13:10:02.893378 27368 consensus_queue.cc:260] T 74072bd16e8d49749ab1f70a27f71bea P df70ca0962e140ba92abead00b2fc477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.894661 27368 ts_tablet_manager.cc:1434] T 74072bd16e8d49749ab1f70a27f71bea P df70ca0962e140ba92abead00b2fc477: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20260709 13:10:02.895597 27368 tablet_bootstrap.cc:492] T a51060799a3243a881c471fc62688815 P df70ca0962e140ba92abead00b2fc477: Bootstrap starting.
I20260709 13:10:02.900409 27368 tablet_bootstrap.cc:654] T a51060799a3243a881c471fc62688815 P df70ca0962e140ba92abead00b2fc477: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:02.901136 27371 raft_consensus.cc:493] T a51060799a3243a881c471fc62688815 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:02.901648 27371 raft_consensus.cc:515] T a51060799a3243a881c471fc62688815 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.904953 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51060799a3243a881c471fc62688815" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477" is_pre_election: true
I20260709 13:10:02.905851 27324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51060799a3243a881c471fc62688815" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16f9474cee3e43c89e5d9e42266cb250" is_pre_election: true
W20260709 13:10:02.906173 27140 leader_election.cc:343] T a51060799a3243a881c471fc62688815 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:02.906563 27324 raft_consensus.cc:2468] T a51060799a3243a881c471fc62688815 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e58e2a069aa48c59858cbadb8ab9394 in term 0.
I20260709 13:10:02.906724 27368 tablet_bootstrap.cc:492] T a51060799a3243a881c471fc62688815 P df70ca0962e140ba92abead00b2fc477: No bootstrap required, opened a new log
I20260709 13:10:02.907235 27368 ts_tablet_manager.cc:1403] T a51060799a3243a881c471fc62688815 P df70ca0962e140ba92abead00b2fc477: Time spent bootstrapping tablet: real 0.012s	user 0.006s	sys 0.003s
I20260709 13:10:02.907325 27371 leader_election.cc:290] T a51060799a3243a881c471fc62688815 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:02.907773 27142 leader_election.cc:304] T a51060799a3243a881c471fc62688815 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250, 3e58e2a069aa48c59858cbadb8ab9394; no voters: df70ca0962e140ba92abead00b2fc477
I20260709 13:10:02.908486 27371 raft_consensus.cc:2804] T a51060799a3243a881c471fc62688815 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:02.908852 27371 raft_consensus.cc:493] T a51060799a3243a881c471fc62688815 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:02.909124 27371 raft_consensus.cc:3060] T a51060799a3243a881c471fc62688815 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:02.909489 27368 raft_consensus.cc:359] T a51060799a3243a881c471fc62688815 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.910036 27368 raft_consensus.cc:385] T a51060799a3243a881c471fc62688815 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:02.910274 27368 raft_consensus.cc:740] T a51060799a3243a881c471fc62688815 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df70ca0962e140ba92abead00b2fc477, State: Initialized, Role: FOLLOWER
I20260709 13:10:02.910952 27368 consensus_queue.cc:260] T a51060799a3243a881c471fc62688815 P df70ca0962e140ba92abead00b2fc477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.912756 27368 ts_tablet_manager.cc:1434] T a51060799a3243a881c471fc62688815 P df70ca0962e140ba92abead00b2fc477: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:10:02.915432 27371 raft_consensus.cc:515] T a51060799a3243a881c471fc62688815 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.917407 27371 leader_election.cc:290] T a51060799a3243a881c471fc62688815 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 election: Requested vote from peers df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:02.918236 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51060799a3243a881c471fc62688815" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477"
I20260709 13:10:02.918453 27324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51060799a3243a881c471fc62688815" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16f9474cee3e43c89e5d9e42266cb250"
I20260709 13:10:02.918751 27251 raft_consensus.cc:3060] T a51060799a3243a881c471fc62688815 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:02.919098 27324 raft_consensus.cc:3060] T a51060799a3243a881c471fc62688815 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:02.925324 27251 raft_consensus.cc:2468] T a51060799a3243a881c471fc62688815 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e58e2a069aa48c59858cbadb8ab9394 in term 1.
I20260709 13:10:02.925320 27370 consensus_queue.cc:1048] T 713697e046ae4150a4047a4bf2c1180f P 16f9474cee3e43c89e5d9e42266cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:02.925691 27324 raft_consensus.cc:2468] T a51060799a3243a881c471fc62688815 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e58e2a069aa48c59858cbadb8ab9394 in term 1.
I20260709 13:10:02.926932 27142 leader_election.cc:304] T a51060799a3243a881c471fc62688815 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250, 3e58e2a069aa48c59858cbadb8ab9394; no voters: 
I20260709 13:10:02.927721 27371 raft_consensus.cc:2804] T a51060799a3243a881c471fc62688815 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:02.928443 27371 raft_consensus.cc:697] T a51060799a3243a881c471fc62688815 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 LEADER]: Becoming Leader. State: Replica: 3e58e2a069aa48c59858cbadb8ab9394, State: Running, Role: LEADER
I20260709 13:10:02.929349 27371 consensus_queue.cc:237] T a51060799a3243a881c471fc62688815 P 3e58e2a069aa48c59858cbadb8ab9394 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:02.941543 27072 catalog_manager.cc:5697] T a51060799a3243a881c471fc62688815 P 3e58e2a069aa48c59858cbadb8ab9394 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "3e58e2a069aa48c59858cbadb8ab9394" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:02.945604 27401 consensus_queue.cc:1048] T 713697e046ae4150a4047a4bf2c1180f P 16f9474cee3e43c89e5d9e42266cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:03.041432 27401 raft_consensus.cc:493] T 7d767accb08c4a38a74ae205bc54d739 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:03.041873 27401 raft_consensus.cc:515] T 7d767accb08c4a38a74ae205bc54d739 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:03.043860 27401 leader_election.cc:290] T 7d767accb08c4a38a74ae205bc54d739 P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461)
I20260709 13:10:03.044595 27174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d767accb08c4a38a74ae205bc54d739" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394" is_pre_election: true
I20260709 13:10:03.044903 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d767accb08c4a38a74ae205bc54d739" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477" is_pre_election: true
I20260709 13:10:03.045326 27174 raft_consensus.cc:2468] T 7d767accb08c4a38a74ae205bc54d739 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 0.
I20260709 13:10:03.045518 27251 raft_consensus.cc:2468] T 7d767accb08c4a38a74ae205bc54d739 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 0.
I20260709 13:10:03.046401 27290 leader_election.cc:304] T 7d767accb08c4a38a74ae205bc54d739 P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250, 3e58e2a069aa48c59858cbadb8ab9394; no voters: 
I20260709 13:10:03.047073 27401 raft_consensus.cc:2804] T 7d767accb08c4a38a74ae205bc54d739 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:03.047384 27401 raft_consensus.cc:493] T 7d767accb08c4a38a74ae205bc54d739 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:03.047649 27401 raft_consensus.cc:3060] T 7d767accb08c4a38a74ae205bc54d739 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:03.053226 27401 raft_consensus.cc:515] T 7d767accb08c4a38a74ae205bc54d739 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:03.054739 27401 leader_election.cc:290] T 7d767accb08c4a38a74ae205bc54d739 P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 election: Requested vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461)
I20260709 13:10:03.055503 27174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d767accb08c4a38a74ae205bc54d739" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394"
I20260709 13:10:03.055656 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d767accb08c4a38a74ae205bc54d739" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477"
I20260709 13:10:03.056116 27174 raft_consensus.cc:3060] T 7d767accb08c4a38a74ae205bc54d739 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:03.056293 27251 raft_consensus.cc:3060] T 7d767accb08c4a38a74ae205bc54d739 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:03.060966 27251 raft_consensus.cc:2468] T 7d767accb08c4a38a74ae205bc54d739 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 1.
I20260709 13:10:03.061270 27174 raft_consensus.cc:2468] T 7d767accb08c4a38a74ae205bc54d739 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 1.
I20260709 13:10:03.061919 27289 leader_election.cc:304] T 7d767accb08c4a38a74ae205bc54d739 P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250, df70ca0962e140ba92abead00b2fc477; no voters: 
I20260709 13:10:03.062764 27401 raft_consensus.cc:2804] T 7d767accb08c4a38a74ae205bc54d739 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:03.063272 27401 raft_consensus.cc:697] T 7d767accb08c4a38a74ae205bc54d739 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 LEADER]: Becoming Leader. State: Replica: 16f9474cee3e43c89e5d9e42266cb250, State: Running, Role: LEADER
I20260709 13:10:03.064013 27401 consensus_queue.cc:237] T 7d767accb08c4a38a74ae205bc54d739 P 16f9474cee3e43c89e5d9e42266cb250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:03.071120 27070 catalog_manager.cc:5697] T 7d767accb08c4a38a74ae205bc54d739 P 16f9474cee3e43c89e5d9e42266cb250 reported cstate change: term changed from 0 to 1, leader changed from <none> to 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3). New cstate: current_term: 1 leader_uuid: "16f9474cee3e43c89e5d9e42266cb250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:03.160009 27401 raft_consensus.cc:493] T e184a0b144684ada99885c4811605036 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:03.160521 27401 raft_consensus.cc:515] T e184a0b144684ada99885c4811605036 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:03.162101 27401 leader_election.cc:290] T e184a0b144684ada99885c4811605036 P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461)
I20260709 13:10:03.162947 27174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e184a0b144684ada99885c4811605036" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394" is_pre_election: true
I20260709 13:10:03.163019 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e184a0b144684ada99885c4811605036" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477" is_pre_election: true
I20260709 13:10:03.163784 27251 raft_consensus.cc:2393] T e184a0b144684ada99885c4811605036 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16f9474cee3e43c89e5d9e42266cb250 in current term 1: Already voted for candidate 3e58e2a069aa48c59858cbadb8ab9394 in this term.
I20260709 13:10:03.164644 27289 leader_election.cc:304] T e184a0b144684ada99885c4811605036 P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250; no voters: 3e58e2a069aa48c59858cbadb8ab9394, df70ca0962e140ba92abead00b2fc477
I20260709 13:10:03.165354 27401 raft_consensus.cc:3060] T e184a0b144684ada99885c4811605036 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:03.168845 27384 raft_consensus.cc:493] T 74072bd16e8d49749ab1f70a27f71bea P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:03.169360 27384 raft_consensus.cc:515] T 74072bd16e8d49749ab1f70a27f71bea P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:03.170662 27401 raft_consensus.cc:2749] T e184a0b144684ada99885c4811605036 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:03.171180 27384 leader_election.cc:290] T 74072bd16e8d49749ab1f70a27f71bea P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461)
I20260709 13:10:03.171766 27174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74072bd16e8d49749ab1f70a27f71bea" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394" is_pre_election: true
I20260709 13:10:03.172128 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74072bd16e8d49749ab1f70a27f71bea" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477" is_pre_election: true
I20260709 13:10:03.172350 27174 raft_consensus.cc:2468] T 74072bd16e8d49749ab1f70a27f71bea P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 0.
I20260709 13:10:03.172726 27251 raft_consensus.cc:2468] T 74072bd16e8d49749ab1f70a27f71bea P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 0.
I20260709 13:10:03.173357 27290 leader_election.cc:304] T 74072bd16e8d49749ab1f70a27f71bea P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250, 3e58e2a069aa48c59858cbadb8ab9394; no voters: 
I20260709 13:10:03.174157 27384 raft_consensus.cc:2804] T 74072bd16e8d49749ab1f70a27f71bea P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:03.174530 27384 raft_consensus.cc:493] T 74072bd16e8d49749ab1f70a27f71bea P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:03.174857 27384 raft_consensus.cc:3060] T 74072bd16e8d49749ab1f70a27f71bea P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:03.176206 27374 consensus_queue.cc:1048] T f49bdf47ae7b41369b04e42ba388c1f1 P df70ca0962e140ba92abead00b2fc477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:03.180780 27384 raft_consensus.cc:515] T 74072bd16e8d49749ab1f70a27f71bea P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:03.183256 27174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74072bd16e8d49749ab1f70a27f71bea" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394"
I20260709 13:10:03.183887 27174 raft_consensus.cc:3060] T 74072bd16e8d49749ab1f70a27f71bea P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:03.183352 27384 leader_election.cc:290] T 74072bd16e8d49749ab1f70a27f71bea P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 election: Requested vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461)
I20260709 13:10:03.188102 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74072bd16e8d49749ab1f70a27f71bea" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477"
I20260709 13:10:03.188987 27251 raft_consensus.cc:3060] T 74072bd16e8d49749ab1f70a27f71bea P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:03.189152 27426 consensus_queue.cc:1048] T f49bdf47ae7b41369b04e42ba388c1f1 P df70ca0962e140ba92abead00b2fc477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:03.190928 27174 raft_consensus.cc:2468] T 74072bd16e8d49749ab1f70a27f71bea P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 1.
I20260709 13:10:03.192222 27290 leader_election.cc:304] T 74072bd16e8d49749ab1f70a27f71bea P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250, 3e58e2a069aa48c59858cbadb8ab9394; no voters: 
I20260709 13:10:03.193148 27384 raft_consensus.cc:2804] T 74072bd16e8d49749ab1f70a27f71bea P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:03.193857 27384 raft_consensus.cc:697] T 74072bd16e8d49749ab1f70a27f71bea P 16f9474cee3e43c89e5d9e42266cb250 [term 1 LEADER]: Becoming Leader. State: Replica: 16f9474cee3e43c89e5d9e42266cb250, State: Running, Role: LEADER
I20260709 13:10:03.194911 27384 consensus_queue.cc:237] T 74072bd16e8d49749ab1f70a27f71bea P 16f9474cee3e43c89e5d9e42266cb250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:03.198002 27251 raft_consensus.cc:2468] T 74072bd16e8d49749ab1f70a27f71bea P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 1.
I20260709 13:10:03.207324 27072 catalog_manager.cc:5697] T 74072bd16e8d49749ab1f70a27f71bea P 16f9474cee3e43c89e5d9e42266cb250 reported cstate change: term changed from 0 to 1, leader changed from <none> to 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3). New cstate: current_term: 1 leader_uuid: "16f9474cee3e43c89e5d9e42266cb250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:03.210125 27384 consensus_queue.cc:1048] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:03.221397 27404 consensus_queue.cc:1048] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:03.223448 27401 consensus_queue.cc:1048] T 7a551aeaa560472abebcd6cc7d456faf P 16f9474cee3e43c89e5d9e42266cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:03.229001 27400 consensus_queue.cc:1048] T acb5efb1d0d2422ba071fc5bc49e6390 P 3e58e2a069aa48c59858cbadb8ab9394 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:03.238173 27400 consensus_queue.cc:1048] T acb5efb1d0d2422ba071fc5bc49e6390 P 3e58e2a069aa48c59858cbadb8ab9394 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:03.255257 27404 consensus_queue.cc:1048] T 7a551aeaa560472abebcd6cc7d456faf P 16f9474cee3e43c89e5d9e42266cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:03.286978 27324 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "713697e046ae4150a4047a4bf2c1180f"
dest_uuid: "16f9474cee3e43c89e5d9e42266cb250"
mode: GRACEFUL
new_leader_uuid: "3e58e2a069aa48c59858cbadb8ab9394"
 from {username='slave'} at 127.0.0.1:48608
I20260709 13:10:03.287586 27324 raft_consensus.cc:606] T 713697e046ae4150a4047a4bf2c1180f P 16f9474cee3e43c89e5d9e42266cb250 [term 1 LEADER]: Received request to transfer leadership to TS 3e58e2a069aa48c59858cbadb8ab9394
I20260709 13:10:03.292649 27324 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "094982455ad24795a7cf35af53248051"
dest_uuid: "16f9474cee3e43c89e5d9e42266cb250"
mode: GRACEFUL
new_leader_uuid: "3e58e2a069aa48c59858cbadb8ab9394"
 from {username='slave'} at 127.0.0.1:48608
I20260709 13:10:03.293121 27324 raft_consensus.cc:606] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 LEADER]: Received request to transfer leadership to TS 3e58e2a069aa48c59858cbadb8ab9394
I20260709 13:10:03.305850 27250 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f49bdf47ae7b41369b04e42ba388c1f1"
dest_uuid: "df70ca0962e140ba92abead00b2fc477"
mode: GRACEFUL
new_leader_uuid: "3e58e2a069aa48c59858cbadb8ab9394"
 from {username='slave'} at 127.0.0.1:49036
I20260709 13:10:03.306378 27250 raft_consensus.cc:606] T f49bdf47ae7b41369b04e42ba388c1f1 P df70ca0962e140ba92abead00b2fc477 [term 1 LEADER]: Received request to transfer leadership to TS 3e58e2a069aa48c59858cbadb8ab9394
I20260709 13:10:03.311637 27324 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7d767accb08c4a38a74ae205bc54d739"
dest_uuid: "16f9474cee3e43c89e5d9e42266cb250"
mode: GRACEFUL
new_leader_uuid: "df70ca0962e140ba92abead00b2fc477"
 from {username='slave'} at 127.0.0.1:48608
I20260709 13:10:03.312187 27324 raft_consensus.cc:606] T 7d767accb08c4a38a74ae205bc54d739 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 LEADER]: Received request to transfer leadership to TS df70ca0962e140ba92abead00b2fc477
I20260709 13:10:03.317028 27324 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7a551aeaa560472abebcd6cc7d456faf"
dest_uuid: "16f9474cee3e43c89e5d9e42266cb250"
mode: GRACEFUL
new_leader_uuid: "df70ca0962e140ba92abead00b2fc477"
 from {username='slave'} at 127.0.0.1:48608
I20260709 13:10:03.317490 27324 raft_consensus.cc:606] T 7a551aeaa560472abebcd6cc7d456faf P 16f9474cee3e43c89e5d9e42266cb250 [term 1 LEADER]: Received request to transfer leadership to TS df70ca0962e140ba92abead00b2fc477
I20260709 13:10:03.329854 27174 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e184a0b144684ada99885c4811605036"
dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394"
mode: GRACEFUL
new_leader_uuid: "df70ca0962e140ba92abead00b2fc477"
 from {username='slave'} at 127.0.0.1:36772
I20260709 13:10:03.330375 27174 raft_consensus.cc:606] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 LEADER]: Received request to transfer leadership to TS df70ca0962e140ba92abead00b2fc477
I20260709 13:10:03.335270 27174 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a51060799a3243a881c471fc62688815"
dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394"
mode: GRACEFUL
new_leader_uuid: "df70ca0962e140ba92abead00b2fc477"
 from {username='slave'} at 127.0.0.1:36772
I20260709 13:10:03.335791 27174 raft_consensus.cc:606] T a51060799a3243a881c471fc62688815 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 LEADER]: Received request to transfer leadership to TS df70ca0962e140ba92abead00b2fc477
I20260709 13:10:03.340056 27400 consensus_queue.cc:1048] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:03.346387 27371 raft_consensus.cc:993] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394: : Instructing follower df70ca0962e140ba92abead00b2fc477 to start an election
I20260709 13:10:03.346825 27371 raft_consensus.cc:1081] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 LEADER]: Signalling peer df70ca0962e140ba92abead00b2fc477 to start an election
I20260709 13:10:03.348477 27251 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e184a0b144684ada99885c4811605036"
dest_uuid: "df70ca0962e140ba92abead00b2fc477"
 from {username='slave'} at 127.0.0.1:49028
I20260709 13:10:03.349074 27251 raft_consensus.cc:493] T e184a0b144684ada99885c4811605036 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:03.349466 27251 raft_consensus.cc:3060] T e184a0b144684ada99885c4811605036 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:03.353870 27400 consensus_queue.cc:1048] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:03.356858 27251 raft_consensus.cc:515] T e184a0b144684ada99885c4811605036 P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:03.359441 27251 leader_election.cc:290] T e184a0b144684ada99885c4811605036 P df70ca0962e140ba92abead00b2fc477 [CANDIDATE]: Term 2 election: Requested vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:03.360816 27174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e184a0b144684ada99885c4811605036" candidate_uuid: "df70ca0962e140ba92abead00b2fc477" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394"
I20260709 13:10:03.360911 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e184a0b144684ada99885c4811605036" candidate_uuid: "df70ca0962e140ba92abead00b2fc477" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "16f9474cee3e43c89e5d9e42266cb250"
I20260709 13:10:03.361351 27174 raft_consensus.cc:3055] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:03.361712 27174 raft_consensus.cc:740] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3e58e2a069aa48c59858cbadb8ab9394, State: Running, Role: LEADER
I20260709 13:10:03.362522 27174 consensus_queue.cc:260] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:03.363384 27174 raft_consensus.cc:3060] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:03.368187 27174 raft_consensus.cc:2468] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df70ca0962e140ba92abead00b2fc477 in term 2.
I20260709 13:10:03.369625 27216 leader_election.cc:304] T e184a0b144684ada99885c4811605036 P df70ca0962e140ba92abead00b2fc477 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e58e2a069aa48c59858cbadb8ab9394, df70ca0962e140ba92abead00b2fc477; no voters: 16f9474cee3e43c89e5d9e42266cb250
I20260709 13:10:03.370303 27426 raft_consensus.cc:2804] T e184a0b144684ada99885c4811605036 P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:03.370771 27426 raft_consensus.cc:697] T e184a0b144684ada99885c4811605036 P df70ca0962e140ba92abead00b2fc477 [term 2 LEADER]: Becoming Leader. State: Replica: df70ca0962e140ba92abead00b2fc477, State: Running, Role: LEADER
I20260709 13:10:03.371547 27426 consensus_queue.cc:237] T e184a0b144684ada99885c4811605036 P df70ca0962e140ba92abead00b2fc477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:03.378819 27070 catalog_manager.cc:5697] T e184a0b144684ada99885c4811605036 P df70ca0962e140ba92abead00b2fc477 reported cstate change: term changed from 1 to 2, leader changed from 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1) to df70ca0962e140ba92abead00b2fc477 (127.14.103.2). New cstate: current_term: 2 leader_uuid: "df70ca0962e140ba92abead00b2fc477" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:03.420274 27400 consensus_queue.cc:1048] T a51060799a3243a881c471fc62688815 P 3e58e2a069aa48c59858cbadb8ab9394 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:03.428006 27400 raft_consensus.cc:993] T a51060799a3243a881c471fc62688815 P 3e58e2a069aa48c59858cbadb8ab9394: : Instructing follower df70ca0962e140ba92abead00b2fc477 to start an election
I20260709 13:10:03.428470 27400 raft_consensus.cc:1081] T a51060799a3243a881c471fc62688815 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 LEADER]: Signalling peer df70ca0962e140ba92abead00b2fc477 to start an election
I20260709 13:10:03.430958 27250 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a51060799a3243a881c471fc62688815"
dest_uuid: "df70ca0962e140ba92abead00b2fc477"
 from {username='slave'} at 127.0.0.1:49028
I20260709 13:10:03.432695 27250 raft_consensus.cc:493] T a51060799a3243a881c471fc62688815 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:03.433100 27250 raft_consensus.cc:3060] T a51060799a3243a881c471fc62688815 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:03.435530 27403 consensus_queue.cc:1048] T a51060799a3243a881c471fc62688815 P 3e58e2a069aa48c59858cbadb8ab9394 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:03.440694 27250 raft_consensus.cc:515] T a51060799a3243a881c471fc62688815 P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:03.445580 27250 leader_election.cc:290] T a51060799a3243a881c471fc62688815 P df70ca0962e140ba92abead00b2fc477 [CANDIDATE]: Term 2 election: Requested vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:03.446187 27174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51060799a3243a881c471fc62688815" candidate_uuid: "df70ca0962e140ba92abead00b2fc477" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394"
I20260709 13:10:03.446802 27174 raft_consensus.cc:3055] T a51060799a3243a881c471fc62688815 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:03.446889 27324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51060799a3243a881c471fc62688815" candidate_uuid: "df70ca0962e140ba92abead00b2fc477" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "16f9474cee3e43c89e5d9e42266cb250"
I20260709 13:10:03.447168 27174 raft_consensus.cc:740] T a51060799a3243a881c471fc62688815 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3e58e2a069aa48c59858cbadb8ab9394, State: Running, Role: LEADER
I20260709 13:10:03.447484 27324 raft_consensus.cc:3060] T a51060799a3243a881c471fc62688815 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:03.447992 27174 consensus_queue.cc:260] T a51060799a3243a881c471fc62688815 P 3e58e2a069aa48c59858cbadb8ab9394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:03.449309 27174 raft_consensus.cc:3060] T a51060799a3243a881c471fc62688815 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:03.454716 27324 raft_consensus.cc:2468] T a51060799a3243a881c471fc62688815 P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df70ca0962e140ba92abead00b2fc477 in term 2.
I20260709 13:10:03.455019 27404 consensus_queue.cc:1048] T 7d767accb08c4a38a74ae205bc54d739 P 16f9474cee3e43c89e5d9e42266cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:03.455873 27217 leader_election.cc:304] T a51060799a3243a881c471fc62688815 P df70ca0962e140ba92abead00b2fc477 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250, df70ca0962e140ba92abead00b2fc477; no voters: 
I20260709 13:10:03.456714 27426 raft_consensus.cc:2804] T a51060799a3243a881c471fc62688815 P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:03.457206 27426 raft_consensus.cc:697] T a51060799a3243a881c471fc62688815 P df70ca0962e140ba92abead00b2fc477 [term 2 LEADER]: Becoming Leader. State: Replica: df70ca0962e140ba92abead00b2fc477, State: Running, Role: LEADER
I20260709 13:10:03.457985 27174 raft_consensus.cc:2468] T a51060799a3243a881c471fc62688815 P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df70ca0962e140ba92abead00b2fc477 in term 2.
I20260709 13:10:03.458020 27426 consensus_queue.cc:237] T a51060799a3243a881c471fc62688815 P df70ca0962e140ba92abead00b2fc477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:03.465245 27072 catalog_manager.cc:5697] T a51060799a3243a881c471fc62688815 P df70ca0962e140ba92abead00b2fc477 reported cstate change: term changed from 1 to 2, leader changed from 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1) to df70ca0962e140ba92abead00b2fc477 (127.14.103.2). New cstate: current_term: 2 leader_uuid: "df70ca0962e140ba92abead00b2fc477" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:03.466711 27401 raft_consensus.cc:993] T 7d767accb08c4a38a74ae205bc54d739 P 16f9474cee3e43c89e5d9e42266cb250: : Instructing follower df70ca0962e140ba92abead00b2fc477 to start an election
I20260709 13:10:03.467233 27384 raft_consensus.cc:1081] T 7d767accb08c4a38a74ae205bc54d739 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 LEADER]: Signalling peer df70ca0962e140ba92abead00b2fc477 to start an election
I20260709 13:10:03.468998 27251 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7d767accb08c4a38a74ae205bc54d739"
dest_uuid: "df70ca0962e140ba92abead00b2fc477"
 from {username='slave'} at 127.0.0.1:49012
I20260709 13:10:03.470108 27251 raft_consensus.cc:493] T 7d767accb08c4a38a74ae205bc54d739 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:03.470491 27251 raft_consensus.cc:3060] T 7d767accb08c4a38a74ae205bc54d739 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:03.475514 27384 consensus_queue.cc:1048] T 7d767accb08c4a38a74ae205bc54d739 P 16f9474cee3e43c89e5d9e42266cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:03.476470 27251 raft_consensus.cc:515] T 7d767accb08c4a38a74ae205bc54d739 P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:03.478523 27251 leader_election.cc:290] T 7d767accb08c4a38a74ae205bc54d739 P df70ca0962e140ba92abead00b2fc477 [CANDIDATE]: Term 2 election: Requested vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:03.479612 27324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d767accb08c4a38a74ae205bc54d739" candidate_uuid: "df70ca0962e140ba92abead00b2fc477" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "16f9474cee3e43c89e5d9e42266cb250"
I20260709 13:10:03.480262 27324 raft_consensus.cc:3055] T 7d767accb08c4a38a74ae205bc54d739 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:03.480556 27324 raft_consensus.cc:740] T 7d767accb08c4a38a74ae205bc54d739 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 16f9474cee3e43c89e5d9e42266cb250, State: Running, Role: LEADER
I20260709 13:10:03.481128 27176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d767accb08c4a38a74ae205bc54d739" candidate_uuid: "df70ca0962e140ba92abead00b2fc477" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394"
I20260709 13:10:03.481348 27324 consensus_queue.cc:260] T 7d767accb08c4a38a74ae205bc54d739 P 16f9474cee3e43c89e5d9e42266cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:03.482590 27324 raft_consensus.cc:3060] T 7d767accb08c4a38a74ae205bc54d739 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:03.490761 27324 raft_consensus.cc:2468] T 7d767accb08c4a38a74ae205bc54d739 P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df70ca0962e140ba92abead00b2fc477 in term 2.
I20260709 13:10:03.491658 27217 leader_election.cc:304] T 7d767accb08c4a38a74ae205bc54d739 P df70ca0962e140ba92abead00b2fc477 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250, df70ca0962e140ba92abead00b2fc477; no voters: 3e58e2a069aa48c59858cbadb8ab9394
I20260709 13:10:03.492527 27426 raft_consensus.cc:2804] T 7d767accb08c4a38a74ae205bc54d739 P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:03.493059 27426 raft_consensus.cc:697] T 7d767accb08c4a38a74ae205bc54d739 P df70ca0962e140ba92abead00b2fc477 [term 2 LEADER]: Becoming Leader. State: Replica: df70ca0962e140ba92abead00b2fc477, State: Running, Role: LEADER
I20260709 13:10:03.493757 27426 consensus_queue.cc:237] T 7d767accb08c4a38a74ae205bc54d739 P df70ca0962e140ba92abead00b2fc477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:03.494850 27370 raft_consensus.cc:993] T 713697e046ae4150a4047a4bf2c1180f P 16f9474cee3e43c89e5d9e42266cb250: : Instructing follower 3e58e2a069aa48c59858cbadb8ab9394 to start an election
I20260709 13:10:03.495371 27384 raft_consensus.cc:1081] T 713697e046ae4150a4047a4bf2c1180f P 16f9474cee3e43c89e5d9e42266cb250 [term 1 LEADER]: Signalling peer 3e58e2a069aa48c59858cbadb8ab9394 to start an election
I20260709 13:10:03.496919 27174 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "713697e046ae4150a4047a4bf2c1180f"
dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394"
 from {username='slave'} at 127.0.0.1:36746
I20260709 13:10:03.497349 27174 raft_consensus.cc:493] T 713697e046ae4150a4047a4bf2c1180f P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:03.497622 27174 raft_consensus.cc:3060] T 713697e046ae4150a4047a4bf2c1180f P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:03.499975 27072 catalog_manager.cc:5697] T 7d767accb08c4a38a74ae205bc54d739 P df70ca0962e140ba92abead00b2fc477 reported cstate change: term changed from 1 to 2, leader changed from 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3) to df70ca0962e140ba92abead00b2fc477 (127.14.103.2). New cstate: current_term: 2 leader_uuid: "df70ca0962e140ba92abead00b2fc477" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:03.502674 27174 raft_consensus.cc:515] T 713697e046ae4150a4047a4bf2c1180f P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:03.504777 27174 leader_election.cc:290] T 713697e046ae4150a4047a4bf2c1180f P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 2 election: Requested vote from peers df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:03.505831 27324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "713697e046ae4150a4047a4bf2c1180f" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "16f9474cee3e43c89e5d9e42266cb250"
I20260709 13:10:03.505998 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "713697e046ae4150a4047a4bf2c1180f" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "df70ca0962e140ba92abead00b2fc477"
I20260709 13:10:03.506469 27324 raft_consensus.cc:3055] T 713697e046ae4150a4047a4bf2c1180f P 16f9474cee3e43c89e5d9e42266cb250 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:03.506543 27251 raft_consensus.cc:3060] T 713697e046ae4150a4047a4bf2c1180f P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:03.506948 27324 raft_consensus.cc:740] T 713697e046ae4150a4047a4bf2c1180f P 16f9474cee3e43c89e5d9e42266cb250 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 16f9474cee3e43c89e5d9e42266cb250, State: Running, Role: LEADER
I20260709 13:10:03.507829 27324 consensus_queue.cc:260] T 713697e046ae4150a4047a4bf2c1180f P 16f9474cee3e43c89e5d9e42266cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:03.509114 27324 raft_consensus.cc:3060] T 713697e046ae4150a4047a4bf2c1180f P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:03.513522 27251 raft_consensus.cc:2468] T 713697e046ae4150a4047a4bf2c1180f P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e58e2a069aa48c59858cbadb8ab9394 in term 2.
I20260709 13:10:03.514716 27140 leader_election.cc:304] T 713697e046ae4150a4047a4bf2c1180f P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e58e2a069aa48c59858cbadb8ab9394, df70ca0962e140ba92abead00b2fc477; no voters: 
I20260709 13:10:03.515372 27403 raft_consensus.cc:2804] T 713697e046ae4150a4047a4bf2c1180f P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:03.515635 27324 raft_consensus.cc:2468] T 713697e046ae4150a4047a4bf2c1180f P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e58e2a069aa48c59858cbadb8ab9394 in term 2.
I20260709 13:10:03.515827 27403 raft_consensus.cc:697] T 713697e046ae4150a4047a4bf2c1180f P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 LEADER]: Becoming Leader. State: Replica: 3e58e2a069aa48c59858cbadb8ab9394, State: Running, Role: LEADER
I20260709 13:10:03.516798 27403 consensus_queue.cc:237] T 713697e046ae4150a4047a4bf2c1180f P 3e58e2a069aa48c59858cbadb8ab9394 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:03.522828 27072 catalog_manager.cc:5697] T 713697e046ae4150a4047a4bf2c1180f P 3e58e2a069aa48c59858cbadb8ab9394 reported cstate change: term changed from 1 to 2, leader changed from 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3) to 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "3e58e2a069aa48c59858cbadb8ab9394" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:03.638288 27384 consensus_queue.cc:1048] T 74072bd16e8d49749ab1f70a27f71bea P 16f9474cee3e43c89e5d9e42266cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:03.650166 27401 consensus_queue.cc:1048] T 74072bd16e8d49749ab1f70a27f71bea P 16f9474cee3e43c89e5d9e42266cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:03.675379 27374 raft_consensus.cc:993] T f49bdf47ae7b41369b04e42ba388c1f1 P df70ca0962e140ba92abead00b2fc477: : Instructing follower 3e58e2a069aa48c59858cbadb8ab9394 to start an election
I20260709 13:10:03.675843 27426 raft_consensus.cc:1081] T f49bdf47ae7b41369b04e42ba388c1f1 P df70ca0962e140ba92abead00b2fc477 [term 1 LEADER]: Signalling peer 3e58e2a069aa48c59858cbadb8ab9394 to start an election
I20260709 13:10:03.677098 27174 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f49bdf47ae7b41369b04e42ba388c1f1"
dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394"
 from {username='slave'} at 127.0.0.1:36762
I20260709 13:10:03.677502 27174 raft_consensus.cc:493] T f49bdf47ae7b41369b04e42ba388c1f1 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:03.677736 27174 raft_consensus.cc:3060] T f49bdf47ae7b41369b04e42ba388c1f1 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:03.683648 27174 raft_consensus.cc:515] T f49bdf47ae7b41369b04e42ba388c1f1 P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:03.685659 27174 leader_election.cc:290] T f49bdf47ae7b41369b04e42ba388c1f1 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 2 election: Requested vote from peers df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:03.686167 27324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f49bdf47ae7b41369b04e42ba388c1f1" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "16f9474cee3e43c89e5d9e42266cb250"
I20260709 13:10:03.686841 27324 raft_consensus.cc:3060] T f49bdf47ae7b41369b04e42ba388c1f1 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:03.687587 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f49bdf47ae7b41369b04e42ba388c1f1" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "df70ca0962e140ba92abead00b2fc477"
I20260709 13:10:03.688568 27251 raft_consensus.cc:3055] T f49bdf47ae7b41369b04e42ba388c1f1 P df70ca0962e140ba92abead00b2fc477 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:03.688923 27251 raft_consensus.cc:740] T f49bdf47ae7b41369b04e42ba388c1f1 P df70ca0962e140ba92abead00b2fc477 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: df70ca0962e140ba92abead00b2fc477, State: Running, Role: LEADER
I20260709 13:10:03.689704 27251 consensus_queue.cc:260] T f49bdf47ae7b41369b04e42ba388c1f1 P df70ca0962e140ba92abead00b2fc477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:03.690627 27251 raft_consensus.cc:3060] T f49bdf47ae7b41369b04e42ba388c1f1 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:03.693094 27324 raft_consensus.cc:2468] T f49bdf47ae7b41369b04e42ba388c1f1 P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e58e2a069aa48c59858cbadb8ab9394 in term 2.
I20260709 13:10:03.694142 27142 leader_election.cc:304] T f49bdf47ae7b41369b04e42ba388c1f1 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250, 3e58e2a069aa48c59858cbadb8ab9394; no voters: 
I20260709 13:10:03.694784 27403 raft_consensus.cc:2804] T f49bdf47ae7b41369b04e42ba388c1f1 P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:03.695218 27403 raft_consensus.cc:697] T f49bdf47ae7b41369b04e42ba388c1f1 P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 LEADER]: Becoming Leader. State: Replica: 3e58e2a069aa48c59858cbadb8ab9394, State: Running, Role: LEADER
I20260709 13:10:03.695230 27251 raft_consensus.cc:2468] T f49bdf47ae7b41369b04e42ba388c1f1 P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e58e2a069aa48c59858cbadb8ab9394 in term 2.
I20260709 13:10:03.696039 27403 consensus_queue.cc:237] T f49bdf47ae7b41369b04e42ba388c1f1 P 3e58e2a069aa48c59858cbadb8ab9394 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:03.702064 27074 catalog_manager.cc:5697] T f49bdf47ae7b41369b04e42ba388c1f1 P 3e58e2a069aa48c59858cbadb8ab9394 reported cstate change: term changed from 1 to 2, leader changed from df70ca0962e140ba92abead00b2fc477 (127.14.103.2) to 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "3e58e2a069aa48c59858cbadb8ab9394" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:03.780107 27384 raft_consensus.cc:993] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250: : Instructing follower 3e58e2a069aa48c59858cbadb8ab9394 to start an election
I20260709 13:10:03.780529 27401 raft_consensus.cc:1081] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 LEADER]: Signalling peer 3e58e2a069aa48c59858cbadb8ab9394 to start an election
I20260709 13:10:03.781976 27174 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "094982455ad24795a7cf35af53248051"
dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394"
 from {username='slave'} at 127.0.0.1:36746
I20260709 13:10:03.782480 27174 raft_consensus.cc:493] T 094982455ad24795a7cf35af53248051 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:03.782804 27174 raft_consensus.cc:3060] T 094982455ad24795a7cf35af53248051 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:03.787998 27174 raft_consensus.cc:515] T 094982455ad24795a7cf35af53248051 P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:03.789595 27174 leader_election.cc:290] T 094982455ad24795a7cf35af53248051 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 2 election: Requested vote from peers df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:03.790350 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "094982455ad24795a7cf35af53248051" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "df70ca0962e140ba92abead00b2fc477"
I20260709 13:10:03.790583 27324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "094982455ad24795a7cf35af53248051" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "16f9474cee3e43c89e5d9e42266cb250"
I20260709 13:10:03.791004 27251 raft_consensus.cc:3060] T 094982455ad24795a7cf35af53248051 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:03.791118 27324 raft_consensus.cc:3055] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:03.791514 27324 raft_consensus.cc:740] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 16f9474cee3e43c89e5d9e42266cb250, State: Running, Role: LEADER
I20260709 13:10:03.792342 27324 consensus_queue.cc:260] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:03.793584 27324 raft_consensus.cc:3060] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:03.798103 27251 raft_consensus.cc:2468] T 094982455ad24795a7cf35af53248051 P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e58e2a069aa48c59858cbadb8ab9394 in term 2.
I20260709 13:10:03.799237 27140 leader_election.cc:304] T 094982455ad24795a7cf35af53248051 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e58e2a069aa48c59858cbadb8ab9394, df70ca0962e140ba92abead00b2fc477; no voters: 
I20260709 13:10:03.799957 27403 raft_consensus.cc:2804] T 094982455ad24795a7cf35af53248051 P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:03.800282 27324 raft_consensus.cc:2468] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e58e2a069aa48c59858cbadb8ab9394 in term 2.
I20260709 13:10:03.800350 27403 raft_consensus.cc:697] T 094982455ad24795a7cf35af53248051 P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 LEADER]: Becoming Leader. State: Replica: 3e58e2a069aa48c59858cbadb8ab9394, State: Running, Role: LEADER
I20260709 13:10:03.801539 27403 consensus_queue.cc:237] T 094982455ad24795a7cf35af53248051 P 3e58e2a069aa48c59858cbadb8ab9394 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:03.807471 27074 catalog_manager.cc:5697] T 094982455ad24795a7cf35af53248051 P 3e58e2a069aa48c59858cbadb8ab9394 reported cstate change: term changed from 1 to 2, leader changed from 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3) to 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "3e58e2a069aa48c59858cbadb8ab9394" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:03.811748 27174 raft_consensus.cc:1275] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Refusing update from remote peer df70ca0962e140ba92abead00b2fc477: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:03.813202 27426 consensus_queue.cc:1048] T e184a0b144684ada99885c4811605036 P df70ca0962e140ba92abead00b2fc477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:03.821017 27324 raft_consensus.cc:3060] T e184a0b144684ada99885c4811605036 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:03.827610 27324 raft_consensus.cc:1275] T e184a0b144684ada99885c4811605036 P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Refusing update from remote peer df70ca0962e140ba92abead00b2fc477: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:03.828881 27402 consensus_queue.cc:1048] T e184a0b144684ada99885c4811605036 P df70ca0962e140ba92abead00b2fc477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:03.843230 27370 raft_consensus.cc:993] T 7a551aeaa560472abebcd6cc7d456faf P 16f9474cee3e43c89e5d9e42266cb250: : Instructing follower df70ca0962e140ba92abead00b2fc477 to start an election
I20260709 13:10:03.843641 27401 raft_consensus.cc:1081] T 7a551aeaa560472abebcd6cc7d456faf P 16f9474cee3e43c89e5d9e42266cb250 [term 1 LEADER]: Signalling peer df70ca0962e140ba92abead00b2fc477 to start an election
I20260709 13:10:03.845057 27251 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7a551aeaa560472abebcd6cc7d456faf"
dest_uuid: "df70ca0962e140ba92abead00b2fc477"
 from {username='slave'} at 127.0.0.1:49012
I20260709 13:10:03.845577 27251 raft_consensus.cc:493] T 7a551aeaa560472abebcd6cc7d456faf P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:03.845890 27251 raft_consensus.cc:3060] T 7a551aeaa560472abebcd6cc7d456faf P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:03.850589 27251 raft_consensus.cc:515] T 7a551aeaa560472abebcd6cc7d456faf P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:03.852160 27251 leader_election.cc:290] T 7a551aeaa560472abebcd6cc7d456faf P df70ca0962e140ba92abead00b2fc477 [CANDIDATE]: Term 2 election: Requested vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:03.853134 27324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a551aeaa560472abebcd6cc7d456faf" candidate_uuid: "df70ca0962e140ba92abead00b2fc477" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "16f9474cee3e43c89e5d9e42266cb250"
I20260709 13:10:03.853127 27174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a551aeaa560472abebcd6cc7d456faf" candidate_uuid: "df70ca0962e140ba92abead00b2fc477" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394"
I20260709 13:10:03.853754 27324 raft_consensus.cc:3055] T 7a551aeaa560472abebcd6cc7d456faf P 16f9474cee3e43c89e5d9e42266cb250 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:03.853881 27174 raft_consensus.cc:3060] T 7a551aeaa560472abebcd6cc7d456faf P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:03.854076 27324 raft_consensus.cc:740] T 7a551aeaa560472abebcd6cc7d456faf P 16f9474cee3e43c89e5d9e42266cb250 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 16f9474cee3e43c89e5d9e42266cb250, State: Running, Role: LEADER
I20260709 13:10:03.854825 27324 consensus_queue.cc:260] T 7a551aeaa560472abebcd6cc7d456faf P 16f9474cee3e43c89e5d9e42266cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:03.855800 27324 raft_consensus.cc:3060] T 7a551aeaa560472abebcd6cc7d456faf P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:03.858974 27174 raft_consensus.cc:2468] T 7a551aeaa560472abebcd6cc7d456faf P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df70ca0962e140ba92abead00b2fc477 in term 2.
I20260709 13:10:03.860112 27216 leader_election.cc:304] T 7a551aeaa560472abebcd6cc7d456faf P df70ca0962e140ba92abead00b2fc477 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e58e2a069aa48c59858cbadb8ab9394, df70ca0962e140ba92abead00b2fc477; no voters: 
I20260709 13:10:03.860929 27402 raft_consensus.cc:2804] T 7a551aeaa560472abebcd6cc7d456faf P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:03.861392 27402 raft_consensus.cc:697] T 7a551aeaa560472abebcd6cc7d456faf P df70ca0962e140ba92abead00b2fc477 [term 2 LEADER]: Becoming Leader. State: Replica: df70ca0962e140ba92abead00b2fc477, State: Running, Role: LEADER
I20260709 13:10:03.861675 27324 raft_consensus.cc:2468] T 7a551aeaa560472abebcd6cc7d456faf P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df70ca0962e140ba92abead00b2fc477 in term 2.
I20260709 13:10:03.862190 27402 consensus_queue.cc:237] T 7a551aeaa560472abebcd6cc7d456faf P df70ca0962e140ba92abead00b2fc477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:03.868552 27072 catalog_manager.cc:5697] T 7a551aeaa560472abebcd6cc7d456faf P df70ca0962e140ba92abead00b2fc477 reported cstate change: term changed from 1 to 2, leader changed from 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3) to df70ca0962e140ba92abead00b2fc477 (127.14.103.2). New cstate: current_term: 2 leader_uuid: "df70ca0962e140ba92abead00b2fc477" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:03.948580 27251 raft_consensus.cc:1275] T 713697e046ae4150a4047a4bf2c1180f P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Refusing update from remote peer 3e58e2a069aa48c59858cbadb8ab9394: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:03.950096 27403 consensus_queue.cc:1048] T 713697e046ae4150a4047a4bf2c1180f P 3e58e2a069aa48c59858cbadb8ab9394 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:03.957561 27324 raft_consensus.cc:1275] T 713697e046ae4150a4047a4bf2c1180f P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Refusing update from remote peer 3e58e2a069aa48c59858cbadb8ab9394: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:03.959131 27403 consensus_queue.cc:1048] T 713697e046ae4150a4047a4bf2c1180f P 3e58e2a069aa48c59858cbadb8ab9394 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:03.980717 27174 raft_consensus.cc:1275] T a51060799a3243a881c471fc62688815 P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Refusing update from remote peer df70ca0962e140ba92abead00b2fc477: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:03.982060 27402 consensus_queue.cc:1048] T a51060799a3243a881c471fc62688815 P df70ca0962e140ba92abead00b2fc477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:03.989068 27325 raft_consensus.cc:1275] T a51060799a3243a881c471fc62688815 P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Refusing update from remote peer df70ca0962e140ba92abead00b2fc477: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:03.990761 27374 consensus_queue.cc:1048] T a51060799a3243a881c471fc62688815 P df70ca0962e140ba92abead00b2fc477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:04.028821 27174 raft_consensus.cc:3060] T 7d767accb08c4a38a74ae205bc54d739 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:04.033785 27174 raft_consensus.cc:1275] T 7d767accb08c4a38a74ae205bc54d739 P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Refusing update from remote peer df70ca0962e140ba92abead00b2fc477: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:04.035080 27374 consensus_queue.cc:1048] T 7d767accb08c4a38a74ae205bc54d739 P df70ca0962e140ba92abead00b2fc477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:04.042964 27325 raft_consensus.cc:1275] T 7d767accb08c4a38a74ae205bc54d739 P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Refusing update from remote peer df70ca0962e140ba92abead00b2fc477: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:04.044298 27402 consensus_queue.cc:1048] T 7d767accb08c4a38a74ae205bc54d739 P df70ca0962e140ba92abead00b2fc477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:04.112069 27251 raft_consensus.cc:1275] T f49bdf47ae7b41369b04e42ba388c1f1 P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Refusing update from remote peer 3e58e2a069aa48c59858cbadb8ab9394: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:04.113273 27403 consensus_queue.cc:1048] T f49bdf47ae7b41369b04e42ba388c1f1 P 3e58e2a069aa48c59858cbadb8ab9394 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:04.120328 27325 raft_consensus.cc:1275] T f49bdf47ae7b41369b04e42ba388c1f1 P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Refusing update from remote peer 3e58e2a069aa48c59858cbadb8ab9394: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:04.121826 27400 consensus_queue.cc:1048] T f49bdf47ae7b41369b04e42ba388c1f1 P 3e58e2a069aa48c59858cbadb8ab9394 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:04.182096 27325 raft_consensus.cc:1275] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Refusing update from remote peer 3e58e2a069aa48c59858cbadb8ab9394: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:04.183202 27400 consensus_queue.cc:1048] T 094982455ad24795a7cf35af53248051 P 3e58e2a069aa48c59858cbadb8ab9394 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:04.190928 27251 raft_consensus.cc:1275] T 094982455ad24795a7cf35af53248051 P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Refusing update from remote peer 3e58e2a069aa48c59858cbadb8ab9394: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:04.193197 27400 consensus_queue.cc:1048] T 094982455ad24795a7cf35af53248051 P 3e58e2a069aa48c59858cbadb8ab9394 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:04.265558 27174 raft_consensus.cc:1275] T 7a551aeaa560472abebcd6cc7d456faf P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Refusing update from remote peer df70ca0962e140ba92abead00b2fc477: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:04.266754 27374 consensus_queue.cc:1048] T 7a551aeaa560472abebcd6cc7d456faf P df70ca0962e140ba92abead00b2fc477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:04.268126 27325 raft_consensus.cc:1275] T 7a551aeaa560472abebcd6cc7d456faf P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Refusing update from remote peer df70ca0962e140ba92abead00b2fc477: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:04.269459 27374 consensus_queue.cc:1048] T 7a551aeaa560472abebcd6cc7d456faf P df70ca0962e140ba92abead00b2fc477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:06.364261 27074 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55752:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:10:06.366263 27074 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:10:06.411855 27165 tablet_service.cc:1511] Processing CreateTablet for tablet 22270c8eaaa648f9b5cbe3125faa8c25 (DEFAULT_TABLE table=soft_deleted_table [id=330588081cce4a9586227b833c776c10]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:10:06.412926 27239 tablet_service.cc:1511] Processing CreateTablet for tablet 22270c8eaaa648f9b5cbe3125faa8c25 (DEFAULT_TABLE table=soft_deleted_table [id=330588081cce4a9586227b833c776c10]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:10:06.413350 27165 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22270c8eaaa648f9b5cbe3125faa8c25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.414215 27239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22270c8eaaa648f9b5cbe3125faa8c25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.420742 27313 tablet_service.cc:1511] Processing CreateTablet for tablet 22270c8eaaa648f9b5cbe3125faa8c25 (DEFAULT_TABLE table=soft_deleted_table [id=330588081cce4a9586227b833c776c10]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:10:06.421120 27164 tablet_service.cc:1511] Processing CreateTablet for tablet f0808b4c74f04ce28b19a78bf4c504a8 (DEFAULT_TABLE table=soft_deleted_table [id=330588081cce4a9586227b833c776c10]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:10:06.421981 27313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22270c8eaaa648f9b5cbe3125faa8c25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.422308 27164 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0808b4c74f04ce28b19a78bf4c504a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.425462 27241 tablet_service.cc:1511] Processing CreateTablet for tablet f0808b4c74f04ce28b19a78bf4c504a8 (DEFAULT_TABLE table=soft_deleted_table [id=330588081cce4a9586227b833c776c10]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:10:06.426704 27241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0808b4c74f04ce28b19a78bf4c504a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.430544 27166 tablet_service.cc:1511] Processing CreateTablet for tablet ac6f6dd4c5f7460bb60fc085a13a1a20 (DEFAULT_TABLE table=soft_deleted_table [id=330588081cce4a9586227b833c776c10]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:10:06.431789 27166 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac6f6dd4c5f7460bb60fc085a13a1a20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.435966 27237 tablet_service.cc:1511] Processing CreateTablet for tablet ac6f6dd4c5f7460bb60fc085a13a1a20 (DEFAULT_TABLE table=soft_deleted_table [id=330588081cce4a9586227b833c776c10]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:10:06.437418 27237 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac6f6dd4c5f7460bb60fc085a13a1a20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.435966 27236 tablet_service.cc:1511] Processing CreateTablet for tablet 9ebd3c36d25e44e293a5d318fa74b4a2 (DEFAULT_TABLE table=soft_deleted_table [id=330588081cce4a9586227b833c776c10]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:10:06.439035 27236 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ebd3c36d25e44e293a5d318fa74b4a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.443775 27163 tablet_service.cc:1511] Processing CreateTablet for tablet 9ebd3c36d25e44e293a5d318fa74b4a2 (DEFAULT_TABLE table=soft_deleted_table [id=330588081cce4a9586227b833c776c10]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:10:06.445019 27163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ebd3c36d25e44e293a5d318fa74b4a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.446417 27235 tablet_service.cc:1511] Processing CreateTablet for tablet 819307e0735043d5b032193af4478edc (DEFAULT_TABLE table=soft_deleted_table [id=330588081cce4a9586227b833c776c10]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:10:06.447616 27314 tablet_service.cc:1511] Processing CreateTablet for tablet f0808b4c74f04ce28b19a78bf4c504a8 (DEFAULT_TABLE table=soft_deleted_table [id=330588081cce4a9586227b833c776c10]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:10:06.448307 27312 tablet_service.cc:1511] Processing CreateTablet for tablet ac6f6dd4c5f7460bb60fc085a13a1a20 (DEFAULT_TABLE table=soft_deleted_table [id=330588081cce4a9586227b833c776c10]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:10:06.448863 27314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0808b4c74f04ce28b19a78bf4c504a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.449473 27312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac6f6dd4c5f7460bb60fc085a13a1a20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.447628 27235 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 819307e0735043d5b032193af4478edc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.452119 27162 tablet_service.cc:1511] Processing CreateTablet for tablet 819307e0735043d5b032193af4478edc (DEFAULT_TABLE table=soft_deleted_table [id=330588081cce4a9586227b833c776c10]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:10:06.452348 27311 tablet_service.cc:1511] Processing CreateTablet for tablet 819307e0735043d5b032193af4478edc (DEFAULT_TABLE table=soft_deleted_table [id=330588081cce4a9586227b833c776c10]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:10:06.453389 27162 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 819307e0735043d5b032193af4478edc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.453649 27311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 819307e0735043d5b032193af4478edc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.452176 27315 tablet_service.cc:1511] Processing CreateTablet for tablet 9ebd3c36d25e44e293a5d318fa74b4a2 (DEFAULT_TABLE table=soft_deleted_table [id=330588081cce4a9586227b833c776c10]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:10:06.456794 27315 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ebd3c36d25e44e293a5d318fa74b4a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.492235 27461 tablet_bootstrap.cc:492] T 22270c8eaaa648f9b5cbe3125faa8c25 P 3e58e2a069aa48c59858cbadb8ab9394: Bootstrap starting.
I20260709 13:10:06.499181 27461 tablet_bootstrap.cc:654] T 22270c8eaaa648f9b5cbe3125faa8c25 P 3e58e2a069aa48c59858cbadb8ab9394: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:06.516227 27165 tablet_service.cc:1511] Processing CreateTablet for tablet 50d0e7c2eeb740c69d8235a9baee5564 (DEFAULT_TABLE table=soft_deleted_table [id=330588081cce4a9586227b833c776c10]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:10:06.519454 27165 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50d0e7c2eeb740c69d8235a9baee5564. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.519743 27237 tablet_service.cc:1511] Processing CreateTablet for tablet 50d0e7c2eeb740c69d8235a9baee5564 (DEFAULT_TABLE table=soft_deleted_table [id=330588081cce4a9586227b833c776c10]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:10:06.521030 27237 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50d0e7c2eeb740c69d8235a9baee5564. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.530370 27241 tablet_service.cc:1511] Processing CreateTablet for tablet 43b0b2e84a7f408fb83a416f9d669635 (DEFAULT_TABLE table=soft_deleted_table [id=330588081cce4a9586227b833c776c10]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:10:06.531665 27241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43b0b2e84a7f408fb83a416f9d669635. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.538448 27311 tablet_service.cc:1511] Processing CreateTablet for tablet 43b0b2e84a7f408fb83a416f9d669635 (DEFAULT_TABLE table=soft_deleted_table [id=330588081cce4a9586227b833c776c10]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:10:06.539834 27311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43b0b2e84a7f408fb83a416f9d669635. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.540421 27462 tablet_bootstrap.cc:492] T 22270c8eaaa648f9b5cbe3125faa8c25 P df70ca0962e140ba92abead00b2fc477: Bootstrap starting.
I20260709 13:10:06.541347 27164 tablet_service.cc:1511] Processing CreateTablet for tablet f53040cb502f490189c5be0990856a58 (DEFAULT_TABLE table=soft_deleted_table [id=330588081cce4a9586227b833c776c10]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:10:06.542627 27164 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f53040cb502f490189c5be0990856a58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.543416 27235 tablet_service.cc:1511] Processing CreateTablet for tablet f53040cb502f490189c5be0990856a58 (DEFAULT_TABLE table=soft_deleted_table [id=330588081cce4a9586227b833c776c10]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:10:06.544730 27235 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f53040cb502f490189c5be0990856a58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.554428 27166 tablet_service.cc:1511] Processing CreateTablet for tablet 4c5d9e4d3ae74d17b4027d97e8f5b01b (DEFAULT_TABLE table=soft_deleted_table [id=330588081cce4a9586227b833c776c10]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:10:06.560848 27166 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c5d9e4d3ae74d17b4027d97e8f5b01b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.556511 27239 tablet_service.cc:1511] Processing CreateTablet for tablet 4c5d9e4d3ae74d17b4027d97e8f5b01b (DEFAULT_TABLE table=soft_deleted_table [id=330588081cce4a9586227b833c776c10]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:10:06.565096 27239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c5d9e4d3ae74d17b4027d97e8f5b01b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.556294 27163 tablet_service.cc:1511] Processing CreateTablet for tablet 43b0b2e84a7f408fb83a416f9d669635 (DEFAULT_TABLE table=soft_deleted_table [id=330588081cce4a9586227b833c776c10]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:10:06.566315 27163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43b0b2e84a7f408fb83a416f9d669635. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.557896 27312 tablet_service.cc:1511] Processing CreateTablet for tablet 50d0e7c2eeb740c69d8235a9baee5564 (DEFAULT_TABLE table=soft_deleted_table [id=330588081cce4a9586227b833c776c10]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:10:06.586745 27312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50d0e7c2eeb740c69d8235a9baee5564. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.580277 27462 tablet_bootstrap.cc:654] T 22270c8eaaa648f9b5cbe3125faa8c25 P df70ca0962e140ba92abead00b2fc477: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:06.559273 27315 tablet_service.cc:1511] Processing CreateTablet for tablet 4c5d9e4d3ae74d17b4027d97e8f5b01b (DEFAULT_TABLE table=soft_deleted_table [id=330588081cce4a9586227b833c776c10]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:10:06.591435 27315 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c5d9e4d3ae74d17b4027d97e8f5b01b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.558761 27461 tablet_bootstrap.cc:492] T 22270c8eaaa648f9b5cbe3125faa8c25 P 3e58e2a069aa48c59858cbadb8ab9394: No bootstrap required, opened a new log
I20260709 13:10:06.558795 27464 tablet_bootstrap.cc:492] T 22270c8eaaa648f9b5cbe3125faa8c25 P 16f9474cee3e43c89e5d9e42266cb250: Bootstrap starting.
I20260709 13:10:06.592546 27461 ts_tablet_manager.cc:1403] T 22270c8eaaa648f9b5cbe3125faa8c25 P 3e58e2a069aa48c59858cbadb8ab9394: Time spent bootstrapping tablet: real 0.101s	user 0.016s	sys 0.002s
I20260709 13:10:06.547806 27314 tablet_service.cc:1511] Processing CreateTablet for tablet f53040cb502f490189c5be0990856a58 (DEFAULT_TABLE table=soft_deleted_table [id=330588081cce4a9586227b833c776c10]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:10:06.593559 27314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f53040cb502f490189c5be0990856a58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:06.596346 27461 raft_consensus.cc:359] T 22270c8eaaa648f9b5cbe3125faa8c25 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.599997 27461 raft_consensus.cc:385] T 22270c8eaaa648f9b5cbe3125faa8c25 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:06.608470 27461 raft_consensus.cc:740] T 22270c8eaaa648f9b5cbe3125faa8c25 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e58e2a069aa48c59858cbadb8ab9394, State: Initialized, Role: FOLLOWER
I20260709 13:10:06.609615 27461 consensus_queue.cc:260] T 22270c8eaaa648f9b5cbe3125faa8c25 P 3e58e2a069aa48c59858cbadb8ab9394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.599488 27464 tablet_bootstrap.cc:654] T 22270c8eaaa648f9b5cbe3125faa8c25 P 16f9474cee3e43c89e5d9e42266cb250: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:06.626935 27461 ts_tablet_manager.cc:1434] T 22270c8eaaa648f9b5cbe3125faa8c25 P 3e58e2a069aa48c59858cbadb8ab9394: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.002s
I20260709 13:10:06.628304 27461 tablet_bootstrap.cc:492] T 819307e0735043d5b032193af4478edc P 3e58e2a069aa48c59858cbadb8ab9394: Bootstrap starting.
I20260709 13:10:06.638263 27462 tablet_bootstrap.cc:492] T 22270c8eaaa648f9b5cbe3125faa8c25 P df70ca0962e140ba92abead00b2fc477: No bootstrap required, opened a new log
I20260709 13:10:06.638906 27462 ts_tablet_manager.cc:1403] T 22270c8eaaa648f9b5cbe3125faa8c25 P df70ca0962e140ba92abead00b2fc477: Time spent bootstrapping tablet: real 0.099s	user 0.009s	sys 0.044s
I20260709 13:10:06.640079 27461 tablet_bootstrap.cc:654] T 819307e0735043d5b032193af4478edc P 3e58e2a069aa48c59858cbadb8ab9394: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:06.640334 27464 tablet_bootstrap.cc:492] T 22270c8eaaa648f9b5cbe3125faa8c25 P 16f9474cee3e43c89e5d9e42266cb250: No bootstrap required, opened a new log
I20260709 13:10:06.640841 27464 ts_tablet_manager.cc:1403] T 22270c8eaaa648f9b5cbe3125faa8c25 P 16f9474cee3e43c89e5d9e42266cb250: Time spent bootstrapping tablet: real 0.082s	user 0.010s	sys 0.005s
I20260709 13:10:06.641965 27462 raft_consensus.cc:359] T 22270c8eaaa648f9b5cbe3125faa8c25 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.643023 27462 raft_consensus.cc:385] T 22270c8eaaa648f9b5cbe3125faa8c25 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:06.643355 27462 raft_consensus.cc:740] T 22270c8eaaa648f9b5cbe3125faa8c25 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df70ca0962e140ba92abead00b2fc477, State: Initialized, Role: FOLLOWER
I20260709 13:10:06.644276 27464 raft_consensus.cc:359] T 22270c8eaaa648f9b5cbe3125faa8c25 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.644265 27462 consensus_queue.cc:260] T 22270c8eaaa648f9b5cbe3125faa8c25 P df70ca0962e140ba92abead00b2fc477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.645107 27464 raft_consensus.cc:385] T 22270c8eaaa648f9b5cbe3125faa8c25 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:06.645826 27464 raft_consensus.cc:740] T 22270c8eaaa648f9b5cbe3125faa8c25 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16f9474cee3e43c89e5d9e42266cb250, State: Initialized, Role: FOLLOWER
I20260709 13:10:06.646461 27462 ts_tablet_manager.cc:1434] T 22270c8eaaa648f9b5cbe3125faa8c25 P df70ca0962e140ba92abead00b2fc477: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:10:06.646694 27464 consensus_queue.cc:260] T 22270c8eaaa648f9b5cbe3125faa8c25 P 16f9474cee3e43c89e5d9e42266cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.648001 27461 tablet_bootstrap.cc:492] T 819307e0735043d5b032193af4478edc P 3e58e2a069aa48c59858cbadb8ab9394: No bootstrap required, opened a new log
I20260709 13:10:06.648093 27462 tablet_bootstrap.cc:492] T ac6f6dd4c5f7460bb60fc085a13a1a20 P df70ca0962e140ba92abead00b2fc477: Bootstrap starting.
I20260709 13:10:06.648695 27461 ts_tablet_manager.cc:1403] T 819307e0735043d5b032193af4478edc P 3e58e2a069aa48c59858cbadb8ab9394: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.003s
I20260709 13:10:06.651834 27461 raft_consensus.cc:359] T 819307e0735043d5b032193af4478edc P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.652616 27461 raft_consensus.cc:385] T 819307e0735043d5b032193af4478edc P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:06.652943 27461 raft_consensus.cc:740] T 819307e0735043d5b032193af4478edc P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e58e2a069aa48c59858cbadb8ab9394, State: Initialized, Role: FOLLOWER
I20260709 13:10:06.653679 27461 consensus_queue.cc:260] T 819307e0735043d5b032193af4478edc P 3e58e2a069aa48c59858cbadb8ab9394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.655434 27462 tablet_bootstrap.cc:654] T ac6f6dd4c5f7460bb60fc085a13a1a20 P df70ca0962e140ba92abead00b2fc477: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:06.656448 27461 ts_tablet_manager.cc:1434] T 819307e0735043d5b032193af4478edc P 3e58e2a069aa48c59858cbadb8ab9394: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:10:06.657547 27461 tablet_bootstrap.cc:492] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 3e58e2a069aa48c59858cbadb8ab9394: Bootstrap starting.
I20260709 13:10:06.660751 27464 ts_tablet_manager.cc:1434] T 22270c8eaaa648f9b5cbe3125faa8c25 P 16f9474cee3e43c89e5d9e42266cb250: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 13:10:06.664407 27461 tablet_bootstrap.cc:654] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 3e58e2a069aa48c59858cbadb8ab9394: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:06.665177 27462 tablet_bootstrap.cc:492] T ac6f6dd4c5f7460bb60fc085a13a1a20 P df70ca0962e140ba92abead00b2fc477: No bootstrap required, opened a new log
I20260709 13:10:06.665778 27462 ts_tablet_manager.cc:1403] T ac6f6dd4c5f7460bb60fc085a13a1a20 P df70ca0962e140ba92abead00b2fc477: Time spent bootstrapping tablet: real 0.018s	user 0.005s	sys 0.011s
I20260709 13:10:06.665943 27464 tablet_bootstrap.cc:492] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250: Bootstrap starting.
I20260709 13:10:06.669565 27462 raft_consensus.cc:359] T ac6f6dd4c5f7460bb60fc085a13a1a20 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.670612 27462 raft_consensus.cc:385] T ac6f6dd4c5f7460bb60fc085a13a1a20 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:06.671047 27462 raft_consensus.cc:740] T ac6f6dd4c5f7460bb60fc085a13a1a20 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df70ca0962e140ba92abead00b2fc477, State: Initialized, Role: FOLLOWER
I20260709 13:10:06.672043 27462 consensus_queue.cc:260] T ac6f6dd4c5f7460bb60fc085a13a1a20 P df70ca0962e140ba92abead00b2fc477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.673435 27464 tablet_bootstrap.cc:654] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:06.675222 27462 ts_tablet_manager.cc:1434] T ac6f6dd4c5f7460bb60fc085a13a1a20 P df70ca0962e140ba92abead00b2fc477: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:10:06.676651 27462 tablet_bootstrap.cc:492] T f0808b4c74f04ce28b19a78bf4c504a8 P df70ca0962e140ba92abead00b2fc477: Bootstrap starting.
I20260709 13:10:06.679124 27461 tablet_bootstrap.cc:492] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 3e58e2a069aa48c59858cbadb8ab9394: No bootstrap required, opened a new log
I20260709 13:10:06.679659 27461 ts_tablet_manager.cc:1403] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 3e58e2a069aa48c59858cbadb8ab9394: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.008s
I20260709 13:10:06.682072 27464 tablet_bootstrap.cc:492] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250: No bootstrap required, opened a new log
W20260709 13:10:06.682346 27122 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:06.682633 27464 ts_tablet_manager.cc:1403] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250: Time spent bootstrapping tablet: real 0.017s	user 0.008s	sys 0.003s
I20260709 13:10:06.682962 27462 tablet_bootstrap.cc:654] T f0808b4c74f04ce28b19a78bf4c504a8 P df70ca0962e140ba92abead00b2fc477: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:06.684629 27461 raft_consensus.cc:359] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.685431 27461 raft_consensus.cc:385] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:06.685798 27461 raft_consensus.cc:740] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e58e2a069aa48c59858cbadb8ab9394, State: Initialized, Role: FOLLOWER
I20260709 13:10:06.685659 27464 raft_consensus.cc:359] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.686543 27464 raft_consensus.cc:385] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:06.686883 27464 raft_consensus.cc:740] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16f9474cee3e43c89e5d9e42266cb250, State: Initialized, Role: FOLLOWER
I20260709 13:10:06.686930 27461 consensus_queue.cc:260] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 3e58e2a069aa48c59858cbadb8ab9394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.687731 27464 consensus_queue.cc:260] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.690207 27461 ts_tablet_manager.cc:1434] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 3e58e2a069aa48c59858cbadb8ab9394: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.003s
I20260709 13:10:06.691233 27461 tablet_bootstrap.cc:492] T f0808b4c74f04ce28b19a78bf4c504a8 P 3e58e2a069aa48c59858cbadb8ab9394: Bootstrap starting.
I20260709 13:10:06.695392 27464 ts_tablet_manager.cc:1434] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.004s
I20260709 13:10:06.696719 27464 tablet_bootstrap.cc:492] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 16f9474cee3e43c89e5d9e42266cb250: Bootstrap starting.
I20260709 13:10:06.699591 27461 tablet_bootstrap.cc:654] T f0808b4c74f04ce28b19a78bf4c504a8 P 3e58e2a069aa48c59858cbadb8ab9394: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:06.706328 27462 tablet_bootstrap.cc:492] T f0808b4c74f04ce28b19a78bf4c504a8 P df70ca0962e140ba92abead00b2fc477: No bootstrap required, opened a new log
I20260709 13:10:06.706964 27462 ts_tablet_manager.cc:1403] T f0808b4c74f04ce28b19a78bf4c504a8 P df70ca0962e140ba92abead00b2fc477: Time spent bootstrapping tablet: real 0.031s	user 0.011s	sys 0.015s
I20260709 13:10:06.709125 27464 tablet_bootstrap.cc:654] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 16f9474cee3e43c89e5d9e42266cb250: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:06.709932 27462 raft_consensus.cc:359] T f0808b4c74f04ce28b19a78bf4c504a8 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.710726 27462 raft_consensus.cc:385] T f0808b4c74f04ce28b19a78bf4c504a8 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:06.711058 27462 raft_consensus.cc:740] T f0808b4c74f04ce28b19a78bf4c504a8 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df70ca0962e140ba92abead00b2fc477, State: Initialized, Role: FOLLOWER
I20260709 13:10:06.712078 27462 consensus_queue.cc:260] T f0808b4c74f04ce28b19a78bf4c504a8 P df70ca0962e140ba92abead00b2fc477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.713827 27462 ts_tablet_manager.cc:1434] T f0808b4c74f04ce28b19a78bf4c504a8 P df70ca0962e140ba92abead00b2fc477: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.002s
I20260709 13:10:06.713981 27464 tablet_bootstrap.cc:492] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 16f9474cee3e43c89e5d9e42266cb250: No bootstrap required, opened a new log
I20260709 13:10:06.714505 27464 ts_tablet_manager.cc:1403] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 16f9474cee3e43c89e5d9e42266cb250: Time spent bootstrapping tablet: real 0.018s	user 0.004s	sys 0.004s
I20260709 13:10:06.715533 27462 tablet_bootstrap.cc:492] T 9ebd3c36d25e44e293a5d318fa74b4a2 P df70ca0962e140ba92abead00b2fc477: Bootstrap starting.
I20260709 13:10:06.716032 27461 tablet_bootstrap.cc:492] T f0808b4c74f04ce28b19a78bf4c504a8 P 3e58e2a069aa48c59858cbadb8ab9394: No bootstrap required, opened a new log
I20260709 13:10:06.716604 27461 ts_tablet_manager.cc:1403] T f0808b4c74f04ce28b19a78bf4c504a8 P 3e58e2a069aa48c59858cbadb8ab9394: Time spent bootstrapping tablet: real 0.026s	user 0.013s	sys 0.004s
I20260709 13:10:06.717355 27464 raft_consensus.cc:359] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.718122 27464 raft_consensus.cc:385] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:06.718464 27464 raft_consensus.cc:740] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16f9474cee3e43c89e5d9e42266cb250, State: Initialized, Role: FOLLOWER
I20260709 13:10:06.719354 27464 consensus_queue.cc:260] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 16f9474cee3e43c89e5d9e42266cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.719559 27461 raft_consensus.cc:359] T f0808b4c74f04ce28b19a78bf4c504a8 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.720430 27461 raft_consensus.cc:385] T f0808b4c74f04ce28b19a78bf4c504a8 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:06.720795 27461 raft_consensus.cc:740] T f0808b4c74f04ce28b19a78bf4c504a8 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e58e2a069aa48c59858cbadb8ab9394, State: Initialized, Role: FOLLOWER
I20260709 13:10:06.721849 27464 ts_tablet_manager.cc:1434] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 16f9474cee3e43c89e5d9e42266cb250: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.004s
I20260709 13:10:06.721545 27461 consensus_queue.cc:260] T f0808b4c74f04ce28b19a78bf4c504a8 P 3e58e2a069aa48c59858cbadb8ab9394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.722892 27464 tablet_bootstrap.cc:492] T f0808b4c74f04ce28b19a78bf4c504a8 P 16f9474cee3e43c89e5d9e42266cb250: Bootstrap starting.
I20260709 13:10:06.724251 27461 ts_tablet_manager.cc:1434] T f0808b4c74f04ce28b19a78bf4c504a8 P 3e58e2a069aa48c59858cbadb8ab9394: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:10:06.725143 27462 tablet_bootstrap.cc:654] T 9ebd3c36d25e44e293a5d318fa74b4a2 P df70ca0962e140ba92abead00b2fc477: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:06.725477 27461 tablet_bootstrap.cc:492] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 3e58e2a069aa48c59858cbadb8ab9394: Bootstrap starting.
I20260709 13:10:06.730129 27464 tablet_bootstrap.cc:654] T f0808b4c74f04ce28b19a78bf4c504a8 P 16f9474cee3e43c89e5d9e42266cb250: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:06.733255 27461 tablet_bootstrap.cc:654] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 3e58e2a069aa48c59858cbadb8ab9394: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:06.737290 27462 tablet_bootstrap.cc:492] T 9ebd3c36d25e44e293a5d318fa74b4a2 P df70ca0962e140ba92abead00b2fc477: No bootstrap required, opened a new log
I20260709 13:10:06.738067 27462 ts_tablet_manager.cc:1403] T 9ebd3c36d25e44e293a5d318fa74b4a2 P df70ca0962e140ba92abead00b2fc477: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.006s
I20260709 13:10:06.741667 27464 tablet_bootstrap.cc:492] T f0808b4c74f04ce28b19a78bf4c504a8 P 16f9474cee3e43c89e5d9e42266cb250: No bootstrap required, opened a new log
I20260709 13:10:06.742301 27464 ts_tablet_manager.cc:1403] T f0808b4c74f04ce28b19a78bf4c504a8 P 16f9474cee3e43c89e5d9e42266cb250: Time spent bootstrapping tablet: real 0.020s	user 0.017s	sys 0.001s
I20260709 13:10:06.741812 27462 raft_consensus.cc:359] T 9ebd3c36d25e44e293a5d318fa74b4a2 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.743206 27462 raft_consensus.cc:385] T 9ebd3c36d25e44e293a5d318fa74b4a2 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:06.743592 27462 raft_consensus.cc:740] T 9ebd3c36d25e44e293a5d318fa74b4a2 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df70ca0962e140ba92abead00b2fc477, State: Initialized, Role: FOLLOWER
I20260709 13:10:06.744428 27462 consensus_queue.cc:260] T 9ebd3c36d25e44e293a5d318fa74b4a2 P df70ca0962e140ba92abead00b2fc477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.745561 27461 tablet_bootstrap.cc:492] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 3e58e2a069aa48c59858cbadb8ab9394: No bootstrap required, opened a new log
I20260709 13:10:06.745229 27464 raft_consensus.cc:359] T f0808b4c74f04ce28b19a78bf4c504a8 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.746109 27464 raft_consensus.cc:385] T f0808b4c74f04ce28b19a78bf4c504a8 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:06.746145 27461 ts_tablet_manager.cc:1403] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 3e58e2a069aa48c59858cbadb8ab9394: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.008s
I20260709 13:10:06.746546 27464 raft_consensus.cc:740] T f0808b4c74f04ce28b19a78bf4c504a8 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16f9474cee3e43c89e5d9e42266cb250, State: Initialized, Role: FOLLOWER
I20260709 13:10:06.746814 27462 ts_tablet_manager.cc:1434] T 9ebd3c36d25e44e293a5d318fa74b4a2 P df70ca0962e140ba92abead00b2fc477: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:10:06.747987 27462 tablet_bootstrap.cc:492] T 819307e0735043d5b032193af4478edc P df70ca0962e140ba92abead00b2fc477: Bootstrap starting.
I20260709 13:10:06.747539 27464 consensus_queue.cc:260] T f0808b4c74f04ce28b19a78bf4c504a8 P 16f9474cee3e43c89e5d9e42266cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.749409 27461 raft_consensus.cc:359] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.750027 27464 ts_tablet_manager.cc:1434] T f0808b4c74f04ce28b19a78bf4c504a8 P 16f9474cee3e43c89e5d9e42266cb250: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 13:10:06.750211 27461 raft_consensus.cc:385] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:06.750629 27461 raft_consensus.cc:740] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e58e2a069aa48c59858cbadb8ab9394, State: Initialized, Role: FOLLOWER
I20260709 13:10:06.751134 27464 tablet_bootstrap.cc:492] T 819307e0735043d5b032193af4478edc P 16f9474cee3e43c89e5d9e42266cb250: Bootstrap starting.
I20260709 13:10:06.751573 27461 consensus_queue.cc:260] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 3e58e2a069aa48c59858cbadb8ab9394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.754314 27461 ts_tablet_manager.cc:1434] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 3e58e2a069aa48c59858cbadb8ab9394: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.003s
I20260709 13:10:06.756381 27462 tablet_bootstrap.cc:654] T 819307e0735043d5b032193af4478edc P df70ca0962e140ba92abead00b2fc477: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:06.757385 27461 tablet_bootstrap.cc:492] T f53040cb502f490189c5be0990856a58 P 3e58e2a069aa48c59858cbadb8ab9394: Bootstrap starting.
I20260709 13:10:06.758213 27464 tablet_bootstrap.cc:654] T 819307e0735043d5b032193af4478edc P 16f9474cee3e43c89e5d9e42266cb250: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:06.767091 27460 raft_consensus.cc:493] T 819307e0735043d5b032193af4478edc P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:06.767385 27461 tablet_bootstrap.cc:654] T f53040cb502f490189c5be0990856a58 P 3e58e2a069aa48c59858cbadb8ab9394: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:06.767879 27460 raft_consensus.cc:515] T 819307e0735043d5b032193af4478edc P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.771843 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819307e0735043d5b032193af4478edc" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477" is_pre_election: true
I20260709 13:10:06.772706 27403 raft_consensus.cc:493] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:06.773425 27403 raft_consensus.cc:515] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.775295 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819307e0735043d5b032193af4478edc" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16f9474cee3e43c89e5d9e42266cb250" is_pre_election: true
I20260709 13:10:06.776875 27403 leader_election.cc:290] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:06.779575 27460 leader_election.cc:290] T 819307e0735043d5b032193af4478edc P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:06.782235 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac6f6dd4c5f7460bb60fc085a13a1a20" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477" is_pre_election: true
I20260709 13:10:06.783095 27251 raft_consensus.cc:2468] T ac6f6dd4c5f7460bb60fc085a13a1a20 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e58e2a069aa48c59858cbadb8ab9394 in term 0.
W20260709 13:10:06.780454 27142 leader_election.cc:343] T 819307e0735043d5b032193af4478edc P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:06.785216 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac6f6dd4c5f7460bb60fc085a13a1a20" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16f9474cee3e43c89e5d9e42266cb250" is_pre_election: true
I20260709 13:10:06.786093 27325 raft_consensus.cc:2468] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e58e2a069aa48c59858cbadb8ab9394 in term 0.
I20260709 13:10:06.787865 27142 leader_election.cc:304] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250, 3e58e2a069aa48c59858cbadb8ab9394; no voters: 
I20260709 13:10:06.788930 27403 raft_consensus.cc:2804] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:06.789469 27403 raft_consensus.cc:493] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:06.789875 27403 raft_consensus.cc:3060] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:06.800027 27401 raft_consensus.cc:493] T 22270c8eaaa648f9b5cbe3125faa8c25 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:06.800529 27403 raft_consensus.cc:515] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.800807 27401 raft_consensus.cc:515] T 22270c8eaaa648f9b5cbe3125faa8c25 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.807004 27174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22270c8eaaa648f9b5cbe3125faa8c25" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394" is_pre_election: true
I20260709 13:10:06.807976 27174 raft_consensus.cc:2468] T 22270c8eaaa648f9b5cbe3125faa8c25 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 0.
I20260709 13:10:06.809803 27290 leader_election.cc:304] T 22270c8eaaa648f9b5cbe3125faa8c25 P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250, 3e58e2a069aa48c59858cbadb8ab9394; no voters: 
I20260709 13:10:06.810947 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22270c8eaaa648f9b5cbe3125faa8c25" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477" is_pre_election: true
I20260709 13:10:06.812863 27251 raft_consensus.cc:2468] T 22270c8eaaa648f9b5cbe3125faa8c25 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 0.
I20260709 13:10:06.815212 27482 raft_consensus.cc:2804] T 22270c8eaaa648f9b5cbe3125faa8c25 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:06.816181 27482 raft_consensus.cc:493] T 22270c8eaaa648f9b5cbe3125faa8c25 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:06.816937 27482 raft_consensus.cc:3060] T 22270c8eaaa648f9b5cbe3125faa8c25 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:06.819021 27140 leader_election.cc:343] T 819307e0735043d5b032193af4478edc P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:06.819569 27140 leader_election.cc:304] T 819307e0735043d5b032193af4478edc P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e58e2a069aa48c59858cbadb8ab9394; no voters: 16f9474cee3e43c89e5d9e42266cb250, df70ca0962e140ba92abead00b2fc477
I20260709 13:10:06.820585 27460 raft_consensus.cc:2749] T 819307e0735043d5b032193af4478edc P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:06.822393 27401 leader_election.cc:290] T 22270c8eaaa648f9b5cbe3125faa8c25 P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461)
I20260709 13:10:06.825724 27403 leader_election.cc:290] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 election: Requested vote from peers df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:06.827057 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac6f6dd4c5f7460bb60fc085a13a1a20" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16f9474cee3e43c89e5d9e42266cb250"
I20260709 13:10:06.828742 27325 raft_consensus.cc:3060] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:06.828958 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac6f6dd4c5f7460bb60fc085a13a1a20" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477"
I20260709 13:10:06.829619 27251 raft_consensus.cc:3060] T ac6f6dd4c5f7460bb60fc085a13a1a20 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:06.832458 27482 raft_consensus.cc:515] T 22270c8eaaa648f9b5cbe3125faa8c25 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.834983 27174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22270c8eaaa648f9b5cbe3125faa8c25" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394"
I20260709 13:10:06.835490 27174 raft_consensus.cc:3060] T 22270c8eaaa648f9b5cbe3125faa8c25 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:06.836786 27251 raft_consensus.cc:2468] T ac6f6dd4c5f7460bb60fc085a13a1a20 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e58e2a069aa48c59858cbadb8ab9394 in term 1.
I20260709 13:10:06.837960 27140 leader_election.cc:304] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e58e2a069aa48c59858cbadb8ab9394, df70ca0962e140ba92abead00b2fc477; no voters: 
I20260709 13:10:06.838982 27460 raft_consensus.cc:2804] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:06.839422 27460 raft_consensus.cc:697] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 LEADER]: Becoming Leader. State: Replica: 3e58e2a069aa48c59858cbadb8ab9394, State: Running, Role: LEADER
I20260709 13:10:06.840595 27460 consensus_queue.cc:237] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 3e58e2a069aa48c59858cbadb8ab9394 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.842778 27174 raft_consensus.cc:2468] T 22270c8eaaa648f9b5cbe3125faa8c25 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 1.
I20260709 13:10:06.844055 27290 leader_election.cc:304] T 22270c8eaaa648f9b5cbe3125faa8c25 P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250, 3e58e2a069aa48c59858cbadb8ab9394; no voters: 
I20260709 13:10:06.839350 27325 raft_consensus.cc:2468] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e58e2a069aa48c59858cbadb8ab9394 in term 1.
I20260709 13:10:06.845266 27401 raft_consensus.cc:2804] T 22270c8eaaa648f9b5cbe3125faa8c25 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:06.847342 27462 tablet_bootstrap.cc:492] T 819307e0735043d5b032193af4478edc P df70ca0962e140ba92abead00b2fc477: No bootstrap required, opened a new log
I20260709 13:10:06.847450 27401 raft_consensus.cc:697] T 22270c8eaaa648f9b5cbe3125faa8c25 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 LEADER]: Becoming Leader. State: Replica: 16f9474cee3e43c89e5d9e42266cb250, State: Running, Role: LEADER
I20260709 13:10:06.848028 27462 ts_tablet_manager.cc:1403] T 819307e0735043d5b032193af4478edc P df70ca0962e140ba92abead00b2fc477: Time spent bootstrapping tablet: real 0.100s	user 0.043s	sys 0.052s
I20260709 13:10:06.848810 27464 tablet_bootstrap.cc:492] T 819307e0735043d5b032193af4478edc P 16f9474cee3e43c89e5d9e42266cb250: No bootstrap required, opened a new log
I20260709 13:10:06.848619 27401 consensus_queue.cc:237] T 22270c8eaaa648f9b5cbe3125faa8c25 P 16f9474cee3e43c89e5d9e42266cb250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.849293 27464 ts_tablet_manager.cc:1403] T 819307e0735043d5b032193af4478edc P 16f9474cee3e43c89e5d9e42266cb250: Time spent bootstrapping tablet: real 0.098s	user 0.025s	sys 0.041s
I20260709 13:10:06.850569 27462 raft_consensus.cc:359] T 819307e0735043d5b032193af4478edc P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.851209 27462 raft_consensus.cc:385] T 819307e0735043d5b032193af4478edc P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:06.851461 27462 raft_consensus.cc:740] T 819307e0735043d5b032193af4478edc P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df70ca0962e140ba92abead00b2fc477, State: Initialized, Role: FOLLOWER
I20260709 13:10:06.847574 27461 tablet_bootstrap.cc:492] T f53040cb502f490189c5be0990856a58 P 3e58e2a069aa48c59858cbadb8ab9394: No bootstrap required, opened a new log
I20260709 13:10:06.852142 27461 ts_tablet_manager.cc:1403] T f53040cb502f490189c5be0990856a58 P 3e58e2a069aa48c59858cbadb8ab9394: Time spent bootstrapping tablet: real 0.095s	user 0.005s	sys 0.023s
I20260709 13:10:06.851999 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22270c8eaaa648f9b5cbe3125faa8c25" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477"
I20260709 13:10:06.852315 27462 consensus_queue.cc:260] T 819307e0735043d5b032193af4478edc P df70ca0962e140ba92abead00b2fc477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.852669 27251 raft_consensus.cc:3060] T 22270c8eaaa648f9b5cbe3125faa8c25 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:06.854584 27462 ts_tablet_manager.cc:1434] T 819307e0735043d5b032193af4478edc P df70ca0962e140ba92abead00b2fc477: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:10:06.855079 27461 raft_consensus.cc:359] T f53040cb502f490189c5be0990856a58 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.856050 27462 tablet_bootstrap.cc:492] T f53040cb502f490189c5be0990856a58 P df70ca0962e140ba92abead00b2fc477: Bootstrap starting.
I20260709 13:10:06.856233 27461 raft_consensus.cc:385] T f53040cb502f490189c5be0990856a58 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:06.856665 27461 raft_consensus.cc:740] T f53040cb502f490189c5be0990856a58 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e58e2a069aa48c59858cbadb8ab9394, State: Initialized, Role: FOLLOWER
I20260709 13:10:06.863305 27461 consensus_queue.cc:260] T f53040cb502f490189c5be0990856a58 P 3e58e2a069aa48c59858cbadb8ab9394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.866346 27461 ts_tablet_manager.cc:1434] T f53040cb502f490189c5be0990856a58 P 3e58e2a069aa48c59858cbadb8ab9394: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.001s
I20260709 13:10:06.867411 27461 tablet_bootstrap.cc:492] T 50d0e7c2eeb740c69d8235a9baee5564 P 3e58e2a069aa48c59858cbadb8ab9394: Bootstrap starting.
I20260709 13:10:06.867048 27464 raft_consensus.cc:359] T 819307e0735043d5b032193af4478edc P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.868113 27464 raft_consensus.cc:385] T 819307e0735043d5b032193af4478edc P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:06.868430 27464 raft_consensus.cc:740] T 819307e0735043d5b032193af4478edc P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16f9474cee3e43c89e5d9e42266cb250, State: Initialized, Role: FOLLOWER
I20260709 13:10:06.869150 27462 tablet_bootstrap.cc:654] T f53040cb502f490189c5be0990856a58 P df70ca0962e140ba92abead00b2fc477: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:06.845438 27482 leader_election.cc:290] T 22270c8eaaa648f9b5cbe3125faa8c25 P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 election: Requested vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461)
I20260709 13:10:06.877640 27251 raft_consensus.cc:2468] T 22270c8eaaa648f9b5cbe3125faa8c25 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 1.
I20260709 13:10:06.872922 27464 consensus_queue.cc:260] T 819307e0735043d5b032193af4478edc P 16f9474cee3e43c89e5d9e42266cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.882758 27464 ts_tablet_manager.cc:1434] T 819307e0735043d5b032193af4478edc P 16f9474cee3e43c89e5d9e42266cb250: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.003s
I20260709 13:10:06.884788 27464 tablet_bootstrap.cc:492] T 43b0b2e84a7f408fb83a416f9d669635 P 16f9474cee3e43c89e5d9e42266cb250: Bootstrap starting.
I20260709 13:10:06.897886 27461 tablet_bootstrap.cc:654] T 50d0e7c2eeb740c69d8235a9baee5564 P 3e58e2a069aa48c59858cbadb8ab9394: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:06.897655 27070 catalog_manager.cc:5697] T 22270c8eaaa648f9b5cbe3125faa8c25 P 16f9474cee3e43c89e5d9e42266cb250 reported cstate change: term changed from 0 to 1, leader changed from <none> to 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3). New cstate: current_term: 1 leader_uuid: "16f9474cee3e43c89e5d9e42266cb250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:06.900323 27464 tablet_bootstrap.cc:654] T 43b0b2e84a7f408fb83a416f9d669635 P 16f9474cee3e43c89e5d9e42266cb250: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:06.905164 27460 raft_consensus.cc:493] T f53040cb502f490189c5be0990856a58 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:06.905700 27460 raft_consensus.cc:515] T f53040cb502f490189c5be0990856a58 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.908061 27460 leader_election.cc:290] T f53040cb502f490189c5be0990856a58 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:06.909587 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53040cb502f490189c5be0990856a58" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477" is_pre_election: true
I20260709 13:10:06.909658 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53040cb502f490189c5be0990856a58" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16f9474cee3e43c89e5d9e42266cb250" is_pre_election: true
W20260709 13:10:06.911190 27140 leader_election.cc:343] T f53040cb502f490189c5be0990856a58 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:06.912071 27142 leader_election.cc:343] T f53040cb502f490189c5be0990856a58 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:06.912521 27142 leader_election.cc:304] T f53040cb502f490189c5be0990856a58 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e58e2a069aa48c59858cbadb8ab9394; no voters: 16f9474cee3e43c89e5d9e42266cb250, df70ca0962e140ba92abead00b2fc477
I20260709 13:10:06.912824 27462 tablet_bootstrap.cc:492] T f53040cb502f490189c5be0990856a58 P df70ca0962e140ba92abead00b2fc477: No bootstrap required, opened a new log
I20260709 13:10:06.913383 27462 ts_tablet_manager.cc:1403] T f53040cb502f490189c5be0990856a58 P df70ca0962e140ba92abead00b2fc477: Time spent bootstrapping tablet: real 0.058s	user 0.008s	sys 0.011s
I20260709 13:10:06.913645 27460 raft_consensus.cc:2749] T f53040cb502f490189c5be0990856a58 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:06.917500 27461 tablet_bootstrap.cc:492] T 50d0e7c2eeb740c69d8235a9baee5564 P 3e58e2a069aa48c59858cbadb8ab9394: No bootstrap required, opened a new log
I20260709 13:10:06.918856 27461 ts_tablet_manager.cc:1403] T 50d0e7c2eeb740c69d8235a9baee5564 P 3e58e2a069aa48c59858cbadb8ab9394: Time spent bootstrapping tablet: real 0.052s	user 0.006s	sys 0.012s
I20260709 13:10:06.917869 27073 catalog_manager.cc:5697] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 3e58e2a069aa48c59858cbadb8ab9394 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "3e58e2a069aa48c59858cbadb8ab9394" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:06.920097 27464 tablet_bootstrap.cc:492] T 43b0b2e84a7f408fb83a416f9d669635 P 16f9474cee3e43c89e5d9e42266cb250: No bootstrap required, opened a new log
I20260709 13:10:06.920591 27464 ts_tablet_manager.cc:1403] T 43b0b2e84a7f408fb83a416f9d669635 P 16f9474cee3e43c89e5d9e42266cb250: Time spent bootstrapping tablet: real 0.036s	user 0.017s	sys 0.000s
I20260709 13:10:06.921958 27461 raft_consensus.cc:359] T 50d0e7c2eeb740c69d8235a9baee5564 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.922704 27461 raft_consensus.cc:385] T 50d0e7c2eeb740c69d8235a9baee5564 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:06.923020 27461 raft_consensus.cc:740] T 50d0e7c2eeb740c69d8235a9baee5564 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e58e2a069aa48c59858cbadb8ab9394, State: Initialized, Role: FOLLOWER
I20260709 13:10:06.923472 27464 raft_consensus.cc:359] T 43b0b2e84a7f408fb83a416f9d669635 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.924237 27464 raft_consensus.cc:385] T 43b0b2e84a7f408fb83a416f9d669635 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:06.924558 27464 raft_consensus.cc:740] T 43b0b2e84a7f408fb83a416f9d669635 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16f9474cee3e43c89e5d9e42266cb250, State: Initialized, Role: FOLLOWER
I20260709 13:10:06.923784 27461 consensus_queue.cc:260] T 50d0e7c2eeb740c69d8235a9baee5564 P 3e58e2a069aa48c59858cbadb8ab9394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.925348 27464 consensus_queue.cc:260] T 43b0b2e84a7f408fb83a416f9d669635 P 16f9474cee3e43c89e5d9e42266cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.927767 27464 ts_tablet_manager.cc:1434] T 43b0b2e84a7f408fb83a416f9d669635 P 16f9474cee3e43c89e5d9e42266cb250: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 13:10:06.928769 27464 tablet_bootstrap.cc:492] T f53040cb502f490189c5be0990856a58 P 16f9474cee3e43c89e5d9e42266cb250: Bootstrap starting.
I20260709 13:10:06.929303 27461 ts_tablet_manager.cc:1434] T 50d0e7c2eeb740c69d8235a9baee5564 P 3e58e2a069aa48c59858cbadb8ab9394: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:10:06.930366 27461 tablet_bootstrap.cc:492] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 3e58e2a069aa48c59858cbadb8ab9394: Bootstrap starting.
I20260709 13:10:06.935737 27464 tablet_bootstrap.cc:654] T f53040cb502f490189c5be0990856a58 P 16f9474cee3e43c89e5d9e42266cb250: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:06.935868 27462 raft_consensus.cc:359] T f53040cb502f490189c5be0990856a58 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.936823 27462 raft_consensus.cc:385] T f53040cb502f490189c5be0990856a58 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:06.937156 27462 raft_consensus.cc:740] T f53040cb502f490189c5be0990856a58 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df70ca0962e140ba92abead00b2fc477, State: Initialized, Role: FOLLOWER
I20260709 13:10:06.938077 27462 consensus_queue.cc:260] T f53040cb502f490189c5be0990856a58 P df70ca0962e140ba92abead00b2fc477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.940831 27462 ts_tablet_manager.cc:1434] T f53040cb502f490189c5be0990856a58 P df70ca0962e140ba92abead00b2fc477: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:10:06.944242 27461 tablet_bootstrap.cc:654] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 3e58e2a069aa48c59858cbadb8ab9394: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:06.945521 27462 tablet_bootstrap.cc:492] T 43b0b2e84a7f408fb83a416f9d669635 P df70ca0962e140ba92abead00b2fc477: Bootstrap starting.
I20260709 13:10:06.947602 27464 tablet_bootstrap.cc:492] T f53040cb502f490189c5be0990856a58 P 16f9474cee3e43c89e5d9e42266cb250: No bootstrap required, opened a new log
I20260709 13:10:06.948197 27464 ts_tablet_manager.cc:1403] T f53040cb502f490189c5be0990856a58 P 16f9474cee3e43c89e5d9e42266cb250: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.004s
I20260709 13:10:06.951931 27462 tablet_bootstrap.cc:654] T 43b0b2e84a7f408fb83a416f9d669635 P df70ca0962e140ba92abead00b2fc477: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:06.952416 27464 raft_consensus.cc:359] T f53040cb502f490189c5be0990856a58 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.953450 27464 raft_consensus.cc:385] T f53040cb502f490189c5be0990856a58 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:06.953827 27464 raft_consensus.cc:740] T f53040cb502f490189c5be0990856a58 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16f9474cee3e43c89e5d9e42266cb250, State: Initialized, Role: FOLLOWER
I20260709 13:10:06.954912 27464 consensus_queue.cc:260] T f53040cb502f490189c5be0990856a58 P 16f9474cee3e43c89e5d9e42266cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.957520 27464 ts_tablet_manager.cc:1434] T f53040cb502f490189c5be0990856a58 P 16f9474cee3e43c89e5d9e42266cb250: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:10:06.958465 27464 tablet_bootstrap.cc:492] T 50d0e7c2eeb740c69d8235a9baee5564 P 16f9474cee3e43c89e5d9e42266cb250: Bootstrap starting.
I20260709 13:10:06.964576 27464 tablet_bootstrap.cc:654] T 50d0e7c2eeb740c69d8235a9baee5564 P 16f9474cee3e43c89e5d9e42266cb250: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:06.972690 27403 raft_consensus.cc:493] T f0808b4c74f04ce28b19a78bf4c504a8 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:06.973330 27403 raft_consensus.cc:515] T f0808b4c74f04ce28b19a78bf4c504a8 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.977078 27426 raft_consensus.cc:493] T 819307e0735043d5b032193af4478edc P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:06.977813 27426 raft_consensus.cc:515] T 819307e0735043d5b032193af4478edc P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:06.981195 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0808b4c74f04ce28b19a78bf4c504a8" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477" is_pre_election: true
I20260709 13:10:06.982036 27251 raft_consensus.cc:2468] T f0808b4c74f04ce28b19a78bf4c504a8 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e58e2a069aa48c59858cbadb8ab9394 in term 0.
I20260709 13:10:06.983569 27140 leader_election.cc:304] T f0808b4c74f04ce28b19a78bf4c504a8 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e58e2a069aa48c59858cbadb8ab9394, df70ca0962e140ba92abead00b2fc477; no voters: 
I20260709 13:10:06.984769 27460 raft_consensus.cc:2804] T f0808b4c74f04ce28b19a78bf4c504a8 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:06.985239 27460 raft_consensus.cc:493] T f0808b4c74f04ce28b19a78bf4c504a8 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:06.985658 27460 raft_consensus.cc:3060] T f0808b4c74f04ce28b19a78bf4c504a8 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:06.989254 27426 leader_election.cc:290] T 819307e0735043d5b032193af4478edc P df70ca0962e140ba92abead00b2fc477 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:06.993672 27174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819307e0735043d5b032193af4478edc" candidate_uuid: "df70ca0962e140ba92abead00b2fc477" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394" is_pre_election: true
I20260709 13:10:06.993937 27324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0808b4c74f04ce28b19a78bf4c504a8" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16f9474cee3e43c89e5d9e42266cb250" is_pre_election: true
I20260709 13:10:06.994556 27174 raft_consensus.cc:2468] T 819307e0735043d5b032193af4478edc P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df70ca0962e140ba92abead00b2fc477 in term 0.
I20260709 13:10:06.994829 27324 raft_consensus.cc:2468] T f0808b4c74f04ce28b19a78bf4c504a8 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e58e2a069aa48c59858cbadb8ab9394 in term 0.
I20260709 13:10:06.996277 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819307e0735043d5b032193af4478edc" candidate_uuid: "df70ca0962e140ba92abead00b2fc477" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16f9474cee3e43c89e5d9e42266cb250" is_pre_election: true
I20260709 13:10:06.997159 27325 raft_consensus.cc:2468] T 819307e0735043d5b032193af4478edc P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df70ca0962e140ba92abead00b2fc477 in term 0.
I20260709 13:10:06.998786 27217 leader_election.cc:304] T 819307e0735043d5b032193af4478edc P df70ca0962e140ba92abead00b2fc477 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250, df70ca0962e140ba92abead00b2fc477; no voters: 
I20260709 13:10:06.991312 27403 leader_election.cc:290] T f0808b4c74f04ce28b19a78bf4c504a8 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:06.999935 27466 raft_consensus.cc:493] T f53040cb502f490189c5be0990856a58 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:07.000573 27426 raft_consensus.cc:2804] T 819307e0735043d5b032193af4478edc P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:07.001329 27426 raft_consensus.cc:493] T 819307e0735043d5b032193af4478edc P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:07.001760 27426 raft_consensus.cc:3060] T 819307e0735043d5b032193af4478edc P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.000578 27466 raft_consensus.cc:515] T f53040cb502f490189c5be0990856a58 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.006460 27466 leader_election.cc:290] T f53040cb502f490189c5be0990856a58 P df70ca0962e140ba92abead00b2fc477 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:07.008646 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53040cb502f490189c5be0990856a58" candidate_uuid: "df70ca0962e140ba92abead00b2fc477" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16f9474cee3e43c89e5d9e42266cb250" is_pre_election: true
I20260709 13:10:07.007202 27174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53040cb502f490189c5be0990856a58" candidate_uuid: "df70ca0962e140ba92abead00b2fc477" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394" is_pre_election: true
I20260709 13:10:07.010108 27174 raft_consensus.cc:2468] T f53040cb502f490189c5be0990856a58 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df70ca0962e140ba92abead00b2fc477 in term 0.
I20260709 13:10:07.011158 27325 raft_consensus.cc:2468] T f53040cb502f490189c5be0990856a58 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df70ca0962e140ba92abead00b2fc477 in term 0.
I20260709 13:10:07.012717 27217 leader_election.cc:304] T f53040cb502f490189c5be0990856a58 P df70ca0962e140ba92abead00b2fc477 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250, df70ca0962e140ba92abead00b2fc477; no voters: 
I20260709 13:10:07.013705 27466 raft_consensus.cc:2804] T f53040cb502f490189c5be0990856a58 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:07.014170 27466 raft_consensus.cc:493] T f53040cb502f490189c5be0990856a58 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:07.014602 27466 raft_consensus.cc:3060] T f53040cb502f490189c5be0990856a58 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.025624 27466 raft_consensus.cc:515] T f53040cb502f490189c5be0990856a58 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.030427 27174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53040cb502f490189c5be0990856a58" candidate_uuid: "df70ca0962e140ba92abead00b2fc477" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394"
I20260709 13:10:07.031219 27174 raft_consensus.cc:3060] T f53040cb502f490189c5be0990856a58 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.032002 27401 raft_consensus.cc:493] T 43b0b2e84a7f408fb83a416f9d669635 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:07.032670 27401 raft_consensus.cc:515] T 43b0b2e84a7f408fb83a416f9d669635 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.036564 27176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43b0b2e84a7f408fb83a416f9d669635" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394" is_pre_election: true
I20260709 13:10:07.039867 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43b0b2e84a7f408fb83a416f9d669635" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477" is_pre_election: true
W20260709 13:10:07.041713 27290 leader_election.cc:343] T 43b0b2e84a7f408fb83a416f9d669635 P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:07.042695 27401 leader_election.cc:290] T 43b0b2e84a7f408fb83a416f9d669635 P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461)
W20260709 13:10:07.044003 27289 leader_election.cc:343] T 43b0b2e84a7f408fb83a416f9d669635 P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:07.044642 27289 leader_election.cc:304] T 43b0b2e84a7f408fb83a416f9d669635 P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250; no voters: 3e58e2a069aa48c59858cbadb8ab9394, df70ca0962e140ba92abead00b2fc477
I20260709 13:10:07.046077 27401 raft_consensus.cc:2749] T 43b0b2e84a7f408fb83a416f9d669635 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:07.046595 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53040cb502f490189c5be0990856a58" candidate_uuid: "df70ca0962e140ba92abead00b2fc477" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16f9474cee3e43c89e5d9e42266cb250"
I20260709 13:10:07.047433 27325 raft_consensus.cc:3060] T f53040cb502f490189c5be0990856a58 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.052028 27401 raft_consensus.cc:493] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:07.052717 27401 raft_consensus.cc:515] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.053627 27461 tablet_bootstrap.cc:492] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 3e58e2a069aa48c59858cbadb8ab9394: No bootstrap required, opened a new log
I20260709 13:10:07.054306 27461 ts_tablet_manager.cc:1403] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 3e58e2a069aa48c59858cbadb8ab9394: Time spent bootstrapping tablet: real 0.124s	user 0.016s	sys 0.012s
I20260709 13:10:07.056396 27176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ebd3c36d25e44e293a5d318fa74b4a2" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394" is_pre_election: true
I20260709 13:10:07.057292 27176 raft_consensus.cc:2468] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 0.
I20260709 13:10:07.059000 27403 raft_consensus.cc:493] T 50d0e7c2eeb740c69d8235a9baee5564 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:07.059661 27403 raft_consensus.cc:515] T 50d0e7c2eeb740c69d8235a9baee5564 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.062816 27325 raft_consensus.cc:2468] T f53040cb502f490189c5be0990856a58 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df70ca0962e140ba92abead00b2fc477 in term 1.
I20260709 13:10:07.064740 27323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d0e7c2eeb740c69d8235a9baee5564" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16f9474cee3e43c89e5d9e42266cb250" is_pre_election: true
I20260709 13:10:07.066061 27403 leader_election.cc:290] T 50d0e7c2eeb740c69d8235a9baee5564 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:07.069357 27466 leader_election.cc:290] T f53040cb502f490189c5be0990856a58 P df70ca0962e140ba92abead00b2fc477 [CANDIDATE]: Term 1 election: Requested vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
W20260709 13:10:07.067528 27142 leader_election.cc:343] T 50d0e7c2eeb740c69d8235a9baee5564 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:07.069144 27174 raft_consensus.cc:2468] T f53040cb502f490189c5be0990856a58 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df70ca0962e140ba92abead00b2fc477 in term 1.
I20260709 13:10:07.071312 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d0e7c2eeb740c69d8235a9baee5564" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477" is_pre_election: true
W20260709 13:10:07.072777 27140 leader_election.cc:343] T 50d0e7c2eeb740c69d8235a9baee5564 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:07.073276 27140 leader_election.cc:304] T 50d0e7c2eeb740c69d8235a9baee5564 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e58e2a069aa48c59858cbadb8ab9394; no voters: 16f9474cee3e43c89e5d9e42266cb250, df70ca0962e140ba92abead00b2fc477
I20260709 13:10:07.066154 27461 raft_consensus.cc:359] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.073873 27461 raft_consensus.cc:385] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:07.072505 27217 leader_election.cc:304] T f53040cb502f490189c5be0990856a58 P df70ca0962e140ba92abead00b2fc477 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250, df70ca0962e140ba92abead00b2fc477; no voters: 
I20260709 13:10:07.074206 27461 raft_consensus.cc:740] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e58e2a069aa48c59858cbadb8ab9394, State: Initialized, Role: FOLLOWER
I20260709 13:10:07.075171 27461 consensus_queue.cc:260] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 3e58e2a069aa48c59858cbadb8ab9394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.075925 27481 raft_consensus.cc:2749] T 50d0e7c2eeb740c69d8235a9baee5564 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:07.076797 27466 raft_consensus.cc:493] T 9ebd3c36d25e44e293a5d318fa74b4a2 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:07.068001 27290 leader_election.cc:304] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250, 3e58e2a069aa48c59858cbadb8ab9394; no voters: 
I20260709 13:10:07.078882 27493 raft_consensus.cc:2804] T f53040cb502f490189c5be0990856a58 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:07.078886 27482 raft_consensus.cc:2804] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:07.080348 27482 raft_consensus.cc:493] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:07.080667 27482 raft_consensus.cc:3060] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.083250 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ebd3c36d25e44e293a5d318fa74b4a2" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477" is_pre_election: true
I20260709 13:10:07.083886 27401 leader_election.cc:290] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461)
I20260709 13:10:07.086779 27426 raft_consensus.cc:515] T 819307e0735043d5b032193af4478edc P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.077428 27466 raft_consensus.cc:515] T 9ebd3c36d25e44e293a5d318fa74b4a2 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.088519 27251 raft_consensus.cc:2468] T 9ebd3c36d25e44e293a5d318fa74b4a2 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 0.
I20260709 13:10:07.090431 27174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819307e0735043d5b032193af4478edc" candidate_uuid: "df70ca0962e140ba92abead00b2fc477" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394"
I20260709 13:10:07.091050 27174 raft_consensus.cc:3060] T 819307e0735043d5b032193af4478edc P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.091779 27482 raft_consensus.cc:515] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.095245 27176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ebd3c36d25e44e293a5d318fa74b4a2" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394"
I20260709 13:10:07.095705 27175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ebd3c36d25e44e293a5d318fa74b4a2" candidate_uuid: "df70ca0962e140ba92abead00b2fc477" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394" is_pre_election: true
I20260709 13:10:07.095897 27176 raft_consensus.cc:3060] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.096720 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ebd3c36d25e44e293a5d318fa74b4a2" candidate_uuid: "df70ca0962e140ba92abead00b2fc477" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16f9474cee3e43c89e5d9e42266cb250" is_pre_election: true
I20260709 13:10:07.027868 27460 raft_consensus.cc:515] T f0808b4c74f04ce28b19a78bf4c504a8 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.097514 27325 raft_consensus.cc:2393] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate df70ca0962e140ba92abead00b2fc477 in current term 1: Already voted for candidate 16f9474cee3e43c89e5d9e42266cb250 in this term.
I20260709 13:10:07.098704 27460 leader_election.cc:290] T f0808b4c74f04ce28b19a78bf4c504a8 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 election: Requested vote from peers df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:07.099071 27466 leader_election.cc:290] T 9ebd3c36d25e44e293a5d318fa74b4a2 P df70ca0962e140ba92abead00b2fc477 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:07.098735 27323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819307e0735043d5b032193af4478edc" candidate_uuid: "df70ca0962e140ba92abead00b2fc477" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16f9474cee3e43c89e5d9e42266cb250"
I20260709 13:10:07.100538 27323 raft_consensus.cc:3060] T 819307e0735043d5b032193af4478edc P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.104571 27217 leader_election.cc:304] T 9ebd3c36d25e44e293a5d318fa74b4a2 P df70ca0962e140ba92abead00b2fc477 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df70ca0962e140ba92abead00b2fc477; no voters: 16f9474cee3e43c89e5d9e42266cb250, 3e58e2a069aa48c59858cbadb8ab9394
I20260709 13:10:07.105599 27482 leader_election.cc:290] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 election: Requested vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461)
I20260709 13:10:07.107015 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ebd3c36d25e44e293a5d318fa74b4a2" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477"
I20260709 13:10:07.107462 27176 raft_consensus.cc:2468] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 1.
I20260709 13:10:07.107825 27466 raft_consensus.cc:3060] T 9ebd3c36d25e44e293a5d318fa74b4a2 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.109508 27290 leader_election.cc:304] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250, 3e58e2a069aa48c59858cbadb8ab9394; no voters: 
I20260709 13:10:07.110486 27482 raft_consensus.cc:2804] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:07.110999 27482 raft_consensus.cc:697] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 LEADER]: Becoming Leader. State: Replica: 16f9474cee3e43c89e5d9e42266cb250, State: Running, Role: LEADER
I20260709 13:10:07.111847 27482 consensus_queue.cc:237] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.112937 27323 raft_consensus.cc:2468] T 819307e0735043d5b032193af4478edc P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df70ca0962e140ba92abead00b2fc477 in term 1.
I20260709 13:10:07.113703 27493 raft_consensus.cc:697] T f53040cb502f490189c5be0990856a58 P df70ca0962e140ba92abead00b2fc477 [term 1 LEADER]: Becoming Leader. State: Replica: df70ca0962e140ba92abead00b2fc477, State: Running, Role: LEADER
I20260709 13:10:07.121199 27493 consensus_queue.cc:237] T f53040cb502f490189c5be0990856a58 P df70ca0962e140ba92abead00b2fc477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.125361 27250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0808b4c74f04ce28b19a78bf4c504a8" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477"
I20260709 13:10:07.126262 27250 raft_consensus.cc:3060] T f0808b4c74f04ce28b19a78bf4c504a8 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.124485 27217 leader_election.cc:304] T 819307e0735043d5b032193af4478edc P df70ca0962e140ba92abead00b2fc477 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250, df70ca0962e140ba92abead00b2fc477; no voters: 
I20260709 13:10:07.117736 27174 raft_consensus.cc:2468] T 819307e0735043d5b032193af4478edc P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df70ca0962e140ba92abead00b2fc477 in term 1.
I20260709 13:10:07.118304 27426 leader_election.cc:290] T 819307e0735043d5b032193af4478edc P df70ca0962e140ba92abead00b2fc477 [CANDIDATE]: Term 1 election: Requested vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:07.119156 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0808b4c74f04ce28b19a78bf4c504a8" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16f9474cee3e43c89e5d9e42266cb250"
I20260709 13:10:07.117775 27462 tablet_bootstrap.cc:492] T 43b0b2e84a7f408fb83a416f9d669635 P df70ca0962e140ba92abead00b2fc477: No bootstrap required, opened a new log
I20260709 13:10:07.134696 27462 ts_tablet_manager.cc:1403] T 43b0b2e84a7f408fb83a416f9d669635 P df70ca0962e140ba92abead00b2fc477: Time spent bootstrapping tablet: real 0.193s	user 0.033s	sys 0.091s
I20260709 13:10:07.130877 27496 raft_consensus.cc:2804] T 819307e0735043d5b032193af4478edc P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:07.133813 27461 ts_tablet_manager.cc:1434] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 3e58e2a069aa48c59858cbadb8ab9394: Time spent starting tablet: real 0.079s	user 0.023s	sys 0.008s
I20260709 13:10:07.139384 27461 tablet_bootstrap.cc:492] T 43b0b2e84a7f408fb83a416f9d669635 P 3e58e2a069aa48c59858cbadb8ab9394: Bootstrap starting.
I20260709 13:10:07.139797 27325 raft_consensus.cc:3060] T f0808b4c74f04ce28b19a78bf4c504a8 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.141960 27462 raft_consensus.cc:359] T 43b0b2e84a7f408fb83a416f9d669635 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.143344 27462 raft_consensus.cc:385] T 43b0b2e84a7f408fb83a416f9d669635 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:07.143657 27462 raft_consensus.cc:740] T 43b0b2e84a7f408fb83a416f9d669635 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df70ca0962e140ba92abead00b2fc477, State: Initialized, Role: FOLLOWER
I20260709 13:10:07.145347 27250 raft_consensus.cc:2468] T f0808b4c74f04ce28b19a78bf4c504a8 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e58e2a069aa48c59858cbadb8ab9394 in term 1.
I20260709 13:10:07.148334 27462 consensus_queue.cc:260] T 43b0b2e84a7f408fb83a416f9d669635 P df70ca0962e140ba92abead00b2fc477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.151353 27464 tablet_bootstrap.cc:492] T 50d0e7c2eeb740c69d8235a9baee5564 P 16f9474cee3e43c89e5d9e42266cb250: No bootstrap required, opened a new log
I20260709 13:10:07.151705 27462 ts_tablet_manager.cc:1434] T 43b0b2e84a7f408fb83a416f9d669635 P df70ca0962e140ba92abead00b2fc477: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:10:07.151873 27464 ts_tablet_manager.cc:1403] T 50d0e7c2eeb740c69d8235a9baee5564 P 16f9474cee3e43c89e5d9e42266cb250: Time spent bootstrapping tablet: real 0.194s	user 0.089s	sys 0.090s
I20260709 13:10:07.153131 27462 tablet_bootstrap.cc:492] T 50d0e7c2eeb740c69d8235a9baee5564 P df70ca0962e140ba92abead00b2fc477: Bootstrap starting.
I20260709 13:10:07.154852 27464 raft_consensus.cc:359] T 50d0e7c2eeb740c69d8235a9baee5564 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.155539 27464 raft_consensus.cc:385] T 50d0e7c2eeb740c69d8235a9baee5564 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:07.145707 27496 raft_consensus.cc:697] T 819307e0735043d5b032193af4478edc P df70ca0962e140ba92abead00b2fc477 [term 1 LEADER]: Becoming Leader. State: Replica: df70ca0962e140ba92abead00b2fc477, State: Running, Role: LEADER
I20260709 13:10:07.155825 27464 raft_consensus.cc:740] T 50d0e7c2eeb740c69d8235a9baee5564 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16f9474cee3e43c89e5d9e42266cb250, State: Initialized, Role: FOLLOWER
I20260709 13:10:07.156548 27464 consensus_queue.cc:260] T 50d0e7c2eeb740c69d8235a9baee5564 P 16f9474cee3e43c89e5d9e42266cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.158633 27464 ts_tablet_manager.cc:1434] T 50d0e7c2eeb740c69d8235a9baee5564 P 16f9474cee3e43c89e5d9e42266cb250: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:10:07.156418 27496 consensus_queue.cc:237] T 819307e0735043d5b032193af4478edc P df70ca0962e140ba92abead00b2fc477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.159559 27464 tablet_bootstrap.cc:492] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 16f9474cee3e43c89e5d9e42266cb250: Bootstrap starting.
I20260709 13:10:07.161336 27140 leader_election.cc:304] T f0808b4c74f04ce28b19a78bf4c504a8 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e58e2a069aa48c59858cbadb8ab9394, df70ca0962e140ba92abead00b2fc477; no voters: 
I20260709 13:10:07.162724 27461 tablet_bootstrap.cc:654] T 43b0b2e84a7f408fb83a416f9d669635 P 3e58e2a069aa48c59858cbadb8ab9394: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:07.162884 27494 raft_consensus.cc:2804] T f0808b4c74f04ce28b19a78bf4c504a8 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:07.163501 27494 raft_consensus.cc:697] T f0808b4c74f04ce28b19a78bf4c504a8 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 LEADER]: Becoming Leader. State: Replica: 3e58e2a069aa48c59858cbadb8ab9394, State: Running, Role: LEADER
I20260709 13:10:07.169178 27494 consensus_queue.cc:237] T f0808b4c74f04ce28b19a78bf4c504a8 P 3e58e2a069aa48c59858cbadb8ab9394 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.180929 27464 tablet_bootstrap.cc:654] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 16f9474cee3e43c89e5d9e42266cb250: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:07.184677 27460 raft_consensus.cc:493] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:07.177738 27466 raft_consensus.cc:2749] T 9ebd3c36d25e44e293a5d318fa74b4a2 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:07.185204 27460 raft_consensus.cc:515] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.207279 27251 raft_consensus.cc:2468] T 9ebd3c36d25e44e293a5d318fa74b4a2 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 1.
I20260709 13:10:07.228914 27323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c5d9e4d3ae74d17b4027d97e8f5b01b" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16f9474cee3e43c89e5d9e42266cb250" is_pre_election: true
I20260709 13:10:07.229974 27070 catalog_manager.cc:5697] T f53040cb502f490189c5be0990856a58 P df70ca0962e140ba92abead00b2fc477 reported cstate change: term changed from 0 to 1, leader changed from <none> to df70ca0962e140ba92abead00b2fc477 (127.14.103.2). New cstate: current_term: 1 leader_uuid: "df70ca0962e140ba92abead00b2fc477" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } health_report { overall_health: UNKNOWN } } }
W20260709 13:10:07.231180 27142 leader_election.cc:343] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:07.227363 27460 leader_election.cc:290] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:07.212940 27325 raft_consensus.cc:2468] T f0808b4c74f04ce28b19a78bf4c504a8 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e58e2a069aa48c59858cbadb8ab9394 in term 1.
I20260709 13:10:07.233333 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c5d9e4d3ae74d17b4027d97e8f5b01b" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477" is_pre_election: true
I20260709 13:10:07.233879 27462 tablet_bootstrap.cc:654] T 50d0e7c2eeb740c69d8235a9baee5564 P df70ca0962e140ba92abead00b2fc477: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:07.216554 27072 catalog_manager.cc:5697] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250 reported cstate change: term changed from 0 to 1, leader changed from <none> to 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3). New cstate: current_term: 1 leader_uuid: "16f9474cee3e43c89e5d9e42266cb250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } health_report { overall_health: HEALTHY } } }
W20260709 13:10:07.239641 27140 leader_election.cc:343] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:07.240144 27140 leader_election.cc:304] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e58e2a069aa48c59858cbadb8ab9394; no voters: 16f9474cee3e43c89e5d9e42266cb250, df70ca0962e140ba92abead00b2fc477
I20260709 13:10:07.241235 27460 raft_consensus.cc:2749] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:07.247416 27464 tablet_bootstrap.cc:492] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 16f9474cee3e43c89e5d9e42266cb250: No bootstrap required, opened a new log
I20260709 13:10:07.247992 27464 ts_tablet_manager.cc:1403] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 16f9474cee3e43c89e5d9e42266cb250: Time spent bootstrapping tablet: real 0.089s	user 0.024s	sys 0.011s
I20260709 13:10:07.249250 27073 catalog_manager.cc:5697] T f0808b4c74f04ce28b19a78bf4c504a8 P 3e58e2a069aa48c59858cbadb8ab9394 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1). New cstate: current_term: 1 leader_uuid: "3e58e2a069aa48c59858cbadb8ab9394" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:07.251575 27464 raft_consensus.cc:359] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.252295 27464 raft_consensus.cc:385] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:07.252552 27464 raft_consensus.cc:740] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16f9474cee3e43c89e5d9e42266cb250, State: Initialized, Role: FOLLOWER
I20260709 13:10:07.253213 27464 consensus_queue.cc:260] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 16f9474cee3e43c89e5d9e42266cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.254755 27464 ts_tablet_manager.cc:1434] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 16f9474cee3e43c89e5d9e42266cb250: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 13:10:07.256819 27461 tablet_bootstrap.cc:492] T 43b0b2e84a7f408fb83a416f9d669635 P 3e58e2a069aa48c59858cbadb8ab9394: No bootstrap required, opened a new log
I20260709 13:10:07.257383 27461 ts_tablet_manager.cc:1403] T 43b0b2e84a7f408fb83a416f9d669635 P 3e58e2a069aa48c59858cbadb8ab9394: Time spent bootstrapping tablet: real 0.120s	user 0.021s	sys 0.024s
I20260709 13:10:07.260073 27461 raft_consensus.cc:359] T 43b0b2e84a7f408fb83a416f9d669635 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.260630 27461 raft_consensus.cc:385] T 43b0b2e84a7f408fb83a416f9d669635 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:07.260867 27461 raft_consensus.cc:740] T 43b0b2e84a7f408fb83a416f9d669635 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e58e2a069aa48c59858cbadb8ab9394, State: Initialized, Role: FOLLOWER
I20260709 13:10:07.261610 27461 consensus_queue.cc:260] T 43b0b2e84a7f408fb83a416f9d669635 P 3e58e2a069aa48c59858cbadb8ab9394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.263628 27461 ts_tablet_manager.cc:1434] T 43b0b2e84a7f408fb83a416f9d669635 P 3e58e2a069aa48c59858cbadb8ab9394: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:10:07.266184 27070 catalog_manager.cc:5697] T 819307e0735043d5b032193af4478edc P df70ca0962e140ba92abead00b2fc477 reported cstate change: term changed from 0 to 1, leader changed from <none> to df70ca0962e140ba92abead00b2fc477 (127.14.103.2). New cstate: current_term: 1 leader_uuid: "df70ca0962e140ba92abead00b2fc477" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:07.275797 27482 consensus_queue.cc:1048] T 22270c8eaaa648f9b5cbe3125faa8c25 P 16f9474cee3e43c89e5d9e42266cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:07.281806 27462 tablet_bootstrap.cc:492] T 50d0e7c2eeb740c69d8235a9baee5564 P df70ca0962e140ba92abead00b2fc477: No bootstrap required, opened a new log
I20260709 13:10:07.282258 27462 ts_tablet_manager.cc:1403] T 50d0e7c2eeb740c69d8235a9baee5564 P df70ca0962e140ba92abead00b2fc477: Time spent bootstrapping tablet: real 0.129s	user 0.017s	sys 0.000s
I20260709 13:10:07.284523 27462 raft_consensus.cc:359] T 50d0e7c2eeb740c69d8235a9baee5564 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.285182 27462 raft_consensus.cc:385] T 50d0e7c2eeb740c69d8235a9baee5564 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:07.285471 27462 raft_consensus.cc:740] T 50d0e7c2eeb740c69d8235a9baee5564 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df70ca0962e140ba92abead00b2fc477, State: Initialized, Role: FOLLOWER
I20260709 13:10:07.286083 27462 consensus_queue.cc:260] T 50d0e7c2eeb740c69d8235a9baee5564 P df70ca0962e140ba92abead00b2fc477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.290329 27462 ts_tablet_manager.cc:1434] T 50d0e7c2eeb740c69d8235a9baee5564 P df70ca0962e140ba92abead00b2fc477: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:10:07.291229 27462 tablet_bootstrap.cc:492] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P df70ca0962e140ba92abead00b2fc477: Bootstrap starting.
I20260709 13:10:07.291719 27482 consensus_queue.cc:1048] T 22270c8eaaa648f9b5cbe3125faa8c25 P 16f9474cee3e43c89e5d9e42266cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:07.296536 27462 tablet_bootstrap.cc:654] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P df70ca0962e140ba92abead00b2fc477: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:07.301995 27462 tablet_bootstrap.cc:492] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P df70ca0962e140ba92abead00b2fc477: No bootstrap required, opened a new log
I20260709 13:10:07.302423 27462 ts_tablet_manager.cc:1403] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P df70ca0962e140ba92abead00b2fc477: Time spent bootstrapping tablet: real 0.011s	user 0.008s	sys 0.001s
I20260709 13:10:07.304554 27462 raft_consensus.cc:359] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.305050 27462 raft_consensus.cc:385] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:07.305277 27462 raft_consensus.cc:740] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df70ca0962e140ba92abead00b2fc477, State: Initialized, Role: FOLLOWER
I20260709 13:10:07.305809 27462 consensus_queue.cc:260] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P df70ca0962e140ba92abead00b2fc477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.307444 27462 ts_tablet_manager.cc:1434] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P df70ca0962e140ba92abead00b2fc477: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20260709 13:10:07.317559 27482 raft_consensus.cc:493] T 50d0e7c2eeb740c69d8235a9baee5564 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:07.318146 27482 raft_consensus.cc:515] T 50d0e7c2eeb740c69d8235a9baee5564 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.320222 27482 leader_election.cc:290] T 50d0e7c2eeb740c69d8235a9baee5564 P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461)
I20260709 13:10:07.320840 27174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d0e7c2eeb740c69d8235a9baee5564" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394" is_pre_election: true
I20260709 13:10:07.321265 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d0e7c2eeb740c69d8235a9baee5564" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477" is_pre_election: true
I20260709 13:10:07.321508 27174 raft_consensus.cc:2468] T 50d0e7c2eeb740c69d8235a9baee5564 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 0.
I20260709 13:10:07.321879 27251 raft_consensus.cc:2468] T 50d0e7c2eeb740c69d8235a9baee5564 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 0.
I20260709 13:10:07.322482 27290 leader_election.cc:304] T 50d0e7c2eeb740c69d8235a9baee5564 P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250, 3e58e2a069aa48c59858cbadb8ab9394; no voters: 
I20260709 13:10:07.323136 27482 raft_consensus.cc:2804] T 50d0e7c2eeb740c69d8235a9baee5564 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:07.323520 27482 raft_consensus.cc:493] T 50d0e7c2eeb740c69d8235a9baee5564 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:07.323844 27482 raft_consensus.cc:3060] T 50d0e7c2eeb740c69d8235a9baee5564 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.328670 27482 raft_consensus.cc:515] T 50d0e7c2eeb740c69d8235a9baee5564 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.330612 27482 leader_election.cc:290] T 50d0e7c2eeb740c69d8235a9baee5564 P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 election: Requested vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461)
I20260709 13:10:07.331375 27174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d0e7c2eeb740c69d8235a9baee5564" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394"
I20260709 13:10:07.331746 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d0e7c2eeb740c69d8235a9baee5564" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df70ca0962e140ba92abead00b2fc477"
I20260709 13:10:07.332058 27174 raft_consensus.cc:3060] T 50d0e7c2eeb740c69d8235a9baee5564 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.332333 27251 raft_consensus.cc:3060] T 50d0e7c2eeb740c69d8235a9baee5564 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.337189 27174 raft_consensus.cc:2468] T 50d0e7c2eeb740c69d8235a9baee5564 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 1.
I20260709 13:10:07.338419 27251 raft_consensus.cc:2468] T 50d0e7c2eeb740c69d8235a9baee5564 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 1.
I20260709 13:10:07.338716 27290 leader_election.cc:304] T 50d0e7c2eeb740c69d8235a9baee5564 P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250, 3e58e2a069aa48c59858cbadb8ab9394; no voters: 
I20260709 13:10:07.339143 27494 consensus_queue.cc:1048] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 3e58e2a069aa48c59858cbadb8ab9394 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:07.339643 27482 raft_consensus.cc:2804] T 50d0e7c2eeb740c69d8235a9baee5564 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:07.340222 27482 raft_consensus.cc:697] T 50d0e7c2eeb740c69d8235a9baee5564 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 LEADER]: Becoming Leader. State: Replica: 16f9474cee3e43c89e5d9e42266cb250, State: Running, Role: LEADER
I20260709 13:10:07.341017 27482 consensus_queue.cc:237] T 50d0e7c2eeb740c69d8235a9baee5564 P 16f9474cee3e43c89e5d9e42266cb250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.350425 27074 catalog_manager.cc:5697] T 50d0e7c2eeb740c69d8235a9baee5564 P 16f9474cee3e43c89e5d9e42266cb250 reported cstate change: term changed from 0 to 1, leader changed from <none> to 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3). New cstate: current_term: 1 leader_uuid: "16f9474cee3e43c89e5d9e42266cb250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:07.355655 27481 consensus_queue.cc:1048] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 3e58e2a069aa48c59858cbadb8ab9394 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:07.388069 27496 raft_consensus.cc:493] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:07.388592 27496 raft_consensus.cc:515] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.390324 27496 leader_election.cc:290] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P df70ca0962e140ba92abead00b2fc477 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:07.390959 27174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c5d9e4d3ae74d17b4027d97e8f5b01b" candidate_uuid: "df70ca0962e140ba92abead00b2fc477" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394" is_pre_election: true
I20260709 13:10:07.391301 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c5d9e4d3ae74d17b4027d97e8f5b01b" candidate_uuid: "df70ca0962e140ba92abead00b2fc477" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16f9474cee3e43c89e5d9e42266cb250" is_pre_election: true
I20260709 13:10:07.391579 27174 raft_consensus.cc:2468] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df70ca0962e140ba92abead00b2fc477 in term 0.
I20260709 13:10:07.391986 27325 raft_consensus.cc:2468] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df70ca0962e140ba92abead00b2fc477 in term 0.
I20260709 13:10:07.392705 27216 leader_election.cc:304] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P df70ca0962e140ba92abead00b2fc477 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e58e2a069aa48c59858cbadb8ab9394, df70ca0962e140ba92abead00b2fc477; no voters: 
I20260709 13:10:07.393496 27496 raft_consensus.cc:2804] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:07.393851 27496 raft_consensus.cc:493] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:07.394167 27496 raft_consensus.cc:3060] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.399811 27496 raft_consensus.cc:515] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.401238 27496 leader_election.cc:290] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P df70ca0962e140ba92abead00b2fc477 [CANDIDATE]: Term 1 election: Requested vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:07.402027 27174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c5d9e4d3ae74d17b4027d97e8f5b01b" candidate_uuid: "df70ca0962e140ba92abead00b2fc477" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394"
I20260709 13:10:07.402165 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c5d9e4d3ae74d17b4027d97e8f5b01b" candidate_uuid: "df70ca0962e140ba92abead00b2fc477" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16f9474cee3e43c89e5d9e42266cb250"
I20260709 13:10:07.402541 27174 raft_consensus.cc:3060] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.402724 27325 raft_consensus.cc:3060] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.406716 27174 raft_consensus.cc:2468] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df70ca0962e140ba92abead00b2fc477 in term 1.
I20260709 13:10:07.407495 27325 raft_consensus.cc:2468] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df70ca0962e140ba92abead00b2fc477 in term 1.
I20260709 13:10:07.407596 27216 leader_election.cc:304] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P df70ca0962e140ba92abead00b2fc477 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e58e2a069aa48c59858cbadb8ab9394, df70ca0962e140ba92abead00b2fc477; no voters: 
I20260709 13:10:07.408545 27496 raft_consensus.cc:2804] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:07.409010 27496 raft_consensus.cc:697] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P df70ca0962e140ba92abead00b2fc477 [term 1 LEADER]: Becoming Leader. State: Replica: df70ca0962e140ba92abead00b2fc477, State: Running, Role: LEADER
I20260709 13:10:07.409642 27496 consensus_queue.cc:237] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P df70ca0962e140ba92abead00b2fc477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.412231 27493 raft_consensus.cc:493] T 43b0b2e84a7f408fb83a416f9d669635 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:07.412706 27493 raft_consensus.cc:515] T 43b0b2e84a7f408fb83a416f9d669635 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.414613 27493 leader_election.cc:290] T 43b0b2e84a7f408fb83a416f9d669635 P df70ca0962e140ba92abead00b2fc477 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:07.416102 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43b0b2e84a7f408fb83a416f9d669635" candidate_uuid: "df70ca0962e140ba92abead00b2fc477" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16f9474cee3e43c89e5d9e42266cb250" is_pre_election: true
I20260709 13:10:07.416503 27174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43b0b2e84a7f408fb83a416f9d669635" candidate_uuid: "df70ca0962e140ba92abead00b2fc477" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394" is_pre_election: true
I20260709 13:10:07.416853 27325 raft_consensus.cc:2468] T 43b0b2e84a7f408fb83a416f9d669635 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df70ca0962e140ba92abead00b2fc477 in term 0.
I20260709 13:10:07.417116 27174 raft_consensus.cc:2468] T 43b0b2e84a7f408fb83a416f9d669635 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df70ca0962e140ba92abead00b2fc477 in term 0.
I20260709 13:10:07.417858 27217 leader_election.cc:304] T 43b0b2e84a7f408fb83a416f9d669635 P df70ca0962e140ba92abead00b2fc477 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250, df70ca0962e140ba92abead00b2fc477; no voters: 
I20260709 13:10:07.418658 27493 raft_consensus.cc:2804] T 43b0b2e84a7f408fb83a416f9d669635 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:07.417987 27070 catalog_manager.cc:5697] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P df70ca0962e140ba92abead00b2fc477 reported cstate change: term changed from 0 to 1, leader changed from <none> to df70ca0962e140ba92abead00b2fc477 (127.14.103.2). New cstate: current_term: 1 leader_uuid: "df70ca0962e140ba92abead00b2fc477" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:07.419073 27493 raft_consensus.cc:493] T 43b0b2e84a7f408fb83a416f9d669635 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:07.419430 27493 raft_consensus.cc:3060] T 43b0b2e84a7f408fb83a416f9d669635 P df70ca0962e140ba92abead00b2fc477 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.424855 27493 raft_consensus.cc:515] T 43b0b2e84a7f408fb83a416f9d669635 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.427152 27493 leader_election.cc:290] T 43b0b2e84a7f408fb83a416f9d669635 P df70ca0962e140ba92abead00b2fc477 [CANDIDATE]: Term 1 election: Requested vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:07.427470 27174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43b0b2e84a7f408fb83a416f9d669635" candidate_uuid: "df70ca0962e140ba92abead00b2fc477" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394"
I20260709 13:10:07.427780 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43b0b2e84a7f408fb83a416f9d669635" candidate_uuid: "df70ca0962e140ba92abead00b2fc477" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16f9474cee3e43c89e5d9e42266cb250"
I20260709 13:10:07.428105 27174 raft_consensus.cc:3060] T 43b0b2e84a7f408fb83a416f9d669635 P 3e58e2a069aa48c59858cbadb8ab9394 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.428336 27325 raft_consensus.cc:3060] T 43b0b2e84a7f408fb83a416f9d669635 P 16f9474cee3e43c89e5d9e42266cb250 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:07.434520 27174 raft_consensus.cc:2468] T 43b0b2e84a7f408fb83a416f9d669635 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df70ca0962e140ba92abead00b2fc477 in term 1.
I20260709 13:10:07.434520 27325 raft_consensus.cc:2468] T 43b0b2e84a7f408fb83a416f9d669635 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df70ca0962e140ba92abead00b2fc477 in term 1.
I20260709 13:10:07.435662 27217 leader_election.cc:304] T 43b0b2e84a7f408fb83a416f9d669635 P df70ca0962e140ba92abead00b2fc477 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250, df70ca0962e140ba92abead00b2fc477; no voters: 
I20260709 13:10:07.436609 27493 raft_consensus.cc:2804] T 43b0b2e84a7f408fb83a416f9d669635 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:07.437018 27493 raft_consensus.cc:697] T 43b0b2e84a7f408fb83a416f9d669635 P df70ca0962e140ba92abead00b2fc477 [term 1 LEADER]: Becoming Leader. State: Replica: df70ca0962e140ba92abead00b2fc477, State: Running, Role: LEADER
I20260709 13:10:07.437758 27493 consensus_queue.cc:237] T 43b0b2e84a7f408fb83a416f9d669635 P df70ca0962e140ba92abead00b2fc477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.444897 27070 catalog_manager.cc:5697] T 43b0b2e84a7f408fb83a416f9d669635 P df70ca0962e140ba92abead00b2fc477 reported cstate change: term changed from 0 to 1, leader changed from <none> to df70ca0962e140ba92abead00b2fc477 (127.14.103.2). New cstate: current_term: 1 leader_uuid: "df70ca0962e140ba92abead00b2fc477" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:07.545256 27493 consensus_queue.cc:1048] T 819307e0735043d5b032193af4478edc P df70ca0962e140ba92abead00b2fc477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:07.555337 27493 consensus_queue.cc:1048] T 819307e0735043d5b032193af4478edc P df70ca0962e140ba92abead00b2fc477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:07.566089 27481 consensus_queue.cc:1048] T f0808b4c74f04ce28b19a78bf4c504a8 P 3e58e2a069aa48c59858cbadb8ab9394 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:07.571391 27482 consensus_queue.cc:1048] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:07.583020 27481 consensus_queue.cc:1048] T f0808b4c74f04ce28b19a78bf4c504a8 P 3e58e2a069aa48c59858cbadb8ab9394 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:07.591445 27466 consensus_queue.cc:1048] T f53040cb502f490189c5be0990856a58 P df70ca0962e140ba92abead00b2fc477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:07.602401 27482 consensus_queue.cc:1048] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:07.616627 27493 consensus_queue.cc:1048] T f53040cb502f490189c5be0990856a58 P df70ca0962e140ba92abead00b2fc477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:07.621789 27323 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "22270c8eaaa648f9b5cbe3125faa8c25"
dest_uuid: "16f9474cee3e43c89e5d9e42266cb250"
mode: GRACEFUL
new_leader_uuid: "3e58e2a069aa48c59858cbadb8ab9394"
 from {username='slave'} at 127.0.0.1:48608
I20260709 13:10:07.622773 27323 raft_consensus.cc:606] T 22270c8eaaa648f9b5cbe3125faa8c25 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 LEADER]: Received request to transfer leadership to TS 3e58e2a069aa48c59858cbadb8ab9394
I20260709 13:10:07.636694 27323 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ebd3c36d25e44e293a5d318fa74b4a2"
dest_uuid: "16f9474cee3e43c89e5d9e42266cb250"
mode: GRACEFUL
new_leader_uuid: "3e58e2a069aa48c59858cbadb8ab9394"
 from {username='slave'} at 127.0.0.1:48608
I20260709 13:10:07.637238 27323 raft_consensus.cc:606] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 LEADER]: Received request to transfer leadership to TS 3e58e2a069aa48c59858cbadb8ab9394
I20260709 13:10:07.648000 27323 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "50d0e7c2eeb740c69d8235a9baee5564"
dest_uuid: "16f9474cee3e43c89e5d9e42266cb250"
mode: GRACEFUL
new_leader_uuid: "df70ca0962e140ba92abead00b2fc477"
 from {username='slave'} at 127.0.0.1:48608
I20260709 13:10:07.648506 27323 raft_consensus.cc:606] T 50d0e7c2eeb740c69d8235a9baee5564 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 LEADER]: Received request to transfer leadership to TS df70ca0962e140ba92abead00b2fc477
I20260709 13:10:07.657527 27251 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4c5d9e4d3ae74d17b4027d97e8f5b01b"
dest_uuid: "df70ca0962e140ba92abead00b2fc477"
mode: GRACEFUL
new_leader_uuid: "16f9474cee3e43c89e5d9e42266cb250"
 from {username='slave'} at 127.0.0.1:49036
I20260709 13:10:07.658015 27251 raft_consensus.cc:606] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P df70ca0962e140ba92abead00b2fc477 [term 1 LEADER]: Received request to transfer leadership to TS 16f9474cee3e43c89e5d9e42266cb250
I20260709 13:10:07.692312 27401 raft_consensus.cc:993] T 22270c8eaaa648f9b5cbe3125faa8c25 P 16f9474cee3e43c89e5d9e42266cb250: : Instructing follower 3e58e2a069aa48c59858cbadb8ab9394 to start an election
I20260709 13:10:07.692770 27401 raft_consensus.cc:1081] T 22270c8eaaa648f9b5cbe3125faa8c25 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 LEADER]: Signalling peer 3e58e2a069aa48c59858cbadb8ab9394 to start an election
I20260709 13:10:07.694548 27174 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "22270c8eaaa648f9b5cbe3125faa8c25"
dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394"
 from {username='slave'} at 127.0.0.1:36746
I20260709 13:10:07.695116 27174 raft_consensus.cc:493] T 22270c8eaaa648f9b5cbe3125faa8c25 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:07.695463 27174 raft_consensus.cc:3060] T 22270c8eaaa648f9b5cbe3125faa8c25 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:07.699780 27174 raft_consensus.cc:515] T 22270c8eaaa648f9b5cbe3125faa8c25 P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.701169 27174 leader_election.cc:290] T 22270c8eaaa648f9b5cbe3125faa8c25 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 2 election: Requested vote from peers df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:07.702049 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22270c8eaaa648f9b5cbe3125faa8c25" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "df70ca0962e140ba92abead00b2fc477"
I20260709 13:10:07.702275 27323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22270c8eaaa648f9b5cbe3125faa8c25" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "16f9474cee3e43c89e5d9e42266cb250"
I20260709 13:10:07.702687 27251 raft_consensus.cc:3060] T 22270c8eaaa648f9b5cbe3125faa8c25 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:07.702911 27323 raft_consensus.cc:3055] T 22270c8eaaa648f9b5cbe3125faa8c25 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:07.703299 27323 raft_consensus.cc:740] T 22270c8eaaa648f9b5cbe3125faa8c25 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 16f9474cee3e43c89e5d9e42266cb250, State: Running, Role: LEADER
I20260709 13:10:07.704046 27323 consensus_queue.cc:260] T 22270c8eaaa648f9b5cbe3125faa8c25 P 16f9474cee3e43c89e5d9e42266cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.704990 27323 raft_consensus.cc:3060] T 22270c8eaaa648f9b5cbe3125faa8c25 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:07.708855 27251 raft_consensus.cc:2468] T 22270c8eaaa648f9b5cbe3125faa8c25 P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e58e2a069aa48c59858cbadb8ab9394 in term 2.
I20260709 13:10:07.709734 27323 raft_consensus.cc:2468] T 22270c8eaaa648f9b5cbe3125faa8c25 P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e58e2a069aa48c59858cbadb8ab9394 in term 2.
I20260709 13:10:07.709851 27140 leader_election.cc:304] T 22270c8eaaa648f9b5cbe3125faa8c25 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e58e2a069aa48c59858cbadb8ab9394, df70ca0962e140ba92abead00b2fc477; no voters: 
I20260709 13:10:07.710762 27481 raft_consensus.cc:2804] T 22270c8eaaa648f9b5cbe3125faa8c25 P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:07.711159 27481 raft_consensus.cc:697] T 22270c8eaaa648f9b5cbe3125faa8c25 P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 LEADER]: Becoming Leader. State: Replica: 3e58e2a069aa48c59858cbadb8ab9394, State: Running, Role: LEADER
I20260709 13:10:07.711921 27481 consensus_queue.cc:237] T 22270c8eaaa648f9b5cbe3125faa8c25 P 3e58e2a069aa48c59858cbadb8ab9394 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.717947 27070 catalog_manager.cc:5697] T 22270c8eaaa648f9b5cbe3125faa8c25 P 3e58e2a069aa48c59858cbadb8ab9394 reported cstate change: term changed from 1 to 2, leader changed from 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3) to 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "3e58e2a069aa48c59858cbadb8ab9394" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:07.811522 27493 consensus_queue.cc:1048] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P df70ca0962e140ba92abead00b2fc477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:07.821278 27426 consensus_queue.cc:1048] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P df70ca0962e140ba92abead00b2fc477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:07.824055 27401 consensus_queue.cc:1048] T 50d0e7c2eeb740c69d8235a9baee5564 P 16f9474cee3e43c89e5d9e42266cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:07.831553 27426 raft_consensus.cc:993] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P df70ca0962e140ba92abead00b2fc477: : Instructing follower 16f9474cee3e43c89e5d9e42266cb250 to start an election
I20260709 13:10:07.832049 27426 raft_consensus.cc:1081] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P df70ca0962e140ba92abead00b2fc477 [term 1 LEADER]: Signalling peer 16f9474cee3e43c89e5d9e42266cb250 to start an election
I20260709 13:10:07.833834 27323 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4c5d9e4d3ae74d17b4027d97e8f5b01b"
dest_uuid: "16f9474cee3e43c89e5d9e42266cb250"
 from {username='slave'} at 127.0.0.1:48598
I20260709 13:10:07.834349 27323 raft_consensus.cc:493] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:07.834700 27323 raft_consensus.cc:3060] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:07.837132 27401 raft_consensus.cc:993] T 50d0e7c2eeb740c69d8235a9baee5564 P 16f9474cee3e43c89e5d9e42266cb250: : Instructing follower df70ca0962e140ba92abead00b2fc477 to start an election
I20260709 13:10:07.837610 27401 raft_consensus.cc:1081] T 50d0e7c2eeb740c69d8235a9baee5564 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 LEADER]: Signalling peer df70ca0962e140ba92abead00b2fc477 to start an election
I20260709 13:10:07.839489 27249 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "50d0e7c2eeb740c69d8235a9baee5564"
dest_uuid: "df70ca0962e140ba92abead00b2fc477"
 from {username='slave'} at 127.0.0.1:49012
I20260709 13:10:07.841593 27249 raft_consensus.cc:493] T 50d0e7c2eeb740c69d8235a9baee5564 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:07.841966 27249 raft_consensus.cc:3060] T 50d0e7c2eeb740c69d8235a9baee5564 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:07.842381 27323 raft_consensus.cc:515] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.843758 27401 consensus_queue.cc:1048] T 50d0e7c2eeb740c69d8235a9baee5564 P 16f9474cee3e43c89e5d9e42266cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:07.845983 27323 leader_election.cc:290] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 2 election: Requested vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461)
I20260709 13:10:07.847038 27176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c5d9e4d3ae74d17b4027d97e8f5b01b" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394"
I20260709 13:10:07.847643 27176 raft_consensus.cc:3060] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:07.849313 27249 raft_consensus.cc:515] T 50d0e7c2eeb740c69d8235a9baee5564 P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.850816 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c5d9e4d3ae74d17b4027d97e8f5b01b" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "df70ca0962e140ba92abead00b2fc477"
I20260709 13:10:07.851850 27251 raft_consensus.cc:3055] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P df70ca0962e140ba92abead00b2fc477 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:07.852155 27251 raft_consensus.cc:740] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P df70ca0962e140ba92abead00b2fc477 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: df70ca0962e140ba92abead00b2fc477, State: Running, Role: LEADER
I20260709 13:10:07.852846 27251 consensus_queue.cc:260] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P df70ca0962e140ba92abead00b2fc477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.853976 27251 raft_consensus.cc:3060] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:07.860284 27251 raft_consensus.cc:2468] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 2.
I20260709 13:10:07.861064 27176 raft_consensus.cc:2468] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 2.
I20260709 13:10:07.861367 27289 leader_election.cc:304] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250, df70ca0962e140ba92abead00b2fc477; no voters: 
I20260709 13:10:07.862237 27401 raft_consensus.cc:2804] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:07.862710 27401 raft_consensus.cc:697] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 16f9474cee3e43c89e5d9e42266cb250 [term 2 LEADER]: Becoming Leader. State: Replica: 16f9474cee3e43c89e5d9e42266cb250, State: Running, Role: LEADER
I20260709 13:10:07.863514 27401 consensus_queue.cc:237] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 16f9474cee3e43c89e5d9e42266cb250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.864375 27174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d0e7c2eeb740c69d8235a9baee5564" candidate_uuid: "df70ca0962e140ba92abead00b2fc477" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394"
I20260709 13:10:07.864698 27323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50d0e7c2eeb740c69d8235a9baee5564" candidate_uuid: "df70ca0962e140ba92abead00b2fc477" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "16f9474cee3e43c89e5d9e42266cb250"
I20260709 13:10:07.865022 27174 raft_consensus.cc:3060] T 50d0e7c2eeb740c69d8235a9baee5564 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:07.865288 27323 raft_consensus.cc:3055] T 50d0e7c2eeb740c69d8235a9baee5564 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:07.865643 27323 raft_consensus.cc:740] T 50d0e7c2eeb740c69d8235a9baee5564 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 16f9474cee3e43c89e5d9e42266cb250, State: Running, Role: LEADER
I20260709 13:10:07.866417 27323 consensus_queue.cc:260] T 50d0e7c2eeb740c69d8235a9baee5564 P 16f9474cee3e43c89e5d9e42266cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.867491 27323 raft_consensus.cc:3060] T 50d0e7c2eeb740c69d8235a9baee5564 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:07.868345 27249 leader_election.cc:290] T 50d0e7c2eeb740c69d8235a9baee5564 P df70ca0962e140ba92abead00b2fc477 [CANDIDATE]: Term 2 election: Requested vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:07.872519 27174 raft_consensus.cc:2468] T 50d0e7c2eeb740c69d8235a9baee5564 P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df70ca0962e140ba92abead00b2fc477 in term 2.
I20260709 13:10:07.873754 27216 leader_election.cc:304] T 50d0e7c2eeb740c69d8235a9baee5564 P df70ca0962e140ba92abead00b2fc477 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e58e2a069aa48c59858cbadb8ab9394, df70ca0962e140ba92abead00b2fc477; no voters: 
I20260709 13:10:07.874266 27323 raft_consensus.cc:2468] T 50d0e7c2eeb740c69d8235a9baee5564 P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df70ca0962e140ba92abead00b2fc477 in term 2.
I20260709 13:10:07.874601 27496 raft_consensus.cc:2804] T 50d0e7c2eeb740c69d8235a9baee5564 P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:07.875173 27496 raft_consensus.cc:697] T 50d0e7c2eeb740c69d8235a9baee5564 P df70ca0962e140ba92abead00b2fc477 [term 2 LEADER]: Becoming Leader. State: Replica: df70ca0962e140ba92abead00b2fc477, State: Running, Role: LEADER
I20260709 13:10:07.875985 27496 consensus_queue.cc:237] T 50d0e7c2eeb740c69d8235a9baee5564 P df70ca0962e140ba92abead00b2fc477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:07.880594 27074 catalog_manager.cc:5697] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 16f9474cee3e43c89e5d9e42266cb250 reported cstate change: term changed from 1 to 2, leader changed from df70ca0962e140ba92abead00b2fc477 (127.14.103.2) to 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3). New cstate: current_term: 2 leader_uuid: "16f9474cee3e43c89e5d9e42266cb250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:07.888363 27070 catalog_manager.cc:5697] T 50d0e7c2eeb740c69d8235a9baee5564 P df70ca0962e140ba92abead00b2fc477 reported cstate change: term changed from 1 to 2, leader changed from 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3) to df70ca0962e140ba92abead00b2fc477 (127.14.103.2). New cstate: current_term: 2 leader_uuid: "df70ca0962e140ba92abead00b2fc477" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:07.901094 27496 consensus_queue.cc:1048] T 43b0b2e84a7f408fb83a416f9d669635 P df70ca0962e140ba92abead00b2fc477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:07.910715 27493 consensus_queue.cc:1048] T 43b0b2e84a7f408fb83a416f9d669635 P df70ca0962e140ba92abead00b2fc477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:08.142513 27249 raft_consensus.cc:1275] T 22270c8eaaa648f9b5cbe3125faa8c25 P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Refusing update from remote peer 3e58e2a069aa48c59858cbadb8ab9394: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:08.143815 27403 consensus_queue.cc:1048] T 22270c8eaaa648f9b5cbe3125faa8c25 P 3e58e2a069aa48c59858cbadb8ab9394 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:08.152179 27323 raft_consensus.cc:1275] T 22270c8eaaa648f9b5cbe3125faa8c25 P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Refusing update from remote peer 3e58e2a069aa48c59858cbadb8ab9394: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:08.153709 27481 consensus_queue.cc:1048] T 22270c8eaaa648f9b5cbe3125faa8c25 P 3e58e2a069aa48c59858cbadb8ab9394 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:08.163780 27484 raft_consensus.cc:993] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250: : Instructing follower 3e58e2a069aa48c59858cbadb8ab9394 to start an election
I20260709 13:10:08.164230 27401 raft_consensus.cc:1081] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 LEADER]: Signalling peer 3e58e2a069aa48c59858cbadb8ab9394 to start an election
I20260709 13:10:08.165763 27174 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9ebd3c36d25e44e293a5d318fa74b4a2"
dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394"
 from {username='slave'} at 127.0.0.1:36746
I20260709 13:10:08.168404 27174 raft_consensus.cc:493] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:08.168726 27174 raft_consensus.cc:3060] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:08.174351 27174 raft_consensus.cc:515] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:08.176204 27174 leader_election.cc:290] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 2 election: Requested vote from peers df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461), 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3:33233)
I20260709 13:10:08.176445 27249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ebd3c36d25e44e293a5d318fa74b4a2" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "df70ca0962e140ba92abead00b2fc477"
I20260709 13:10:08.176767 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ebd3c36d25e44e293a5d318fa74b4a2" candidate_uuid: "3e58e2a069aa48c59858cbadb8ab9394" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "16f9474cee3e43c89e5d9e42266cb250"
I20260709 13:10:08.177083 27249 raft_consensus.cc:3060] T 9ebd3c36d25e44e293a5d318fa74b4a2 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:08.177387 27325 raft_consensus.cc:3055] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:08.177724 27325 raft_consensus.cc:740] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 16f9474cee3e43c89e5d9e42266cb250, State: Running, Role: LEADER
I20260709 13:10:08.178514 27325 consensus_queue.cc:260] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:08.179762 27325 raft_consensus.cc:3060] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:08.183914 27249 raft_consensus.cc:2468] T 9ebd3c36d25e44e293a5d318fa74b4a2 P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e58e2a069aa48c59858cbadb8ab9394 in term 2.
I20260709 13:10:08.185050 27140 leader_election.cc:304] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 3e58e2a069aa48c59858cbadb8ab9394 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e58e2a069aa48c59858cbadb8ab9394, df70ca0962e140ba92abead00b2fc477; no voters: 
I20260709 13:10:08.185704 27481 raft_consensus.cc:2804] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:08.186043 27325 raft_consensus.cc:2468] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e58e2a069aa48c59858cbadb8ab9394 in term 2.
I20260709 13:10:08.186208 27481 raft_consensus.cc:697] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 LEADER]: Becoming Leader. State: Replica: 3e58e2a069aa48c59858cbadb8ab9394, State: Running, Role: LEADER
I20260709 13:10:08.187062 27481 consensus_queue.cc:237] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 3e58e2a069aa48c59858cbadb8ab9394 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:08.193194 27074 catalog_manager.cc:5697] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 3e58e2a069aa48c59858cbadb8ab9394 reported cstate change: term changed from 1 to 2, leader changed from 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3) to 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1). New cstate: current_term: 2 leader_uuid: "3e58e2a069aa48c59858cbadb8ab9394" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:08.374179 27174 raft_consensus.cc:1275] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Refusing update from remote peer 16f9474cee3e43c89e5d9e42266cb250: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:08.375349 27401 consensus_queue.cc:1048] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 16f9474cee3e43c89e5d9e42266cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:08.382486 27249 raft_consensus.cc:1275] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Refusing update from remote peer 16f9474cee3e43c89e5d9e42266cb250: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:08.383831 27484 consensus_queue.cc:1048] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 16f9474cee3e43c89e5d9e42266cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:08.399533 27174 raft_consensus.cc:1275] T 50d0e7c2eeb740c69d8235a9baee5564 P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Refusing update from remote peer df70ca0962e140ba92abead00b2fc477: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:08.400697 27493 consensus_queue.cc:1048] T 50d0e7c2eeb740c69d8235a9baee5564 P df70ca0962e140ba92abead00b2fc477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:08.408877 27323 raft_consensus.cc:1275] T 50d0e7c2eeb740c69d8235a9baee5564 P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Refusing update from remote peer df70ca0962e140ba92abead00b2fc477: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:08.410459 27466 consensus_queue.cc:1048] T 50d0e7c2eeb740c69d8235a9baee5564 P df70ca0962e140ba92abead00b2fc477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:08.602638 27249 raft_consensus.cc:1275] T 9ebd3c36d25e44e293a5d318fa74b4a2 P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Refusing update from remote peer 3e58e2a069aa48c59858cbadb8ab9394: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:08.603796 27460 consensus_queue.cc:1048] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 3e58e2a069aa48c59858cbadb8ab9394 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:08.611299 27325 raft_consensus.cc:1275] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Refusing update from remote peer 3e58e2a069aa48c59858cbadb8ab9394: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:08.614286 27531 consensus_queue.cc:1048] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 3e58e2a069aa48c59858cbadb8ab9394 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:10.661459 27074 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55752:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:10:10.684988 27074 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:55752:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:10:10.741019 14748 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:10:10.842115 27551 log.cc:927] Time spent T ac6f6dd4c5f7460bb60fc085a13a1a20 P 3e58e2a069aa48c59858cbadb8ab9394: Append to log took a long time: real 0.061s	user 0.005s	sys 0.000s
I20260709 13:10:10.876817 27574 tablet.cc:1724] T f0808b4c74f04ce28b19a78bf4c504a8 P 16f9474cee3e43c89e5d9e42266cb250: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:10.880023 27570 tablet.cc:1724] T f0808b4c74f04ce28b19a78bf4c504a8 P 3e58e2a069aa48c59858cbadb8ab9394: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:10:10.979414 27216 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.1:38667, user_credentials={real_user=slave}} blocked reactor thread for 81697.5us
I20260709 13:10:11.008876 27584 tablet.cc:1724] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:11.028106 27586 tablet.cc:1724] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 3e58e2a069aa48c59858cbadb8ab9394: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:11.049220 27597 tablet.cc:1724] T 9ebd3c36d25e44e293a5d318fa74b4a2 P df70ca0962e140ba92abead00b2fc477: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:11.043999 27589 tablet.cc:1724] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 3e58e2a069aa48c59858cbadb8ab9394: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:11.018047 27587 tablet.cc:1724] T 22270c8eaaa648f9b5cbe3125faa8c25 P 3e58e2a069aa48c59858cbadb8ab9394: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:11.042495 27590 tablet.cc:1724] T 22270c8eaaa648f9b5cbe3125faa8c25 P 16f9474cee3e43c89e5d9e42266cb250: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:11.043612 27595 tablet.cc:1724] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 16f9474cee3e43c89e5d9e42266cb250: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:11.128894 27604 tablet.cc:1724] T f53040cb502f490189c5be0990856a58 P df70ca0962e140ba92abead00b2fc477: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:11.133978 27589 tablet.cc:1724] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 3e58e2a069aa48c59858cbadb8ab9394: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:11.114709 27174 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "acb5efb1d0d2422ba071fc5bc49e6390"
dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394"
mode: GRACEFUL
new_leader_uuid: "16f9474cee3e43c89e5d9e42266cb250"
 from {username='slave'} at 127.0.0.1:39838
I20260709 13:10:11.141283 27174 raft_consensus.cc:606] T acb5efb1d0d2422ba071fc5bc49e6390 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 LEADER]: Received request to transfer leadership to TS 16f9474cee3e43c89e5d9e42266cb250
I20260709 13:10:11.144512 27587 tablet.cc:1724] T f53040cb502f490189c5be0990856a58 P 3e58e2a069aa48c59858cbadb8ab9394: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:11.150079 27574 tablet.cc:1724] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 16f9474cee3e43c89e5d9e42266cb250: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:11.164907 27603 tablet.cc:1724] T 819307e0735043d5b032193af4478edc P df70ca0962e140ba92abead00b2fc477: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:11.169976 27601 tablet.cc:1724] T ac6f6dd4c5f7460bb60fc085a13a1a20 P df70ca0962e140ba92abead00b2fc477: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:11.173669 27570 tablet.cc:1724] T 819307e0735043d5b032193af4478edc P 3e58e2a069aa48c59858cbadb8ab9394: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:11.175987 27561 raft_consensus.cc:993] T acb5efb1d0d2422ba071fc5bc49e6390 P 3e58e2a069aa48c59858cbadb8ab9394: : Instructing follower 16f9474cee3e43c89e5d9e42266cb250 to start an election
I20260709 13:10:11.176565 27561 raft_consensus.cc:1081] T acb5efb1d0d2422ba071fc5bc49e6390 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 LEADER]: Signalling peer 16f9474cee3e43c89e5d9e42266cb250 to start an election
I20260709 13:10:11.178498 27325 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "acb5efb1d0d2422ba071fc5bc49e6390"
dest_uuid: "16f9474cee3e43c89e5d9e42266cb250"
 from {username='slave'} at 127.0.0.1:48600
I20260709 13:10:11.179070 27325 raft_consensus.cc:493] T acb5efb1d0d2422ba071fc5bc49e6390 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:11.179425 27325 raft_consensus.cc:3060] T acb5efb1d0d2422ba071fc5bc49e6390 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:11.184967 27584 tablet.cc:1724] T f53040cb502f490189c5be0990856a58 P 16f9474cee3e43c89e5d9e42266cb250: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:11.192535 27325 raft_consensus.cc:515] T acb5efb1d0d2422ba071fc5bc49e6390 P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } }
I20260709 13:10:11.212214 27574 tablet.cc:1724] T 819307e0735043d5b032193af4478edc P 16f9474cee3e43c89e5d9e42266cb250: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:10:11.224373 27289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.103.2:34461, user_credentials={real_user=slave}} blocked reactor thread for 65780.6us
I20260709 13:10:11.226096 27603 tablet.cc:1724] T 43b0b2e84a7f408fb83a416f9d669635 P df70ca0962e140ba92abead00b2fc477: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:11.234395 27250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb5efb1d0d2422ba071fc5bc49e6390" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "df70ca0962e140ba92abead00b2fc477"
I20260709 13:10:11.229708 27325 leader_election.cc:290] T acb5efb1d0d2422ba071fc5bc49e6390 P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 2 election: Requested vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461)
I20260709 13:10:11.229954 27175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acb5efb1d0d2422ba071fc5bc49e6390" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394"
I20260709 13:10:11.236407 27175 raft_consensus.cc:3055] T acb5efb1d0d2422ba071fc5bc49e6390 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:11.236824 27175 raft_consensus.cc:740] T acb5efb1d0d2422ba071fc5bc49e6390 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3e58e2a069aa48c59858cbadb8ab9394, State: Running, Role: LEADER
I20260709 13:10:11.237713 27175 consensus_queue.cc:260] T acb5efb1d0d2422ba071fc5bc49e6390 P 3e58e2a069aa48c59858cbadb8ab9394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } }
I20260709 13:10:11.239007 27175 raft_consensus.cc:3060] T acb5efb1d0d2422ba071fc5bc49e6390 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:11.247367 27597 tablet.cc:1724] T 50d0e7c2eeb740c69d8235a9baee5564 P df70ca0962e140ba92abead00b2fc477: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:11.251850 27601 tablet.cc:1724] T f0808b4c74f04ce28b19a78bf4c504a8 P df70ca0962e140ba92abead00b2fc477: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:11.252566 27603 tablet.cc:1724] T 22270c8eaaa648f9b5cbe3125faa8c25 P df70ca0962e140ba92abead00b2fc477: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:11.247251 27590 tablet.cc:1724] T 50d0e7c2eeb740c69d8235a9baee5564 P 16f9474cee3e43c89e5d9e42266cb250: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:11.224140 27586 tablet.cc:1724] T 50d0e7c2eeb740c69d8235a9baee5564 P 3e58e2a069aa48c59858cbadb8ab9394: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:11.233661 27587 tablet.cc:1724] T 43b0b2e84a7f408fb83a416f9d669635 P 3e58e2a069aa48c59858cbadb8ab9394: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:11.231931 27595 tablet.cc:1724] T 43b0b2e84a7f408fb83a416f9d669635 P 16f9474cee3e43c89e5d9e42266cb250: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:11.235055 27250 raft_consensus.cc:3060] T acb5efb1d0d2422ba071fc5bc49e6390 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:11.268923 27250 raft_consensus.cc:2468] T acb5efb1d0d2422ba071fc5bc49e6390 P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 2.
I20260709 13:10:11.269863 27175 raft_consensus.cc:2468] T acb5efb1d0d2422ba071fc5bc49e6390 P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 2.
I20260709 13:10:11.270344 27289 leader_election.cc:304] T acb5efb1d0d2422ba071fc5bc49e6390 P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250, df70ca0962e140ba92abead00b2fc477; no voters: 
I20260709 13:10:11.271986 27607 raft_consensus.cc:2804] T acb5efb1d0d2422ba071fc5bc49e6390 P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:11.272663 27607 raft_consensus.cc:697] T acb5efb1d0d2422ba071fc5bc49e6390 P 16f9474cee3e43c89e5d9e42266cb250 [term 2 LEADER]: Becoming Leader. State: Replica: 16f9474cee3e43c89e5d9e42266cb250, State: Running, Role: LEADER
I20260709 13:10:11.273618 27601 tablet.cc:1724] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P df70ca0962e140ba92abead00b2fc477: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:10:11.275837 27607 consensus_queue.cc:237] T acb5efb1d0d2422ba071fc5bc49e6390 P 16f9474cee3e43c89e5d9e42266cb250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } }
I20260709 13:10:11.281605 27250 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e184a0b144684ada99885c4811605036"
dest_uuid: "df70ca0962e140ba92abead00b2fc477"
mode: GRACEFUL
new_leader_uuid: "16f9474cee3e43c89e5d9e42266cb250"
 from {username='slave'} at 127.0.0.1:54118
I20260709 13:10:11.282274 27250 raft_consensus.cc:606] T e184a0b144684ada99885c4811605036 P df70ca0962e140ba92abead00b2fc477 [term 2 LEADER]: Received request to transfer leadership to TS 16f9474cee3e43c89e5d9e42266cb250
I20260709 13:10:11.283823 14748 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:10:11.284451 14748 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:11.314435 27070 catalog_manager.cc:5697] T acb5efb1d0d2422ba071fc5bc49e6390 P 16f9474cee3e43c89e5d9e42266cb250 reported cstate change: term changed from 1 to 2, leader changed from 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1) to 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3). New cstate: current_term: 2 leader_uuid: "16f9474cee3e43c89e5d9e42266cb250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:11.319705 27072 catalog_manager.cc:6242] T 00000000000000000000000000000000 P dcdd1fd6000d46a4b7bd49014ad3e41f: soft_deleted_table [id=330588081cce4a9586227b833c776c10] alter complete (version 1)
I20260709 13:10:11.349035 27536 raft_consensus.cc:993] T e184a0b144684ada99885c4811605036 P df70ca0962e140ba92abead00b2fc477: : Instructing follower 16f9474cee3e43c89e5d9e42266cb250 to start an election
I20260709 13:10:11.349589 27581 raft_consensus.cc:1081] T e184a0b144684ada99885c4811605036 P df70ca0962e140ba92abead00b2fc477 [term 2 LEADER]: Signalling peer 16f9474cee3e43c89e5d9e42266cb250 to start an election
I20260709 13:10:11.350997 27325 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e184a0b144684ada99885c4811605036"
dest_uuid: "16f9474cee3e43c89e5d9e42266cb250"
 from {username='slave'} at 127.0.0.1:48598
I20260709 13:10:11.351425 27325 raft_consensus.cc:493] T e184a0b144684ada99885c4811605036 P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:11.351689 27325 raft_consensus.cc:3060] T e184a0b144684ada99885c4811605036 P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:11.356271 27325 raft_consensus.cc:515] T e184a0b144684ada99885c4811605036 P 16f9474cee3e43c89e5d9e42266cb250 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:11.357820 27325 leader_election.cc:290] T e184a0b144684ada99885c4811605036 P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 3 election: Requested vote from peers 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667), df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461)
I20260709 13:10:11.358683 27175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e184a0b144684ada99885c4811605036" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3e58e2a069aa48c59858cbadb8ab9394"
I20260709 13:10:11.358889 27250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e184a0b144684ada99885c4811605036" candidate_uuid: "16f9474cee3e43c89e5d9e42266cb250" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "df70ca0962e140ba92abead00b2fc477"
I20260709 13:10:11.359339 27175 raft_consensus.cc:3060] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:11.359529 27250 raft_consensus.cc:3055] T e184a0b144684ada99885c4811605036 P df70ca0962e140ba92abead00b2fc477 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:11.359988 27250 raft_consensus.cc:740] T e184a0b144684ada99885c4811605036 P df70ca0962e140ba92abead00b2fc477 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: df70ca0962e140ba92abead00b2fc477, State: Running, Role: LEADER
I20260709 13:10:11.360776 27250 consensus_queue.cc:260] T e184a0b144684ada99885c4811605036 P df70ca0962e140ba92abead00b2fc477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:11.361938 27250 raft_consensus.cc:3060] T e184a0b144684ada99885c4811605036 P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:11.365429 27175 raft_consensus.cc:2468] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 3.
I20260709 13:10:11.366683 27290 leader_election.cc:304] T e184a0b144684ada99885c4811605036 P 16f9474cee3e43c89e5d9e42266cb250 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16f9474cee3e43c89e5d9e42266cb250, 3e58e2a069aa48c59858cbadb8ab9394; no voters: 
I20260709 13:10:11.366765 27250 raft_consensus.cc:2468] T e184a0b144684ada99885c4811605036 P df70ca0962e140ba92abead00b2fc477 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16f9474cee3e43c89e5d9e42266cb250 in term 3.
I20260709 13:10:11.367447 27607 raft_consensus.cc:2804] T e184a0b144684ada99885c4811605036 P 16f9474cee3e43c89e5d9e42266cb250 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:11.367997 27607 raft_consensus.cc:697] T e184a0b144684ada99885c4811605036 P 16f9474cee3e43c89e5d9e42266cb250 [term 3 LEADER]: Becoming Leader. State: Replica: 16f9474cee3e43c89e5d9e42266cb250, State: Running, Role: LEADER
I20260709 13:10:11.368824 27607 consensus_queue.cc:237] T e184a0b144684ada99885c4811605036 P 16f9474cee3e43c89e5d9e42266cb250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } }
I20260709 13:10:11.375877 27072 catalog_manager.cc:5697] T e184a0b144684ada99885c4811605036 P 16f9474cee3e43c89e5d9e42266cb250 reported cstate change: term changed from 2 to 3, leader changed from df70ca0962e140ba92abead00b2fc477 (127.14.103.2) to 16f9474cee3e43c89e5d9e42266cb250 (127.14.103.3). New cstate: current_term: 3 leader_uuid: "16f9474cee3e43c89e5d9e42266cb250" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16f9474cee3e43c89e5d9e42266cb250" member_type: VOTER last_known_addr { host: "127.14.103.3" port: 33233 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:11.689640 27250 raft_consensus.cc:1275] T acb5efb1d0d2422ba071fc5bc49e6390 P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Refusing update from remote peer 16f9474cee3e43c89e5d9e42266cb250: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:11.690872 27607 consensus_queue.cc:1048] T acb5efb1d0d2422ba071fc5bc49e6390 P 16f9474cee3e43c89e5d9e42266cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:11.702739 27175 raft_consensus.cc:1275] T acb5efb1d0d2422ba071fc5bc49e6390 P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Refusing update from remote peer 16f9474cee3e43c89e5d9e42266cb250: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:11.704427 27602 consensus_queue.cc:1048] T acb5efb1d0d2422ba071fc5bc49e6390 P 16f9474cee3e43c89e5d9e42266cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:11.772300 27250 raft_consensus.cc:1275] T e184a0b144684ada99885c4811605036 P df70ca0962e140ba92abead00b2fc477 [term 3 FOLLOWER]: Refusing update from remote peer 16f9474cee3e43c89e5d9e42266cb250: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:11.773478 27602 consensus_queue.cc:1048] T e184a0b144684ada99885c4811605036 P 16f9474cee3e43c89e5d9e42266cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df70ca0962e140ba92abead00b2fc477" member_type: VOTER last_known_addr { host: "127.14.103.2" port: 34461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:11.782150 27175 raft_consensus.cc:1275] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394 [term 3 FOLLOWER]: Refusing update from remote peer 16f9474cee3e43c89e5d9e42266cb250: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:11.783878 27605 consensus_queue.cc:1048] T e184a0b144684ada99885c4811605036 P 16f9474cee3e43c89e5d9e42266cb250 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e58e2a069aa48c59858cbadb8ab9394" member_type: VOTER last_known_addr { host: "127.14.103.1" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:12.345570 14748 tablet_server.cc:179] TabletServer@127.14.103.1:0 shutting down...
W20260709 13:10:12.368947 27290 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 12 similar messages]
W20260709 13:10:12.386610 27290 consensus_peers.cc:597] T e184a0b144684ada99885c4811605036 P 16f9474cee3e43c89e5d9e42266cb250 -> Peer 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667): Couldn't send request to peer 3e58e2a069aa48c59858cbadb8ab9394. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:12.388849 14748 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:12.389887 14748 tablet_replica.cc:333] T 43b0b2e84a7f408fb83a416f9d669635 P 3e58e2a069aa48c59858cbadb8ab9394: stopping tablet replica
I20260709 13:10:12.390955 14748 raft_consensus.cc:2243] T 43b0b2e84a7f408fb83a416f9d669635 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:12.391662 14748 raft_consensus.cc:2272] T 43b0b2e84a7f408fb83a416f9d669635 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.394083 14748 tablet_replica.cc:333] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 3e58e2a069aa48c59858cbadb8ab9394: stopping tablet replica
I20260709 13:10:12.394726 14748 raft_consensus.cc:2243] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:12.395198 14748 raft_consensus.cc:2272] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.398165 14748 tablet_replica.cc:333] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 3e58e2a069aa48c59858cbadb8ab9394: stopping tablet replica
I20260709 13:10:12.398754 14748 raft_consensus.cc:2243] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:12.399585 14748 raft_consensus.cc:2272] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.401620 14748 tablet_replica.cc:333] T f0808b4c74f04ce28b19a78bf4c504a8 P 3e58e2a069aa48c59858cbadb8ab9394: stopping tablet replica
I20260709 13:10:12.402156 14748 raft_consensus.cc:2243] T f0808b4c74f04ce28b19a78bf4c504a8 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:12.402835 14748 raft_consensus.cc:2272] T f0808b4c74f04ce28b19a78bf4c504a8 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.405049 14748 tablet_replica.cc:333] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 3e58e2a069aa48c59858cbadb8ab9394: stopping tablet replica
I20260709 13:10:12.405704 14748 raft_consensus.cc:2243] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:12.406632 14748 raft_consensus.cc:2272] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.409243 14748 tablet_replica.cc:333] T 22270c8eaaa648f9b5cbe3125faa8c25 P 3e58e2a069aa48c59858cbadb8ab9394: stopping tablet replica
I20260709 13:10:12.409915 14748 raft_consensus.cc:2243] T 22270c8eaaa648f9b5cbe3125faa8c25 P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:12.410719 14748 raft_consensus.cc:2272] T 22270c8eaaa648f9b5cbe3125faa8c25 P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.412849 14748 tablet_replica.cc:333] T f53040cb502f490189c5be0990856a58 P 3e58e2a069aa48c59858cbadb8ab9394: stopping tablet replica
I20260709 13:10:12.413424 14748 raft_consensus.cc:2243] T f53040cb502f490189c5be0990856a58 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:12.413858 14748 raft_consensus.cc:2272] T f53040cb502f490189c5be0990856a58 P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.416028 14748 tablet_replica.cc:333] T a51060799a3243a881c471fc62688815 P 3e58e2a069aa48c59858cbadb8ab9394: stopping tablet replica
I20260709 13:10:12.416601 14748 raft_consensus.cc:2243] T a51060799a3243a881c471fc62688815 P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:12.417309 14748 raft_consensus.cc:2272] T a51060799a3243a881c471fc62688815 P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.420466 14748 tablet_replica.cc:333] T 74072bd16e8d49749ab1f70a27f71bea P 3e58e2a069aa48c59858cbadb8ab9394: stopping tablet replica
I20260709 13:10:12.421097 14748 raft_consensus.cc:2243] T 74072bd16e8d49749ab1f70a27f71bea P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:12.421576 14748 raft_consensus.cc:2272] T 74072bd16e8d49749ab1f70a27f71bea P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.423591 14748 tablet_replica.cc:333] T 7d767accb08c4a38a74ae205bc54d739 P 3e58e2a069aa48c59858cbadb8ab9394: stopping tablet replica
I20260709 13:10:12.424181 14748 raft_consensus.cc:2243] T 7d767accb08c4a38a74ae205bc54d739 P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:12.424621 14748 raft_consensus.cc:2272] T 7d767accb08c4a38a74ae205bc54d739 P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.426637 14748 tablet_replica.cc:333] T acb5efb1d0d2422ba071fc5bc49e6390 P 3e58e2a069aa48c59858cbadb8ab9394: stopping tablet replica
I20260709 13:10:12.427330 14748 raft_consensus.cc:2243] T acb5efb1d0d2422ba071fc5bc49e6390 P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:12.427765 14748 raft_consensus.cc:2272] T acb5efb1d0d2422ba071fc5bc49e6390 P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.430565 14748 tablet_replica.cc:333] T 7a551aeaa560472abebcd6cc7d456faf P 3e58e2a069aa48c59858cbadb8ab9394: stopping tablet replica
I20260709 13:10:12.431121 14748 raft_consensus.cc:2243] T 7a551aeaa560472abebcd6cc7d456faf P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:12.431587 14748 raft_consensus.cc:2272] T 7a551aeaa560472abebcd6cc7d456faf P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.433670 14748 tablet_replica.cc:333] T 50d0e7c2eeb740c69d8235a9baee5564 P 3e58e2a069aa48c59858cbadb8ab9394: stopping tablet replica
I20260709 13:10:12.434165 14748 raft_consensus.cc:2243] T 50d0e7c2eeb740c69d8235a9baee5564 P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:12.434619 14748 raft_consensus.cc:2272] T 50d0e7c2eeb740c69d8235a9baee5564 P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.436847 14748 tablet_replica.cc:333] T 094982455ad24795a7cf35af53248051 P 3e58e2a069aa48c59858cbadb8ab9394: stopping tablet replica
I20260709 13:10:12.437381 14748 raft_consensus.cc:2243] T 094982455ad24795a7cf35af53248051 P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:12.438378 14748 raft_consensus.cc:2272] T 094982455ad24795a7cf35af53248051 P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.440529 14748 tablet_replica.cc:333] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394: stopping tablet replica
I20260709 13:10:12.441035 14748 raft_consensus.cc:2243] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:12.441490 14748 raft_consensus.cc:2272] T e184a0b144684ada99885c4811605036 P 3e58e2a069aa48c59858cbadb8ab9394 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.444916 14748 tablet_replica.cc:333] T 819307e0735043d5b032193af4478edc P 3e58e2a069aa48c59858cbadb8ab9394: stopping tablet replica
I20260709 13:10:12.445477 14748 raft_consensus.cc:2243] T 819307e0735043d5b032193af4478edc P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:12.445942 14748 raft_consensus.cc:2272] T 819307e0735043d5b032193af4478edc P 3e58e2a069aa48c59858cbadb8ab9394 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.448308 14748 tablet_replica.cc:333] T 713697e046ae4150a4047a4bf2c1180f P 3e58e2a069aa48c59858cbadb8ab9394: stopping tablet replica
I20260709 13:10:12.448820 14748 raft_consensus.cc:2243] T 713697e046ae4150a4047a4bf2c1180f P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:12.449596 14748 raft_consensus.cc:2272] T 713697e046ae4150a4047a4bf2c1180f P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.451606 14748 tablet_replica.cc:333] T f49bdf47ae7b41369b04e42ba388c1f1 P 3e58e2a069aa48c59858cbadb8ab9394: stopping tablet replica
I20260709 13:10:12.452080 14748 raft_consensus.cc:2243] T f49bdf47ae7b41369b04e42ba388c1f1 P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:12.452844 14748 raft_consensus.cc:2272] T f49bdf47ae7b41369b04e42ba388c1f1 P 3e58e2a069aa48c59858cbadb8ab9394 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.492333 14748 tablet_server.cc:196] TabletServer@127.14.103.1:0 shutdown complete.
W20260709 13:10:12.531900 27216 consensus_peers.cc:597] T 7d767accb08c4a38a74ae205bc54d739 P df70ca0962e140ba92abead00b2fc477 -> Peer 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667): Couldn't send request to peer 3e58e2a069aa48c59858cbadb8ab9394. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38667: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:12.532269 14748 tablet_server.cc:179] TabletServer@127.14.103.2:0 shutting down...
W20260709 13:10:12.542989 27290 consensus_peers.cc:597] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 16f9474cee3e43c89e5d9e42266cb250 -> Peer 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667): Couldn't send request to peer 3e58e2a069aa48c59858cbadb8ab9394. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38667: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:12.561632 27216 consensus_peers.cc:597] T a51060799a3243a881c471fc62688815 P df70ca0962e140ba92abead00b2fc477 -> Peer 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667): Couldn't send request to peer 3e58e2a069aa48c59858cbadb8ab9394. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38667: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:12.569310 14748 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:12.570356 14748 tablet_replica.cc:333] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P df70ca0962e140ba92abead00b2fc477: stopping tablet replica
I20260709 13:10:12.571321 14748 raft_consensus.cc:2243] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:12.571938 14748 raft_consensus.cc:2272] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.574700 14748 tablet_replica.cc:333] T 43b0b2e84a7f408fb83a416f9d669635 P df70ca0962e140ba92abead00b2fc477: stopping tablet replica
W20260709 13:10:12.575361 27216 consensus_peers.cc:597] T f53040cb502f490189c5be0990856a58 P df70ca0962e140ba92abead00b2fc477 -> Peer 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667): Couldn't send request to peer 3e58e2a069aa48c59858cbadb8ab9394. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38667: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:12.575500 14748 raft_consensus.cc:2243] T 43b0b2e84a7f408fb83a416f9d669635 P df70ca0962e140ba92abead00b2fc477 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:12.576511 14748 raft_consensus.cc:2272] T 43b0b2e84a7f408fb83a416f9d669635 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.578500 14748 tablet_replica.cc:333] T 9ebd3c36d25e44e293a5d318fa74b4a2 P df70ca0962e140ba92abead00b2fc477: stopping tablet replica
I20260709 13:10:12.579089 14748 raft_consensus.cc:2243] T 9ebd3c36d25e44e293a5d318fa74b4a2 P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:12.579517 14748 raft_consensus.cc:2272] T 9ebd3c36d25e44e293a5d318fa74b4a2 P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.581444 14748 tablet_replica.cc:333] T f0808b4c74f04ce28b19a78bf4c504a8 P df70ca0962e140ba92abead00b2fc477: stopping tablet replica
I20260709 13:10:12.581979 14748 raft_consensus.cc:2243] T f0808b4c74f04ce28b19a78bf4c504a8 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:12.582413 14748 raft_consensus.cc:2272] T f0808b4c74f04ce28b19a78bf4c504a8 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.584290 14748 tablet_replica.cc:333] T ac6f6dd4c5f7460bb60fc085a13a1a20 P df70ca0962e140ba92abead00b2fc477: stopping tablet replica
I20260709 13:10:12.584870 14748 raft_consensus.cc:2243] T ac6f6dd4c5f7460bb60fc085a13a1a20 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:12.585291 14748 raft_consensus.cc:2272] T ac6f6dd4c5f7460bb60fc085a13a1a20 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.587173 14748 tablet_replica.cc:333] T 22270c8eaaa648f9b5cbe3125faa8c25 P df70ca0962e140ba92abead00b2fc477: stopping tablet replica
I20260709 13:10:12.587754 14748 raft_consensus.cc:2243] T 22270c8eaaa648f9b5cbe3125faa8c25 P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:12.588184 14748 raft_consensus.cc:2272] T 22270c8eaaa648f9b5cbe3125faa8c25 P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.590080 14748 tablet_replica.cc:333] T f53040cb502f490189c5be0990856a58 P df70ca0962e140ba92abead00b2fc477: stopping tablet replica
I20260709 13:10:12.590773 14748 raft_consensus.cc:2243] T f53040cb502f490189c5be0990856a58 P df70ca0962e140ba92abead00b2fc477 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:12.591646 14748 raft_consensus.cc:2272] T f53040cb502f490189c5be0990856a58 P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.594163 14748 tablet_replica.cc:333] T a51060799a3243a881c471fc62688815 P df70ca0962e140ba92abead00b2fc477: stopping tablet replica
I20260709 13:10:12.594975 14748 raft_consensus.cc:2243] T a51060799a3243a881c471fc62688815 P df70ca0962e140ba92abead00b2fc477 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:10:12.595697 27216 consensus_peers.cc:597] T 7a551aeaa560472abebcd6cc7d456faf P df70ca0962e140ba92abead00b2fc477 -> Peer 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667): Couldn't send request to peer 3e58e2a069aa48c59858cbadb8ab9394. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38667: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:12.596060 14748 raft_consensus.cc:2272] T a51060799a3243a881c471fc62688815 P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.598667 14748 tablet_replica.cc:333] T 74072bd16e8d49749ab1f70a27f71bea P df70ca0962e140ba92abead00b2fc477: stopping tablet replica
I20260709 13:10:12.599337 14748 raft_consensus.cc:2243] T 74072bd16e8d49749ab1f70a27f71bea P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:12.599887 14748 raft_consensus.cc:2272] T 74072bd16e8d49749ab1f70a27f71bea P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.602545 14748 tablet_replica.cc:333] T 7d767accb08c4a38a74ae205bc54d739 P df70ca0962e140ba92abead00b2fc477: stopping tablet replica
W20260709 13:10:12.602720 27216 consensus_peers.cc:597] T 819307e0735043d5b032193af4478edc P df70ca0962e140ba92abead00b2fc477 -> Peer 3e58e2a069aa48c59858cbadb8ab9394 (127.14.103.1:38667): Couldn't send request to peer 3e58e2a069aa48c59858cbadb8ab9394. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.1:38667: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:12.603252 14748 raft_consensus.cc:2243] T 7d767accb08c4a38a74ae205bc54d739 P df70ca0962e140ba92abead00b2fc477 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:12.604169 14748 raft_consensus.cc:2272] T 7d767accb08c4a38a74ae205bc54d739 P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.606170 14748 tablet_replica.cc:333] T 7a551aeaa560472abebcd6cc7d456faf P df70ca0962e140ba92abead00b2fc477: stopping tablet replica
I20260709 13:10:12.606740 14748 raft_consensus.cc:2243] T 7a551aeaa560472abebcd6cc7d456faf P df70ca0962e140ba92abead00b2fc477 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:12.607510 14748 raft_consensus.cc:2272] T 7a551aeaa560472abebcd6cc7d456faf P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.609622 14748 tablet_replica.cc:333] T acb5efb1d0d2422ba071fc5bc49e6390 P df70ca0962e140ba92abead00b2fc477: stopping tablet replica
I20260709 13:10:12.610163 14748 raft_consensus.cc:2243] T acb5efb1d0d2422ba071fc5bc49e6390 P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:12.610622 14748 raft_consensus.cc:2272] T acb5efb1d0d2422ba071fc5bc49e6390 P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.613543 14748 tablet_replica.cc:333] T 50d0e7c2eeb740c69d8235a9baee5564 P df70ca0962e140ba92abead00b2fc477: stopping tablet replica
I20260709 13:10:12.614084 14748 raft_consensus.cc:2243] T 50d0e7c2eeb740c69d8235a9baee5564 P df70ca0962e140ba92abead00b2fc477 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:12.614842 14748 raft_consensus.cc:2272] T 50d0e7c2eeb740c69d8235a9baee5564 P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.616854 14748 tablet_replica.cc:333] T 094982455ad24795a7cf35af53248051 P df70ca0962e140ba92abead00b2fc477: stopping tablet replica
I20260709 13:10:12.617342 14748 raft_consensus.cc:2243] T 094982455ad24795a7cf35af53248051 P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:12.617775 14748 raft_consensus.cc:2272] T 094982455ad24795a7cf35af53248051 P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.619817 14748 tablet_replica.cc:333] T 819307e0735043d5b032193af4478edc P df70ca0962e140ba92abead00b2fc477: stopping tablet replica
I20260709 13:10:12.620308 14748 raft_consensus.cc:2243] T 819307e0735043d5b032193af4478edc P df70ca0962e140ba92abead00b2fc477 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:12.621057 14748 raft_consensus.cc:2272] T 819307e0735043d5b032193af4478edc P df70ca0962e140ba92abead00b2fc477 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.622972 14748 tablet_replica.cc:333] T e184a0b144684ada99885c4811605036 P df70ca0962e140ba92abead00b2fc477: stopping tablet replica
I20260709 13:10:12.623472 14748 raft_consensus.cc:2243] T e184a0b144684ada99885c4811605036 P df70ca0962e140ba92abead00b2fc477 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:12.623888 14748 raft_consensus.cc:2272] T e184a0b144684ada99885c4811605036 P df70ca0962e140ba92abead00b2fc477 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.626713 14748 tablet_replica.cc:333] T 713697e046ae4150a4047a4bf2c1180f P df70ca0962e140ba92abead00b2fc477: stopping tablet replica
I20260709 13:10:12.627228 14748 raft_consensus.cc:2243] T 713697e046ae4150a4047a4bf2c1180f P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:12.627795 14748 raft_consensus.cc:2272] T 713697e046ae4150a4047a4bf2c1180f P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.629771 14748 tablet_replica.cc:333] T f49bdf47ae7b41369b04e42ba388c1f1 P df70ca0962e140ba92abead00b2fc477: stopping tablet replica
I20260709 13:10:12.630250 14748 raft_consensus.cc:2243] T f49bdf47ae7b41369b04e42ba388c1f1 P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:12.630699 14748 raft_consensus.cc:2272] T f49bdf47ae7b41369b04e42ba388c1f1 P df70ca0962e140ba92abead00b2fc477 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:12.643159 27289 consensus_peers.cc:597] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 16f9474cee3e43c89e5d9e42266cb250 -> Peer df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461): Couldn't send request to peer df70ca0962e140ba92abead00b2fc477. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:12.672868 14748 tablet_server.cc:196] TabletServer@127.14.103.2:0 shutdown complete.
I20260709 13:10:12.715309 14748 tablet_server.cc:179] TabletServer@127.14.103.3:0 shutting down...
W20260709 13:10:12.739852 27289 consensus_peers.cc:597] T acb5efb1d0d2422ba071fc5bc49e6390 P 16f9474cee3e43c89e5d9e42266cb250 -> Peer df70ca0962e140ba92abead00b2fc477 (127.14.103.2:34461): Couldn't send request to peer df70ca0962e140ba92abead00b2fc477. Status: Network error: Client connection negotiation failed: client connection to 127.14.103.2:34461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:12.757684 14748 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:12.758410 14748 tablet_replica.cc:333] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 16f9474cee3e43c89e5d9e42266cb250: stopping tablet replica
I20260709 13:10:12.759172 14748 raft_consensus.cc:2243] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 16f9474cee3e43c89e5d9e42266cb250 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:12.760092 14748 raft_consensus.cc:2272] T 4c5d9e4d3ae74d17b4027d97e8f5b01b P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.762332 14748 tablet_replica.cc:333] T 43b0b2e84a7f408fb83a416f9d669635 P 16f9474cee3e43c89e5d9e42266cb250: stopping tablet replica
I20260709 13:10:12.762928 14748 raft_consensus.cc:2243] T 43b0b2e84a7f408fb83a416f9d669635 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:12.763366 14748 raft_consensus.cc:2272] T 43b0b2e84a7f408fb83a416f9d669635 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.765415 14748 tablet_replica.cc:333] T f0808b4c74f04ce28b19a78bf4c504a8 P 16f9474cee3e43c89e5d9e42266cb250: stopping tablet replica
I20260709 13:10:12.765985 14748 raft_consensus.cc:2243] T f0808b4c74f04ce28b19a78bf4c504a8 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:12.766424 14748 raft_consensus.cc:2272] T f0808b4c74f04ce28b19a78bf4c504a8 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.768350 14748 tablet_replica.cc:333] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 16f9474cee3e43c89e5d9e42266cb250: stopping tablet replica
I20260709 13:10:12.768903 14748 raft_consensus.cc:2243] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:12.769342 14748 raft_consensus.cc:2272] T ac6f6dd4c5f7460bb60fc085a13a1a20 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.771145 14748 tablet_replica.cc:333] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250: stopping tablet replica
I20260709 13:10:12.771777 14748 raft_consensus.cc:2243] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:12.772204 14748 raft_consensus.cc:2272] T 9ebd3c36d25e44e293a5d318fa74b4a2 P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.773995 14748 tablet_replica.cc:333] T 22270c8eaaa648f9b5cbe3125faa8c25 P 16f9474cee3e43c89e5d9e42266cb250: stopping tablet replica
I20260709 13:10:12.774524 14748 raft_consensus.cc:2243] T 22270c8eaaa648f9b5cbe3125faa8c25 P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:12.774962 14748 raft_consensus.cc:2272] T 22270c8eaaa648f9b5cbe3125faa8c25 P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.776897 14748 tablet_replica.cc:333] T f53040cb502f490189c5be0990856a58 P 16f9474cee3e43c89e5d9e42266cb250: stopping tablet replica
I20260709 13:10:12.777433 14748 raft_consensus.cc:2243] T f53040cb502f490189c5be0990856a58 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:12.777858 14748 raft_consensus.cc:2272] T f53040cb502f490189c5be0990856a58 P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.779574 14748 tablet_replica.cc:333] T a51060799a3243a881c471fc62688815 P 16f9474cee3e43c89e5d9e42266cb250: stopping tablet replica
I20260709 13:10:12.780159 14748 raft_consensus.cc:2243] T a51060799a3243a881c471fc62688815 P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:12.780663 14748 raft_consensus.cc:2272] T a51060799a3243a881c471fc62688815 P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.782835 14748 tablet_replica.cc:333] T 74072bd16e8d49749ab1f70a27f71bea P 16f9474cee3e43c89e5d9e42266cb250: stopping tablet replica
I20260709 13:10:12.783342 14748 raft_consensus.cc:2243] T 74072bd16e8d49749ab1f70a27f71bea P 16f9474cee3e43c89e5d9e42266cb250 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:12.784215 14748 raft_consensus.cc:2272] T 74072bd16e8d49749ab1f70a27f71bea P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.786020 14748 tablet_replica.cc:333] T f49bdf47ae7b41369b04e42ba388c1f1 P 16f9474cee3e43c89e5d9e42266cb250: stopping tablet replica
I20260709 13:10:12.786553 14748 raft_consensus.cc:2243] T f49bdf47ae7b41369b04e42ba388c1f1 P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:12.786995 14748 raft_consensus.cc:2272] T f49bdf47ae7b41369b04e42ba388c1f1 P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.788960 14748 tablet_replica.cc:333] T 7d767accb08c4a38a74ae205bc54d739 P 16f9474cee3e43c89e5d9e42266cb250: stopping tablet replica
I20260709 13:10:12.789479 14748 raft_consensus.cc:2243] T 7d767accb08c4a38a74ae205bc54d739 P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:12.789912 14748 raft_consensus.cc:2272] T 7d767accb08c4a38a74ae205bc54d739 P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.791810 14748 tablet_replica.cc:333] T 713697e046ae4150a4047a4bf2c1180f P 16f9474cee3e43c89e5d9e42266cb250: stopping tablet replica
I20260709 13:10:12.792302 14748 raft_consensus.cc:2243] T 713697e046ae4150a4047a4bf2c1180f P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:12.792774 14748 raft_consensus.cc:2272] T 713697e046ae4150a4047a4bf2c1180f P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.794770 14748 tablet_replica.cc:333] T 819307e0735043d5b032193af4478edc P 16f9474cee3e43c89e5d9e42266cb250: stopping tablet replica
I20260709 13:10:12.795286 14748 raft_consensus.cc:2243] T 819307e0735043d5b032193af4478edc P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:12.795744 14748 raft_consensus.cc:2272] T 819307e0735043d5b032193af4478edc P 16f9474cee3e43c89e5d9e42266cb250 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.797578 14748 tablet_replica.cc:333] T e184a0b144684ada99885c4811605036 P 16f9474cee3e43c89e5d9e42266cb250: stopping tablet replica
I20260709 13:10:12.798115 14748 raft_consensus.cc:2243] T e184a0b144684ada99885c4811605036 P 16f9474cee3e43c89e5d9e42266cb250 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:10:12.798864 14748 raft_consensus.cc:2272] T e184a0b144684ada99885c4811605036 P 16f9474cee3e43c89e5d9e42266cb250 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.800694 14748 tablet_replica.cc:333] T 50d0e7c2eeb740c69d8235a9baee5564 P 16f9474cee3e43c89e5d9e42266cb250: stopping tablet replica
I20260709 13:10:12.801216 14748 raft_consensus.cc:2243] T 50d0e7c2eeb740c69d8235a9baee5564 P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:12.801647 14748 raft_consensus.cc:2272] T 50d0e7c2eeb740c69d8235a9baee5564 P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.803478 14748 tablet_replica.cc:333] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250: stopping tablet replica
I20260709 13:10:12.803967 14748 raft_consensus.cc:2243] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:12.804415 14748 raft_consensus.cc:2272] T 094982455ad24795a7cf35af53248051 P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.806319 14748 tablet_replica.cc:333] T 7a551aeaa560472abebcd6cc7d456faf P 16f9474cee3e43c89e5d9e42266cb250: stopping tablet replica
I20260709 13:10:12.806769 14748 raft_consensus.cc:2243] T 7a551aeaa560472abebcd6cc7d456faf P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:12.807197 14748 raft_consensus.cc:2272] T 7a551aeaa560472abebcd6cc7d456faf P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.809048 14748 tablet_replica.cc:333] T acb5efb1d0d2422ba071fc5bc49e6390 P 16f9474cee3e43c89e5d9e42266cb250: stopping tablet replica
I20260709 13:10:12.809515 14748 raft_consensus.cc:2243] T acb5efb1d0d2422ba071fc5bc49e6390 P 16f9474cee3e43c89e5d9e42266cb250 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:12.810361 14748 raft_consensus.cc:2272] T acb5efb1d0d2422ba071fc5bc49e6390 P 16f9474cee3e43c89e5d9e42266cb250 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.850871 14748 tablet_server.cc:196] TabletServer@127.14.103.3:0 shutdown complete.
I20260709 13:10:12.888592 14748 master.cc:562] Master@127.14.103.62:44831 shutting down...
I20260709 13:10:12.916955 14748 raft_consensus.cc:2243] T 00000000000000000000000000000000 P dcdd1fd6000d46a4b7bd49014ad3e41f [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:12.917680 14748 raft_consensus.cc:2272] T 00000000000000000000000000000000 P dcdd1fd6000d46a4b7bd49014ad3e41f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:12.918087 14748 tablet_replica.cc:333] T 00000000000000000000000000000000 P dcdd1fd6000d46a4b7bd49014ad3e41f: stopping tablet replica
I20260709 13:10:12.939450 14748 master.cc:584] Master@127.14.103.62:44831 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (11513 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (25294 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (365301 ms total)
[  PASSED  ] 7 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] LeaderRebalancerTest.AddTserver

 1 FAILED TEST
I20260709 13:10:13.002008 14748 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 423 messages since previous log ~14 seconds ago
I20260709 13:10:13.002295 14748 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 422 messages since previous log ~14 seconds ago
I20260709 13:10:13.002485 14748 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 420 messages since previous log ~14 seconds ago
I20260709 13:10:13.002682 14748 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~33 seconds ago
I20260709 13:10:13.002938 14748 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~33 seconds ago
I20260709 13:10:13.003178 14748 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 1 messages since previous log ~123 seconds ago
I20260709 13:10:13.003459 14748 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 9 messages since previous log ~0 seconds ago
I20260709 13:10:13.003799 14748 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 122 messages since previous log ~33 seconds ago
